Touchscreen Usage Discussion

[BradM]

Introduction

I wanted to get the pulse on who's using touchscreen displays with some of NI's RT Targets along with some details on experience and driving motivators for the display purchased. Basically, NI's considering purchasing different display types to refine the experience for future software releases (and to document setup for existing software releases). The existing, long-standing discussion on eGalax-based touch controller screens prompted this work, but we want to see if there are other issues out there.

What we're looking for from you

If you've purchased an NI RT controller with the intent on using the embedded display functionality with touchscreen input:

1. Which monitor or monitors did you purchase?
   • What were some of the driving factors that led you to this monitor or monitors?
2. Did it work out of the box for you?
3. If not, were you able to get it working? Using what resources?
4. A listing of the command lsusb -v so we know what touch controller's being used (or, if it's not a USB device, note how the device connects to the controller)
   

We'll use the results of the feedback from this document to guide our work.

[BradM]

Anyone? Bueller? I know there are folks using touchscreens, and it can only help if NI has a better understanding of the touchscreens in use.

[simplemaan]

Hi.  I just completed doing some testing using a cRIO-9030 loaned to me by my NI DSM (Thanks Bryan Heslop!)  This was for the express purpose of investigating whether it could be used for a potential client application.

I used it with LabVIEW 2015 RT, an Asus VT207N 19.5in touchscreen monitor with mini-dP to VGA converter and the USB input.  My experience was good on the side of touchscreen input for pushing buttons and so forth.  I was impressed that the cRIO recognized the touchscreen on boot and offered to calibrate.

I think it is a viable platform but currently have these questions that I've been working to resolve:

1. Keyboard input:  I couldn't find a workable solution for keyboard entry in touchscreen (only) mode.  I can, of course plug a USB keyboard into the cRIO, but that defeats the idea of a panel HMI.  I'm in the process of googling, searching NI community and the xfce documentation, but no one seems to have even asked this question that I can see.

2. Pull-down menus:  Contrary to what has been suggested in RT documentation I've read, I've successfully implemented a customized run-time menu for a LabVIEW application, but it only works correctly if the VI is a startup application.  If the VI is stopped and restarted, some of the menu items get overwritten with the default LabVIEW menu items.  Also, I can't find a setting to increase the text size on the menu items.  They are awfully small for an adult index finger.

3. I completely gave up on using touchscreen for these things: double-taps, changing chart/graph axes limits.  Setting the focus area to maximum (15 pixels) didn't seem to help and also what good is it to select the text if I can't edit it.

4. Other items:  I wasn't able to get correct functionality from a VI invoke node to maximize the front panel.  This means in order to run in Kiosk mode, I would have to statically size the front panel to the maximum resolution.  Not a huge deal.  Also, although I did try using splitter bars so the a graph area and a bottom of window splitter bar would work with resizing and everything displayed ok, but I wasn't actually able to change the window size.  Also it only displayed correctly if the VI was a startup and there were minor positioning errors (probably because of scrollbar hiding) if I restarted a VI.  I'm not sure splitter bars are intended to be supported, but I think programmatically maximizing a front panel should be and splitter bars would make this more flexible with different sized monitors.  Again, this is workable through static VI sizing.

[BradM]

simplemaan wrote:

...

I used it with LabVIEW 2015 RT, an Asus VT207N 19.5in touchscreen monitor with mini-dP to VGA converter and the USB input.  My experience was good on the side of touchscreen input for pushing buttons and so forth.  I was impressed that the cRIO recognized the touchscreen on boot and offered to calibrate.

I'm glad to hear that not all non-NI monitors have had the sorts of issues that seem to popup on the forums. We do intend on improving the experience for all, which is the very purpose of this discussion. Thanks for the input.

I think it is a viable platform but currently have these questions that I've been working to resolve:

1. Keyboard input:  I couldn't find a workable solution for keyboard entry in touchscreen (only) mode.  I can, of course plug a USB keyboard into the cRIO, but that defeats the idea of a panel HMI.  I'm in the process of googling, searching NI community and the xfce documentation, but no one seems to have even asked this question that I can see.

There was some work done a bit back, did you see Florence Virtual Keyboard ?

2. Pull-down menus:  Contrary to what has been suggested in RT documentation I've read, I've successfully implemented a customized run-time menu for a LabVIEW application, but it only works correctly if the VI is a startup application.  If the VI is stopped and restarted, some of the menu items get overwritten with the default LabVIEW menu items.  Also, I can't find a setting to increase the text size on the menu items.  They are awfully small for an adult index finger.

3. I completely gave up on using touchscreen for these things: double-taps, changing chart/graph axes limits.  Setting the focus area to maximum (15 pixels) didn't seem to help and also what good is it to select the text if I can't edit it.

4. Other items:  I wasn't able to get correct functionality from a VI invoke node to maximize the front panel.  This means in order to run in Kiosk mode, I would have to statically size the front panel to the maximum resolution.  Not a huge deal.  Also, although I did try using splitter bars so the a graph area and a bottom of window splitter bar would work with resizing and everything displayed ok, but I wasn't actually able to change the window size.  Also it only displayed correctly if the VI was a startup and there were minor positioning errors (probably because of scrollbar hiding) if I restarted a VI.  I'm not sure splitter bars are intended to be supported, but I think programmatically maximizing a front panel should be and splitter bars would make this more flexible with different sized monitors.  Again, this is workable through static VI sizing.

I bucket all of these issues into something that NI is aware of, and I think that's all that can be said of these issues. Getting a better feel for how many are affected by these sorts of problems can help prioritize the fixes.

[simplemaan]

I knew this would happen AFTER I posted, but yes I did find the Florence Virtual keyboard.  Unfortunately, I won't have a chance to test it this go-around.  I will tell my customer though.

I'm going to make a comment that I'm sure NI has also thought of, but IMHO the virtual keyboard functionality should be included "out of the box" as the touchscreen calibration routine is.  I recognize that one of the cool things about Linux RT is the ability to add packages for those who need/want that functionality, but a virtual keyboard should be a core feature that is only upgraded/maintained when the cRIO firmware is changed through the usual upgrade procedure through NI MAX.  Asking someone to use download a package (even though it is NI-maintained) and then enter console commands doesn't engender the same confidence.  If someone starts customizing the OS, then they are willing to accept this, but for someone who wants a "stock" LabVIEW RT experience it's a downgrade.

Thanks for your quick answers to my feedback!  Please consider me very interested in this topic!

[BradM]

It may not engender the same confidence, but that's reasonable as it has not seen as much testing time. It was unclear whether the normal use-case for these UI-enabled targets would need an onscreen keyboard or if they were going to be chiefly for simple I/O type of controls/indicators (large buttons, slide switches, etc.)

That said, this overall is just the sort of information I was looking for with this, thank you.

[James@Work]

simplemaan wrote:

...but IMHO the virtual keyboard functionality should be included "out of the box" ...

Exactly!

[felipefoz]

We'be trying to use Florence Virtual Keyboard and we have not succeded.

First of all, the default package installed is not showing correctly, as some buttons in keyboard appears in black. Some solutions in community advised to even recompile the code for this OS.

Second, we were not able to integrate this keyboard with system, as it needs to be launched and it sticks in the screen independent of what is going on in there, so it messes up the usability of the labview software running in the touch screen.

So far, we've been researching a workaround for this, as this solution was already chosen, we can't look for alternatives in the moment.

[BradM]

felipefoz, James@Fann,

This is the type of feedback that I was looking for, thanks for sharing. It may be worth checking out the configuration UI by launching florence with the -c flag from a terminal.

[felipefoz]

Thanks for the reply.

This launches florence settings, right? It is useful but it doesn't solve all our problems.

I have noticed the configurations diverge a litte bit from those shown in florence web site. I.e. missing behaviour tab.