LabVIEW
SOLVED - Remote front panel disconnecting from inactivity causes stalls in continuous data logging due to invoke nodes
Hey everyone,
I wanted to share a new post about my solution I found to this post I had made the other day (https://forums.ni.com/t5/LabVIEW/Text-file-appears-to-have-gotten-too-large-to-write-to-it-fast/m-p/... ) since it ended up having nothing to do with what I originally thought it was about. This way if anyone in the future has issues with a remote front panel and stalled code they can find this new post and hopefully get some use out of my headache. Turns out I was stalling a couple sections of the code with invoke nodes that stall when the remote front panel becomes disconnected.
The problem I was facing...
I have a program I am working on for a coworker. It's an adaptation of some code I had written for my own test stand since his test stand shares a lot of functionality. But because he needs a much higher data save rate there were some unforeseen issues that came up.
My code is set up with a QMH architecture to handle all the file I/O features. There are various other subVIs and while loops in the main block diagram that handle conversions, front panel stuff, writing to XY plots, etc. What I was seeing was that every now and again I would see the indicator which tells me the QMH queue size start to increase, and the XY plots would go inactive. Then the program would go back to normal and process the backlog of "Acquire" events in a fraction of a second (the saved data in the text file would show 60 data points with the same timestamp), and the XY plots would begin plotting data only with a blip where the while loop was stalled.
This was frustrating because it meant my coworker would lose a minute of data seemingly randomly.
The investigation (and eventual solution)...
I noticed one time which this occurred that Notepad was not able to open the text file because it was too large (~55 MB), and wondered if file size was getting in the way of LabVIEW logging data. Hence the title of the original post.
I quickly ruled that out though when I witnessed the bug occur with a new save file that was only a couple hundred KB. Also it didn't make much sense anyway since LabVIEW should be able to handle a few 10,000 MB of data especially since I don't close and re-open the file reference between saves (1 Hz for roughly 20 days had me concerned that I was going to stall the program with file I/O). On top of that, why was the XY plot writing loop - which had nothing to do with the file I/O - getting stalled too?
Since file size was ruled out, I took a look at the things which were shared with both the XY plot writing loop and the "Acquire" state in my consumer loops. I have a subVI (circled in red below) which pulls specific data from a few global variable clusters. The subVI is used in the "Acquire" state and in the XY plot writing loop to pull the data to log to the file and to pull the data to display on the XY plots.
On a whim I checked the VI properties and noticed in Context Help for Execution that "Non-reentrant execution" suggested to used "Preallocated clone reentrant execution" on Real-Time targets. Since I'm running the program on a cRIO I was sure that this was my issue and I was running into a problem where the loops were both trying to use the same subVI at the exact same time or something to that effect.
I updated the properties of the subVI and deployed the code. Everything seemed fine at first, but then the next day I opened up the front panel to be sure and witnessed the blip on the XY plots again. So reentrancy was not the cause.
After scratching my head for a few days I made an observation about the timing of the blips. I noticed that it always occurred shortly after the front panel became disconnected. If my computer went to sleep or I just left the front panel open too long I would go back and see this screen. All I had to do was refresh the page and I'd be back to my front panel.
This got me thinking that if the bug had to do with the front panel, I should look at what similarities the loops have regarding actions done on the front panel. That's when I realized the invoke nodes in both loops. I have these invoke nodes set up to reinitialize a couple boolean controls to essentially latch them until they're processed. Scroll up a couple images back in this post and you'll see the invoke nodes. I removed these from the code temporarily, ran the code on the cRIO, induced a front panel disconnect, and sure enough there was no blip to be seen!
I came up with a quick fix. Since I don't need to reinitialize these controls every second - they only need reinitialized when they are true and after they've been processed - I put them inside a case structure which is connected to the value of the control and voila. Now the invoke nodes will only invoke anything when the buttons are in the true state, and the buttons will only be in the true state when the operator puts them into the true state which means that the front panel will be active and no blip will occur.
Maybe there's a more robust solution that involves priorities or another consumer loop to handle front panel invoke/property nodes, but for now this will do the trick perfectly. If anyone has any suggestions for a more robust solution they would still be appreciated!
Thanks!
