LabVIEW
Text file appears to have gotten too large to write to it fast enough even with the file reference open between saves
Solved!
I have a program I'm developing where I write to a text file. There's a save time which queues an "Acquire" state for the consumer loop to handle. I've been testing the limits of the program before I hand it off to my coworker who will be operating it, and I've run into an issue that I'm having trouble tracking down the root cause.
The save timer is set to 1 second, so every second the consumer loop goes through its Acquire state and logs data to a text file which is kept open until the test is completed. I've been letting the program run for a few days at a time while keeping an eye on file sizes and the size of the queue as well. For a while everything seemed to be fine, and the file i/o actions didn't clog up the queue, but this morning I found that the queue had become backed up and was growing at a steady rate of one element per second. This suggested to me that there was a function somewhere in the code that was preventing an Acquire state from completing.
My thought is that the text file I'm writing to is getting too large. After a few days of writing to the file it gets to be a few 10,000 KB - upwards of 50,000. Since the write to file occurs every second, I elected to keep the file reference open instead of running the Open File VI and Close File VI every Acquire occurrence.
I've watched the queue size, and it never seemed to falter. And when it did falter it was overnight so I couldn't see what might have happened. I was under the impression that keeping the file reference open would help avoid issues, but was I wrong? Is there still a point where the file could get too large to handle even if it's already open? The files I'd been writing to were about 55,000 KB this morning, and since the queue size was steadily increasing I believe this is the size the file was when the program failed.
Let me know if there's any subVIs in the attached VI that you'd like to see.
Thank you for the reply! I should have added more information, this is running on a cRIO 9045, so it has nothing to do with Windows. I no longer think that there's an issue with the file size - in fact I think that I have discovered the issue.
I have seen behavior before where there's a stretch of time on the XY plot that doesn't have data being written to it once per second. See below at 9:09 am.
I copied the XY plot data to my clipboard and moved that to Excel, and there were missing data points around 9:09. Then I went into the text file and checked around 9:09 am, and sure enough there were about 60 data points all at 9:09:58 am. This suggested to me that the queued-up Acquire states all unloaded in the span of a second (see following picture).
I thought it was odd that the XY plot writing loop and the consumer loops both got stalled, but the other loops in the program were working fine (queue size kept increasing which meant the save timer was working, front panel wasn't stuck on any particular tab, and indicators were updating normally), so I looked at the XY plot writing loop and the Acquire state of the consumer loops to see what they both shared in common.
In both loops I have a subVI that pulls various data from several global variables. In the Acquire stat this subVI pulls the data and then sends it to the text file. In the XY plot writing loop it pulls the data that is sent to the XY plot. On a whim I checked the VI properties and noticed that in the context help for "Non-reentrant execution" it mentions that you should only use "Preallocated clone reentrant execution" on Real-Time targets. Since I'm using a cRIO I think that this means I need to go into all of my subVIs and set them all to Preallocated clone reentrant execution.
Based on some of the things I'm reading online, multiple calls to a Non-reentrant subVI can cause hangups in the loops they are being called at, and this agrees with the fact that only the loops with this subVI were stalled.
I appreciate you taking the time to reply to me! I'm going to try changing all of the subVIs that I have to Preallocated clone and see if that solves things.
Back to say that even converting all my subVIs that are called multiple times did not fix the issue. I had the same thing occur where the queue got clogged up with a minute's worth of Acquire state messages.
The weird thing is that it's been exactly a minute's worth both times that I've caught it.
Made an interesting observation just now....could be coincidence, but could be something
So I'm running a remote front panel so that my coworker who I'm building this for can run the program without having the development software installed. Sometimes I will open the front panel and see this. It's just a matter of refreshing the page to open it up again. BUT I noticed a few times that the weird blip on the XY plot a couple comments up would have occurred shortly before losing the connection with the remote panel. I thought it was just a coincidence, and it could still be because I don't know if it's possible to force this to occur programmatically to check it, BUT it seemed like an interesting coincidence no less. Best I can do so far is put my computer to sleep for ~5 min and hope that when I come back to the remote panel it's been disconnected.
I'm going to fiddle around with the remote panel property invoke nodes to see if I can find a way to programmatically induce this connection loss and then see if this is what's causing the blip.
