02-18-2016 02:58 PM
I've been debugging this issue for some time and cannot find a reason to explain why.
I am using a cDAQ-9178 with eight slots. Three slots are thermocouple (TC) cards, 9213 and the other two are a voltage and current card, 9375 and 9208 respectively.
Two loops are reading the different cards depending on the data type. One loop monitors the voltages to count pulses from flow meters, the second reads the TC and current cards.
The voltage loop continuously counts falling edge pluses and loads the total onto a queue every second. When the voltage loop triggers a total to be loaded, a notifier trips the bottom loop to record the TC and current reading.
The code is continuously running and dequeuing elements that are sent to a database table, where a new row is generated with the newly recorded data.
Around the same time, been this way the last two days, the DAQmx read vi will freeze and does not trigger the timeout error. When opening the code and using the highlight operation, the code shows green arrows on the two DAQmx read vi's in the bottom loop. See the screen shot of the bottom loop below. (Sorry don't have a screen shot of the actual frozen code)
As I said, this has happened the last two days at around 4:30 am. The strange thing is, I cannot stop the code from the main front panel because any controls are not registered by the main vi due to the fozen DAQmx read vi.
I can either unplug the cRIO which allows the front panel to be read, and upon shutdown, an USB communication error pops up. If I am not at the PC, I have to restart the computer to restart the code. Otherwise, if ctrl-alt-del is used to shut down the main vi, upon restart, the system cannot recognize the cRIO being connected to the PC, and hence must be shutdown.
Any advice or steps to help debug the issue would be great.
Thank you in advance for any help, I've been left scratching my head what the problem could be.
Memory limition on the PC or cRIO or the USB connection is too slow. The strange thing is, the code works well 99% of the time.
I've attached the two main vi's that should cover the scope of the problem.
02-19-2016 09:36 AM - edited 02-19-2016 09:36 AM
Hey scaskey,
I am looking over your post and believe I understand the issue, but to get some context could you clarify a few things?
1. Where is the cRIO in this system? You mentioned at the beginning you were using a cDAQ chassis with 5 cards but I am unsure of how the cRIO fits in this system.
2. What modules/tool kits are you using with this? It seems some of the VIs are missing when trying to open the attached files. (Perhaps there are custom subVIs)
3. Have you monitored PC usage or memory while running this VI to see if you are exceeding your capabilities?
02-19-2016 11:14 AM
Hello Eric,
Ok great! Any ideas you have would be very helpful.
1. The cRIO is next the PC and is connected via a USB connection. The 5 cards in the cRIO are monitoring temperatures, relative humidities, and water flow rates in a home. The VI accesses the cRIO every second via a DAQmx read subVIs and then separately records the readings.
2. These are custom subVIs. I can attach if needed but I don't think they are causing the issue. They are either applying the calibration fit to the temperature points, or monitoring the voltage reading to generate a count during a falling edge. The only toolkit that should be used is the DAQmx for the main VI.
3. I have not tried doing this. The PC running the main VI has 16 GB of RAM. Do you have a good way to monitor the PC usage outside of the display available when pressing ctrl-alt-del?
Thank you!
Stephen
02-19-2016 12:23 PM - edited 02-19-2016 12:24 PM
Hello again scaskey,
Thanks for the quick response!
1. What is the chassis you are using (model number). You mentioned a cDAQ - 9178, but then mentioned that you have these modules in a cRIO. Which is correct?
2. Don't worry about loading any custom subVIs, as the next step should do for you what I hoped to do on my end.
3. To monitor performance and memory allocation, in your VI, you can navigate to the menu bar at the top and select TOOLS >> PROFILE >> PERFORMANCE AND MEMORY. This will open up a pop up with information that is blank, and then you can run the VI and this will populate with data. This will allow you to see how much memory and cpu usage is occuring while running the VI.
02-19-2016 01:14 PM
Hello Eric,
1. My apologies. It is a cDAQ - 9178 with 5 modules taking up the 8 available slots.
2. Ok sounds great.
3. I've followed your steps and have started the monitoring. I can click the snapshot button and see the various VI times, and the average bytes used throughout the code. Is there a level that should not be exceeded on the memory from the various components of the VI?
I've attached two images that show the VI memory usage from the various VIs, subVIs, and other taks, as well as a screenshot of the PC system resources I'm running the VI on.
Please let me know if anything here looks to be a cause of concern or if the code requires rewriting to reduce the load on memory.
Thank you again!
Stephen
02-19-2016 01:37 PM
scaskey,
Thanks for getting back so quickly. Not a problem, I am glad we've clarified the system.
Are you receiving any error on the front panel in your Error Out cluster when this occurs? It looks like the default timeout on your DAQmx Read.vi is the default 10 seconds. This should then throw an error if it is unable to find a piece of data to read, which would appear in the Error Out cluster.
Another question: Have you tried running this setup on a different PC? I've seen issues like this in the past where the issue was computer specific. I'd be curious to see if this issue arises on a different computer. Do you have access to one for this option?
Thanks for your cooperation!
02-19-2016 02:09 PM
Hello Eric,
That is the strange thing. No error is received. As you mentioned, I would expect a timeout error. I ran one version of this code where I gave a 4 second input to the DAQmx read to see if this would produce an error. No luck there.
When I see the VI has froze, I can access the wiring pane, select highlight execution, and I see within the data_writing subVI, there are green arrows on the DAQmx read subVI. The front panel stop button on the main cannot be recognized since it is hung up within the reading subVI.
During previous versions of the code, I could generate one error that would be registered during this frozen state and allow the main VI to generate the stop notifer. This occured when I unplugged the cDAQ. An USB communication error would pop-up upon shutdown. Other than this, I had no other way to stop the VI other than a hard shutdown using ctrl-alt-del or by manually closing the front panel with the close window button.
I have not tried running this on another PC. Unforantely this would not be an easy task since the software on the PC would need to be copied over to the second PC.
Did you see anything of concern with the memory usage of the VI?
Thank you,
Stephen
02-19-2016 02:22 PM
Hey Stephen,
I didn't see anything too fishy in the Resource Monitor, but I'd like to see a screenshot of the Memory tab in that. I am hoping the graph on this page for memory history shows a steady value. If it shows an increasing value, there may be a memory leak that could be causing this issue. If you could attach a screenshot of that memory tab in the resource monitor, that would be fantastic.
Your CPU usage is slightly high, but doesn't look high enough to cause issues with this program.
Also, what versions of LabVIEW and DAQmx are you using?
Thanks!
02-19-2016 02:41 PM
Hello Eric,
Ok that is good to hear then. I've attached a screenshot of the memory tab for your reference.
The version of LabVIEW being used is 13.0.1f2 (32-bit). Memory Allocated 121054K.
The version of DAQmx being used is 15.0.1.
Also the code recently froze on me again. I've attached two more images on this. The first shows the location where the code is hung up at, frozen_state_vi. Here you can see three of the four DAQmx reads are where the code is at.
The second image, ...error_generated_by_unplugging_cDAQ, is how I can kick the VI out of its frozen state by plugging the cDAQ. The top loop DAQmx read VI is the first to generate an error code followed by the bottom two read VIs. Then I can saftely shut the program down and restart the code without needing to restart the PC. Otherwise, if I am not a the PC, and have to force LabVIEW to shut-down, upon restart, the cDAQ isn't recognized by the VI.
Hopefully this helps to add to the discussion.
Thank you,
Stephen
02-22-2016 09:35 AM
Hello again scaskey,
Have you tried downloading DAQmx 15.1.1f3? This is the newest version and most up to date. It is compatible with LabVIEW 13.0.1f2. As 15.0.1 is also compatible with LabVIEW 13.0.1f2, I can't guarantee this will fix it, but it is always good to have the latest version, as bug fixes are applied through these updates.
Have you tried running any of the shipping examples of similar acquisition type for an extended period of time? I'd be curious to see if your DAQmx Read VIs freeze when just using doing the DAQmx portion (2 Digital Input tasks and 2 AI tasks).
Let me know if either of these suggestions help!