RT datalogging (saving) strategy

[charmand79]

Hi all,

I have been working on a RT project (using cRIO 9074 together with two 9014 via ethercat) for quite a while now and I wanted to ask this question since I started.. 

After lots of forum searching and practicing I have decided that the best way to proceed with data logic, conditioning and saving would be to have four separate timed loops; one loop running at 25 ms dealing with datalogging and logic for 10 channels, another loop running at 250ms dealing with 20+ channels, one at 1s dealing with 250+ channels, and the saving loop, running every 10 secs.. Each loop builds a 2D array (using Uninitialised Shift Register) passing it to the 10 sec save loop.The save loop was then appending all new BINARY data to ONE file, which was created at the beginning of the vi and closed upon vi exit.

It all worked sufficiently (no finished lates and hang ups) until I decided to introduce the NI 9802 module to increase my storage (had to do it as the 300MBs of the cRIO hard drive was way too little). It appeared that appending the approx. 250x300 array of binary SGL data was perhaps too much for it as I started getting hang ups every 5-6 iterations or so. So instead of appending the data to the same file, I decided to write the data each time to a new file (i.e. 10 sec loop now creates a new file writes the 250x300 SGL array and then closes the file). There was considerable improvement, hang ups are now less frequent, but still I DO get some RIO hang ups.

If you havent been bored to death by now, could anyone suggest an alternative strategy? Is there something wrong with my strategy? I cannot believe that cRIO cannot deal with my datalogging requirements. And i say that because if I dont have my saving routine running, then all my loops finish at <10% of the timed loop period setting!! So, its definitely my saving that messes things up.. 

I have added my whole project just in case somebody wanted to have a look, main VIs of interest:

running on RIO "AMR-cRIO9074/RIO/VIs/Main Comms/RIO_Main/Save_All_Data"

and running on HOST "My Computer/Host/UI/Host_PC"

I have to warn you though. The project IS V.V. LARGE so I'd stick to a general suggestion...

One last question: What is the exact effect of the Network Publishing Period value on the performance of the cRIO system??  I have set mine at 100ms, while my scan engine is at 25 ms. My "fastest" net variable in my whole project is being read every 250 ms...

Thanks very much for your time,

Harry

[Sanjay_C.]

Hi charmand,

Are you accessing the SD card from the RT (as in you're mounting it as a drive and then writing to it) or are you accessing the SD card from the FPGA? Honestly, I think that the resolution to this issue is going to lie in you doing some benchmarking with a few differeent approaches in writing to the SD card. It's interesting to find that writing a number of separate files is less taxing on the controller than one large file, as it suggests that keeping the reference open to the SD card is expensive. I'd also recommend that you try to build off of/steal from the shipping examples which illustrate how to read/write to the SD card. You can find these examples in the example finder by following through: Hardware Input and Ouput » CompactRIO » Module Specific » File Input and Output. You'll find that there are shipping examples which illustrate how to properly communicate with the 9802 from the RT as well as from the FPGA. It also seems that if you are accessing the SD card directly from the RT, you must have an FPGA VI (even if it's just a blank).

[charmand79]

Sanjay,

I am accessing the card from the RT and I do have an FPGA vi which I had to recompile upon introduction of the 9802 module. To be honest I have now decided to skip the 9802 module and the SD card option and return back to saving in the RIO hard drive.. It seems that the 9802 module is way way way too slow c.f. to the specifications given by NI (i.e. 2MB/s for FPGA usage and 300 or so kB/s for RT usage). I am positive I am doing things the right way (checked all examples and searched all relative posts) but it just doesnt work right...

Thanks for the reply,

Harry

[Sanjay_C.]

Hey,

This is a fairly new product, and as such, it's very possible that it's far from as efficient as it will be/needs to be. You might see a very meaningful gain by passing the information from the RT down to the FPGA via a DMA FIFO so that you can make use of the much better (2 MB/s) data rate and write to the SD card from the FPGA. I'm not sure if it's a feasible option for your application, but it might avoid all the issues you are facing with the lag on the RT side when trying to write to the SD card since you're taking the load off the processor, and implementing the write to file directly in hardware (FPGA).