Faster way to write waveform data to file

[egyptiandude]

Hello everyone! This is my first post here so please be gentle =] 

I am creating a data acquisition + digital output system and am running into issues acquiring data AND writing the data to a tdms file. 

-The module must divide a 10 second time interval into 20 500ms blocks. Data will be acquired during the entire 10 second interval and written to file.

-During the first block (block 0), a digital output is set high

-Based on the number of peaks detected at the input during block 0, another digital output will then be turned on for the channel(s) meeting the peak condition for the duration of a given number of blocks (set by user).

-At the end of 10 seconds, this process will loop

If you look at my code, the while loop with the read function normally has a "write to measurement file" vi in it, however, when it's there, I get the error: "Attempted to read samples that are no longer available. You can fix this by increasing the buffer size, etc."

As soon as I take that write to file out of the loop, the module runs fine with no errors, IF I am not using a program in the background, like Chrome (this also triggers the same error).

I need this program to run at a minimum of 24 hours, so it's important that this can run for a long period of time. One idea I thought of was to buffer the data and then write it to file all at once after the 10 seconds... but am unsure how to implement this. Also, I don't know if something about the way I've written the code kills processing power and causes this error.

Any help is greatly appreciated!

[Todd_Lesher]

Why are you writing to the terminals and to the local variables? Are you running this on an RT system? A couple of things to try:

  Put a "DAQmx Configure Logging.vi" before the While Loop.

  Get rid of the Write To Measurement File express VI and do your own file management

  Create a Queue outside the While Loop, enqueue the data inside the While Loop, and write to file in another loop that dequeues the data.

[egyptiandude]

Hi Todd, thanks for such a swift response.

I am not running this on a real time system. The DAQ is sampling 16 channels and based on the peak count of each channel, I want to have a digital output go high, which I will add in the empty while loop. In that while loop, I will access the values in the local variables for each channel.

Out of curiosity, why use a queue as opposed to an array? Doesn't the mxread output a waveform which can be easily decomposed into an array? I'm new to labview so please, bear with me.

I'll give it a shot and let you know what I come up with.

[Todd_Lesher]

---

@egyptiandude wrote:
I am not running this on a real time system. The DAQ is sampling 16 channels and based on the peak count of each channel, I want to have a digital output go high, which I will add in the empty while loop. In that while loop, I will access the values in the local variables for each channel.

---

Writing to a local variable is the same thing as writing to a terminal - only slower. No need to do it twice in the same place.

The Queue has a data type. In your case it would be an array of waveforms.

[johnsold]

What Todd is describing is a simplified form of the Producer/Consumer architecture. This uses parallel loops to separate the data acquisition and file write operations. The queue is used to pass the waveform data from one loop to the other.

By using the hardware clock on the data acquisition device and parallel loops you should be able to get reliable acquisition and logging without timed loops.

Also your two regular while loops are what is called "greedy" loops.  They contain no Wait functions and will try to run as fast as possible  and may consume all available CPU resources. Place a Wait (ms) function inside each loop. 100 milliseconds is probably a reasonable value for each loop. They do not actaully seem to do anything. Are they there as placeholders for something you will be doing later?

Lynn

[johnsold]

Why not simply expand the # found array indicator to display 16 elements rather than making 16 copies of the data?

Lynn

[egyptiandude]

Hi John, The first loop keeps track of the individual "blocks" that the time is being broken into. This is necessary for future logic such as: if (block == 1){execute}

The second loop was to store the peak values to do similar logic: if (block == 1 && peak>3){etc}

Because I am working with 500 ms blocks, won't 100 ms wait times kill the synchronization that I'm trying to achieve and result in a loss of 1/5 of my data?

Thanks for the help,V

[egyptiandude]

Good idea, I didn't know that was an option! Have only been using the program for a week, so I am still picking things up 🙂

[Todd_Lesher]

Your 500ms loop isn't doing anything except updating the Block Number display. It doesn't affect anything else in the code. The LED STOP loop doesn't really do anything at all. What do you expect it to do?

[Todd_Lesher]

---

@egyptiandude wrote:
Because I am working with 500 ms blocks, won't 100 ms wait times kill the synchronization that I'm trying to achieve and result in a loss of 1/5 of my data?

---

No, because the Elapsed Time VI will simply run less often.