How do I save data as I aquire without getting read error?

[jgolden]

I am using a USB 6251, programming with CVI 8.  I want to acquire 4 channels at 60-90kHz for 5 min.  I want to save the acquired data, so I set It up to save formatted ascii data each time the "everyNsamples" function runs, which runs whenever the board has acquired a certain number of samples.  After its done acquiring, the user can get that data and save it into a file.   The problem is that I'm getting an "attempt to read sample that is no longer available" error while its saving during the acquire operation.   I've set up the number of samples such that it runs the "everyNsamples" routine about once a second. 

I've seen the example programs that show you can display data on screen, but none of them show you how to save data while you acquire.  How do people normally save acquired data?  Do you save into RAM while acquiring?  If you save to a file while acquiring, how do you avoid the read error?  Do I have to learn how to do multithreading (ack!)?

This seems like a very basic problem that should have a very simple common solution. 

thanks,

Joel

[ebalci]

Hi Joel,

USB-6251 has an onboard memory for only 4095 samples (look at the Specifications tab at http://sine.ni.com/nips/cds/view/p/lang/en/nid/202597).

Since you configure the EveryNSamples function to be about called every second, this corresponds to 60-90 thousand samples, which is way more than the board can store.

Hence your samples are overwritten.

Probably that's why you get that error. Try to assign a lower N, like 2048. This corresponds to a 22-35 millisec of acquisition period, which is slow enough for most PCs.

Hope this helps, 

[jgolden]

Thanks for your response.  I tried changing the number of samples to 2048 as you suggested, but I got a message: “Requested everyNsample event interval is not supported for the given data transfer mechanism and buffer size”

 

I’m a little confused.  I am doing continuous sampling:  DAQmxCfgSampClkTiming sample mode set to DAQmx_Val_ContSamps.  The description in the CVI help says that the samples per channel to acquire is set to certain sizes depending on the sample rate.  For 60kHz the chart says the number of samples is 100,000, which is way larger than 4095.  Does this mean I cant do continuous sampling with the 6251?

[ebalci]

I am not experienced with USB data acquisition devices, but I know DMA cannot be used as a transfer mechanism for USB.

So they may not be very fast , but you should surely be able to perform continious sampling.

Apperantly, the configured sample count for the event and the buffer size do not get along.

What's your buffer size? Can you put some of your code in your next post?

Maybe you can post the question to the DAQ forum, too.

Besides there are other ways of getting informed as samples are taken.

Have you tried using DAQmxRegisterSignalEvent function with DAQmx_Val_SampleCompleteEvent as the signal parameter?

[jgolden]

I also tried making the buffer an integer multiple of the sample size, and even tried making it the same size, to no avial.

Before I try posting code (which is on a different computer than this one) I will say that I am convinced the problem is related to the data saving procedure in the everyNsamples function.   If I sample at 60 or 600 Hz, it all works perfectly.  If I sample at 60kHz, it gives the error immediately.  If I comment out the fprintf command, I dont get an error until after its done taking all the allotted samples.   So perhaps it is the time that it takes to save the data that is making a delay so long that the sample data isnt available anymore. 

If this is the case, how does anyone save data at 1Ms/s?

Joel

[CharlesD]

ebalci is correct in that USB devices cannot do DMA transfers, but only USB single streams and Programmed I/O.  Also your USB 6251 should be able to sample at a maximum rate of

- 1.25 MS/s single channel,
- 1.00 MS/s multi-channel
(aggregate)

The on board buffer size is 4095 but this is a "backlog" buffer size, meaning that if you cannot transfer the data to your computer fast enough it allows the card to store some of the data until your computer is ready to transfer the data to the computer (4095 samples).  In all cases if your DAQ device is sampling at 60kHZ it will try to transfer I believe that the error code that you are getting is actually because of the DAQmx buffer being overwritten and not thecard's buffer being overwritten.

usually there are two buffer errors possible:

1.  ERROR 200279(labview error #) Attempted to read samples that are no longer available. The requested sample was previously available, but has since been overwritten. Icreasing the buffer size, reading the data more frequently, or specifying a fixed number of samples to read instead of reading all available samples might correct the problem.

2.   Measurements: Data was overwritten before it could be read by the system.  If Data Transfer Mechanism is Interrupts, try using DMA or USB Bulk. Otherwise, divide the input signal before taking the measurement.

The second error corresponds to the data not being transfered from the card to the computer fast enough (problem with that 4095 buffer).  On the other hand the first error which is the one I believe you are getting is actually corresponding to the daqmx buffer being overwritten because your program is not pulling the data off quickly enough from the DAQmx buffer.  Basically the solution to this is the following: 

1. Increase the DAQmx buffer size using the DAQmx_Buf_input_BufSize function.

2. Increase the number of samples being read every iteration or read all of the samples available each iteration to ensure best operation.

3. Do not write data to file at the same time (same iteration as your daqmx read)  this is important because each time you read from the daq device you are increasing the time in between each iteration and thereforenot pulling the data off of the card quickly enough. If you need to do this the best way to do this is a producer-consumer architecture (multi-threading). More information on this is found here:

http://zone.ni.com/devzone/cda/tut/p/id/3023

Example finder: CVI Help --> Find Examples --> Optimizing Applications --> Multithreading

Let us know how this shapes up!

2.DAQmx_Buf_Input_BufSize

[jgolden]

Thanks for your suggestions.

Yes, the error is "attempted to read samples that are no longer available".

1.  I'm confused about the different buffers.  There's the 4095 sample buffer on the device, and a DAQmx buffer, and a RAM buffer that I create with a malloc command which is where the data goes during the EveryNCallback using the DAQmxReadAnalogF64 command.  If I increase the DAQmx buffer, doesnt that just increase the number of samples for the EveryNCallback function?

2.  If I increase the number of samples being read for every iteration, doesnt that just increase the number of samples for the EveryNCallback function?  It will take even longer to write to disk.

3.  If I dont write the data to file during the EveryNCallback, which is where the daqmx read is, where else in the program can I write the data to disk? 

It looks like I'm going to have to learn how to do multithreading, which could take some time.  I dont mind learning something new, but I am not convinced this is the correct solution and I'm worried it will be a waste of time.  It seems to me this is a very basic problem that everyone (but me) has solved and there should be a simple example somwhere of how to save the data you acquire. 

 thanks,

Joel

[ebalci]

Hi Joel,

First of all be sure, you are running the release version executable. Debug versions are much slower.

Second, create a thread using CmtScheduleThreadPoolFuncAdv so that you can adjust the priority of your thread and set the priority to 'highest'.

[ I deliberately did not say 'time critical' since I heard that may ruin other things, but still you can give it a try ]

And try to code your application as polling loop (this loop is executed by the thread created above) instead of the callback approach.

Third, do not use a text file and try to format the data before you write. Just stream all data as binary and process the file later when required.

You can consider using the NI TDM Streaming library, which is supposed to be faster than text or binary. However, I never tried that myself.

These may improve things considerably.

Hope this helps, 

[CharlesD]

Hello jGolden,

1. The DAQmx buffer is the RAM buffer.  it is the buffer allocated by DAQmx in RAM memory. 

2. So yes you are correc that it will take longer to write that amount of stuff to file but at the same time you will have more room in your buffer to fill up because you have taken more samples out of the buffer because you read more samples.  

So the problem here is that you are trying to write this data to file while reading the data.  One option could be to store the data in an array for the duration of your daqmx read.  Once youare doing acquiring the data then call a function that will save all of that data to a file, (post-processing).

The second option - the more elegant option would be to learn how to multi-thread. 

Good luck!