DAQmx Overwrite Samples Bug

[GregFreeman]

When using DAQmx overwrite samples there is what we believe to be a bug in the buffer. We are reading the same channel 10 times every 2 milliseconds. Over time, the same data point should shift ~2ms to the left. Instead, we see multiple bugs. The first is that when the exact number of samples to read is set to a constant which equals the size of the buffer, we read the same waveform 10 different times. This is shown by the data from every read lying exactly on top of the data from every subsequent read. There is no shift.

 

Now, if we put an arbitrary property node before doing a read or we wire up -1 to the number of samples to read (which is the whole buffer) the waveform shifts almost as expected. However, you can see a massive shift in the data between point 1 and point 2 of about 60 ms and from that point forward there is the expected 2 ms time difference. If we put timing in the attached sample VI, however, the loop only takes 2 ms to execute each time. This tells us that there isn't some long wait, but instead the data is being shifted too far in the circular buffer. 

You won't be able to reproduce this with simulated hardware. We are using a real 9234 and a single slot 9171 cDAQ chassis.

[GregFreeman]

Could only post 3 attachments. Here is the final image showing the long shift when my loop time was only 2 ms but the data point shifted 60 ms

[Kevin_Price]

Very curious indeed.   Got a few comments and observations from this end from some testing on a PCIe X-series card and a PCI M-series card.  Made a couple small mods as follows:

• Added DAQmx Buffer property node to explicitly set buffer size to 10000.
• Added enum & case to explicitly read entire buffer (10000 samples) rather than the magic 4267
• Performed a simple voltage reading, got errors when trying to directly designate accelerometers

1. Explicitly requesting the entire buffer size worth of samples always produced an error for me.  Data was returned on the first iteration only.  Are you sure you got 10 overlaid iterations or did you only produce data on the first?   (Either way would look exactly the same as the 1st plot would be on top of, and obscure any identical plots underneath).

2. The call with -1 does *not* read the entire buffer.  It merely reads all *available* samples.   Due to the settings for "RelativeTo" and "Offset", you are always telling it to read from the offset marker point to the most recent sample.  That # samples is always 4267.   So it should behave the same as an explicit request for 4267 samples using the same "RelativeTo" and "Offset".

   Over here, it does.   I never saw the anomalies with the explicit request for 4267 samples over here.  It behaved the same as the -1 request with pretty impressively consistent 2msec intervals.  Below are screencaps from the explicit request case, one each with an X-series and M-series board.

3. Dunno why the property node read should matter, over here there's no anomaly to compare against so that case just behaves properly like the other ones.

-Kevin P

Screencaps show the ordered spikes at ~2msec spacing, first X-series then M-series.

[GregFreeman]

Thanks very much for looking into this and adding what you found. Later on we did put a while loop and read multiple times. I don't remember all the possible test cases we went through when we had the while loop there, but I don't believe we ever saw an error. Either way, something funny is going on because a property node changes our response haha! I do know those single slot cDAQs are pretty new so maybe the issue is with that?

To your point number 2, you are right I just worded it wrong. I think I meant "it should read all available samples which is equal to the entire buffer in this case because it is always filled." Unless of course I am still misunderstanding -1 in the case of this configuration, and then I stand to be corrected 
 

I believe our sales rep has gotten an NI AE involved and pointed them at this thread so hopefully the will see your response as well.

[drummaniac83]

Thanks for testing.  I'm working on this with Greg.  My speculation is it has something to do with the USB/DMA buffer not being updated correctly.

We did confirm there are actually 10 graphs in the error case(made other plots invisible to see hidden ones).

The exact number of samples you read doesn't matter, but more of the fact you're reading an explicit number of samples vs just reading all available.  The actual number of samples returned in both cases is the same.  By setting the offset to a negative value (N) from "most recent" the read function always has N samples available immediately.

I'm glad this doesn't show up in other card types.  I would be VERY interested if someone had a USB X-series to see if this is related to cDAQ or maybe the USB driver stack.

Thanks so much for testing this!

Josh

[drummaniac83]

---

@Kevin_Price wrote:

Very curious indeed.   Got a few comments and observations from this end from some testing on a PCIe X-series card and a PCI M-series card.  Made a couple small mods as follows:

• Added DAQmx Buffer property node to explicitly set buffer size to 10000.
• Added enum & case to explicitly read entire buffer (10000 samples) rather than the magic 4267
• Performed a simple voltage reading, got errors when trying to directly designate accelerometers

Can you post your VI mods so we can see if we get a behavior change?

Thanks!

Josh

[Kevin_Price]

Greg:

Try out this slight mod and you'll see that the -1 setting reads 4267 samples just like when you explicitly ask for 4267, due to the combination of "RelativeTo==MostRecentSample" and "Offset==-4267".

Josh: I attached my test code.  Just added a post-read query for "# Available Samples" to illustrate the point above.  Remove it if it interferes with reproducing the behavior Greg sees.

Further, FWIW, I noticed that my explicit requests to read the full 10000 samples from a size 10000 buffer behaved a little different with M-series and X-series, PCI-6251 and PCIe-6341 respectively.  The M-series consistently returned data only on the 1st iteration and thereafter gave me error -200279.  The X-series board usually returned no data but would occasionally give me 2 or 3 iterations worth before returning the same error.  I don't think either result is an issue, the issue is me asking the driver to deliver the entire buffer when it can only do so without error before the very next sample arrives in less than 1/17000 sec.  The way I see it, that's on me.  There are better ways to get the whole buffer of data using multiple reads.

Final note: I have a nagging thought that the choices for "RelativeTo" have changed.  Didn't there used to be selection for "FirstUnreadSample" or something like that?  I thought I'd once worked out a fairly straightforward scheme where one loop kept reading from a task in a continuous lossless manner while another loop could occasionally swoop in, grab a chunk of most recent samples, but then I could easily set the properties back where the other loop needed them to be for continued lossless acquisition.

I just recently was trying to do a similar thing and found a much more complicated solution than I think I remember doing in the past, but it seems there are some properties or settings missing now.   Any ideas of a simple-ish way to accomplish this?

-Kevin P

[drummaniac83]

Kevin,

It seems like the addition of the property node, even after the read, changes something behind the scenes and pseudo-fixes the problem.  It fixed the major issue of no buffer updates for me, but did not fix the minor issue of the large shift between first and sucessive reads.

Can you try your test code without the property node to see if you still see normal behavior?

Thanks,

Josh

[Kevin_Price]

I did most of my testing and experimenting before adding that post-read property node, but just to be sure I took it out and did a bunch of runs again.  Results were just like the earlier screencaps I posted.  The time intervals are all pretty consistent and the first one looks just like the rest.

-Kevin P

[drummaniac83]

Kevin,

Thanks a bunch. So it does sound like this is limited to either USB/cDAQ or even just the single slot module.

I'm going to try and find a USB daq and see if I can see the issue with that.

Thanks,

Josh