Producer consumer continuous data retrieval and storage

[Witschas82]

Dear all,

I am rather new in using Labview for sophisticated data acquisition and hope to not ask extremely stupid questions. What I am currently trying to do is the following:

I am working with a "Lidar" instrument which emits 500 laser pulses per second. From each laser pulse, I get an atmospheric return (time signal) which I sample with the niscope 5162 for 100 µs (50000 samples, 500 MHz sample rate, 8 bit). So my goal is just to store the data of each single laser pulse return on my HDD.

My first attempts ended in the attached .vi, which is principally saving the data very fast. But after a certain time (~1 minute) the memory runs full. 

So my question is, if there is an easy approach to get rid of that issue. E.g. using the laser trigger to start the acquisition of the 100µs and using the time to the next laser pulse to reinitialize the queue?

I hope that the explanation of my problem is reasonable and the you can provide help.

Any ideas are highly appreciated. 

All the best, Ben

[GerdW · Accepted solution]

Hi Ben,

---

@Witschas82 wrote:

My first attempts ended in the attached .vi, which is principally saving the data very fast. But after a certain time (~1 minute) the memory runs full. 

---

Which function gives you the "memory full" error?

Does it also happen when you switch off the plotting in the XY graph?

Btw. you don't need to set the file position to "current", that's a no-op! (And some error handling regarding file access might be useful…)

[Witschas82]

Hi Gerd, thanks a lot for the very fast and constructive reply. Indeed, when I switch off the plotting (and the power spectrum calculation also also had added), there is absolutely no problem with writing the data. This helped me a lot.

As it is very helpful to have the outgoing laser pulse monitored, I am wondering what you would suggest. Would it be possible to average 750 pulses (1 second) and update the plot only every second? This shouldn't affect the memory significantly, right?

Thanks again for your fast and helpful reply and have a nice weekend,

Ben

[Kevin_Price]

I don't really know my way around the niScope products, but it appears to me that you're hoping to re-Initiate Acquisition for each individual trigger while they're arriving every 2 msec.  I don't know if it matters for you, but I'd expect there's a decent chance you aren't catching every single trigger.

A lot of DAQmx devices would support a hardware retriggering mode where there would only be one software call to start the acquisition task.  Thereafter you could just loop over calls to Read 50k samples without the overhead of re-starting the task.  Dunno if niScope or your device supports a similar mode of operation, but I think it could help you to avoid missing triggers.

Another thought for you to consider is to run your consumer loop a little differently.   Instead of pacing the loop solely with the Dequeue function (which *is* the typical way to run a consumer), I might pace it with a longer wait time in the 100-500 msec range.  Then I'd flush the queue to retrieve all the built-up contents, write them to disk, and perform whatever other processing you want before updating a live display for the operator.

    I think this tends to hit a sweet spot for display update rate.  Once a second can feel kinda sluggish to a user, ten times a second is faster than they can interpret what they saw before it's replaced by something new. 

-Kevin P