TDMS Write Function slowing loop

[dmkopeck]

Hi,

I'm counting edges of TTL pulses coming from a photon counting photomultiplier tube module on my PCI-6602. I'm trying to sample at 4 kHz, but at 2 kHz my consumer loop starts to lag behind. Without the TDMS Write Function I can sample at > 4 kHz, so I believe that to be the issue. Is there a more efficient way to write the data or have I reached a limit for writing my data?

Thanks,

Dillon Kopecky 

[Kevin_Price]

Your description doesn't match your code or the GUI values on your front panel.  So I don't know what to respond to.

1. You say your consumer loop lags behind the producer when TDMS Write is called.  But the front panel shows that their # iterations were the same (the consumer gets one extra iteration to detect the queue error you force after ending the producer).

2. You make mention of 2 kHz and 4 kHz rates, but your front panel shows a 10 msec value for your Wait function as well as a 10 msec measurement of time between iterations.  That's only 100 Hz.

3. The Wait function *CAN"T* be used to wait for fractions of a millisec, only integer quantities.  And it won't be fully reliable under Windows.  I have no idea what led to your belief that you were able to control your software-timed sampling to either a 2 or 4 kHz rate.  But I can almost guarantee that you were wrong.

4. Your producer loop rate will be limited by your software-timed on-demand DAQmx calls to read one sample at a time.

5. I'm kinda doubtful that the consumer loop would lag behind the producer.  Writes to TDMS are pretty well optiimized, including inherent write-caching behind the scenes such that I'd expect it to have no real problem supportting multi-kHz function call rates.   Admittedly, I haven't personally tested this though.

6. You could always add your own caching within the consumer loop.  Use something like the "Elapsed Time" vi to set up a disk writing pace (maybe once a second or so). 

7. After discussing this overall app with you on several other threads, it's really time that you need to take the plunge into hardware timing.  And it's also time to switch over to the newer, more capable 6612 you said you had available.

8. With the 6612 you can do edge counting with count reset *AND* a hardware sample clock.  Your app wants the count reset and your timing needs demand a hardware clock.  The 6602 makes you choose between one OR the other.  That's NOT what you need.

9. The hardware clock is your only realistic way to sync photon counts from your counter task with your external imaging device.   All this mucking about with software timing just isn't ever gonna get you there.  So there isn't much point trying to dig deep and solve the problem you posed in this thread.

   Once you set up hardware timing, you also get a buffer and won't be needed to try to iterate loops at multiple kHz under Windows.

-Kevin P

[dmkopeck]

Sorry, I should elaborate a bit more. (I've decided to a different route than my previous post and deal with the memory issues of my device in favor of proper timing) I have a digital micromirror device, DMD module (DLPLCR4500) programmed to display a sequence of patterns. I am counting the edges of TTL pulses coming from a photon counter module which measures the light reflecting off the DMD. After the pattern swap a TTL trigger is sent out. I've used the count reset function to reset the edge count count for each image, therefore obtaining photon count and image correlation.

I'm unsure how to save the GUI values so I'll attach my front panel.

This is all likely the result of my misconception of the wait function in this vi. You can see in my front panel that a loop time of less than or equal to 0.5 ms causes this issue. If I remove the TDMS write I can go faster. Although I can't wait less than 1 ms accurately, It was my understanding that the sample rate and wait function were equivalent in this vi.

I was mistaken, I don't have the 6612 but a PXI-6289. Would that help?

[Kevin_Price · Accepted solution]

I don't think the 6289 will particularly help.  That's an M-series and other than some of the USB models, I think they also have a very small hardware counter FIFO onboard, much like the 6602.

I'm still confused with what you're trying to do.  It seems like if the DMD sends out a TTL pulse at the end of a pattern, you'd want to use that signal as a hardware clock to *sample* your count value.  That'd give you a total # of photon counts at the end of each image.

   What you're doing now is rapidly querying the count value in real time, and I suppose you are just figuring that the highest value you see before reset is the value you actually care about?

With the 6602, you have to choose between hardware-based count reset and hardware-based sampling.  Sampling is *MUCH* more important.  Without reset, you'll keep capturing a total cumulative # of photon counts.  But it's really pretty easy to do a finite difference on those values in software, resulting in the same data you'd get if you could have done the count reset in hardware.

OR

There's *also* a way to take advantage of counters' flexibility to get both to happen in hardware on a 6602.  I didn't bring it up before because it requires a kind of "inside-out" thinking about the counter signals and that tends to be confusing to people.  So I don't tend to bring it up until / unless necessary.  And here we are.

    A version of it was illustrated in one of the links I gave you before.  Namely, this one.   You set up for period measurement, using units of "Ticks", and you swap the roles normally played by the count signal and the sampling signal.   That's the inside-out part.

   You end up measuring the period between your DMD device's "end of image" pulses, and you measure it in units of "# photon pulses".   In period measurement mode, the count resets after each measurement automatically.  Exactly what you want, actually!   Just kinda complicated to explain, but we've gone far enough now that it's probably time.

   You won't use the linked code exactly as is, but you *can* learn from it to see how you would set up period measurement.  The period terminal would be that DMD pulse and the timebase would be your photon pulses.  Try reading through the whole thread, it's a least partly similar to what you're doing.

-Kevin P

[dmkopeck]

You were right on point with what I was trying to do.

I really like this alternative. I can't believe I didn't think about using the DMD pulse as the period terminal! In addition, this greatly simplifies my data processing. It seems to be working well for my app.