multiple recording of waveforms using an external trigger signal

[beam]

Dear DF Gray,

thank you for your help and last comments which solved the problem. During further VI development several questions occured. I will explain my situation using the VI where the "niScope MultiFetch WDT.vi" is OUTSIDE the for-loop (please see attached screenshot "wfm display and scalar measurement_outside loop").


Situation:
As you know, I am fetching the waveforms on CH0, feed them through my fitting-routine and save the fitting values in an 1D-array.

Now, I have a voltage signal from a power meter connected to CH1 (voltage On/OFF). For each waveform fitting value (acquired form CH0), I need the CORRESPONDING Peak-to-Peak voltage value from the power meter (acquired from CH1). So in the end, I will have 1D-array with my waveform fitting values (CH0) and a CORRESPONDING 1D-array with the Peak-to-Peak voltage values (CH1).

I solved the situation as follows:
a) First, I only need to display the voltage signal (CH1) on a waveform graph so I know how it looks like (no further processing or saving). This is done by using the second "niScope MultiFetch WDT.vi" and auto-indexing the waveforms using the for-loop). Please see attachment "wfm display and scalar measurement_outside loop".

b) Second, I need to perform the voltage Peak-to-Peak measurement for each of the fetched waveforms on CH1. I am doing this by using the "niScope Fetch Measurement.vi" and performing a scalar measurement which gives a 1D-array.


Question:
1. Can I fetch the SAME waveforms (CH1) for SEVERAL TIMES from the onboard memory of the digitizer? The waveforms will stay in the memory after I fetched them once, right?

2. Do I have to fetch the voltage waveforms TWICE to display the voltage waveforms from CH1 AND perform a Peak-to-Peak voltage measurement for each of these waveforms (like I did it)? Is there a better way to do this, so I only have to fetch the waveforms once for display and measurement?


An other question:
As suggested above, I placed the niScope Fetch WDT.vi into the for-loop to avoid buffer problems (slow down the program). So, I am fetching and fitting 1 waveform per loop iteration (please see attachment: "wfm display and scalar measurement_outside loop").

I compared the speed of these two VI's (recoding 256 waveforms, each with 2000 data pts,) by using the Profile Tool in LV. I did not see a big difference in execution speed between these two VI versions (250ms vs. 187ms). It seems to me that even if the 1MB buffer is more than full the program with the external for-loop is still quite fast?


It would be very nice if you could give me some comments.


Kind regards,

beam

[DFGray]

1. You can fetch the same waveform multiple times from the digitizer. It will stay in the card's memory until you initiate another acquisition.


2. If you absolutely want to fetch only once, and you have the full or pro versions of LabVIEW, use the Amplitude and Levels.vi on your data to find the peak-to-peak value. I don't know whether the NI-SCOPE driver will only fetch the waveform once for both a fetch and a measurement. I know it caches the waveform if you do multiple measurements, so it only fetches to the internal cache once for any number of measurements. I don't know if a waveform fetch also fills the measurement cache. I would try it and find out. Set the number of points to something large (like 200,000 or 1,000,000) and single step through your program, watching the memory use using the Windows Task Manager. I could create good arguments for both behaviors, so I don't know what was picked.


3. I am not surprised that your speeds are similar. In your case, the real time killer is probably the fitting function. You will probably get more performance increase by optimizing it than optimizing fetches. This problem will only get worse as the number of points gets higher, as Levenberg-Marquardt uses data spaces that are the square of the input data size. You may want to try other algorithms than a fit to get your data. You could see some huge performance gains.

[beam]

Dear DF Gray,

It would be very nice if you could give me again some of your professional help and advice regarding a modification in the multiple-recordVI I am currently using for the multiple-recording of my waveforms (please see attached example “Multi_record from Beam_3”).

My situation is the same as described in our conversation on 04/21/2005 (please see conversation above).

A short reminder: Per loop-iteration, I am fetching ONE of the multiple-recorded waveforms (using the niScope Fetch WDT.vi inside the for-loop to avoid buffer problems), feed the waveform through the fitting-routine and save the fitting value in a 1D-array. This is done until all multiple-recorded waveforms are processed. At the same time, I acquire for each waveform fitting value the CORRESPONDING voltage average value from the power meter (acquired from CH1). So in the end, I will have a 1D-array with my waveform fitting values (CH0) and a CORRESPONDING 1D-array with the voltage average values (CH1).

Situation: Instead of passing the fetched waveforms on-by-one through the fitting-routine, I would like to average first a consecutive number of fetched waveforms and then pass only the average waveform through the fitting-routine. For example: The first 8 waveforms (wfm’s 0-7) from the 128 multiple-recorded waveforms are fetched one-by-one and the average waveform is created. After that, the average waveform is passed through the fitting-routine and the fitting value is saved in the 1D-array. Now the next 8 waveforms (wfm’s 8-15) are fetched and averaged. The average waveform is again passed through the fitting-routine……. this repeats itself until all 128 multiple-recorded waveforms are processed (16 fitting values in the 1D-array). In parallel the CORRESPONDING voltage average values from the power meter (acquired from CH1) are treated in the same way.

The goal of this exercise is to reduce the time which is needed by the fitting-routine to process the multiple-recorded waveforms.

I still would like to fetch only ONE of the multiple-recorded waveforms per loop-iteration, using the niScope Fetch WDT.vi inside the for-loop structure to avoid buffer problems.

It would be very nice if you could give me some help with this issue. How should I do this? What is the most efficient way to handle the waveforms and average the waveforms / voltage average values?

Kind regards,

Beam

[DFGray]

This could be a complex or simple solution, depending on your needs.  It also may not give you any speed.  The simple solution is to add another FOR loop inside your current one to take all the data.  It will run eight times and the outside loop will run 16 times (128 total fetches).  Calculating the record number to fetch is now a bit more complex.  Put a shift register on both loops.  Initialize it with an I32 set to 0.  Put a +1 operator in the inner loop to increment it.  Use the value before the +1 to set the record number (currently set by the loop counter).  To average the waveforms, use another shift register, this time only on the inner loop.  Initialize it with a waveform of the correct size that is all zeroes.  At each iteration of your inner loop, take your output waveform, divide by 8 (use the divide operator - it works with arrays), then add to the value of the array in the shift register.  When the inner loop finishes, the output of the shift register will be the averaged array.  Use it as you are currently using the actual array.  Note that this array is reinitialized to zeros at every iteration of the outer loop.

The problem with this method is that it smears your waveform.  This is because the reference position is usually randomly distributed in the time space between your last pretrigger point and your first post-trigger point, not one fixed point.  If you are interested in peak or valley values, this may produce unacceptable results.  This, too, is solvable, by means of the Resample Waveforms (single shot).vi.  Use the reference position information to generate the values needed for the resample.  The spline mode will give you the best results, but is also the slowest.  Do the resample, then the average as mentioned above.

If the first method is acceptable to you, check out the waveform measurement capabilities of the NI-SCOPE driver.  They include statistics, so you may have to do very little.

Let me know if you need more info.

[beam]

Dear DFGray,

Thank you very much for your reply and helpful comments. I wrote two versions of the LabVIEW program (please see attached library “Multi_record from Beam_4”). The program “Multi_record_dual channe9.vi” is using my way to determine the record number to fetch. The second program “Multi_record_dual channe9a.vi” is using your proposed method to determine the record number to fetch (I hope I understood your way correctly).

Problem:

After I average e.g. 5 waveforms in the inner For-loop (using the shift register to add the 1D arrays and then divide the final 1D array by factor of 5), I do not get my 1D array of the averaged waveform out (which has to be passed to the waveform fitting routine). There are only zero’s in the array which is passed to the waveform fitting routine. What is my mistake?

It would be very nice if you could have a look at the programs to check if I made any more mistakes.

Which is the better way to determine the record number to fetch?

I certainly will have questions about the resampling of waveforms to avoid the smearing, but first I need to check if this “simple method” is acceptable in my main program. Is there another way than resampling to average the waveforms (For example using a certain value in the waveform 1D array as reference position, then aligning the waveforms 1D arrays to each other and average them in a last step)?

Kind regards, and thank you very much for your help!

Beam

[beam]

Ooops, I think that I forgot to initialize a 1D array for the shift register. Please see attached program "Multi_record from Beam_5". Is this the way to solve this problem?

Kind regards,

Beam

p.s. Is it better to do the averaging of waveforms by using the 1D-arrays of the waveforms or the waveform data type directly as you suggested earlier?

[DFGray]

My apologies for the late reply.  I have been on vacation.  It appears you figured it out in any case.  The only change I would make is to move the array initialization outside the main loop.  This will give you a little speed (maybe, LabVIEW is pretty good about this type of optimization, try it and find out - make sure you turn off autoindexing when you feed the array into the first loop).

Your second example did get the increment for the record selection correct.  Your method is also fine.  The increment is probably a hair faster, but the difference will be in the noise compared to everything else you have going on.

Adding the raw 1D arrays will be faster than adding waveform datatypes.  However, to make this worthwhile, you may want to fetch 1D arrays, not waveforms.  The problem is that any time you extract an array from a waveform, you may (probably will) copy it.  This takes time.  Even though you are not dealing with large data, you may want to check out the tutorial Managing Large Data Sets in LabVIEW. It will give you some hints on how to minimize data copies, which translates directly into speed.

Let me know if you have any more problems.  I will try to reply sooner next time.

[beam]

Dear DFGray

Thank you very much for your reply. Your comments are always very helpful!

I included the changes you suggested in my program i.e. array initialization outside main loop and fetching 1D arrays instead of wfm’s (please see attached program “Multi_record from Beam_6”). I hope that I made no mistakes.

It seems that I can save ~1 second per processed data point in my main program when I fetch 1D arrays instead of wfm’s.

Some additional questions:

1. To confirm my own understanding: Is it correct that in the way I am “Initiate Acquisition” and “Fetch no. of records” in my program, the fetching of the waveforms/1D-array starts as soon as waveforms/1D-arrays are available from the Initiate Acquisition process (fetching while acquiring waveforms)?

2. Is there any way to increase the fetch speed so the waveforms/1D-arrays are quicker available for the adding and curve-fitting process? The main goal is to increase the processing speed without changing my curve-fitting routine.

3. Resampling of waveforms to avoid smearing:

a) In your reply from 07/08/2005 you wrote: ”Use the reference position information to generate the values needed for the resample” What do you mean by this?

b) Where would I place the “Resample Waveforms (single shot).vi” in my program to resample in the most efficient way?

c) Could I also do the resampling using 1D-arrays instead of waveforms?

Thank you very much for all your help & comments

Kind regards,

Beam

[DFGray]

Your changes look good.  There is one minor issue that may cause you problems.  The array size should be numSamples, not min record length.  This will keep the arrays the same size.

Since you are not using the Channel 1 waveform, you may be able to save some time by either eliminating it or moving it outside the loop to only fetch the last waveform for display.  I believe the measurement will work without fetching the waveform (although the driver will actually fetch the waveform to take the measurement, it will not get copied into the LabVIEW memory space, saving time and memory).  I could be wrong, though.  It has been awhile since I worked with NI-SCOPE measurements.

1. As long as you are not using a 5102 or 5911, fetch should start as soon as a waveform is available.  You can fetch while acquiring with all NI-SCOPE devices but the 5102 and 5911.
2. You can't really increase the speed of acquisition beyond what you are currently doing.  However, you can change your program so the acquisition and analysis do not wait on each other.  You currently take data, analyze, then take data again.  You can set up your program so you have an acquisition loop and an analysis loop.  Since analysis is probably your slow point, this will allow acquisition to take place quicker and not wait on analysis.  However, if you are CPU bound already, this may not help.  If you want to try this, do the following.
   1. Create a queue whose data element is a cluster containing a double (xIncrement) and a 1D double array (data). This will be the method of passing data from your acquisition loop to your analysis loop.
   2. Move the fitting routine out of your current loop and replace it with an enqueue of the data that would normally have gone into the fitting routine.
   3. Create another loop. Inside it, place a dequeue to get the data from the first loop. Put your fitting routine here fed by the data from the dequeue. Queues are nice in that they wait for data.
   4. Close the queue reference after your analysis loop finishes. The easiest way to do this is to wire the dequeue inside the loop to the close outside the loop.
   You now have parallel acquisition and analysis. If you have dual processors or a hyperthreaded processor, this program will take advantage of them. You may get a lot of performance from this. You may get almost nothing. It depends on where your bottleneck is. This method will also increase your memory usage, since each queue element contains a full waveform. You may want to limit the queue size (number of queue elements) if you have memory limitations (you shouldn't with the default values on your front panel).
3. 1. The only reason you are resampling is to line the waveforms up in time to avoid smearing them, which tends to reduce the peak values. For example, take a hypothetical waveform of five points with an acquistion rate of 1. If the trigger occurred on the middle point, the reference position would be -2.0. If the trigger occurred exactly between the second and third points, the reference position would be -1.5. Assuming the waveform acquired was exactly the same in both cases, the actual data will be different because it was acquired at different locations in the waveform. To average, you have to line the waveforms up again. In this case, one of the waveforms needs to be shifted in time 0.5 periods, the difference in the reference positions.
   2. Place the resample VI between the acquisition and the addition.
   3. Unfortunately, it appears you will have to go back to using waveforms to use the LabVIEW resample VI.  I hate it when that happens
      
   I am not convinced that Resample Waveforms (single shot).vi is the fastest way to shift your waveform in time. An “easier” way is to take the Fourier transform of your data, convert the output to r,θ format, shift the phases of each element the correct amount to effect the time shift, then convert back to imaginary numbers and do an inverse Fourier transform. This works very well for sub-sample-width shifts, although you may have minor ringing artifacts on the ends of your data. You can solve this by adding a few more data points to the ends of the array and trimming them when done. NOTE: all of this will take time and probably make your final time worse, not better. If you don't need to resample, don't do it.

[beam]

Dear DFGray,

thank you very much for your quick and extensive reply! I think I got now a pretty good picture of my situation.

I will try the parallel acquisition and analysis at a later stage, but I have one additional question regarding the array size of my "Initialize Array" function:

You wrote: "array size should be numSamples and not min record length". If I wire numSamples to my "Initialize Array" function, the initialized array has 0 elements (no data in array). I think, this is because the numSamples is passing a -1 to the "Initialize Array" function which means that the maximum no. of samples is fetched from each waveform (-1 : fetch all available samples).

How can I use numSamples to set my array size? What am I doing wrong?

Kind regards,

Beam