Decimation - aaaarrrggghhhh, the horror, the horror!!!

[RPJ]

Hello!

Before trying sample compression vi again, I have a couple of questions. If you open the front panel and then look into the diagram of the sample compression vi, you can see (attached picture):

1. Reset: I suppose this is true when you want a fresh start or when you do not want to include the points left from previous decimation in case the decimating factor was not an exact submultiple of the number of samples coming in? How would you explain

2. Enable: this enables or disables the decimation. correct?

3. Size: this is the decimating factor and therefore it is a number? in the original sample compression vi it is shown as a constant - this is the only input I changed it so I give the option of an external control for the user: for example, if the user wants to run the vi and save data using one decimating factor and then re-run the vi using another decimating factor, the user would have to open the block diagram, double click on the express vi and input a different decimating factor. But what if the user is not you or if you want to build an installable program which should allow the user to easily change whatever decimating factor he wants? Not giving the option to control the decimating factor or the output from a main interface is inconvenient. Any way, I guess opening the front panel and making that a control would solve this problem. Correct? (see second picture with "size" made a control)

4. Mode -  this one is a multiple choice and although there are three possible options (always, sliding block, and non-sliding block) only the non-sliding block is selected by default and no explanation is given in the help file. What is this for? is the right mode selected by the vi depending on the relationship between the size of the array being input and the size of the decimating factor? Can anybody explain what does this do?

5. Signals - this goes into the read dll block and to a conditional disable structure. What is that structure for? what's its role? It would have been great to actually have the option to input wveforms directly rather than 'just signals'. You can easily convert the waveform to a dynamic signal but it always makes you wonder what's going to happen with all that waveform information, especially t0 and dt. Any suggestions how to carry this over? and remember, the dt at the exit has to be 'aligned' to the output signals...

6. Error in

On the output side now:

7. Waveform output: is this what remains after decimation or is this waveform carring the output you selected in the main panel of the subvi: minimum, maximum, last, mean, median? If you select on the main panel that pops up any of these possible outputs, they will show on the subvi's icon. However, I could not locate any waveform output by itself; maybe this is dependent on the mode (see 4)?

8. Sum: when does this output become active? (does it depend on the mode too?)

9. Mean

10. Data valid? The help file says "Indicates whether the data is valid." - what does this mean? What is valid data and what is invalid data?

11. Minimum

12. Maximum

13. Last

14. Median

15. Error out

Then there is

16. Tab control - I kinda figured what is this for... I don't mind having this explained too

17. Functions: what is this for and why is set to 3 (see control panel)? Why not a bigger or a smaller value?

Finally, what is the property node for in the diagram enable  (what does it do?; what's its role?)

I would greatly appreciate detailed or at least explicative answers. Unless I understand how the 'black box' works I cannot program my application properly.

Thank you for your efforts.

[DSPmchen]

Hi RPJ,

In the example VI, Continuous Decimating VI, the reset and averaging control does not expose to user.
Because when you change the reset or averaging, the timing of input signal and decimated signal need to be re-synchronized.

You will find that there is an event case for "Decimating factor" value change.  If the decimating factor changes, the reset control is set to TRUE in that iteration and new t0 is used for input signal and decimated signal.

If you just change the reset and averaging in this example VI, this VI will show incorrect result because there is no corresponding operation in event structure.

Could you please point me out how I can reproduce the problem in your application? 
Your attached VI contains one block data.   However, I don't know how to reproduce the problem when the output array has more than 2 values.

Also I write a VI in LV 8.6 to show the use of Decimate (continuous) VI. 
Currently I set # of sample to 1000, the decimating factor to 400.   This setting will make the output array size switches between 2 and 3.

You can also change the decimating factor to 500.  In this case, the output array size is always 2.

帖子被DSPmchen在 09-01-2009 01:36 AM

时编辑过了

[RPJ]

DSPmchen,

Thank you for your explanations.

You ask "Could you please point me out how I can reproduce the problem in your application? 
Your attached VI contains one block data.   However, I don't know how to reproduce the problem when the output array has more than 2 values."

Unfortunately I do know how to reproduce the problem because I do not know how or why it happens. For a fraction of a second I se the output array going from 2 values to more values. If the decimation vi is correct the only explanation would be that the number of samples coming in are either bigger or smaller than the prescribed 1000 samples per channel....

Can you write a similar vi using sample compression.vi? It intrigues me that although sample compression should be more powerful and it offers more output options, there is no example for it (or maybe I did not search enough?)

Thanks

[DSPmchen]

Hi RPJ,

Here is the VI using sample compression VI.

I use "last" decimating method to show the result.  The key is to set the timing of decimated array correct.  Here I use different t0 for decimated array than that in my previous VI.

There is an example for sample compression VI, Data Reduction VI. But it is quite simple.

[RPJ]

This is interesting. Thank you for uploading an example with sample compression vi

I still have some questions:

1. The vi works great for 400 decimating factor and for 'last' value option. However, if you change to 'mean' value then the red dots (decimated waveform) do not align timewise with the original waveform unless you use a decimating factor that is a submultiple of the samples read per channel for example 250. Maybe I did not take enough time to analyze the example you sent but I think the time misalignement happens also for maximum, minimum and median value. The key may be in your statement "Since we use "last" method for decimation, we need to correct the t0 of the decimated array."

I guess this could be explained like this: since we want to display the last value superimposed over the original waveform we need to add the time of the decimated samples to the t0 of the set of samples entering (or bein generated) in one 'turn' of the while loop. What should be done in case of the other options?

2. How should I save the data in a file? Should I take the output waveform as it is right now or should I lo the data without changes to the t0 given that the dt is being adjusted. I guess this will influence how I am going to read the data later on. Is is going to be a difference if I save this data to a TDMS file versus a spreadsheet file? Will I be able to properly display the file use TDMS viewer?

Sorry, I am a little under pressure this morning and maybe I am not asking the right questions; in any case, could you discuss a bit the two topics? Thank you.

[DSPmchen]

Hi RPJ,

I use 'last' method because this method picks the last sample every M samples, where M is the decimating factor. In other words, the decimated sample is one of the input samples.  So you can easily check if the resulting decimated array aligns the input array. 

For other options, I think there is one assumption that the input signal should not change much in one demicate interval.

First, I want to understand your word "However, if you change to 'mean' value then the red dots (decimated waveform) do not align timewise with the original waveform unless you use a decimating factor that is a submultiple of the samples read per channel for example 250."

To me, there is no difference between 400 and 250 decimating factor.

For example, in my VI, the input signal is sine wave, if you use 400 decimating factor, and use 'mean' option, the result is as follows.

 

The VI averages the first 400 samples (in yellow rectangle) to get the mean value (the first red dot), and I put it align with the last sample of this interval.

Then the VI averages the second 400 samples (in blue rectangle) to get the second mean value (the second red dot), and I put it align with the last sample.

If I use 250 decimating factor, the result is as follows.

Both of them seems not close to original waveform.  This is because they don't meet the above assumption - the input signal should not change much in one decimate interval.

Then see another example.  The input signal is slowly changed, and with some noise, like the real signal.
With the slowly changed waveform, I still use 'mean' option. Decimating factor is 400 and 250, respectively.  I still align the decimated sample with the last sample in that interval.  The result is more acceptable.

 

Thus, my opinion is,  if you use any option other than 'last', the input waveform should meet the assumption. If so, then you can align the decimated samples at beginning, at center, or at end of the decimate interval. This does not matter much.

For your second question,  at least you need to record the t0 for the first decimated sample when you start decimating or reset decimating.
But to be safer, you can log the decimated waveform to the file. Just append the data to the file in each iteration.
I am not familiar with TDMS, but I think this is just a file format. You can log the data to any format, TDMS, spreadsheet, or txt file.

帖子被DSPmchen在 09-02-2009 10:33 PM

时编辑过了

帖子被DSPmchen在 09-02-2009 10:38 PM

时编辑过了

[RPJ]

If you look at your second example, the one with the 'real' signal, for the one with the 400 decimating factor, there is no red dot where the waveform ends (at the right end of the chart). I would suppose this is because there are a number of samples left that are less than 400. I can live with that. For mean, median, minimum, or maximum I was not concerned that the decimating signal is not superimposed over the original waveform in terms of amplitude (Y axis) but in terms of time (X axis); the amplitude will definitely be different if you use mean, median, minimum, or maximum. If you use 'last' the point should belong to the original waveform both as amplitude and time.

I will try to implement this solution to my case and will share with you the result. It may take a while but be sure I'll be back.

Have a good, relaxing Labor day.

[DSPmchen]

Hi RPJ,

Yes, you are correct.

In the second example, there is no red dot at the end of the waveform, since the left samples are less than 400.  Because the Sample Compression VI does not know if there will be next block data coming or not.

If you do the decimating block-by-block, this may occur at the last point of the last block data.  This needs special processing for the last block data. 

Look forward to your result!   And have a great Labor day!