how to use SavitzkyGolayFiltering

[Wolfgang]

Hello all,

I am trying to find the peak maxima of a noisy input signal. Using simple threshold search (peaks exceeding a certain width and amplitude) returns some estimates, but I have data that has very broad peaks as compared to the noise, so this simple algorithm does not work well in such cases: it will find lots of narrow peaks, but never the broad 'hump'.

So I thought to apply some filter, e.g. the SavitzkyGolayFiltering function. Since I could not find an example I am wondering what parameter one should choose for 'frame size', in other words, the number of datapoints used for fitting. Does it make sense to use all data points, as I might expect a very broad peak covering say 60% of my input data? But I might have a narrow peak of say 100 data points (in a data set of 16000 data) as well... The same holds for the polynomial order: For large data sets, I could use a large order, say 7 or 9, permitting sharp peaks...? But this polynomial fit will generate articifial peaks, right? So may be I should simply use a running average? How should I set up these two parameters frame size and polynomial order to achieve a running average? Sorry for these beginners questions...

Thanks a lot,

Wolfgang

[Wolfgang]

why do I have to finish my comment in three minutes? Taking longer all the text is lost...... is there a trick how to operate this forum?? 

Ok, so I will repeat from my memory:

I have read more about this kind of filtering and wanted to share this knowledge, as, unfortunately, no background information is given in the NI help files.

It turnes out, that peaked signals such as in spectra can be nicely smoothed using SG, typically best with 4th order and the number of averaging data points roughly the FWHM of the most narrow peak to be expected.

Still, with this knowledge, I can not use the functions in the advanced math library:

- it is unclear to me if I first have to calculate the filter coefficients; thats what the help of SGFiltering says; however, it does not provide a suitable parameter for this coefficent matrix that I obtain from SGcoeff

- it seems that the number of sidepoints is symmetric in the NI implementation, that is, the number of leftward and rightward points is equal; no doc could be found on this aspect

- SGCoeff help says that 2*framesize+1 is framesize, which is a typo, of course, and should read: 2*sidepoints+1 is framesize

cheers,

Wolfgang

[Wolfgang]

Ok... I understand that the advanced analysis library is not the main focus of this forum, but may be someone from NI could comment on it?

It appears to me that I am using the function SavitzkyGolayFiltering now with some success, which in turn means that the help file for this function is wrong.

It says:

This function smooths a noisy signal using piece-by-piece fitting of a polynomial function to the signal. The fitting is done by least squares minimization. See SavitzkyGolayCoef for how the Savitzky-Golay filter coefficients are obtained to apply them here to produce the filtered signal.

It seems to me that the function SavitzkyGolayFiltering is NOT using any filter coefficients...?

Wolfgang

[DianaS · Accepted solution]

Hi Wolfgang,

the SavitzkyGolayCoef-Function is part of the SavitzkyGolayFiltering. But there are some more functions around it. 

If you can use labview also there are 2 good examples in it in the labview example finder "Design and Apply Savitzky-Golay Filter.vi" and "Savitzky-Golay Filtering.vi"

Attached you get a screenshot in labview where the  SavitzkyGolayCoef-Function is used and some other code. This together is the SavitzkyGolayFiltering-Function.

[Wolfgang]

Hi DianaS,

Thank you for addressing the topic.

1) If the SavitzkyGolayCoef function is used internally by SavitzkyGolayFiltering, what is the purpose of providing the SavitzkyGolayCoef function: I may calculate some coefficients, but cannot use them for anything?

2) It might be useful to adjust the wording of the help file; I would even suggest to add some comments (such as provided in my earlier post) to give some rough ideas what numbers to use for order and averaged points;  

3) It might be even a good idea to provide an example in C/CVI; while I am familiar with LabView, I do not have it installed on my system and thus can not have a look at the examples...

4) If I am not mistaken, the explanation from the helpfile '(2*frameSize +1) is frameSize,' is 'misleading', at least mathematically it is not really correct; it should read something like '2*sidepoints+1'

Wolfgang

[DianaS]

Hi Wolfgang,

the coefficients depend on the sidepoints. Here you can see a little bit more theory about this filter:

http://www.statistics4u.info/fundstat_germ/cc_filter_savgolay.html

Its in german but I think you can find something in english too if you want.

The framesize/sidepoint thing is corrected here:

http://zone.ni.com/reference/en-XX/help/370051K-01/cvi/cvisavitzkygolayfiltering/

In addition I will write a product suggestion for adding this examples in on of the next versions of CVI

[Wolfgang]

Hi Diana,

1) I am aware of the relation between coefficients and sidepoints. But as you explained to me the coefficients are already calculated in the filtering function SavitzkyGolayFiltering, what for should one calculate the coefficients with the extra function SavitzkyGolayCoef? Phrased differently, is there any application where one might need the SavitzkyGolayCoef routine?

2) The wrong explanation, as cited in my earlier posts, is found here: http://zone.ni.com/reference/en-XX/help/370051K-01/cvi/cvisavitzkygolaycoef/

In CVI 9 it's still the same.

Wolfgang

[DianaS]

Hi Wolfgang,

I'm sure there is a kind of application where somebody might need the SavitzkyGolayCoef-function.
With this function you have better access on the coefficients.
But of course if you don't need this function you don't have to use it.

[Wolfgang]

the documentation is still wrong in CVI2010, the relevant part reads:

frameSize

int

(2*frameSize +1) is frameSize,

It should be instead: (2*nSidePoints+1) is frameSize

It appears that the documentation was o.k until CVI 8.5, but then for some reason it has changed for the worse...

[Wolfgang]

The documentation of CVI2012 has been corrected