nonlinear curve fitting how to get the parameter standard deviation

[wolfkil]

Hello I have tried to figure out (also here in the forum) how to get the standard deviation for the parameters of a nonlinear curve fitting (Lev-Mar). The most interesting I have found was concerning the Covariance Matrix. Here the most reliable information came from DSPGuy (Message 8). But in the example VI he first takes the square root of the diagonal elements of the covariance matrix and then multiplies these values with the RMSE (using the "goodness of fit" VI) to get the standard deviation of the parameters. In the thread most the time it was told that one has first to multiply the diagonal elements with the MSE directly from the Lev-Mar-VI.

And one other question is still around for me. What already came up in the previous mentioned thread:

how is DOF defined?

a: N_MeasPoints - N_FittingParameters

b: N_MeasPoints - ( N_FittitngParameters - 1 )

As I understand the help for the "goodness of fit" VI a: should be right, also as DSPGuy states. But I always thought b: should be right in the same way as  Peter Vijn2 stated in the formerly mentioned thread.

Best thanks in advance

Wolfgang

[altenbach]

Hi Wolfgang,

These are all good questions and I have just formed a Community Group wher this can be discussed in detail and all in one place. Sorry, it is still under heavy construction, but please join and provide feedback on the charter document.

Sorry, I am very busy with other things at the moment, Maybe Jim can chip in in the meantime. 😉

[wolfkil]

Hello "alternate" ,

 thanks for the fast reply. I have looked at "Nonlinear Fitting" Group, but as I have never joined such a group jet I do not know how one should attribute to it. So I will first look what is going on further an see later if I can help or state questions there.

 OK I have to admit that my first question on the LevMar was somehow "silly" or better to say blindness has come to me. To make it clear to other readers (in the case that someone else has such questions) and to somehow ask if I have understood it right I want to show what I have understood so far:

Assuming a function with n fit parameters one gets a n x n Covariance matrix M. As given by DSPGuy (Jim) one should take the diagonal elements M_ii and multiply them by the sum-of-square error (SSE) and divide by the degree of freedom (DOF) and finally take the square root to get the standard deviation of each fit parameter:

delta_par = sqrt ( M_ii * SSE / DOF )

but as the root mean square error (RSME) is defined as

  RSME = sqrt ( SSE / DOF )

it is for sure the same to take

delta_par = sqrt ( M_ii ) * RSME

but here I am asking myself is it necessary to call the "goodness of fit" VI to get RSME if you automatically get the "weighted mean square error" (MSE=SSE/N) from the LevMar-VI (by the way I think in the German help of the Lev-Mar.VI there is an error in translation as it says the MSE output is the "mittleren gewichteten Fehler" so there it is missing the square - who is responsible for such things?). So why not simply calculating

RSME = sqrt ( MSE * N / DOF )  and then calculating delta_par?

So there is still the question on how is DOF really defined ??? !!!

For sure all this is only true assuming that the fit model describes the data well - as mentioned by "altenbach" - and that the data are Gaussian (white noise) distributed. The other remark of "altenbach" that in the case of nonlinear function it may happen to have strongly nonsymetric errors is for sure true and I get those errors in the good old "PAW" (a very old CERN tool which is doing fitting based on MINUIT). But right now I first see the problem for a daily user to get the right results from a - in mathematical sense probable perfectly right programmed - Levenberg Marquart routine.

 I am also still not totally sure how I can feed in the errors of the measured data. For sure one has to calculate the weights. But how are they defined? Taking the manual from MINUIT I would take weights=1/error^2. In the document "Overview of Curve Fitting Meodels and Methods in LabVIEW" I only can find that these weights cane be used to set smaller weights for outliners (which seems to be mathematically somehow "wishi washi").

So in general I would really appreciate to have the possibility to switch between a Lev-Mar.VI with weighted inputs or absolute error input. An in addition to the covariance matrix the Lev-Mar.VI should internally calculate the errors of the fit parameters.

[Topper_Harley]

Hi Wolfgang,

I am sorry, but I did not quite understand your request. Did you encounter an error with Lev-Mar algorithm or its documentation? Do you have a suggestion on how to improve a VI that implements it?

Maybe these links are of any use for you:

Incorrect Results from the Nonlinear Lev-Mar Fit VI in LabVIEW

http://digital.ni.com/public.nsf/allkb/5BF55E4887A063B086256CAE0073A12E?OpenDocument

The Convergence Algorithm for the Nonlinear Lev-Mar Fit VI and Small Scale Data Sets

http://digital.ni.com/public.nsf/allkb/F133A090A43EF7D986256688005E040B?OpenDocument

To make suggestions on LabVIEW, please visit this page:

ni.com/ideas

If this does not help you, please state your issue in a precise and comprehensive way, yet give the context where the problem occurs.

Regards, Topp

[wolfkil]

Dear Topp,

 I have several "problems" with the "Nonlinear Curve Fit.VI":

1) in the German Help the translation for the "MSE" output seems for me to be wrong

2) it is not clear how to calculate DOF for the calculation of the sigma for each fit parameter:

    a: N_MeasPoints - N_FittingParameters

    b: N_MeasPoints - ( N_FittitngParameters - 1 )

    and all the calculation of the parameter error should be given in the help for this VI

3) I can't find a documentation on how to feed errors of measurement data into this VI. Probable via the "Weighting" input.

4) I would appreciate to have this VI with the choice of ether a "weighting" input or an "error" input. And in general a output "error of best fit coefficient"

Hopefully this list is "precise and comprehensive" enough. Although I not really see were I have not been precise. Maybe that I did not structure it sufficiently. This is due to the fact that I constantly - also during writing - try to get more information on this problem.

Regards Wolfgang

[wolfkil]

I have done so:

NI Ideas

so if anyone is interested give "cudos"

[altenbach]

1. Can you quote the german help page for the MSE output? Which function has a MSE output?

2. The help states: "degree of freedom is the length of Y minus the number of coefficients in the fitted model. The default is –1. If degree of freedom is less than or equal to 0, this VI sets degree of freedom to the length of Y minus 2."

Since this is an input, you can change the value if you feel you need a different definition.

3. Try the following conversion: weight=1/variance (See also). In the very old fitting tools (pre-8.0) we had a variance input and it internally converted to a weight. One problem with this was that a weight of zero would have required a SD of infinity, causing numerical issues.

[wolfkil]

Hello "altenbach",

1) It seems that there is constantly something changin from one LV version to another.

- In the old "Levenberg Marquardt.VI" (LV 7) the MSE was given as output as discussed in the old thread (see my first post). 

- As I found on the web at least from the version  8.2 on the output was called "residue"  and "returned the weighted mean error of the fitted model" (LV 8.2 help). So was this  sqrt(MSE)=sqrt(SSE/N) with N the numbers of data points?

- Now I find in the English version of the "Nonlinear Curve Fit VI" (LV 2010 Help) the output is called again "residue" but now "returns the weighted mean squared error between the best nonlinear fit and Y". So is this again MSE? When opening in the German LV 2010 the Help of the "Nichtlineare Kurvenanpassung (VI)" this output is called "Restbetrag" (allready a somehow unclear translation) and explained to be the "mittlerer gewichteter Fehler zwischen Bester nichtlinearr Anpassung und beobachteten Y-Werten". So here nothing is stated on squared. So for me it is totally unclear.

2) Probable you are citing the help of the "goodness of fit.VI". Yes there it is stated how DOF should be calculated. But as "Peter Vijn2" has stated in the thread on the covariance matrix (message 10) DOF "is equal to the number of data-points minus (the number of model-parameters-1)". And I also think that I have earlyer heard that for checking the goodnes of a fit and the given measurement errors one should take the chi^2 value and devide it by DOF and this value should be in the order of one. And here DOF was N_MeasPoints-(N_FitParam-1). But I will take it as LV tells me:-)

3) That weight=1/variance=1/sigma^2 is exactly what I have found in MINUIT (see Message 3 in here)

[DSPGuy · Accepted solution]

Adding to Christian's post.

1.  If our documentation is incorrect then we will fix it.

2.  I verified that our source code for the CLN in the Goodness of Fit.vi implements the DOF as our help states.  They are in agreement.  As Christian states, if a different definition is appropriate for a given application, then you have the option to supply that value for the DOF.  I will add that the results we return for the example linked in Wolfgang's original post agree with the results that NIST certifies for that problem to 8 decimal places. 

3.  As Christian mentioned, out old implementation of Lev-Mar had an input called "Standard Deviation".  We mapped that immediately to a weight by using weight=(1/std. dev.)^2 so Christians formula for mapping the variance to weight is exactly what we used before.

-Jim

[altenbach]

residue != MSE because it includes weighting.

Sorry, I did not mention the question about parametere error estimates.

The correct way to obtain parameter errors is outlined in Jim's example (from here) showing agreement with the NIST lanczos3 dataset. This is what I use in my wrapper.

You can easily verify that using a different DOF calculation would give incorrect paramter errors. I recommend a method that agrees with the NIST standard. 🙂