LabVIEW
"Variance PtByPt.vi" generates negative values for certain input values
Solved!
With certain sequences of input values the Variance PtByPt.vi can generate negative values. This is due to the finite precision in the calculations.
Because it is very common to calculate the standard deviation by taking the square root of the variance, this behavior can take you by surprise as the result will be NaN's and not almost 0 as you would expect. Depending on the data, this can be frequent or extremely in frequent. In general - a nuisance to debug.
The cure is simple: Take the Abs() value before the square root. Attached is a vi which demonstrates the case.
I would suggest to include this correction (or similar) into the "Variance PtByPt.vi" because it does give rise to unexpected and hard to find errors.
By definition - variance cannot be negative.
Intersting. I agree that this is just some random instability due to the limitations of floating point math. For example if you would change the last digit of the repetitive input numbers from 1 to 3, everything is positive.
From a purely mathematical point of view, the math is clean, but from a software engineering view, maybe this could be improved by either rearranging the algorithm a little bit or by brute force (coercing the variance to >=0, ;)). Someody needs to study this.
