Linear fit PtByPt slope degradation

[Jonny93]

Hi!

I'm having some issues with the slope output of the NI_PtByPt.lvlib: Linear Fit PtByPt.vi.

I feed it single x and y values, between 50 - 100 times per second, and after running for some time (10-15 minutes), it starts behaving weirdly.

I'm trying to find the slope of the x and y values, which typically changes slowly over time.

I've attached an example VI which recreates the issue in two separate while loops, running at different speeds.

The issue there is displayed as a reduced slope estimation as the number of datapoints is increased, even though the sample size is the default 100, so it should not be affected.

In the example file, I have set up an increment variable, by which the "value" increases each second. This increment variable is also increased every 10 seconds, to simulate how the input data changes over time.

Can anyone point out how I'm using the linear fit wrong here?

[GerdW · Accepted solution]

Hi Jonny,

---

@Jonny93 wrote:

Can anyone point out how I'm using the linear fit wrong here?

---

I guess it's a problem of numeric accuracy when numbers get larger and larger. You are trying to fit values that differ by some powers of 10…

This is what I get when replacing the GetTimestamp function:

[Jonny93]

Thank you for the reply!

I believe you are correct in that my issue is working with high numbers, so I've rewritten my code to always work around 0 instead, which seems to have solved the issue.

I never thought I'd have floating point issues when working with numbers in the million range, but I guess they can show up there as well.

Thank you again, for your quick response!