LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

Subract bug or limitation

Solved!
Go to solution

I have two values with exactly 1 unit of difference, by example (X+1 and X+2), when I subtract one other number (Y) from the others the results are not with exactly 1 unit of differece ((X+2-Y) and (X+1-Y) don't have exacly one 1 unit of difference), there are some changes in some decimal numbers. In the VI attached the 18th number after the , is different. All the controls/indicators have the same representation/precision. This seams to apens only when i add 1. (and if subtract one from another the result is 1)

Can someone explain the reason this appens?

Very strange...

See example attached

0 Kudos
Message 1 of 4
(2,521 Views)
Solution
Accepted by topic author IDILIO

Please see Greg McKaskle's discusion of epsilon for an explanation of what you are seeing.

 

Ben

Retired Senior Automation Systems Architect with Data Science Automation LabVIEW Champion Knight of NI and Prepper LinkedIn Profile YouTube Channel
Message 2 of 4
(2,516 Views)

This is the well known result of the finite representation of numbers in the computer.  For a Double the representation is 64 bits containing sign, mantissa, and exponent. Some numbers displayed to 16-18 digits in decimal form cannot be exactly represented in this binary format.  The result is what you see.

 

There are many posts in the Forum regarding this.

 

Lynn 

Message 3 of 4
(2,515 Views)

Ok. Thank you for the confirmation.

I didn't get to find answer in forum because i didn't know exactly how/what to search.

0 Kudos
Message 4 of 4
(2,501 Views)