11-19-2008 08:20 AM
I have two values with exactly 1 unit of difference, by example (X+1 and X+2), when I subtract one other number (Y) from the others the results are not with exactly 1 unit of differece ((X+2-Y) and (X+1-Y) don't have exacly one 1 unit of difference), there are some changes in some decimal numbers. In the VI attached the 18th number after the , is different. All the controls/indicators have the same representation/precision. This seams to apens only when i add 1. (and if subtract one from another the result is 1)
Can someone explain the reason this appens?
Very strange...
See example attached
Solved! Go to Solution.
11-19-2008 08:25 AM
Please see Greg McKaskle's discusion of epsilon for an explanation of what you are seeing.
Ben
11-19-2008 08:30 AM
This is the well known result of the finite representation of numbers in the computer. For a Double the representation is 64 bits containing sign, mantissa, and exponent. Some numbers displayed to 16-18 digits in decimal form cannot be exactly represented in this binary format. The result is what you see.
There are many posts in the Forum regarding this.
Lynn
11-19-2008 08:57 AM
Ok. Thank you for the confirmation.
I didn't get to find answer in forum because i didn't know exactly how/what to search.