number rounding bug in LabVIEW

[ThinkG]

There is a bug in how LabVIEW rounds numbers for 1 decimal place precision when using "Display Format..." for front panel control/indicator.  The bug is also present when using "Format Into String" with %.1f format string.

In both cases, LabVIEW rounds up when > 5 rather than > or = to 5, which is convention.

e.g.

4.55    => 4.5
4.551   => 4.6

This is a LabVIEW bug that the rounding does not follow convention.  The same test in Excel results in 4.6 in both cases.

[johnsold]

It is probably due to the actual representation of the number in binary.

If you change the display format to show many digits, the number entered as 4.55 becomes 4.54999999999999982, which correctly rounds to 4.5

There have been many threads on the Forum regarding finite binary representation of numbers.

Lynn 

[Ben]

Check it again with odd numbers (like 3.5 not 4.5). There is some number theory thing that says round up for even and down for odd.

Ben

[ThinkG]

Thanks for the replies.  I guess it's not a bug per se but is an unexpected behavior.  If the last digit of display is critical then it seems we should always round in code to the desired decimal place prior to display.  Something to remember.  Thanks again.

  - Chris

[smercurio_fc]

---

ThinkG wrote: 

In both cases, LabVIEW rounds up when > 5 rather than > or = to 5, which is convention.

---

http://en.wikipedia.org/wiki/Rounding

[altenbach]

See also

LabVIEW uses bankers rounding, but as was already mentioned, in your case the problem is due to limited resolution and the number is actually slightly smaller than 4.5.