NI TestStand

cancel
Showing results for 
Search instead for 
Did you mean: 

Transfering the results from labview to teststand. How it works?

Solved!
Go to solution

Hi,

 

Sometimes, despite the tact that my LV code module returns the data with -exactly - two significant digits of precision, in TS report I see a number with sixteen digits of precision.

 

For example the data returned (not  only displayed) by the code module is -0.09 (DBL), but TS reports displays it to me as -0.0899999999999999.

 

Why it is like that?

0 Kudos
Message 1 of 2
(4,350 Views)
Solution
Accepted by topic author MimiKLM

Because you are transferring a double precision floating point number.  The display has nothing to do with the actual data.


GCentral
There are only two ways to tell somebody thanks: Kudos and Marked Solutions
Unofficial Forum Rules and Guidelines
"Not that we are sufficient in ourselves to claim anything as coming from us, but our sufficiency is from God" - 2 Corinthians 3:5
Message 2 of 2
(4,349 Views)