LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

precision of formula node?

I need to calculate a simple formulae.

But different results between two program.

Can you check which one is correct using other program?

or tell me the principle of calculation with formula node?

Enclosed is the example.

 

LabVIEW: 8.0614613965

Excel: 8.0614616898

 

-labmaster

 

 

Download All
0 Kudos
Message 1 of 14
(3,732 Views)

@labmaster wrote:

I need to calculate a simple formulae.

But different results between two program.

Can you check which one is correct using other program?

or tell me the principle of calculation with formula node?

Enclosed is the example.

 

LabVIEW: 8.0614613965

Excel: 8.0614616898

 

-labmaster

 

 


yes-

LabVIEW is complainant with IEEE-745. for the type that you involke.   how many digits ?


"Should be" isn't "Is" -Jay
0 Kudos
Message 2 of 14
(3,727 Views)

Actually, the next calculation needs the highest resolution. Maybe, higher than 8th digit should the same.

However, I need to know which way is the more correct in my calculation.

 

- labmaster.

0 Kudos
Message 3 of 14
(3,722 Views)

If you do all the calculations in regular LV functions rather than in the formula node, you can use Extended Precision data types.  That changes the 17th digit right of the decimal point.

 

When I look at the data at intermediate steps (the individual terms summed to get e_ITS90) I see that the spreadsheet matches to 7 to 13 significnat digits, depending on the exponent.  Note that for the third term (with exponent = 0), the spreadsheet calculation does not return the g2 coefficient: g2 = 19.542636 and g2*x^0 = 19.54263612. I set up a column in the spread sheet with the values equal to the g[i] coefficients but displayed with as many digits as the spreadsheet allows. g1, g2, and g3 differ from what is entered in column C in the first several digits to the right of the last digit entered. When I do the same thing in LV, the differences tend to be in the 15th or 16th digit.

 

All of these things are the result of the way numeric data is represented in the computer, as Jeff pointed out earlier.  It appears that you can get somewhat more precison in LV than in the spreadsheet. I did not do any accuracy checking.

 

Lynn

Message 4 of 14
(3,708 Views)

Do you mean the formula node is not compatible with the regular functions of LV?

However, my question is about the ability or limitation of formula node (in case of definition float32 or 64).

From the consideration of datatype, I have never seen the warning message of using formula node.

Still, I need to check if which one is correct in real calculation.

 

-labmaster.

0 Kudos
Message 5 of 14
(3,684 Views)

all three methods in LabVIEW agree within the limitations of rounding errors (Another nickel for Dennis' retirement fund) and are "Correct" results for floating point processor implementation of IEEE 754.

 

Excel differs from IEEE 754 for operations on denomalized numbers.

More Here

 

A Slide rule would probably offer yet another set of answers.  For a truely "Mathematically Correct" answer you cannot use a computer but, you can work it out longhand with a pencil.


"Should be" isn't "Is" -Jay
0 Kudos
Message 6 of 14
(3,669 Views)

The formula node is compatible with other LV functions.  The advantage of most of the built in math functions is that they are polymorphic, meaning that they can work with any numerical representation and often with arrays as well.  The formula node inputs and outputs (and, presumably, does all internal calculations) in DBL precision.

 

Jeff, I think that is Saverio's fund, but I am sure Dennis would not mind! Edit: Hey, If we help their retirement funds, does that give us better chances of some day catching up with their post counts?

 

Lynn

0 Kudos
Message 7 of 14
(3,653 Views)

johnsold wrote: Edit: Hey, If we help their retirement funds, does that give us better chances of some day catching up with their post counts?

 

Lynn



Your logic is escaping me here.  Retirement=More free time=Fewer posts to the forum? Smiley Happy

0 Kudos
Message 8 of 14
(3,647 Views)

Maybe they will go on a cruise or something?

 

Lynn

0 Kudos
Message 9 of 14
(3,645 Views)

Retirement = no more LabVIEW license.

0 Kudos
Message 10 of 14
(3,639 Views)