greater than or equal function not working properly

[Wolleee]

Hi all,

Im curious if anyone has seen this before or knows why its not working.  I have a greater than or equal function and i am comparing two doubles.  Ive probed the numbers entering the function so i know that the 2 numbers entering the function are the same.

Problem is i dont get a true from the function until the comparasin value is greater than the number entering.

Using LV2011.

Any reason this happens?

Thank you,

Matt

[Wolleee · Accepted solution]

Guess i answered my own question

http://zone.ni.com/reference/en-XX/help/371361H-01/lvconcepts/comparison_funcs/

[GerdW]

Ok - and what was the exact solution?

[Wolleee]

I actually dont know.  For me i just lowered the number it was compared to ha in the link i posted it just says funny behavior can be expected.  I didnt really look any further into it.  What exactly would be the solution to this?

Thanks,

matt

[crossrulz]

The problem with floating points is that they are not exact.  If they are off by a bit or two, your numbers are not equal.  This is because a large majority of numbers cannot be accurately saved in 64 bits.

[Wolleee]

That makes a lot of sense and I assumed something like that. The driver I was working with for an Agilent was using floating point inputs and outputs. That's why I left them in that representation. In this circumstance would it be a good idea to convert them to integers? I know some rounding will of course happen but I'm not too worried with the precision.

[GerdW]

Hi Matt,

would it be a good idea to convert them to integers?

That depends on your numbers - you didn't tell us anything about them before…

When Agilent chose DBL datatype they did this for a reason (atleast I guess). So using integers instead might lead to other problems!