Auto range control & comparison of low numbers

[John_Deans]

I am trying to create a control function that changes scale each time the knob passes from high to low index or low to high index. This works well except when the number gets to the range 0.001 to .01 then It  sticks. It would appear that the compare does to output true when the inputs are 90.000E-3. acopy of the contol funtion in a stand alone vi is attached

[smercurio_fc]

Your fundamental problem is that you are performing an equality comparison on floats. This is incorrect when dealing with computers. You must, instead, perform an "approximately equal" based on the machine's epsilon value (found in the Numeric palette) as shown below:

 

Also:

• The Stop button should be placed inside the "Value change" event for it so that the button can be read and its value reset.
• You do not need to use a reference constant to access the scale control. The event case provides that already as the "CtrlRef" parameter.

Message Edited by smercurio_fc on 11-27-2009 10:37 AM

[cirrusio · Accepted solution]

This will be an issue with double precision values - change the equal to in the increment to greater than or equal to and the equal in the decrement to less than or equal to.  Cheers, Matt