LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

precision of formula node?


@Darin.K wrote:

johnsold wrote: Edit: Hey, If we help their retirement funds, does that give us better chances of some day catching up with their post counts?

 

Lynn



Your logic is escaping me here.  Retirement=More free time=Fewer posts to the forum? Smiley Happy


I'm hoping post count is stored as a U16- the only dang way I'm likely to come close- even ploting on a log scale!

 


"Should be" isn't "Is" -Jay
0 Kudos
Message 11 of 14
(583 Views)

Hello

 

I am trying to generate a sinus signal (and others) by getting an ouput array (with a formula node) that sholud fit the maximal resolution of my card (if I am not wrong the resolution is 20V/2**16 =>305uV).

 

The problem is that I get only 5 decimals on the output (float32). Why?

 

-Labview 2013

-PCI X 6321

 

Thanks

 

 

0 Kudos
Message 12 of 14
(537 Views)

I can't open your VI because it is too new.

 

But I'm going to guess that you the 5 decimals you are seeing is because that is the display format of your indicator.  Try changing the display format to show more decimals.

0 Kudos
Message 13 of 14
(523 Views)

Hello

 

Yes, you are right. Sorry for the stupid question.

 

Thanks.

0 Kudos
Message 14 of 14
(511 Views)