LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

Issue with significant digits during calculations

Solved!
Go to solution

Hello,

 

I noticed this behaviour in a VI I built - when dividing, I get numbers a group of numbers rounded up (and hence "identical). At first, I tried to control this by increasing the significant digits on all the controls, but it didn´t do the trick - using probes, I could see that the correct numbers got to the division, but after that, everything was rounded up to two significant digits. I tried a couple of tricks (https://forums.ni.com/t5/Example-Code/Set-the-Precision-Digits-on-a-Double-Precision-Wire-in-LabVIEW...) I saw on the forum, but they didn´t work. 

 

Any idea why is that? The VI is annexed.

 

Thanks in advance

0 Kudos
Message 1 of 2
(116 Views)
Solution
Accepted by topic author Flonares

I found out the issue! I managed to get the appropriate respons after formatting the Numeric Properties of the numeric control of the output array.

 

Thanks!

0 Kudos
Message 2 of 2
(112 Views)