do numerical indicators display extended precision floats correctly?

[tst]

Nicely formed question. Hopefully someone will have an answer.

---

Root Canal wrote:

Delightfully, the flatten to string function appears to scramble the bits into “noncontiguous” pieces

---

Try looking at your endianess. LabVIEW is big endian and Windows is little endian. The flatten primitive should have an input for changing the endianess, but you can also use the type cast primitive if you don't like the results there.

[CharlesK]

For Windows on the Intel Platform, an extended precision number (EXT) has a machine epsilon of ~1.1e-19. This means that at least 19 digits of precision should be expected.
The indicators that you see showing you 16 digits of precision is a bug in our display code.  I have submitted a Correction Action Request (#117387) to fix this bug in a future version of LabVIEW.
Attached is a VI that computes the machine epsilon for EXT precision.  If you run the VI, you will see that we successfully calculate the machine epsilon but fail to display it correctly.  Under the hood, LabVIEW does the right computations; otherwise, it would be incapable of computing this value. The data shown in the indicator is different from the data on the wire. Using this wire as an input to other functions keeps the extended precision of the value.

For more info on Machine Epsilon:
http://en.wikipedia.org/wiki/Machine_epsilon




Message Edited by Charles K on 06-23-2008 01:33 PM

Message Edited by Charles K on 06-23-2008 01:33 PM

Message Edited by Charles K on 06-23-2008 01:33 PM

Message Edited by Support on 06-24-2008 12:53 PM

[Root_Canal]

Thanks tst, you were right, I just had bitslexia. Attached is a vi that provides alternative readouts for extended precision numbers.

Mr. K, please include the numeric controls in your Correction Action Request. One may properly enter extended precision numbers into numeric controls, but the control reinterprets the entered value and displays a double precision value that does not accurately represent the entered extended precision value in memory.

Message Edited by Root Canal on 06-23-2008 02:49 PM

Message Edited by Root Canal on 06-23-2008 02:49 PM

Message Edited by Support on 06-24-2008 12:53 PM

[Root_Canal]

oops. I guess I should make sure that my vi actually works before posting it. I think the attached one is ok.

[Root_Canal]

Third time's a charm. this version seems to work correctly. sorry 'bout that.

[LaurenLL]

This was reported to R&D (#117387) for further investigation.  I added this post so that R&D could search the forums for this particular corrective action request.  Thanks for the feedback.

Regards,

Lauren

[X.]

WTH is going on with "I have submitted a Correction Action Request (#117387) to fix this bug in a future version of LabVIEW (CharlesK, NI, 2008-06-23)"?

4 years and 4 LabVIEW versions later, LV is still unable to display extended numerics with more tjan 16 digits precision, AFAIK...

[Henrik_Volkers]

---

@X. wrote:

WTH is going on with "I have submitted a Correction Action Request (#117387) to fix this bug in a future version of LabVIEW (CharlesK, NI, 2008-06-23)"?

4 years and 4 LabVIEW versions later, LV is still unable to display extended numerics with more tjan 16 digits precision, AFAIK...

---

Hey, it's still a correct answer 😉

Don't trust your expectation if you get answers like this or:

a.s.a.p.      (could still be not possiple)

before [chrismas, new year, easter, ..]      (year??)

Quality is the moment where expectation meets reality 😄

[X.]

I sent a PM to CharlesK, who seems to still be working at NI AND reading these forums (if not participating a lot)...