Data Logging Set Precision

[jtVulcan]

Hi all, 

I have been looking around for setting precision on the data being logged. Here is what my data looks like. 

I am using TDMS write to capture all the data, and I would like the data to be captured as a floating point with a 2 digit of precision. 

I know that TDMS captures in binary, but is there no way to set precision before the data is captured? 

Thanks!

[RTSLVU]

I save all my data in the full precision then simply format the cells in Excel to how ever many decimal places are relevant.

A long time ago when I first started out LabVIEW programming I wanted to do like you do and only save data with say two decimal places. The engineer I was working for at the time was very adamant that he wants the data in the highest precession, then he will decide how to format it.

Because when you lower precision the extra decimal places are not just truncated, the resulting number is rounded up or down. 

[jtVulcan]

Hi RTSLVU, 

Okay, I see. 

I would still like to set precision prior to the data logging. Due to our sensors are not as precise or fine-tuned, and I believe that it is not reading as precise past thousandth digit. We would like the data set to a reasonable precision. Thank you for being concerned on information or data loss. 

Do you know what's the best way of set decimal precision for the values?

[RTSLVU]

I haven't used TDMS in years so I'm not really sure. But in general LabVIEW uses the full resolution of the numeric type (double numeric is a 64 bit umber) internally. Even though you can format the display to show less decimal places.

This is why even though two number may look the same on the screen DBL-X= 64.32 and DBL-Y = 64.32 but if you compare them LabVIEW could say they are NOT equal because DBL-X is actually 64.3215621 and DBL-Y is actually 64.3215622  

[mcduff]

---

@jtVulcan wrote:

Do you know what's the best way of set decimal precision for the values?

---

Only if you use ASCII values can you set the precision; so instead of TDMS you would write a text file. If you want to save disk space and reduce precision, use single floating numbers, TDMS can save them and they are only 32 bits instead of 64 bits, so your data file will be cut in half. Unless you are using fixed point numbers, then no real way to set binary numbers to an arbitrary precision.