TDMS scale data

[Freddy.]

I tried using the TDMS Create Scaling Information.vi to create the polynomial conversion coefficients first. Later, to update the coefficients, I directly used the TDMS Set Property function to update the three coefficients, NI_Scale[0]_Polynomial_Coefficients[0], NI_Scale[0]_Polynomial_Coefficients[1], and NI_Scale[0]_Polynomial_Coefficients[2], to the new conversion coefficients.

[mcduff]

Are you still having an issue? Your picture never loaded.

[Hooovahh]

---

@mcduff wrote:

Are you still having an issue? Your picture never loaded.

---

Virus scanning on the forums has been a bit crazy lately and NI is apparently looking into it.  I can see why you might want to apply different scaling.  I heard of a situation years ago where the data was calibrated, then logged.  Then later found the calibrated equipment wasn't calibrated correctly.  They calibrated it properly, then applied the new scaling, to the old raw data to verify that it was working correctly.  Had they only logged the scaled data, with no way to remove the scale, they would have had a much harder time, applying the new calibration to verify the data.

Still I think these cases are quite rare and I've personally never had a need to apply different scales to data.  I experimented with changing, and removing TDMS scaling, and I proposed an idea about it on the Idea Exchange.

[mcduff]

---

@Hooovahh wrote:

Had they only logged the scaled data, with no way to remove the scale, they would have had a much harder time, applying the new calibration to verify the data.

---

This really depends on how the data was taken and the how the scale was applied.

For a normal TDMS file, where the programmer specifies the scale, the raw data can be retrieved by setting the Number of Scales to 0. This works all the time. The other case comes from TDMS files created by DAQmx Logging feature. In this case the number of scales is 2 and if you set to 0, LabVIEW will crash upon reading the file. I do not know if the situation has changed since I have not tried recently. There are ways to recover the raw data from a DAQmx Logged file though; the folks who wrote the TDMS reader for python can easily do this. Surprisingly, NI/Emerson makes it hard for their software to have this feature.

[jyoung8711]

Needing to change the scaling factors is actually an interesting use case.  While I've never really wanted to "change" the scaling, there have been times where I've wanted to remove it (e.g. just read the underlying raw data).  When I've done this, I will actually open the TDMS file as an "in-memory" file (TDM Streaming >> Advanced TDMS >> TDMS In Memory >> "TDMS In Memory Open.vi")  This VI can take a TDMS path, and will actually load the entire file into memory, you can then read/write as you would on disk.

This isn't required (and obviously won't work for VERY large files), but I do this specifically because I don't want to actually edit the TDMS file when I'm working with it.

I've then set the property "NI_Number_Of_Scales" = 0 and had good luck reading the raw data w/out it crashing.

I've also had some luck using the property "NI_Scaling_Status" which I believe has the values "scaled" and "unscaled".  I vaguely remember the polarity being the opposite of what I initially expected though.

To update the scaling, I think the most recent post from the OP would be what I would do, editing the internal properties directly to the new values (though you may need to trigger that "NI_Scaling_Status" option again).

My understanding is that that "scaling" polymorphic VI is just editing those properties under the hood anyway (with some logic for multiple scales) -- there were some performance issues with it a few years ago, and the advise I received from NI was "just add the properties yourself until it's resolved"

[mcduff]

---

@jyoung8711 wrote:

I've also had some luck using the property "NI_Scaling_Status" which I believe has the values "scaled" and "unscaled".  I vaguely remember the polarity being the opposite of what I initially expected though.

---

From the help file; does not make a whole lot of sense.

Specifies whether LabVIEW scales data when reading the data from a .tdms file. If the value is unscaled and the .tdms file contains one or more scales, LabVIEW scales the data. If the value is scaled, LabVIEW does not scale the data. For DAQmx data channels, LabVIEW cannot return unscaled data even when NI_Scaling_Status is unscaled.

[jyoung8711]

I can't speak to the DAQmx use cases, as I wasn't using it related to that.

Off the top of my head, I think I needed to force it to say "scaled", which basically made the API think that it "was already scaled" so it didn't apply the scaling factor -- essentially then providing the raw data (like I said, my memory is that it was counter-intuitive)

I looked for the VI where I was testing this and unfortunately couldn't find it.

The help file blurb you posted doesn't make much sense to me.

[jyoung8711]

FWIW - I always found it a little weird that there wasn't a switch on the VIs to force the API to read the unscaled data... My understanding is that this is all embedded in the API/Client itself, the file is just encoding the information needed to do the scaling.

[mcduff]

---

@jyoung8711 wrote:

FWIW - I always found it a little weird that there wasn't a switch on the VIs to force the API to read the unscaled data... My understanding is that this is all embedded in the API/Client itself, the file is just encoding the information needed to do the scaling.

---

The pyTDMS API allows this. If you look into their API you can see there are things NI did not expose to us. For example, the NI_Datatype uses different values for RAW DAQmx data.

Also, I cannot change the property to unscaled in a simple example. The viewer always shows scaled. Setting the number of scales to 0 seems to work without a hitch.

[jyoung8711]

Interesting on the DAQmx datatypes.

For the scaling, I think you misread my post.

If you actually set the "NI_Scaling_Status" to "scaled", the API then interprets the channel as "already scaled" and so won't scale it on data read, giving you the original data.  If you set it to "unscaled" the API thinks it NEEDS to scale the data.

It's a bit of a weird one.  The API seems to intercept the value.  No matter what you 'set' if you try to read the value, it seems to always show up as "scaled".  But the behavior changes depending on what you set.

See the attached VI as reference (LV 2018+)