TDMS Delete Properties

[Nicolas_Bats]

---

@GerdW  a écrit :

as has been said several times before you cannot delete properties, when you don't want to create a copy of your file.

OK thanks, this is what I was afraid of. I just wanted an "official" answer from NI.

---

@GerdW  a écrit :

What about setting the properties to a certain value to mark it as "invalid" for your further data processing?

---

As I said I am just a small piece in the puzzle, I can change the code but not the specifications !

[GerdW]

Hi Nicolas,

---

@Nicolas_Bats wrote:

---

OK thanks, this is what I was afraid of. I just wanted an "official" answer from NI.

---

Keep in mind: my comment is not an "official answer from NI"…

Did you call the NI support to ask this very question?

[Nicolas_Bats]

OK I meant "official answer" or from advanced LabVIEW users.

---

@GerdW  a écrit : Did you call the NI support to ask this very question?

I never had chance or valuable answer from NI French support 🙂 I prefer the forum. I will ask them anyway. 

[LiesbethvO]

Well, there's been a typo in the property name, which I want to correct.

[JÞB]

---

@LiesbethvO wrote:

Well, there's been a typo in the property name, which I want to correct.

---

Simple problem!  Thanks for restating that!  

Unfortunately, this isn't an easy thing to fix.  You WILL NEED TO copy all of the file to another file correcting the Property Name inside your code.  As simple as reading all properties then editing the property names before writing.

It is a bit like the old joke:

What is the difference between a light bulb and a pregnant woman?  You can unscrew a light bulb.

[LiesbethvO]

Still, I think it is quite a missed chance for NI. It really makes sense to store measurement data as .tdms files, but if you can't make these simple changes, the advantage of saving properties together with measurement data does not weigh up to the disadvantage of building workarounds when analyzing the data. I use Matlab for data analysis, and in my Matlab files, I have quite some rmfield commands to remove those fields that I cannot remove with labview in the 100s of GB of .tdms file data. It makes the code unnecessary complicated. I guess in the future, I will also acquire the data in matlab and store the results as .mat files because I think that will be easier. 

[JÞB]

---

@LiesbethvO wrote:

Still, I think it is quite a missed chance for NI. It really makes sense to store measurement data as .tdms files, but if you can't make these simple changes, the advantage of saving properties together with measurement data does not weigh up to the disadvantage of building workarounds when analyzing the data. I use Matlab for data analysis, and in my Matlab files, I have quite some rmfield commands to remove those fields that I cannot remove with labview in the 100s of GB of .tdms file data. It makes the code unnecessary complicated. I guess in the future, I will also acquire the data in matlab and store the results as .mat files because I think that will be easier. 

---

Actually,  that is one of the advantages of *.tdms!  It is a binary protocol for saving data acquired during a test!  The disadvantage,  as you noted,  is that the TEST itself ( Read that as "scientific experiment") should be properly prepared before starting the data analysis. 

So, changes to properties, adding or deleting them posthumously to data gathering, smacks of BS B as in Bull, S as in Shit! DOX, Design Of eXperiment, must be rigorous!  TDMS enforces that DOX rule by prohibiting post design changes like "Renaming " properties. 

Design the experiment right first! Then no changes are needed.

[mcduff]

---

@JÞB wrote:

---

TDMS enforces that DOX rule by prohibiting post design changes like "Renaming " properties. 

---

But you can still change the value of property after the test. (Also NI added a delete Channel feature.)

@LiesbethvO wrote:

Still, I think it is quite a missed chance for NI. It really makes sense to store measurement data as .tdms files, but if you can't make these simple changes, the advantage of saving properties together with measurement data does not weigh up to the disadvantage of building workarounds when analyzing the data.

---

I think the issue with deleting properties lies in the structure of the TDMS file. Lengths and offsets are set in the internal structure; when you delete a property all those offsets/lengths would need to be recalculated and moved in the file. It does not appear there is a way to fill the deleted property with null characters or something similar.

[LiesbethvO]

Hi Jay, you're right that DoX must be rigorous. I also wouldn't start driving a car without first determining the itinerary. Nevertheless, I still wouldn't buy a car that doesn't have a reverse gear, even if I never plan to drive backwards. 

The problem, as I've been performing experiments for 8 years, and redesigning a measuring device based on the results, is that I cannot guarantee 100% that

1) the design of the experiments is always ideal and systematic: 

8 years ago I couldn't use the insights I have today. Sometimes, unexpected results occur, and I become aware of variables in my experiments which I didn't realize were variables. In these cases, I still want to analyze the results in the most appropriate way, in order to learn. 

2) the experiments are always carried out exactly according to plan. E.g., 2 weeks ago an intern used different variable names and values. After that I changed it to an indicator instead of a control, so he won't be able to do that anymore, but so this is how (bull)**bleep** happens!

[JÞB]

---

@LiesbethvO wrote:

Hi Jay, you're right that DoX must be rigorous. I also wouldn't start driving a car without first determining the itinerary. Nevertheless, I still wouldn't buy a car that doesn't have a reverse gear, even if I never plan to drive backwards. 

The problem, as I've been performing experiments for 8 years, and redesigning a measuring device based on the results, is that I cannot guarantee 100% that

1) the design of the experiments is always ideal and systematic: 

8 years ago I couldn't use the insights I have today. Sometimes, unexpected results occur, and I become aware of variables in my experiments which I didn't realize were variables. In these cases, I still want to analyze the results in the most appropriate way, in order to learn. 

2) the experiments are always carried out exactly according to plan. E.g., 2 weeks ago an intern used different variable names and values. After that I changed it to an indicator instead of a control, so he won't be able to do that anymore, but so this is how (bull)**bleep** happens!

---

I probably didn't mean that quite as harsh as it came out in text. However, this simple fact remains true "Once you identify that a property in an experiment, that property remains."  Deletion of a property is rather difficult to justify. Much better to simply add an additional property duplicating the original property value and document the change.  That preserves the change history and shows data integrity.

I think SCOUT TDMS Editor does implement a property delete feature, I would avoid using it. And certainly,  any xlsx file is easy to edit without compromise of the source TDMS file.