TDMS Append Raw Data: Possible without meta data?

[10Things_Rob]

It says in the whitepaper on TDMS file format (http://zone.ni.com/devzone/cda/tut/p/id/5696😞

"If meta information between segments doesn’t change, the lead in and meta information parts can be completely omitted and raw data can just be appended to the end of the file. "

Is this actually true? Doesn't seem to work for me.

In my application I am writing small amounts of data to a TDMS file infrequently. This results in a HUGE header-to-data ratio. I would like to omit the repetitive meta information from each write since the raw data content is the same each time I write.

Has anyone had any experience trying this? I can try to re-write the GTDMS Write function, but it looks tedious...

[YongqingYe]

Which TDMS nodes you are using? Is that the nodes on programming -> file I/O ->TDM Streaming?

 
And, which version of LabVIEW you are using? This functionality begins to be supported from LabVIEW 9.0.

If you are using other versions, I would recommend you use the property of "NI_MinimumBufferSize" and you can find the information about this in the help of TDMS Set Properties.

Please let me know if you have any other question. 

[10Things_Rob]

I am using LV 8.6.1.

I have used both the TDMS functions in the pallette that you referred to and the G TDMS that you can download (http://zone.ni.com/devzone/cda/tut/p/id/6471)

As for the MinimumBufferSize idea, do I understand right that the data is only written to disk when the BufferSize is full? If so, that will not fit my use case at all since I am writing data very infrequently (over a long period of time).

So you are saying that the TDMS 2.0 API suppports omitting the meta data?

[MIG]

Are you opening and closing the tdms file each time you write to it?

I only have LabVIEW 8.2.1 so this may not apply to later versions, but I notice that if I open and close the file each time I write to it there is vastly more meta data written than if I open the file once, write to it many times, and then close it.

Also, if you can temporarily put up with a bloated file while you are writing to it, you can always defragment it afterwards, which will remove all the redundant meta information.

[JackBao]

I had the same problem when I try to follow the example in the white paper http://www.ni.com/white-paper/5696/en. I even entered the hex contents listed in the white paper. It worked fine with the first write iteration, the tdms file could be opened with DIAdem. But after I appended the new set of data, without adding new lead in, it couldn't be opened by DIAdem. This is the only post I could find through Google, and seems the problem was not closed. Can anyone provide suggestions?

Thanks.

[deppSu]

Can you upload your TDMS file? We can have a check and see whether it conforms to TDMS File Format.

[JackBao]

please see the attached files. The two files were created by entering the hex from tables in the white paper in the section "In cremental Meta Information Example", iteration 1 and 2. I hope I didn't enter wrong numbers.

[Tianbin_Xu]

Hi Jack,

Since new raw data values are appended to the existing segment, the old value in Lead In telling the size of total raw data values should be updated accordingly.

The updated TDMS file for the "iteration 2" write is in the attachment.