Why are my TDMS files so large?

[blessedk]

Hello Herbert,

I am very new to Labview but I find this topic very interesting. I tried using the NI_minimum buffer size property. Kept getting an error message " invalid file reference. I looped the channels ( since I want all channels to be buffered) . I had my "TDMS set property" in between TDMS open and TDMS write. I also noticed that it demanded a mandatory input for the group names ( which does not apply to me) once the a channel name is defined. I put in 10000 for buffer value. Can you think what may have gone wrong ?

[M0535]

Hi blessedk,

Are you using your TDMS files in LabVIEW? If so, I would recommend posting in the LabVIEW section of the forums as you might get a better/quicker response.

[Brad_Turpin]

Hi blessedK,

Why do you say the mandatory group name input doesn't apply to you?  If you're looping over all the channels and assigning this property, then you need to provide the group name input each time you provide the channel name input.  That's how the TDMS VIs work.  Both inputs are required for the Channel level, only the group input is required for the Grorup level, and neither input is required for the File level.

Brad Turpin

DIAdem Product Support Engineer

National Instruments

[blessedk]

Brad, so glad to hear back from you. I actually assigned a group name and also I don't have the "invalid file" error ( Wrong wiring ) but I have new concerns which are two fold.
1. The file size did not change even after applying the property.
2. I had previously written/ read a 700 MB TDMS file seamlessly within seconds, but now when I try to do the same at some other point on the same VI, it takes an unacceptable amount of time. I have a feeling that i really need storage at this second point ( in between two For loops) because the second For loop takes time to start running even after the first loop is done and then stops intermittently while running before completing the iteration.

[blessedk]

Note that the 700 MB file I successfully wrote/read ( and still works good) at my first my storage point was absolutely without buffering.

[Brad_Turpin]

Hi blessedk,

You never mentioned what value you assigned to that property-- it makes a big difference.  It needs to be a value big enough that you're not constantly writing binary headers yet small enough that if your acquisition crashes you won't be miffed about the amount of buffered data in memory you just lost.  Would it be possible for you to post you data acquisition code, at least the part that saves TDMS files?

Brad Turpin

DIAdem Product Support Engineer

National Instruments

[blessedk]

Hi Brad,

I tried 1000, 10000, and 100000 of buffer sizes. Attached are two screenshots of the storage sections of the VI.

The first write (one shown with buffer) writes/reads about 720MB  pretty quick even without a buffer. (No improvement seen with buffer). 

The second write consists of four 2D array dbl  inputs and one ID array (at the bottom), with all four 2d arrays leading into the same For loop.

This second write/read is slow and hangs. I have also tried this with dfferent configurations ( have tried dedicated For loops for each of the four write/reads in parallel, as well as wiring the error clusters in series so that each write/read  is followed by the next. - All to no avail.

I am new to LABVIEW so it is very possible that I am missing something.

[blessedk]

Please note that i intentionally blanked out the file path name, so don't consider that a problem.

[Brad_Turpin]

Hi blessedk,

Your VI screenshot shows that you are not sending a GroupName to the TDMS Write.vi

There was a wonky mode programmed into the TDMS VIs that allowed this scenario and then auto-looped the result, but the short story is that you never want to do this.  What Group do you want those TDMS Write.vis to write to, exactly?

I don't know if that's the problem, but let's address this first and then see if you still have issues.

Brad Turpin

DIAdem Product Support Engineer

National Instruments