TDMS: "Rows before columns"

[bockdoug]

Hey, I am writing a bunch of data to a TDMS file. The problem I am facing is that TDMS is organised in channels. Let me explain. I am writing some arrays to the file of known and repeatable size (e.g. a temperature array with four entries follwed by a pressure array of 12 entries).This works fine and as expected. Then I need to write some arrays that I do not know the length of, on top of which the length may change from "write-call" to "write-call" (e.g I may be writing an array of length 8 one time, followed by an array of length 13 the next time TDMSWrite is called.) The problem is that if I write a longer array after a short array, the difference in length between the arrays is not written to the corresponding row when viewed with excel, but rather at the corresponding columns ("channels" as tdms calls them). Please see the attached excel files for clarification.

In 'EverythingsOK.xlsx', everything's OK. In 'PeakPopsUp.xlsx' an additional peak showed up at some time in the scan, but rather than being written to the correct row, it got written to the (still corect but) meaningless channel. I chnaged my program so that now each excel column corresponds to a TDMS channel see (InadequateWorkaround.xlsx), but that means that the end user is limited to 16,384 cloumns (as is the limit in excel workbooks), which may not be enough.
So my question is, how can I fix this issue? At first I thought the decimated/interleaved on the TDMSWrite.vi input looked promising but that has nothing to do with where the data is written, only when its written... I could of course move to csv files, but that would require rather large changes to my code.

Any help would be great.
Cheers 
Doug

[Blokk]

Hmm, I might be not understand 100% the explanation in this very morning, but I explain what I usually do. It might help you might not 🙂

So if I need to write data arrays with varying length as channels to a TDMS, I keep the following logic so I am able to access my data easy later on. Since the length of the channels (columns in the converted Excel) are not the same, I rather keep an absolute time stamp channel for each data channel. I know this ~doubles the size of the TDMS file, but I did not have crazy amount of channels so far in my projects. So what you will get in Excel in columns: Timestamp1, Data1, Timestamp2, Data2, Timestamp3, Data3, etc...

Since I always have absolute time stamps, it is easy to do data mining later, and pick the required time intervals with the existing data there. If you use Excel for data analysis/display, you might be interested in the following NI product too: http://www.ni.com/diadem/

Do not forget another useful feature in TDMS files, that there are groups (groups appear as "Sheets" in your converted Excel file)! You can have your channels under multiple groups thus you can increase the maximum number of total channels in one converted excel file. I often divide up my measurement channels into multiple categories, and keep them in their corresponding TDMS Group. This gives you more "room to play"...