save contonous data in TDMS

[Runess]

Hello 

I have a live stream of undeterminate lenght with 3 channels X,Y,Z. To save the data to HDD in TDMS I created the file with 3 channels and save the data when i read it, but  when channels are full stops working. 

The only way I find to save all data is creating new channels so my file has X1,X2...Xn Y1,Y2......Yn.......

There is any way to save all data without creating more channels?

J.C.

[Hooovahh]

We need to see the code.  You say when the channels are full it stops working.  What is the error, where does it stop?  What gets full?  You should have no problem continually logging to the same channels for a very long time.

But if you do have to create new files all the time, then you should know TDMS files can be merged, and the channels will be concatenated.  Combining TDMS files is just one of the many features of my Tremendous TDMS Toolkit found on VIPM.IO

[santo_13]

---

@Runess wrote:

Hello 

I have a live stream of undeterminate lenght with 3 channels X,Y,Z. To save the data to HDD in TDMS I created the file with 3 channels and save the data when i read it, but  when channels are full stops working. 

The only way I find to save all data is creating new channels so my file has X1,X2...Xn Y1,Y2......Yn.......

There is any way to save all data without creating more channels?

J.C.

---

You should be able to append data to the same channel, please share the code you have.

[Runess]

Hello

Thanks for the reply, this is the code. It's working fine until the channel has 2145484800 samples, then the function AppendDataValues returns -6219.

DDC_MaximumNumberOfDataValuesExceeded = -6219, // The request would exceed the maximum number of data values for a channel.

 IntPtr pTDMSFile, pChannel, pChannelGroup;
 float[] buffer;
 
 TDMLib.TDM.DDC_CreateFile("1.tdms", "TDMS", "", "", "", "", ref pTDMSFile);
 TDMLib.TDM.DDC_AddChannelGroup(pFile, "stream", "", out pChannelGroup);
 TDMLib.TDM.DDC_AddChannel(pChannelGroup, TDMLib.TDM.DDCDataType.DDC_Float, "SIGNAL", "", "ut", out pChannel);

 while (!isStopped)
 {	
	int readedBytes = readBytes(out buffer);
	if (readedBytes > 0)
	{
		TDMLib.TDM.DDC_AppendDataValues(pChannel, buffer, readedBytes);
		totalbytes += readedBytes;
	}
 } 
 
 TDMLib.TDM.DDC_SaveFile(pFile);

[santo_13]

---

@Runess wrote:

Hello

Thanks for the reply, this is the code. It's working fine until the channel has 2145484800 samples, then the function AppendDataValues returns -6219.

DDC_MaximumNumberOfDataValuesExceeded = -6219, // The request would exceed the maximum number of data values for a channel.

 IntPtr pTDMSFile, pChannel, pChannelGroup;
 float[] buffer;
 
 TDMLib.TDM.DDC_CreateFile("1.tdms", "TDMS", "", "", "", "", ref pTDMSFile);
 TDMLib.TDM.DDC_AddChannelGroup(pFile, "stream", "", out pChannelGroup);
 TDMLib.TDM.DDC_AddChannel(pChannelGroup, TDMLib.TDM.DDCDataType.DDC_Float, "SIGNAL", "", "ut", out pChannel);

 while (!isStopped)
 {	
	int readedBytes = readBytes(out buffer);
	if (readedBytes > 0)
	{
		TDMLib.TDM.DDC_AppendDataValues(pChannel, buffer, readedBytes);
		totalbytes += readedBytes;
	}
 } 
 
 TDMLib.TDM.DDC_SaveFile(pFile);

---

Oh wow, that is a pretty large dataset, what is your application? looks like you're reaching the I32 max limit. Why do you need so many samples?

[Hooovahh · Accepted solution]

Wow that is a large amount of data in one channel.  I'd find reviewing the data post-test to be quite difficult.  That being said it is only 35 minutes of continuous data collection, if you sample at 1MHz.  You are going to have to try to either start logging into another channel like "Data (1)" and "Data (2)", or make a new file and log to that. 

That being said the TDMS offset and length support the I64 data type.  I just made a TDMS file with a single channel in it that had 3,006,476,200 samples in it.  The data type was a U8 and the file size was about 3GB which checks out.  I was then able to open a new reference to the file, and query the channel length, and read the last handful of values in the file.  This was all within LabVIEW.  Sorry I don't know enough about the .NET implementation, but the file type supports these large amounts of data.

[santo_13]

It is possible that the .NET libraries for TDMS could be bottlenecked evident from the error description that you're reaching max samples per channel.

[Runess]

I'm using the 64bits lib and it seems that still uses I32 for indexing. For now I'm creating new channels as needed.

Thanks.