Memory usage problem with TDMS files

[ethen99]

Hi,

I have used TDMS write file to save my real time data .  I need to leave my VI,  running   for a long time (even one day ) then I  process the acquired data in a seperate vi.

My problems are

I olways get memory usage message during data  acquization. Also even if I finish acquization ( for example make a 2GBytes TDMS file I get the same message when I want to try to read it. 

To read the TDMS fisrt I open ,configure ,read and then close the TDMS ref.Then I use the data from the TDMS as input to my loop.

By using this method I olways get (Memory usage message ) and I need to stop the process loop. Would you please help me how can I avoid this problem

[P@Anand]

Obviously you will get Memory error (Be happy your LV didn't crash while reading the file). The reason is you may write the data to a same file continously at specified interval while doing that you have to be sure that you are not holding large data in code, if you do so you will get memory error while writing. Then while reading the file LV tries to keep the data in the dynamic memory and since your memory is not that big you wil get that memory error while reading.

Simple solution:

Try to write the data into different files for say every 1 hour so for 24 hours you will have 24 file. This you can still extend depending on the rate and number of samples /sec you need to log.

good luck.

[ethen99]

Many thanks for your answer but can you show me with a simple example how can I write my data in seperate TDMS files and how can I read from them and attach all data together

[ethen99]

Better to say  how can I  log data into multiple files . I have 2 parralel loops in my VI . one loop dis for data acquization and the other one process the data using queue structure. Then I need to save the processed data in TDMS format.

To log into multiple files I tried to use a timer inside my process loop so when the specified time( 1 hour ) has elapsed then I trigger an event to change the TDMS address outside the loop but it is not working . Would you please show me with a very basic example how I can change the address of the TDMS file using events or case structure or any other methods.

Many thanks

 

[rolfk]

your problem is the attempt to attach all the data in one big goo in memory. A 2GB TDMS file needs probably a little more than 2GB in memory if you want to read all its contents at once. That is BAAAD! You should try to write your analysis in a way that can read the data in smaller chunks and process each chunk individually instead of trying to load the entire GIGA blob at once. Computers can have a lot of memory nowadays but that does not mean that you should attempt to operate on GigaByte data in memory in one go.

[ethen99]

Thanks rolfk

yes you are right and as I mentioned in my previous email now I would like to log my data in multiple files but I don't know how!

DAQmx Configure Logging can easily do the job if you use DAQ but I am using USRP and NI-RFSG and there is no specific block like the daq one for these devices.

As I said I don't know how to change the TDMS file name every one hour -or when the file size reach to a specific value - during data acquization

[deppSu]

What error do you get? If you want to change the file address you must use TDMS Close to close the tdms file first.