TDMS read error-getdatatype

[ktklabtech]

Brief background on my project.  I have a PXI system setup to read signals from DSA devices, S-Series, and an Encoder.  I am using matlab to make function calls to the NI Libraries in DAQmx, and nilibddc.h.  I collect data and stream data to disk with the ability to view data during acquisition.  Acquisition occurs for nearly a minute over 37 channels at 100ksamp/sec.  So the .TDMS files are fairly large.  I synchronize my systems by a sort of hybrid master slave device configuration.  I use automatic synchronization across similar device types, and to synchronize different device types i use the sample clock and start trigger from a DSA device to sync my other devices.  Since I use automatic synchronization the .TDMS files can have anywhere between 1 channel or 20 channels in one file.  This makes for a very large file and Microsoft Excel Importer doesn't cut it due to the limitation on size and time it takes to load.  So I am using the nilibddc.dll to read the data from the files which seems to be pretty smooth so far. I have not attempted to read a file with a minute worth of data but worse case I can read chunks of the data file at a time so I am not terribly worried about that.  My problem occurs when I read data from my .tdms file containing my encoder data.

The part where it gets hung up is my function calling for DDC_GetDataType.  Now when I use my code to read data from analog .tdms files it works smoothly and does everything I want it to.  The error I get for my counter signals is an unexpected error (-6204.)  Now the interesting thing is when I use labview 2009 on another computer, i can read the files with no problem.  I am guessing my labview 2009 uses different libraries though.  For example when I read the file in labview, i go to the functions pallete, select file i/o, tdms streaming, and use either read .tdms or tdms viewer function.  When I am on my primary computer, I am using a call to the DIAdem Connectivity Library I downloaded from NI. The microsoft excel importer seems to also be hung up at the same location (getdatatype).  I found this when inspecting the log file it creates when I try to read it.  Any help would be appreciated.  I have attached the error log I get from microsoft .tdms importer.  I also attached a sample .tdms file with encoder position data in it.  If trying to open outside of labview, don't be fooled if it tells you there is no data available.  It lies, using labview 2009 you can read all 500ksamps without a problem.  Using TDMS viewer or read tdms atleast.  Thanks.

[nickb · Accepted solution]

Hello -

I wanted to preface what follows with the fact that I'm assuming you're using the TDM C DLL downloaded here.  If this is the case, the following should resolve things for you - if not, please let me know where you've gotten the ddc library from.

Unfortunately, the TDMS dll that is downloaded through the link above is a little outdated and contains the bug you've reported.  I've created an internal action request to update the download with the latest version we have available.  In the meantime, I've attached the necessary tdms dll to resolve the issue for you.  You will need to use the attached dll to replace the dll in ".\Dominonilibddc\dev\bin\<bitness>\tdms_ebd.dll". 

One potential problem here is if you're using the included merge modules to redistribute your product.  If you are, we'll need to get you some updated merge modules as well - please let me know if this is the case for you and what the urgency level is.

Please let me know if this does not resolve the issue for you.

Thanks -

NickB

National Instruments

[ktklabtech]

I went ahead and used the tdms_ebd.dll that you attached and tried it out with my code.  Unfortunately I am still getting hung up at the same location on TDMS files with CI data stored in them.  The getdatatype functions returns an unexpected error.  If I try to bypass this in my code by manually setting the datatype, it then runs into the issue where it acts as if there is no data in the file.  I believe the functions asking for the length of the data is where it returns a 0. Now I did download the TDMS C DLL from the link to NI's website.  This is the one I am currently using with the tdms_ebd.dll that you attached in your response. 

I am trying to read the information into matlab.  I have code written up that is very similar to what most people have done, and I believe might even be what NI has released as a starting point for TDMS reading in matlab. One example that is closely related to my code is found here. Import into Matlab. My code is not as heavily formatted and I would have to go through the same process in order to release my code.  My code is basically the code though that he refers to earlier in his message found in the NI devzone. One thing I wanted to note is this though, depending on where the TDMS C DLL is downloaded, the C formatting for the functions differs slightly.  For example I have found that there are header files with a _m added to the file.  I assume these files were made specifically for usage with matlab, since they are a lot easier to work with then the actual header files.  I do have both types and have tried both types and still no luck. 

Let me know if you need any more information, but as of right now, the update seems to not have fixed my problem.  Thanks.

[nickb]

Unfortunately, I don't have access to Matlab, so I'm unable to test that at the moment.  However, I can give you the c program I wrote, to verify whether it does or does not work on your end.  I've included the exe that I wrote, alongside the file that you originally attached to this forum.  If you could copy these two files into your bin directory (32 or 64 bit, whichever you're using) and run the app from the command line, I'd be curious to see whether the app works or not.  It should fail with the earlier version of the tdms dll (2.0.xxx), and work with the version I attached (2.2.xxx).

Please let me know if you see something different.  If my code does indeed work, and yours continues to fail, I could take a look at your matlab code to see if we can determine why it's failing when the c code appears to be working.

NickB

National Instruments

[nickb]

I just realized the app I attached earlier was built in Visual Studio 2010, and you might not have the correct run-time components to run it.  Attached below is the same app built in VS2005, which you should hopefully be able to run.

NickB

National Instruments

[ktklabtech]

I am a bit unsure what was going on but for whatever reason matlab was still finding the old _ebd file.  I now have it working and things are much easier now.  Thanks for the assistance.  For future readers, the new _ebd file attached in a previous response allows access to counter data.  The examples that NI has produced for reading .tdm/.tdms files and coverting to matlab will allow users to read in the files no matter how large they are.  It's important to note the type of machine you are running on.  I have not had a chance to test this out on a 64-bit machine, but within a 32-bit machine, I can use what NI has provided to read in a .tdms file in matlab with data over 6Msamples. I can then save it to a .mat file and open it up and view the data on a 64-bit machine.  There are limitations in matlab as to how much data you can read in.  For example if I had 8 channels of 6Msamples, I would run out of memory on my 32-bit machine.  When working with very large .tdms files, I would recommend reading in a .tdms file one channel at a time, and possibly a certain number of samples at a time. The reading and saving process is not very long for what I have tested with. On the order of a few seconds.  Once I go through the proper channels, I will attempt to post code with the appropriate libraries.  For the most part, everything you need to read .tdms into matlab and save as a .mat file is out there and just takes time to find.  Thanks again Nick for the help.   

[jwins]

Has an update been released for the TDMS problem? I'm writing an application using Visual Studio 2005  (C++) that uses these DLLs to read tdms files.  When I try to open a tdms and/or tdm file I get the following errors: 

IfI try to open it as a “TDMS”, I get the “storage could not be opened” error. 

If I try to open it using the “TDM” config, I get the “read only” error.

Any help would be greatly appreciated.  Thank you all in advance for your time and consideration.

[Bile]

Dear nickb,

Is it possible to get the project from which you have created the *.exe file?

I have a problem that I can run your exe file, but my program still shows the same error ("Error occurred when loading the DIAdem library DLL (nilibddc.dll)")

Thank you

B.