Combining syncrounous and async data?

[mak90]

Here is my problem: I have two sources of data which I desire to be logged, one source is data being logged with NI provided analog and digital input -generated in wonderful hardware timed intervals. I perform continous reading on this data and am certain of its delta time. The other source of data is ni-switch status (which calls a ni-switch .dll to query the hardware state), and data coming from other (what i call 'software timed' data sources) .dll read resources. Specifically resolver and rvdt positions which are monitored by 3rd party pci cards without any hardware buffering. So every time i call the provided .dll read function, i get the data. This data is generated when the read loop (software) gets around to performing its iteration (which we know in a non-RT system can be who knows when). Naturally i want to combine this hardware and software timed data into one logged file which merrily can be viewed with my data viewer.. Have any of ya'll done this kindof thing before? Or how would suggest doing this? I have done this a number of ways in the past, and forsee doing it again in the future but non of my implementations are imho very robust. I have considered perhaps timestamping all my software-read data and trying to interleave it with the hardware timed data, interpolating the software timed data when it is non-existant for a hardware timed data point... And perhaps this is the best way, but look before you leap kindof thing here. Thanks all!

[hecmar.arreola]

First of all, the complexity of this problem (and if a solution is reasonably possible) is related to your sample rate.  If you require 1ms precision then you are going to have a hard time with that and unbuffered software-timed hardware.  What kind of delta-t between samples are we talking about here?

Secondly, you should consider using the Waveform datatype.  The Waveform datatype is very easy to use, and allows for a timestamp for each data point (or a collection of evenly spaced datapoints where the first sample is timestamped).

As an example, I take data from a DAQmx device, and from a software timed USB thermocouple reader.  What I do is take them all into the waveform datatype, and then I coerce the USB thermocouple reader data points to the time stamp of the DAQmx device.  There timing is about the same, and I'm generally dealing with large dt's between samples (250-500ms typically).  So if my temperature measurement comes in at T=900ms and my DAQmx device measurement comes int at T=1000ms, then the temperature measurement is recorded at T=1000ms.  I do this by combining the two waveform arrays and using the "align time stamps" VI on the new waveform array.

This is very simplistic, and it can be because my application does not need the precision.

[mak90]

Good point, i knew I would forget to mention something.

I would love to get data as quickly as possible, I plan on sampling the hardware timed data at 1khz, and then interleaving the software timed data into it. The software timed data loops I have running as quickly and efficiently as possible but I doubt i'm getting better than 200hz out of them when the cpu is really loaded.

thanks for the waveform idea -

jacob

[johnsold]

jacob,

If it is practical to do so, create or use a signal which is common to both the hardware and software timed acquisitions.  This becomes a synchronizing signal.

For example if your position sensing devices have a once per revolution pulse, feed that to an unused analog input channel on your hardware timed device as well as recording it through the software timed system.  You can then align the data to that pulse.

Lynn 

[Jarrod_S.]

You should look into TDMS for your logging needs. This format has a very simple API based on channels. Each channel can have it's own notion of time. You can either tag it with a dt like a waveform, or you can use two channels to form an XY pair.

Most importantly, you don't have to have one sampling rate or timing scheme in your file. You can log two channels at two rates. It even supports multi-threaded access, so multiple loops can write to the same file.

Using TDMS will be much easier than trying to resample and sychronize all data that gets logged.