Large data array to disk

[sjandrew]

I have also found the source of my "setting to zero" problem. I tell the device how many samples I would like, and it's supposed to give me that many. I have a numeric control on the front panel of my top-level VI called Num Samples, and it feeds this value to some subVis which make the proper function call at the DLL level. If I change Num Samples from 10,000 to 100,000, the subVi just ignores the fact that I have updated (verified by opening the front panel of the subVi after exectuion completes - it still has the old value of 10,000). However, if I have the front panel of the subVi open when I'm running the top-level VI, it will update to the new value of 100,000. Hence, I don't get out the number of samples that I was expecting.

To be clear - this is a simple wire from one numeric control to an input terminal on one subVI's connector pane, which then connects to a control on its front panel. The most basic of function parameter passing, but something prevents this from working properly.

[nathand]

Again, it would really help to see your code - even a mock-up of it. Who handles the array allocation - you, or the DLL? If the DLL expects to receive a pointer to an array large enough to hold all the data, and you pass it an empty array, you're almost guaranteed to get some weird behavior eventually, since the DLL will be writing to memory that was never properly allocated. If the DLL allocates the memory, then you need some way (such as MoveBlock) to copy it into an array that LabVIEW knows about. Also, if your DLL is passing back singles in little-endian, and you're just writing that data directly to disk without doing any processing/display in LabVIEW, it may not make sense to receive it into LabVIEW as an array of singles because LabVIEW will do the unnecessary endianness conversion for you. You could instead receive the data as an array of bytes and write those directly to disk; then when you read them in Matlab or whatever as little-endian, the data will be fine. Just remember that you need 4 times as many bytes as singles in the array (since a single-precision float is 32 bits).

[RavensFan]

It took until message 28 of this thread to find out that the array is relaled to a DLL call?  That might have been an important piece of information to hear about in message 1, or at least message 3. 

The array is supposedly 1 million elements according to the dll and LabVIEW thinks it is 10 millions elements?  That is a huge discrepancy, I'd try to solve that problem first before I'd worry about how long it takes to write the array to the file.

[sjandrew]

I'm not sure who allocates the array - the "call library function" node just has an array of doubles as an output, and no array on the input, which would make it seem that the DLL is allocating the memory. This array of doubles then gets converted to an array of singles (a conversion which I deleted, noticed no time improvement, then put back).

I'm working with the DLL vendor to see if I can do this at the C level; maybe it's just my prejudice, but I have a feeling that LabView is the perpetrator here.

[nathand]

---

@sjandrew wrote:

I'm not sure who allocates the array - the "call library function" node just has an array of doubles as an output, and no array on the input, which would make it seem that the DLL is allocating the memory. This array of doubles then gets converted to an array of singles (a conversion which I deleted, noticed no time improvement, then put back).

I'm working with the DLL vendor to see if I can do this at the C level; maybe it's just my prejudice, but I have a feeling that LabView is the perpetrator here.

---

You need better documentation on how the DLL works. I doubt the problem is LabVIEW; more likely you're just not calling the DLL properly. Who wrote the LabVIEW VI that calls the DLL - you, or the vendor? If you, did the vendor provide any documentation for the function? How is the Call Library Function Node (and in particular, that array parameter) configured? If you find out how to do it properly in C, you can make LabVIEW do the same thing. My guess is that you need to preallocate the array in LabVIEW (and the same would be necessary in C).

Why convert to the array of singles, though, if you're just going to write the data directly to disk?

[nathand]

Just curious - did you ever resolve this?

[sjandrew]

Yes and no. I got a new driver from the vendor, which uses VISA VIs (which just wrap up a text-based command syntax, so you can see what it's doing) to acquire the data (instead of IVI, which called a dll, which did who knows what). The long and short of that is that the data is always in the array, no more problems there. (Before, the data was dropping out).

Still having trouble with the File I/O, though - using the VISA driver, if I try to write to disk using a binary write, I get error code 6: "Write to Binary File in test.vi<APPEND>test.bin" (where test.vi is the LabView code, and test.bin is where I'm trying to write). This makes no sense, because I opened the file (with the Open File VI) with option 4, which is "Create or Replace" - i.e., no appending. This only happens if I try to write 1,000,000 or more data points; if I try to write fewer than that, I get no error.

I then wrapped the Binary Write in a for loop, and I get no error, although it still takes a long time (about 4 minutes for 1 million points).