LabVIEW and large data sets are incompatible. T/F? I say T.

[*3d0g]

dca (debug continues, appended)

Ok, 2nd hairball gone.  Everything still ok, except the CH1 issue still has the workaround.  (No, it wasn't magically fixed.)

[*3d0g]

No, the shift registers have not worked, so far.  Still looking for the solution.

But I still need to figure out why I have to use this workaround, though, and this is why I say "so far."  They could still work once I figure out this need for the workaround. JÞB: I have not heard of your function, before today.  I tried searching on it earlier in help, and that's when I ended up reading the memory management document. ...I fell down a rabbit hole and read and read. It'd seem your function would be more time effective vs creating a substitute model vi. I just need to find it, now. 

[*3d0g]

One last thing, for now, unless I suddenly get a solution, of course, is what I'm doing right now technically didn't hi-jack my own thread, technically, for what I am doing right now is exactly what I would be doing with the LeCroy, as far as I know.  I'd be drilling down to the raw data, at the API, bringing it out and processing it.  Their presented vi can't handle this number of samples, at least not with what I have to work with, bits-wise.  (There's actually a term for that, but I can't recall it and haven't for a while; it's not "bits-wise."  It's a single word. Nor is it bitness.)

Change to 64-bit LabVIEW!  I can't do that!  That is not an option.  If there is a solution, I want to find it -- I'd like to find it with your assistance, because that would reduce the time I must search and test hypotheses.  Granted, yes, I started this thread with a scenario, but it's actually real.  I'm trying to solve a real problem.  I figured in the end I'd either solve the LeCroy problem, or I'd solve the PicoScope problem; the thread was just launched in a generic/hypothetical situation. I was asking you, experts and novices alike, what would you do to prove LabVIEW, in my situation (32-bit), can handle this?  So far the only hypotheses for solving this real problem are change to 64-bit LabVIEW or use shift registers.  Hence, so far, reflecting back to the title of my thread, the answer is T.    

[*3d0g]

looking into this Always Copy idea, I found this, which led to a suspected bug investigation, CAR #737887 

I'm still digging, though.  I still don't see how to use Always Copy, aside from dropping it on a wire and seeing what happens.

[altenbach]

---

@*3d0g wrote:

The solution to a problem isn't a statement of words unless the statement is proven to be correct by a result.    

---

Our issue is still that the problem is just a statement of words because we haven't see anything graphical.

Words are incapable of describing LabVIEW code implementations even if you use 1k words 😄 . So you tried "shift registers". There a millions of way to use them and some are better than others. If you used an "always copy" we still don't know how you used it!

[santo_13]

And even after so many posts, you have never attached the VI you're working with.

[*3d0g]

altenbach:

"we haven't see anything graphical"

Thank you for participating in my thread.  It will be a huge pain, as I see it (I don't think LabVIEW like you guys,) to construct a model of my problem so that you will have a graphic.  It is too bad that my saying the thing won't go through the wall isn't good enough.  I thought for sure there'd be ideas when I said that I could bypass the issue using a local variable.  You know LabVIEW backwards and forwards, especially Rolfk it appears from what I've read.  I see it as a simple, "there's an open at the wall."  It looks like a bug, but a bug that was previously reported.  The solution has been "Always Copy."  As far as how to use it, I figure I'll just drop it prior to the array exiting and see what happens, forcing a copy of the array to the other side, hopefully.

As far as my issue with memory, I'm seeing I'm going to have to go to the disk.  LabVIEW just can't handle it and remain 32-bit LabVIEW.

What do I need?  What is the requirement?  That has been said.  I need two differential captures of 32MSa or better, and I need to be able to process them.  I'm sampling at 500MSa/s, no higher.   

[*3d0g]

santo_13:

"And even after so many posts, you have never attached the VI you're working with."

It is disappointing that the signal won't go through the wall isn't good enough, but as I just said to altenbach, it'd be too much of a pain, and time-wise, to come up with a vi for you guys.  The signal won't go through the wall.  I had to cheat and use a local variable to bypass it.  It seems a simple description, to me.  And it looks like a bug that I might be able to "fix" via "Always Copy."

As far as the huge memory, LabVIEW 32-bit can't do it.  I'm going to have to go to the disk.

(BTW, of course I'm talking about post processing.  I've never said a thing about streaming or speed aside from the sample rate.)

[mcduff]

Just returned from vacation, I'll show some pictures.

I don't have your instrument, I simulated a USB-6366 X series DAQ. It has 8 channels and can sample at 2 MSa/s. I took data for 16 seconds, so that should give me 8 channels of 32 MSa. Each sample is 16 bit.

I displayed the data and did some stats; note the stats are identical for the data because it is simulated.

I am using LabVIEW 2021, 64Bit, but looking at the memory usage, could fit into 32 bit with the 3GB option. But you only have 2 channels so I exceed your requirements by 6 channels, so I say it can fit into 32 bit.

8 Channels

32 MSa Each Channel

2.048G to store in memory as doubles

512M to store in I16

2.56G just for data

Am using about 600 Mb for LabVIEW, display, processing, etc, see screenshot

Here are the screen shots.

Display data. Decimated data before display.SImple stats on 32 million points for each channel.Memory

[*3d0g]

mcduff: Thank you for your participation in the thread.

simulation and math...

I'm whittling away as I can.  LabVIEW is quite greedy I am learning.  The slightest thing can increase the memory usage, and it doesn't give it back.  I tweak a little and I test.  Right now I am working with two channels.  In the end it needs to be four.  500MSa/s to produce 32MSa per channel. It will be two differential signals.

As things stand right now, the raw data is had into memory, but I'm having to get more creative getting it out.  The shift registers aren't going to work, because each needs a buffer of appropriate size, using the present code -- that has to change.  I'm going to have to retrieve and store, channel by channel, reusing the buffer for each.  When the vi exits I'll need to request deallocation and move to a post-processing vi, probably.  I'll see.