How to speed up array manipulation

[thehun]

I need to generate two arrays of up to 20 million samples to simulate the signals of a quad encoder. I start with an array of position samples and for each sample I create an array with the representation of a single pulse and insert it in to the final array as many times as necessary. I have tried to optimize this code in many ways (using "replace array subsets" instead of "insert into arrays" and leaving as much code as possible out of the FOR loop), but it is still painfully slow to execute (it takes about 15 minutes just to process about 10% of the whole array on an i5 machine). 

Is there any faster way to manipulate this array? I read somewhere that "insert into arrays" shouldn't be used inside a loop, but when I initialize an array with the total number of samples and use instead "replace array subsets" to insert the samples the execution gets even slower. 

Cheers

[GerdW]

Hi hun,

- Whenever someone uses ArraySize and the loop iterator to determine the stop condition of a while loop (s)he didn't understand AUTOINDEXING of a FOR loop very well…

- Whenever someone uses two ArraySize functions to determine the size of the very same array in two locations (s)he didn't think very well on optimization…

- Whenever someone determines an ArraySize in each iteration of the loop for an array that never changes while the loop runs (s)he didn't think very well on optimization…

- Why do you use InsertIntoArray when you have created an array of the needed size before your loop? Don't you want to ReplaceArraySubsets instead?

Yes, there surely is enough to improve on that VI. But you only show an image and we don't want to paint images here…

[thehun]

Hello Gerd,

you are right in that there is a lot to optimize, but you mostly refer to the outermost WHILE loop which isn't the one causing trouble and therefore I didn't dedicate too much thought to it. About the last point, as you will see if you look closely I am no longer using Insert Into Array but Replace Array Subsets.

The other problem I am now having is trying to represent both output signals on a Mixed Signal Graph. I can plot one of them but if I try to show both the system apparently runs out of memory. 

[JÞB]

What Gerd said.  Plus what version of LabVIEW is this?  (You may need an always copy on that copied array feeding both SRs into the while loop, which, should be a for loop)

[thehun]

Hello Jeff,

I am using Labview 2014. As far as I can see I'm not having any trouble with the SRs correctly storing the values. 

[GerdW]

Hi hun,

attach your VI with some meaningful default data…

[thehun]

Thanks Gerd,

here is the VI. After restarting Labview now it seems to be executing pretty fast :mansurprised:

However, I still have the memory problem trying to represent both output signals on a MSG. 

About default data: I think the input array "Position" and other variables should be filled, let me know if they aren't!

[PaulG.]

I got a "memory full" message right off the bat running this vi.  

[thehun]

Hi Paul,

that is strange, I only get that when trying to represent the signal. How much RAM do you have (I have 8GB)?

Although in total there are 2 arrays of 20M samples of 8 bytes/sample (=320MB=2.56Gb). Enough to fill up the memory in many cases I guess...

[BowenM]

---

@PaulG. wrote:

I got a "memory full" message right off the bat running this vi.  

---

Huh.  Today I learned... try to initialize an array with "NaN" and it gives you memory full error.  Does the same thing when you try to initialize an array of size "inf" - imagine that!