Queue Memory Allocation Weirdness

[Ben]

---

@mcduff wrote:

Ben,

Thanks for your insights. However, please try this example while looking at the task manager in Windows.

 I am using LabVIEW 2011 SP1 and Win7 with a Core2Duo.

The buffer allocations do not show anything unusal or unexpected. The only strange part is the last loop; the memory keeps increasing even though there is only 1 element in the queue at anytime.

Cheers,

mcduff

---

Sorry Mr mcduff but the only machine I use to access these forums is limited to LV 7.1.

my suggestion about making those array sizes coming from a control rather than a constant is "Constant folding" where LV will cheat and precompile code at edit time rather than at run time. Changing it to a control, LV can not cheat (your benchmark).

The next part comes with looking at the "Buffer allocations" goe along with the narative tha goes with the code as in...

Crate a queue thtat can handle ten elements.

Take a COPY of the data (its a copy since we can not queue the same buffer to five entries in the queue without frightful reprecusions) and point to it as a queue element.

Remove an element of from the queue and release the lock on the buffer holding the data.

Queue up empty buffers for  the queue to point at.

Remove the entries that point at the empty buffers.

Put a copy of yet another static array in the queue.

Verify it is only one.

Empty the queue

Crate yet another copy and put it in the queue....

As much as I can do without bending over backwards,

Ben

[nathand]

Constant folding can be an issue for timing benchmarks, but in this case for memory use, I don't think it's having any effect.

I've done a bit more testing.  My guess is the queue (fixed-size, anyway) is implemented as a circular buffer.  This is a pretty common way to implement any sort of FIFO.  It uses two pointers - a read pointer, and a write pointer.  Every enqueue increments the write pointer, every dequeue increments the read pointer, and they wrap at when the reach the last element.  There's a bit more logic, and each FIFO slot probably has a flag indicating if it's full, but those pointers are the key part.

Based on the above, here's what I think is happening.  When you enqueue the single elements in the last loop, you're actually allocating a different element of the FIFO (because the write pointer increments each time).  As previously noted, the queue won't release the memory associated with an element when that element is dequeued, so that a new element can be queued without a new allocation.  So, you're allocating space for all the elements in the queue, even though it only ever contains a single element.  I'm not sure why my code, with the initial enqueue outside the loop, doesn't show this - perhaps there's some optimization that notices that the same element being dequeued is immediately enqueued and reuses the same slot.

I hope someone with actual knowledge from within NI will comment.

[Ben]

---

@nathand wrote:

...the same slot.

I hope someone with actual knowledge from within NI will comment.

---

Ditto that!

All I can ever do is read the tea leaves and contrive some narative that fits the nature emperical testing shows.

Where is Dr VI when you need him?

Ben 

[mcduff]

Nathand,

Your explanation sounds completely plausible and seems to explain the results that I am seeing. Thanks for your help.

Interestingly, if you set the queue size to unlimited, the memory will only increase to the number of pointers that have been previously allocated in the loops before and not increase indefinitely.

Thanks to you and Ben.

Cheers,

mcduff