@Matt_AM wrote:
@avogadro5 wrote:
... but by flushing the queue every time you're forcing a full copy anyway.
Just wondering, how am I flushing the queue? Is it when I am querying all elements in the queue? This is where my knowledge of LV lacks; the "nuts and bolts" side of things.
If I am doing this wrong, what is the correct way to use the queue as I want, or is it just I am not using an efficient method for doing what I want? Would using a shift register with an IPE to update the XY bundle be the correct methodology? I can add a feedback node to my queued circ buffer or I can add a shift register to the loop outside and pump the "old XY" into the circ buffer. I'd most likely go with a feedback node since this way I keep the queued circ buffer self contained, and all it needs is the new data and num points to run.
Digressing to the current queue circ buffer I am using, I thought that queues basically were a list of points and data, which point to where the next set of pointers/data is at. By creating a 1000 elements and doing a lossy enqueue, I am assuming that the queue has to update the initial "start here" pointer (element 0), and the "old stop here" pointer (give more details below). So when lossy enqueuing, I wouldn't need to create an array, then remove data from the array, then send that array to my XY bundle, I'd just need to update pointers and query the queue for all elements.
To be verbose with the "old stop here" pointer, say a queue is size 100. "start here point" is element 0 and "end here" pointer is element 99. If I lossy enqueue 3 elements, the "old start here" pointer references the "old element 2" (now element 0 due to the lossy enqueue removing the 1st 3 elements) and the "old end here", which reference element 96 now, would be updated to point to element 97, which continues to the "stop here" pointer of element 99 again. Hopefully this makes sense, I tried giving a clear example of what I meant by "updating the old stop here pointer" being updated once a lossy enqueue happened.
Thanks,
Matt
Yeah I'm using "flush" as shorthand for "get all elements" - I think they're the same thing other than "flush" removes the elements as it goes through the queue.
I think you're correct as far as enqueuing (I don't want to think through all the pointer logic), now think of what has to happen in order to turn all the data in the queue into an array as is done in your screenshot: we need to allocate space for all the elements then go element by element through the queue, copying each one into the array that you bundle into the cluster. The whole game here is avoiding copying all the elements and you just guaranteed it will happen at least once every time the VI runs. Because your output is an array the only way to actually keep the data in place and avoid copies is to preallocate an array to the size you need and use something like, as others suggested, a circular/ring buffer - here's a good library that makes it easy: https://www.vipm.io/package/jdp_science_malleable_buffer/.
The situation where a queue can help is if you enqueue frequently, but flush (get all elements) infrequently - for instance you might acquire data and insert at 1kHz but because your monitor only runs at 60Hz, it makes a lot of sense to keep the "for display" data in a queue and then only flush the queue when you go to display the data. Then you only pay the copy penalty a fraction as often as you insert data.
This page might help - a queue is understood to be a linked list under the hood: https://www.geeksforgeeks.org/linked-list-vs-array/ the key performance benefit is inserting a single element is far faster than "resizing" an array which in a low level language is actually creating a new array and copying the old one to the new one (barring malloc f-word-ery). I'll say if you run benchmark tests with large data sets in LabVIEW by timing various operations - which I strongly suggest you do to get a handle on this topic! - it's clear that a lot of things that look like they might force a copy operation (like build array node) actually don't always because the compiler seems to be smart enough to, for instance, allocate extra space in memory for the array to "grow into."
