Queue Data

[owallis1]

Hello,

I am building a queue VI which should take in my incoming results one at a time add them to a queue, which that be processed by my consumer loop. When the remaining number in queue reaches 0, then the queue loop should exit. 

I was thinking something like:

Create queue

Queue data in producer loop

Dequeue data in consumer loop

Wait for queue to become empty before exiting

I am having trouble with passing the results into the producer one at a time (and not though an auto-indexing array) and also stopping the while loops once all the results have been passed in

Could I use a event based queue? For example, when a new result comes in then it will be added to the queue.

Any tips would be appreciated. Cheers,

Oliver

[Bob_Schor]

You don't quite "get" the idea of the Producer/Consumer design.  The Consumer (which is dequeuing) is supposed to empty the Queue and not exit.  That's where the design gets its power.  For this design to work, the Consumer must "consume" faster than the Producer "produces" (otherwise the Queue eventually takes all the available memory and everything stops) -- under these conditions, the Queue must become empty periodically.

What this loop does is to optimize the timing regularity of the Producer, which is often a Data Producing loop that needs to operate on a strict and regular schedule.  To facilitate this, you want to "get rid of the data" as soon as it comes in, as any data manipulation may take time, may delay the next cycle of the Producer, and will almost certainly introduce "timing irregularities".  The Consumer, on the other hand, usually has no time restrictions other than to be, on average, faster than the Producer.  Consider a "log data to disk" operation.  How long does it take to write the data?  That depends on various things, including whether you need to also open a new file, whether the user has to input the name of the file, etc.  Say you have data coming in at 1000 points/second.  it is easy to write 1000 points/second to disk, but may take a few seconds to open the file.  If your Queue can accomodate 10,000 samples, this gives you 10 seconds to fiddle with getting the file open -- you can even let the Queue be unbounded in size and get lots of time.

As the previous paragraph notes, while you put things into a Queue "one at a time", that "one" can be "one array of 1000 points".  However the Producer packages and "produces" the data should determine the nature of the Queue.

Hope this helps.

Bob Schor

[crossrulz]

---

@owallis1 wrote:

When the remaining number in queue reaches 0, then the queue loop should exit.

---

Do NOT do that.  Your producer should be sending your consumer a command of some sort to tell it to stop.  There are a few ways to do this.  If you do not care about keeping all of your data, destroy the queue and the consumer loop will have an error and know to stop.  If you do care about all of your data, actually send a command through the queue that the consumer loop is looking for to know when to stop.  What that command looks like really depends on what your data in the queue looks like.

[owallis1]

The purpose of my queue is to be able to handle multiple inputs from different test blocks. For example instead of having 30 test inputs, i was thinking I could pass a cluster with all the test information into a queue. So I am not really using the queue for handling the differing speeds between producer and consumer. 

The queue is intended to go around the save results part of my code, whereby I will take in all of the results via the queue and once all the of the results have been processes then it will pass an array to the rest of my code.

So maybe I could send a 'end of results' message at the end of the results which could be used to tell the queue that the results have finished?

[crossrulz]

---

@owallis1 wrote:

The purpose of my queue is to be able to handle multiple inputs from different test blocks. For example instead of having 30 test inputs, i was thinking I could pass a cluster with all the test information into a queue. So I am not really using the queue for handling the differing speeds between producer and consumer.

---

So this is more of a Queued Message Handler.  That is fine.  I do it all the time.

Based on what you have stated here, I would make the data type of the queue be a cluster that contains an enum and a variant.  The enum is used as the command such as "Clear All Data", "Report All Data", "Stop", "Set X Data".  And then the variant can hold any data you need.  You just use the Variant To Data to convert the data to whatever you need it to be.

[owallis1]

Okay Cheers! It's now working all well and good.

Thank you for your time