Producer/Consumer vs Master/Slave Data loss

[johnji]

Hello Labview users,

I have been experimenting with P/C vs M/S lately in the context of data aquistion. Lots of posts have made it sound (Correct me if I'm wrong) that P/C is the correct architecture to if data preservation is desired, so that's what I've tried to implement.

In the producer loop, I've included 3 DAQ assitant VIs (2AO, 1 AI), one that actively generates data and is being enqueued. I have 3 other consumer loops, two of which are consumer loops 1) Writes data to TDMS 2) Outputs to a graph 3) Non producer: Event structure, monitors changes to UI.

When I run data aquisition, the file generated are consistently half i.e. aquire half the number of sample that  I have specified regardless of the sampling rate. I also have a weird graph being displayed (See attached image). 

However,

When I replace the architecture with a Master/Slave, this seems to solve the whole problem.

Question therefore is: has anyone come across such problems? Are there any downsides to sticking with M/S as opposed to P/C in terms of data loss?

Thanks

[James.M]

Can you please elaborate more on what you have differently between your Producer/Consumer and Master/Slave code? Producer/Consumer is a subset of Master/Slave and so dominant these days that I don't hear the term Master/Slave used much anymore.

If you're losing data then there's something you must be doing wrong. Please include a Snippet of your code, or attach your VIs, so that we can better assist you.

[dkfire]

From the picture you have attached, you only have one queue.

Do you have multiple loops that are dequeueing from the same queue? That is a no go!

You need to have a queue for every consumer that you have.

[johnji]

Didn't know you could do a snippet. Here goes. I believe that I am properly dequeueing 

[crossrulz · Accepted solution]

---

@johnji wrote:

I believe that I am properly dequeueing 

---

You are not.  Once an item is dequeued, it is gone.  Since you have three loops dequeueing from the same queue, each loop will get ~1/3 of the data.  If all of the loops are to process everything, then you will want a different queue for each loop and have the producer write to each queue.  Alternatively, you could use an User Event.

[crossrulz]

---

johnji wrote:

Question therefore is: has anyone come across such problems? Are there any downsides to sticking with M/S as opposed to P/C in terms of data loss?

Thanks

---

Back to the original question.

The main difference between Master/Slave and Producer/Consumer is that M/S using a Notifier while a P/C uses a queue.

A Notifier is a broadcast message, so everybody can get it.  However, you can miss a notification if your loop is slow.  The Notifier is lossy.  You overwrite a notification each time you write to the notifier.

A Queue is a lossless FIFO.  It is meant to only be read from 1 consumer.

So if you truely want no data loss, then you need to use a queue.  But since you are trying to have many loops dequeue the data, you really need a different queue for each consumer loop.

[Yamaeda]

---

crossrulz wrote:

So if you truely want no data loss, then you need to use a queue.  But since you are trying to have many loops dequeue the data, you really need a different queue for each consumer loop.

---

To expand on Cross' text: Queues are a Many-to-one relationship, Events are One-to-many. If you want to signal many loops you should use an Event driven producer/consumer, or use individual queues for the relationship reasons just stated.

/Y