parallelized for loop progress bar

[Bob_Albern]

I have a for that takes a long time to execute and enabled parallelism to speed things up. I also want a progress bar. Before enabling parallelism, I simple had an indicator wired to the iteration counter and the array size that is being processed. This gave a good percent complete indicator. With the for loop parallelized, the indicator bounces around depending on which instance of the for loop happens to update the indicator. Is there a way to have an indicator inside a parallelized for loop?

[VirtualWorld]

From your description what I can understand is you might want 

Front Panel - Modern - Numeric - Horizontal Progress Bar or Horzontal Graduated Bar, you can define a property node to reinitialize that with Empty and Go forward with increase or you can do it other way too.

Just to make sure it takes U8 as an input. 

If thats not exactly what you want may be if you can paste your code and little bit in detail description. 

Kudos are welcomed 

[altenbach]

The point is that with parallelization enabled, there is no longer a simple concept of %complete, because each parallel instance has no information on the progress of the others.

What is your LabVIEW version? How is the parallel for loop partitioned?

In LabVIEW 2011, you can obtain the parallel instance ID, so you would update the progress bar for only one instance using a case structure, assuming that it only needs to do 1/N the work, where N is the number of processors. This assumes of course that each iiteration completes in a similar time.

A better idea is probably a simple action engine that increments with each call, incrementing an integer in a feedback node, and returns the total count. Since it is not reentrant, it will be shared by all loop instances.

[Matt_W1]

A DVR could be used inplace of an action engine as well.

[altenbach · Accepted solution]

---

@altenbach wrote:

A better idea is probably a simple action engine that increments with each call, incrementing an integer in a feedback node, and returns the total count. Since it is not reentrant, it will be shared by all loop instances.

---

Here's a very quick draft. See if it works for you. (Warning, only use it in one FOR loop overall). 😄

Probably needs a few tweaks....

[Bob_Albern]

I tried to find a way to control the progress bar from only one instance of the process, which was your first suggestion. The simple action engine is a great solution. I wired the inputs from your example and it worked right out of the box. Thanks.

[mfletcher]

It can be unsafe to call subVIs containing state from a parallel For Loop. (If you show warnings, you will see that LabVIEW gives a warning about calling the subVI from the loop.) There is a race condition in the loop involving the call to Incrementer.vi and the indicator.

One possible execution sequence:

1. Loop instance A increments the progress to 1
2. Loop instance B increments the progress to 2
3. Loop instance B writes 2 to the indicator
4. Loop instance A writes 1 to the indicator

You would need to make updating the state and writing to the indicator atomic to eliminate this race condition. One way to make those operations atomic would be to pass a reference to the indicator into Incrementer.vi and update the indicator inside that nonreentrant subVI. Aristos Queue posted a progress bar to LAVA that keeps track of the progress and updates a progress indicator in one nonreentrant VI.

Another thing to keep in mind is that writing to an indicator in a loop can degrade performance, so you might decrease performance by updating a progress bar.

[altenbach]

---

@mfletcher wrote:

It can be unsafe to call subVIs containing state from a parallel For Loop.

---

Yes, that's why I said: "Warning, only use it in one FOR loop overall".

I probably should also have left a diagram comment inside the subVI to that effect. Sorry for this omission. 😉

(It can be used in multiple FOR loops only if you can guarantee that they never run concurrently)

(Potential IDEA for the idea exchange: Maybe we should have a new kind of reentrancy setting: "pool parallel instances" or similar. Now we could make the subVI reentrant, but within parallel loop instances they would share one clone. Would that be generally useful? Let me think about that....)

[mfletcher]

I was explaining that it is unsafe even if you only have it in one parallel for loop because one parallel for loop creates multiple loop instances. Calling the subVI has side effects that affect the other loop instances.  There is a race from the subVI call to the indicator. If you take out the Wait primitive and you run ParallelProgress.vi continuously, you will see that the progress bar occassionally does not end at 99.

[altenbach]

Understood!

For a progress bar it is however sufficient. The final value is just cosmetic and most likely not used in computations.

If the loop instances execute very fast, we don't care about the progress, and if the loop is very slow we are more likely to get an asynchronous update. Yes, it could in the worst case end smaller by the number of parallel instances.