multithread blocking

[macsmithAgain]

I'm having trouble executing 3 instances of a program, lets call it crunch.exe.  The intention is to feed data sets to the 3 instances and for them to run in parallel.  When finished, each loop should obtain another work package from the queue and invoke crunch.exe again.  This should continue until all the datasets have been processed by the 3 instances.  See the block diagram below.

To explain what actually happens, lets assume that dataset 1 takes 10 seconds, dataset 2 takes 20 seconds and dataset 3 takes 30 seconds.  Whilst monitoring Taskmanager:

• when the vi starts, 3 instances of crunch appear in Taskmanager. 
• After 10 seconds, one of the instances disappears but there is no output on its standard output. 
• After 20 seconds, another of the instances disappears from task manager but there is no output from standard output 2. 
• After 30 seconds, the last instance of crunch disappears from task manager, all three standard outputs report their results, all three loops obtain a new dataset and three new instances of crunch appear in Taskmanager.
• this continues until all datasets have been processed.

So each time around the loop, the loop pauses after processing its dataset until its 2 sibling loops have completed processing their datasets.  I've tried placing the Cmd action into a re-entrant subvi but the behavour is the same.  I've also tried putting the subvi into a different Preferred Execution System with differrent (higher) priorities to the main vi but again the behaviour is the same.

I'm running labview 14 on windows 10 pro.  The PC has 2 physical cpus, each with 6 physical cores and hyperthreading is enabled.  There is 96Gbytes of ram available.

Can any one help (please!).

Ian

[crossrulz]

It is sounding like your crunch.exe is not "thread safe".  In other words, it doesn't like to run things in parallel.  A stupid experiment to try would be to make 2 copies of the executable and have each of your loops call a different copy.

[Alexander_Sobolev]

If it takes 30 seconds to execute three datasets, then they execute in parallel. Otherwise they would execute 20+30+10 seconds.

I would say it is system exec problem waiting for all instances of one executable

macsmithAgain, Can you try to start 3 cmd sessions in windows and send each command there?

[macsmithAgain]

thanks crossrulz, an interesting thought.  I did as you said and created crunch0.exe, crunch1.exe and crunch2.exe, calling them instead of crunch.exe.  Sadly the result was the same.  😞

[macsmithAgain]

thanks Alexander_Sobolev.  I too think it's something to do with the System Exec but have been unable to find out and whatever it is.

Re the anticipated runtimes, what should happen is that after 10 seconds, the first completing thread should pickup another dataset off of the queue and start running that immediately.  Instead it waits until the other two have finished (30 secs) before running again.  So it is running in parrallel but for some reason, awaiting the completion of the other two before doing the next job.

[Bob_Schor]

Whatever it is that "Crunch" is doing, could you do it with LabVIEW code?  Then you could make it a VI, set it to be a Pre-allocated Reentrant Clone, and call as many copies as you wanted (or would fit), letting LabVIEW ensure the parallelism for you.

Bob Schor

[macsmithAgain]

I've also just tried running three CMD consols simultaneously and they complete in the correct amount of time, ie, they don't wait on the other two.

So perhaps it's labview's implementation of CMD.

Ian

[macsmithAgain]

crunch (not its real name) is actually a commercial product, so I can't replicate it in labview, unfortunately.

Ian

[cbutcher]

Although this is a fairly dissimilar test, the following returns the "return code" at different times for the 3 calls (and of course the total matches the longest duration).

So it appears that the System Exec node doesn't always cause the issue you're seeing...

[cbutcher]

A somewhat closer test also doesn't show the problem you describe (at least for me).

Spoiler

Note the images below don't show the evidence of this - you'd have to run it and check the update times of the indicators.

So perhaps the problem is related to crunch.exe? (or at least, "timeout" behaves as expected).

The attached file is backsaved to 2015 - I've not checked what happened to the Sync Data Flow VIM...