Improving for loop performance

[MancPablo]

Morning All,

I have a For loop executing 5,000 times nested in an event structure. I have parallelisation enabled as each iteration is completely indepentant of all other iterations. I am performing a constrained optimisation in each iteration. I have several multidimensional arrays as inputs which I initially index outside of the For loop and then auto-index on the For loop. I then build up several arrays with the results from each iteration as outputs on the For loop. Run time is between 40 - 60 minutes, where as the same code by a colleague of mine using Python takes 10 mins. Is there anything I can do to improve performance? I am running a quad core, i7 laptop with a lot of RAM, so not a hamster in a wheel driving the code.

Thanks,

Paul

[udka]

If possible add vi or snippet.

-Unless what is inside each iteration of for loop it is difficult to give any suggestion.

-How big are arrays?

-What data type?

[Sam_Sharp]

Yeah, without posting your code it's impossible to see if you have missed something that might allow you to extract some extra performance.

[MancPablo]

Ok,

Attached is a PPT of the main for loop with the optimisation routine and the function it is calling. Hopefully it is clear.

Data types are mainly double precision. arrays vary between 1D, 5000 point and 80 x 5000 points.

Paul

[udka]

If you don't know how to create snippet and attach in forum : http://www.ni.com/tutorial/9330/en/

[Sam_Sharp]

1) What is happening in the unnamed/default icon SubVIs in the diagrams? If you had attached your code...rather than a powerpoint containing screenshots, we would be able to see.

2) I don't think that formula/script nodes are the most efficient way of doing maths - see if replacing them with LabVIEW primitives improves the performance (e.g. why are you multiplying 1e6 x 1e6 x 1e6...shouldn't that just be 1e18?)

3) Is your 'objective function VI' that you're passing into the constraint function set to reentrant? If not, it will serialise the calls so you won't be able to run truely in parallel

4) Try to move anything you can outside of the For loops (e.g. the FGV calls) (again, the FGV has to be non-reentrant so it will serialise these calls)

[Bob_Schor · Accepted solution]

I agree with saying that (a) you should post code (the simplest way is to attach your VI, not a picture, certainly not PowerPoint), and (b) you should not use Formula Nodes if you want speed.  I'll wager a dime this code will beat yours many times over.

[This, incidentally, is a LabVIEW Snippet, a .PNG "picture" that will turn into executable LabVIEW code, due to some NI "Magic", if you drop it in a LabVIEW 2016 (version shown in the upper-right corner) Block Diagram.]

Bob Schor

[MancPablo]

Hi All,

As you were busy typing I was busy recoding and seeing the effect of moving the FGV outside the for loops and feeding them as variant data to the model. Went from 40mins to ~40 second execution. I will look at changing the coding away from formula nodes.

1e6 x 1e6 x 1e6 - yes you are right. It is legacy on my part to ensure I am convert to the correct units.

VIs are set to non-reentrant at the moment. Will change that now. I'll try and remember to share code in future

Many thanks