Array to typedef cluster (is too slow)

[EMCCi]

Hello,

I have an array of interleaved data coming from a DMA fifo in a compact rio. This data has 27 variables, and 100 values for each variable.

I want to put this data in a type def cluster of arrays, 27 arrays (1 array for each variable) of length=100 each one. I have developed the following code (based on this) but is too slow:

It takes almost 4 seconds to execute, and I need to execute this code every second. Here you have the detailed timming in nanoseconds:

As you can see, is the loop that is setting the cluster values which is taking too much time to execute. I don't know why is taking that long. The most strange part is that the cpu is far away from 100% during execution. It is between 10/20% (NI MAX values), so I don't know what is happening here.

Someone knows what is failling here? Or how to do it faster?

Thanks and regards,

EMCCi

[bseguin]

because you use reference to the object and you update it every iteration of the for loop.

Change your strategy. convert to data and then update once your indicator.

Benoit

[mikeporter]

Your code is doing several things that are slow, but the killer seems to be where you update all this data by reference. It's taking 3.5 seconds all on its own.

Also, it looks like the other loop is turning a 1D array into a 2D array. The way you are doing it is also very wasteful as there is a built-in function (Reshape Array) that does that very thing.

Mike...

PS: Don't use express VIs.

[wiebe@CARYA]

Notice that the transpose only sets a flag.

That doesn't mean the orientation of the 2D array doesn't make a difference, it does. But you won't notice until you do something with the array...

For instance, getting the first row (or column, I never remember) from a 2D array means copying a contiguous block of data to another block of memory. Once transposed, it means copying a value, skip the stride, copy the next value, skip the stride, etc.. Not sure how it's implemented exactly, it might be just the cache misses on the memory, but it makes a difference.

So you might be better of by permanently transposing the input array, and\or inserting the right way just to avoid the transpose (which doesn't seem to cost anything) just to speed up the auto indexing in the last loop.

Just saying, it's a factor. Probably not significant compared to the value property in the loop.

[EMCCi]

Ok, so reading what you have answered me, it seems that the big part of the waste is in updating the data by reference in a loop. I didn't know that write values by reference was slow... I'm doing it by first time.

Then, as Benoit said, I should first build the cluster with the "analog data", and then write this cluster once in the typdef cluster by reference? Like this?

But the problems is how to build this cluster from the 2D array in an easy way?

[paul.r]

Use 'Set Control Values by Index' instead of the value property node. Its significantly faster. 

As far as building up the cluster, I am sure you could figure out some programmatic way to do it, but the fastest way is probably to just create one big ugly VI where you index each element of the array and bundle it into the cluster.  

[arteitle]

One side note, are those timing measurements really in nanoseconds? Because if they are, then it isn't taking four seconds to execute, it's taking four milliseconds (3.63 x 10^6 nanoseconds).

[EMCCi]

Okey, I have created the "big ugly" VI with 27 index array, 27 wires and a bundle by name of 27 elements... and it works. But this is just what I was trying to avoid. Today there are 27, but what happens when there are 270? It's not easily scalable.

I have another case were I'm having the same problem, but the input is not a floating point array from a DMA, but a string where inside there is different data type: Numeric, floating point, boolean, string variables... So I can't create an array and then index and bundle it. I should do an array for each data type... Wich is not praticall at all. Now I'm doing it like this, but takes 15 seconds to execute (for a cluster of 57 elements):

How can be this done programatically? It was my first time working with references, but they do not seem to be an option due to the high time cost.

arteitle: No, sorry. They are in microseconds.

Best regards and thanks for your responses,

EMCCi

[paul.r]

Why would you have a cluster of 270 elements? Why do you have a cluster of 57 elements?

As I mentioned before, using the control value property node is extremely slow. How is the string array mapped? 

[mcduff]

Do you need to use references?

Can you use something like this instead?

Cannot test it, since you did not include any vi or cluster.

mcduff