how to create a large group of arrays all different lengths

[MechaCool]

Hello All,

I want create a 2D array that has two coloums and an X number of rows (possibly in the hundreds). The first coloum should have a calculated average corresponding to that row. The second coloum should have a standard deviation assosiated with average. I have trouble getting to this point. I know how to aqquire the set of values that should be considered for each average and std and I also know how to calculate the average and std. 

A loop will tell me what value to tag on and what row within X to put it in. I want to be able to have the X number of arrays each representing a row with the values before i calculate the avg and std. Each array would be of a different length.

I dont know how to tag on the single values from the loop to the approriate row in order to create the group of X arrays.

The only way i can think of doing this is initializing an exessvly large 2D array and then replacing the first avaliable zero in the row with the value form the loop. Then I just wouldnt consider the zeros when I am making the final 2D array. What i do not like about this is that it would be exesive and there is the possibility that there wouldnt be enough zeros for all the values i wanted consider in a specific row.

I dont wanna calculate the mean and std for each row every time I add a value from the loop either. That would allow me to skip the groups of arrays but would also slow down the program. Can anyone give me some advice on how to execute this? Thank you in advance!

MechaCool

[rpursley8 · Accepted solution]

Use a cluster.  In your case, the cluster will contain an array of data and two numeric inputs for the average and std dev.  Look at the attached.  I created a Type Def control of the data type I think you want.  In the vi, you can see how to use the cluster to write your data.  For several data sets, you would have an array of this cluster.  Each element could have an array of different lengths and its associated mean and std dev.

[Darin.K]

A little bit slower on the coding, but my idea would be the same, use a cluster with a 2D data array and a 1D array to keep track of length.  In this example I show a few other tips I find useful:  I initialize the data arrays with NaN before filling and use a shift register.  As noted many times on the forum, filling arrays in loops, especially when clusters are involved, is fraught with memory usage peril.  Instead, I replace elements in the data array.  Just choose the initial size to comfortably fit your data, otherwise this simple vi will fail.  The 'Go Slow' option lets you watch the arrays fill.  I update only the necessary row in the mean/std. dev array as well.  One final note, in a real application I would probably not put the indicators inside the array.

[altenbach]

OK, a 2D array with 2 columns and a few hundred rows is tiny and not even worth worrying about in terms of performance.

Why do you think you have a performance problem? Are you sure the problem is with the 2D array?

The primary problem is to accululate the raw data used to calculate the mean and SD. You can do this with nearly no memory if you do a ptbypt technique, e.g. as shown here.

It would really help if you could attach a simple example of what you are doing. A simple VI is worth a few paragraphs of descriptions. Most of your post is not very clear at all and a VI would certainly help. Where is the data coming from?

[MechaCool]

Thanks everyone!

I did not know very much about this cluster modifying concept before. I appreciate your help as it has been very enlightening. What I have decided to do is replace arrays within clusters every time I get a new value for it, that way I wouldn't have to worry about initializing a big enough array to begin with. I can also apply the average and std vi after I have aquired all the arrays. Since this whole iteration would happen many times I wanted it to be as simple as possible. Thanks for all your help!!

-MechaCool

[altenbach]

---

MechaCool wrote:

I did not know very much about this cluster modifying concept before. I appreciate your help as it has been very enlightening. What I have decided to do is replace arrays within clusters every time I get a new value for it, that way I wouldn't have to worry about initializing a big enough array to begin with. I can also apply the average and std vi after I have aquired all the arrays. Since this whole iteration would happen many times I wanted it to be as simple as possible.

---

This is probably the worst case in terms of memory thrashing. Constantly resizing arrays inside complicated data structures is extremely inefficient and I wouldn't be surprised if you are a few orders of magnitude slower than necessary. Well, if your computer can keep up and nothing else is starved for CPU it does not really matter until you suddenly need to work with much larger data structures in the future.

As I said already, you can do a point-by-point algorithm that does not need the save the large arrays at all. That would be much more efficient.

[pri2686]

hi...

i am having an issue.... i need to average elements... the process should go like this....

first--average of a[1] 

next step is average of a[1] and a[2]

next: average of   a[1],a[2],a[3]

next: average a[1],a[2],a[3],a[4] and so on...

and finally i need to have all these averages in an array....

can anyone help me?? i am new to labview....

Thanks

[RavensFan]

You should have created a new thread ofr this instead of hijacking someone else's thread.

By a[1], I am going to assume you mean the first element of array a, which is actually index 0.

Here is one way to do it.

Message Edited by Ravens Fan on 08-26-2009 10:20 PM

[Darin.K]

You could do this.

Oops, too slow with the pictures...

Message Edited by Darin.K on 08-26-2009 09:29 PM

[pri2686]

thnks a lot...

it works...

i having a trouble with arrays.. here is my problem....i am acqurind data from agilent 34970a unit and the readings in the 1d array are being replaced with the new ones.... i mean in a[0]... they are not getting stored in the next index....

wat shld i do?

im attaching the vi here.... the 1D array here is the DC current readings...