Parsing data and averaging unique X-inputs to their multiple Y-outputs

[jvaldez12]

My system is currently actively collecting data from two sensors through Arduino and displaying it on a graph. However, the problem I am facing is that the data is being read and duplicating because of how fast I have the sampling rate at but I can't change that due to some restrictions in my coding. I was wondering if there was a way to use the text file that's created (I included a test file: S001 as an example) and parse through the data to average all the Y values that correspond to the one X value.

I have the steps written out as to how I think it can be accomplished but I don't know which functions to use: Read text file, parse the data with pattern match or scan from string, group the y values corresponding to the x values with an array or use a loop to find all the x and y values, find the mean of the y values, and finally reprint all of the data to a new text file. 

If anyone could help that would be great. Thank you.

[altenbach]

Well, the correct way would probably be to average the incoming data as long as x is the same, but here's a quick draft how your file could be processed. (Please verify correct operation!)

There are quite a few odd constructs, for example it does not make sense to append an ever-growing string and rewrite the entire file from scratch. It would be more reasonable to open the file before the loop, then just append the new lines and close the file when done. Also not that you don't need to use build array before converting to dynamic data. You just need to configure the conversion node to accept a scalar.

[jvaldez12]

If it’s possible to average the data actively while the VI is running, then yes that is ideal. However, because the test will start at angle 0 up to a max of 180, at the rate the data is collecting, there is bound to be more than y value corresponding to a single X which I believe you addressed with the average adjacent.

A question I have with the average adjacent. If I wanted to repress the “All data” icon you have on the left to swap between different files would I just be adding a path to it to upload a different text file?

Thank you for being so helpful!   

[altenbach]

---

@jvaldez12 wrote:

A question I have with the average adjacent. If I wanted to repress the “All data” icon you have on the left to swap between different files would I just be adding a path to it to upload a different text file?

---

Yes, Just use "read delimited spreadsheet" to read the data and wired its output to the 2D array connected to "all data". At this time, the wired will break, but just right-click the "all data" control and change it to an indicator and everything will be fine.

You can use exactly my code skeleton to average data as it arrives and only save when X changes.

[jvaldez12]

I did what you said and added the Read Delimited Spreadsheet. I tried to have it read the "test" txt document but the data isn't plotting anymore. Did I add the Read Delimited Spreadsheet.vi wrong? Or does it need to be an active stream of data? Thanks.

[altenbach]

Maybe try to autoscale the xy graph axes 😄

(I get the feeling that some of the x-values should be negative, but that's just me)

[altenbach]

---

@altenbach wrote:

Maybe try to autoscale the xy graph axes 😄

(I get the feeling that some of the x-values should be negative, but that's just me)

---

Here's how the averaged trace would look like if we swap the sign of x whenever x=0. Seems more reasonable.

(Of course you need to adjust your experiment so the data is correct, this is just a very fragile Band-Aid to illustrate the idea)