I need help w/ speed up parsing of spreadsheet string

[jon_mcbee]

Thats right, I removed the loop and replaced it with a Search and Replace String.vi, this created the problem of replacing each \s with a \t.  In order to remove the "duplicate" \t I tried using the Search and Replace Pattern.vi.  When I say "duplicate" i mean that I want each group of consecutive \t's to be replaced with just one \t.  The Search and Replace Pattern.vi was able to do this, but it gets the execution time back up to where it was with the initial vi with all the loops.  If you have any suggestions or tricks I would love to hear them.  Thank you for all of your help!

Cheers!

[Ray.R]

The code prior to the while loop, is it like the suggested solution?  I will look at it..

[Ray.R]

I don't see any \s (space) in the text following the Search and Replace in my example.   See attched image.

Did you use the one I loaded in my previous post?

It's supposed to remove white spaces..  or.. did you change that one to have tabs??  You may have to post your new code...

If you changed it from empty string to tabs.. okay.. humm..

Message Edited by JoeLabView on 09-14-2007 02:20 PM

[altenbach]

---

@jmcbee wrote:
 When I say "duplicate" i mean that I want each group of consecutive \t's to be replaced with just one \t. 

---

Easiest would be to use "scan strings for tokens". It automatically contracts consecutive delimiters into one. Here's a quick example. Maybe you can adapt it to your exact requirements. In this case it also stepts through the entire string only once, while all these consecutive "replace all" operations need to start from the beginning.

In this case it gets each line and then separates all elements with tabs. In this example case, it elmininates any extra \s, \t, comma, hyphen, |.

Modify as needed. I don't really know what you need exactly.

Message Edited by altenbach on 09-14-2007 11:31 AM

[altenbach]

Maybe we can all take a step back and ask what you actually want to do with all this. For example, the body of your table is all "fixed with" fields, so a properly designed "scan from string" on each line might do the job also very efficiently. Maybe you don't need the entire huge cleaned-up string in memory.

What is the final purpose! Do you e.g. need certain data columns as 1D arrays, for example?

[altenbach]

---

@altenbach wrote:

---

Sorry, there is a small bug! Please remove the two "=0" wired to [i] (inner and outer loop)!

Since delimiters are diagram constants, we probably don't need those anyway. In any case, even if we wire the "cached?" input, it should be a "!=0" (not equal zero). Check the online help.

[Ray.R]

I agree with Altenbach.

I created an example code, but unless we know what you want to accomplish in the end, I might be sending you in the wrong direction..

Thanks Christian 🙂

[jon_mcbee]

I apologize for not getting back sooner.  Attached is the new code I am using.  In the end I do only need to parse out three of the columns as 1D arrays, with each 1D array corresponding to a run number (if you look at the log file the run number is the first column).  The log file updates with a new row about once a minute, I need to be able to look at the file after it updates and pull out the last run number (sometimes there will be three run numbers intervowen e.g. 121A 122A 123A 121A 122A 123A... but in a column) and if that is the case I need to be able to pull out those three run numbers.  I use those "Last Run Numbers" to parse the array and pull out the 3 1D columns for the appropriate runs.  I then turn these columns into numerical arrays and plot them as xy-graphs.  The whole thing is working but a bit slow on the initial parsing.  The easiest solution seems to be speeding up the Search and Replace Pattern.vi, as changing anything else would mean a redo of parsing for the xy-graph.  I hope this helps, feel free to follow up with questions.

Cheers!

[jon_mcbee]

JoeLabView,

To answer your post, I wanted to parse out hyphens and not spaces in the original example, I may have included the wrong subvi sorry if that is the case.  When I remove all spaces there is no longer a delimiter between columns so I cant parse the data.  I hope that clarifies.

Cheers!

[Matthew_Kelton]

One problem you are going to have is that you have spaces in your data in the example file you posted.  You have a trial "16 L" and entire Source ID column has spaces ("AB - MGK-1").  If you are trying to write a generic fixed width spreadsheet reader, you are going to have to get a bit more robust.  Even Excel has issues importing fixed width column text files on occasion.

So, your code is going to have "16\tL" and "AB\t-\tMGK-1" whcih you will interpret incorrectly when you go to convert it to an array.

If you are trying to write an interpreter for this specific type of file, then you should look into using scan from string in the string pallete.  Depending on what you want for output from your VI, you can build all the 1D arrays in the correct data type (i.e. DBL or string).  If you KNOW that your data won't have a string in it, you can use "Scan string of characters in set" with a space for the character string, and then use a "Scan string for characters not in set" and again use a space for the character string.  Throw away the first string from the scan and the second will have your data.

If your data will have spaces, then use the "Scan string (abc)" and specify the column width.  Then use Trim whitespace to remove the beginning and trailing spaces.  Then you can have your data with spaces in it.

If you scan all of your data as strings, you can turn it into a 2D array of strings to pass out and your code will be a drop in replacement for the Read in Spreadsheet File VI, but only supporting a 2D-array of strings.  The user can then convert the strings to whatever data type it should be.

This method won't work for a file with a different layout since the column widths will vary.