LabVIEW
Best and fast way to edit and save big arrays to file?
Hi Anna,
@labviewis3 wrote:
Is this fast and efficient or is there a better solution? I could also use the "index array" function and then "build array".
But I guess this will be slower?What do you think?
I'm happy about any advice
What have you tried? Have you tried several methods and did you compare them?
Why don't you attach your code?
My advice is to actually try all the different methods and see which one is faster. (If they're all too fast for it to be obvious which is the fastest, we can help you with code to help you determine which is fastest.)
I feel this is an exercise you must complete yourself (but of course we can help you arrive at an answer) because it sounds an awful lot like homework to me.
@labviewis3 wrote:
Hey guys
I do have 2d arrays with data that I'd like to save to file. It's supposed to be quick and efficient.
I have n columns and 100.000 rows in the array like that:
a b c d ... n
...
It always helps to be as specific as possible. What are a, b, c etc.? What is the datatype? How are they stored in the file?
Formatted and delimited numbers (integer or floating point)? Delimited string words? Data in a flat binary file? How long are the rows? Are the fields fixed or variable length?
If these are formatted numbers, a large expense would be to scan them into a numeric datatype and format the result afterwards. It is also potentially lossy, because you need to ensure that the output format matches the original data. It might be more efficient to operate exclusively on the strings of each line.
We can help significantly better if you would attach a short but representative data file. (maybe 100 rows) so we can see what you are dealing with. The best solution strongly depends on the details.
@labviewis3 wrote:
My approach is to use "delete from array function" and save to file afterwords.
Is this fast and efficient or is there a better solution? I could also use the "index array" function and then "build array".
Using "delete from array" or "index array/built array" both seem bad ideas. I would definitely process one row at a time using a FOR loop to keep processing steps on a small scale. Successive column deletions each required large memory reallocations because most data will shift in memory. It also needs to be done carefully because each deletion will change the indices of the higher columns.
