Read file in chunks

[GerdW]

Hi gigi,

do you read the context help of the used functions at any time? For ReadFile it says:

---

If line mode is TRUE, the function reads until it encounters an end of line marker or end of file, or until it reads count characters, if count is wired and greater than 0.

---

So when you wire "10" at count input it will read just 10 bytes. To read a full line from the file you have to wire a sufficient big number (as said before), atleast it should exceed the expected number of bytes per line...

---

EOF function give me the size of the file and not the offset of the EOF?

---

Reading the help is useful here too as it says:

---

offset indicates the location of the EOF relative to the beginning of the file.

---

Think about that and you will agree that "position of EOF relative to beginning of the file" is the same as the filesize...

[pincpanter]

Hi Gigi:

- don't wire the count input: it will read one complete line

- terminate the loop only when Read returns an error (usually it should be an EOF)

- process data when

i+1 mod 1000 = 0    (i.e. you accumulated 1000 more lines) 

OR   

EOF AND String<>Null  (this is necessary for the last file chunk, where probably the number of lines is not 1000)

P.S. in this pseudocode <> means "different than"

[gigi85]

well, changin' a function - read lines from file - i have made my VI easier 🙂

now i have to manipulate my datas in the while loop directly, right?

if i have to make my chekings and handlings each time i have data from my output i have this problem 🙂

i have to check if the start word is present (it's present 1 or 2 times)...and i have to analyze only data's after the second start (if present) or after the first one (if in my file is present only 1 start)

to solve this problem i'm going to save the offset...and i would read the file again only from this offset..is a correct way to do it?

in my solution i have the problem that i'm going to read my file two times 😞 i don't thinks is very efficient

[GerdW]

Hi gigi,

did you read the context help for "Read Lines From File"? You should have noticed that it provides an "EOF?" output...

You can stop the while loop using that output. No need determine filesize and to compare with current read position. Well, it has been mentioned before (by pincpanter just in the message before yours)...

[pincpanter]

Once again: don't use EOF offset to terminate the loop! This number is the file size in bytes! What's the point in reading 5000 lines millions of times? Almost all of those readings will be beyond the file end!

The Read Lines vi has an EOF? output: use this one.
About the "Start" word: I don't know what's the structure of this file because you never explained it in detail - you should have described it just from your first post - so it' not obvious how to proceed.

However, definitely you don't need to read the file twice. First,  I think you should keep treating data line-by-line (use Pick Line to get individual lines from your multiline string).

When I need to locate data into specific sections of a file, I use a state machine. There are plenty of documentation about state machines in LabVIEW.

Basically, you should maintain a state information (typically a number or - better - an enum in a shift register). Depending on the state, you perform different operations, that will be contained into different cases of a Case structure When you're in the first state (say state 0), you search a line containing "Start" (or beginning with "Start" or whatever you need), then you jump to the next state (say state 1) where you begin processing data according to your needs. When finished, you jump to a third state (state 2) or even back to state 0 if you need to search another "Start", and so on.

[gigi85]

hello...

sad to tell but i can't read in chunks...sorry to have waste your time 😞

for big file they want me to read just a part of the file...so before reading i check the file size, if it's bigger than a particular size i have to choose which part i want to read...

so i want that a window appear...where he advise me that the file is too big and let me to choose the part of the file i want to read (using property node)

i have done this - look the attachment

but it doesn't work...the window doesn't appear and it choose always 0 - offset 😞

why?

another little question...now when i press esc button to my mainVI it stop running but also it close the VI...what do i change for mistake? 🙂 i would terminate that the VI run only

thanks again

[GerdW]

Hi gigi,

did you test your VI using debugging tools like highlighting?

The VI shown in the picture just runs once without waiting for any user input. When you want to VI to wait for some user action you should also program it that way...

[gigi85]

The VI shown in the picture just runs once without waiting for any user input

i know 😞

When you want to VI to wait for some user action you should also program it that way...

i know too...i ask you to all you just to learn 🙂 i don't know in which way can i program in this way 😞

[GerdW]

Hi gigi,

how would you program a wait sequence in any other programming language?

Maybe some pseudocode will help:

REPEAT
  wait(100ms)
UNTIL new UserAction? (like: OK-button pressed?)

 Either use a while loop or an event structure.

When you don't know how to program something in LabVIEW you can always search the example finder for suitable example VIs...

[Yamaeda]

I still think you're staring yourself blind at the chunk reading issue, what is it you're really trying to do, or rather what is this sub-vi supposed to achieve?

/Y