Saving large amounts of data in a single file, and as it is collected

[CatDoe]

If the file IO primitives are used, how are they affected when the program crashes? In the past, I had file that opened, wrote, but never closed so they never saved anywhere, and the data was lost in the void. 

In this program's case, I need to write after each message queue completes. This is not when all the data for all the runs is taken. Bob_Schor's example is a much better version of saving the file at the end of all of the data runs than mine. I can easily modify this example to accommodate a single line of data, but that brings me back to the speed question.

[Bob_Schor]

@CatDoe -- I just sent you a Private Message in the Forums with a question -- it should show up as a red Exclamation Point by your name when you are logged into the Forums.

Bob Schor

[CatDoe]

As a follow up to anyone else needing a solution. The best solution @Bob_Schor and I discussed was parallel loops. I had an event loop with a single message handling loop. I will add a second message handling loop with an independent queue to save data. The data saving message handling loop will save data in parallel with the original (now "data acquiring") message loop. The two messaging loops need independent message queues to avoid message queue conflicts (one loop will dequeue a message so the other never receives it etc...).

The only trick I am now working on is how to end both message handling loops using a single user event. I think I have a solution but need to work out a few kinks.

[Hooovahh]

---

@CatDoe wrote:

The only trick I am now working on is how to end both message handling loops using a single user event. I think I have a solution but need to work out a few kinks.

---

This shouldn't be a problem at all.  User Events are 1:N, where you can generate an event, and have N loops all react to it.  It is common for a publisher/subscriber design because of it.  Just have each loop register to the same User Event, then have them each unregister when closing, then have the User Event itself get destroyed after the loops are done running.  This is unlike Queues which are 1:1, or N:1.  In those cases whoever dequeues the message first gets it, and all other loops won't know it was ever there.