Problem reading from large .hpf file

[jcannon]

Hi everyone,

I am having some problems reading/displaying data from a large .hpf file. The file has about 16 channels of data with each channel sampled at 2kHz for around 40mins; so total file size is about 450Mb.

I am using the open data storage vi with the .hpf dataplugin (smaller files of the same type as the above open ok with this configuration). Obviously, file will not open using normal means ("Not enough memory" error)... I know I need to break the file up into "chunks" to be able to read it, but not sure what the best way of doing this it (I have read some white papers etc on using the get properties function but do not understand them)... At the end of the day, I would like to display each channel and performance various analyses (filters, rms, threshold detection, among other functions)... I am happ do to this to each channel separately, but sure how to acess the data.

Attached it what I have done... The timer function is used to determine how long the data takes to load.... Any suggestions would be greatly apprectiated.

Regards,

Jack

[Basil-B]

Hi Jack,

Sounds like you're on the right track, but I'm not clear on something. Does the setup you have right now work but it's really slow, or does it not work at all?  Are you getting any errors?

Property nodes allow you to manipulate properties of an object (each object has a set of properties which can be acted upon) programmatically within LabVIEW. I'm not sure how familiar you are with them but an HPF object will have a specific set of properties which can be programmatically modified and manipulated. If you right click and create a property or method you might find what you're looking for under there.

Please let me know what happens.

[jcannon]

Hi basil,

Thanks for the reply...

My current work works for smaller files of the same format/configuration... They open virtually instantaneously... With the large files, the vi runs but does nothing for about 15-20 mins or so then I get an error: "Not enough memory" and the vi stops... This happens with and without the chart..

Thus, I would like to try and load the file in sub-sections; such as try and import only channel 1; if that does not work try and import the first x number of data points from channel one, etc...

Any ideas on how this can be done?

Regards,

Jack

[Mark_Yedinak]

Have you looked at the VIs in the Advanced Storage pallete. It looks like you can use those to select specific file from the storage file.

[altenbach]

Is there a reason you are opening and reading the same file every 100ms, but never close it? You are probably generating a new reference every time.

the HPF plugin is in VBScript, so I am not sure how memory efficient that is.

Then you are seing 450MB woth of data to a waveform graph. That does not sound very reasonable. I doubt your screen even has 450M pixels. :o. Have a look at this note.

[jcannon]

Hi Altenbach,

I have removed the while loop, added a Close.vi, and removed the chart.... Error still occurs after about 6 mins of loading (code below):

I tried an NI example that ships with LabVIEW with slightly better success (Read Channel Groups.vi); it uses a series of nested for loops to get the data in chunks (I think?). It would load channel properties etc, but only after I removed the chart (see code below)...

Also read the link you provided, but I think only the need to decimate data for display was apropriate in my case; althugh I think the above code reduces memory usage as smaller chunks are obtained...

Regards,

Jack