parse xml performance

[Sapiophile]

Hi all,

The standard xml parse funciton from labview is incredible slow. to read an 800 kb file takes at least 5 min. I tried everything to speed it up, but no way. On one hand I cannot pre-initialize the arrays, so this makes it a bit slower, the library itself is super slow and the library does not allow multithreading, so multiple cores could process the information.

Can anyone help? Thank you.

[OEMa]

Hi Adrenalin,

could you be a bit more specific about what you're doing with which VIs, maybe by posting a VI and example XML File?

Anyway, it looks like the XML Performance issue is known. Here a Community page with faster solution.

Community: Parsing XML is too slow, or is it? - National Instruments
https://decibel.ni.com/content/docs/DOC-30377

And here a similar Forum thread.

Why does reading XML take so long? - Discussion Forums
http://forums.ni.com/t5/LabVIEW/Why-does-reading-XML-take-so-long/td-p/2056752

Best,

Marc

[JÞB]

We really do need to know more... Which functions?

The lower vi (open the vi heirarchy) depends on "DOMUserDefRef.DLL" The upper one does not but only works with the defined LabVIEW XML Schema

Niether gets around limitations of actually accessing data on disk!  Parallel disk access is not supported on most OS's (There really is only limited hardware!) and then the lazy OS's just jamb the data in the files wherever it seams convienient at the moment (Hence we have defragmentation utilities) finally there are often other services (anti-virus software...)  that slow up file IO.  

With all that said,  300Sec for 800kB seams incredably excessive!  I would really question the hardware first, then the AntiVirus or other IT "Junk" on board ,then a virus that has been missed by IT.  

[Sapiophile]

There is the code and a sample file. Backgroumd: I have a reader that creats xml files as a result of the measurement. I do parse that file, to structre the data in a way I can use them to calculate enzyme activities and store the results in the db.

I have to admit that the code is not nicely wirtten, I was simple exausted and no more patient at all as it didn't perfomed better after 6h of optimization.

Thank you all for your support!!!

[JÞB]

I absolutly hate when I have to say it but, You should have considered TDMS rather than XML.  From a brief glance TDMS would be the better file format for your needs.  And just how HUGE is your monitor?

[Sapiophile]

Jep, your right, a 27" mac display does not help u to programm nice codes in labview 😉

The thing is, that the xml file from the reader can't be changed. It's fix. Okay, I can also use xls files, but since Tecan is then using the activex interface and might produce alot of data over a long time, excel usually crashes and all the data from ur experiment are gone. xml is save, but labview proceeds it slooooooow ...

[tst]

I didn't look at your VI or your XML file too closely, and I have little experience with parsing XML, but my recommendation would be to read up a bit on XPath. The XML functions accept XPath syntax and can use it to parse the data out of the XML and it's much faster than iterating through the hierarchy.