Tree control: best (fast!) way to initialize?

[Christ0phe]

Hello Labviewers,

My application monitors up to 150.000 tags. I am giving the users the possibility to select a tag out of this list of 150.000 tags (in fact there are also several subgroups of tags to ease the selection process). To be as possible closest to a 'Windows-like' selection environment I have implemented a Tree Control that I feed with all my tags in an initializing stage of the application. Then the user just browse through the tree and double-click on the item he wants to select. The problem is that this init phase lasts for more than 20min (!) on a very decent and recent PC full of RAM,... The process of feeding the Tree Control itself seems very slow. I have attached the VI in this message, I have left default in the front panel (in this case less than 150.000 tags, but it is to show the structure and the selection process)

1.  Is the Tree Control not the good choice / structure? Are there alternatives?
2.  Is my programming the problem?

Thanks a lot for your help

Christophe

[Norbert_B]

Christophe,

first of all, you are talking about a huge amount of memory you have to allocate for handling the items. So it will take some time to do so.

20 minutes do sound too much in the first place, so you definetly want to dig in the real source for this amount of time.

That being said, your current code cumulates THREE different tasks into a single module:

1.) Loading files. Plural, as there are obviously several files read.

2.) Parsing the strings read from each file.

3.) Populating the tree regarding of the parsing result.

Please rework your example as such that you can benchmark each individual task.

Norbert

[Christ0phe]

Nobert,

You are right that I should have sequenced my code like you said BUT I already performed all the statistics to assess which part of the code takes time. The only part that is responsible for more than 99% of the total execution time is : Edit Tree Items.Add Multiple Items to End. I have reduced my VI focussing only on this aspect. In the Vi attached, if you change the N (number of items to add to the Tree Control) you will notice that above a certain threshold the 'time per item' starts to increase and after N>30.000 (on my PC) the PC even gets stalled and when it eventually comes to the end of the execution of the VI the 'time per item' has exploded. Best 'time per item' is 0.035ms for N=5.000.

I hope someone can help me, I think it is related to the node Edit Tree Items.Add Multiple Items to End and the way it adds, it is as if if uses a sort of build string command that worsens more and more with the size of the Tree Control.

I have also made another test: instead of trying to write say 100.000 items 'at once', I tried to split the writing into smaller bunches of 5.000 or 10.000 items but past the first iterations the Tree Control starts to fill up too much and the same problem as for the single bunch of 100.000 items occur (slow, stalled,...). Even if I defer Front Panel updates...

Thanks

Christophe

[tst]

Although my experience with it is limited, I believe that the LV tree control is known to perform badly with a large number of elements, as you've seen. I would suggest you do a search and you can probably find some threads on the topic.

Relevant suggestions:

1. Don't fill the entire tree in advance. Fill in the top level and use the tree events to fill additional items as the user selects them.
2. Don't use the LV tree control. You can probably use a .NET control which will perform better (although I would suggest testing).
3. Don't use a tree control at all, as I'm not sure that's very good for navigating such a large number of items. Instead, use a listbox with a filter box above - you type into the box and only the relevant items are shown in the listbox. Note that as far as I know, the listbox also has issues with a large number of rows, so you might want to limit that, but I have a vague recollection that this might have been fixed around 2012-3. You might want to have a filter even if you do keep the tree.
4. When performing multiple operations on the control, defer the panel updates. This shouldn't be relevant in your example, since you only perform a single action (although you could try it to see if it helps), but it's relevant in others.

[RWD]

Better late than never.  It's not you!  I have come to the conclusion that there is no (fast!) way to generate tree content programmatically.  I'm populating a tree from a DB (~178K items in tree).  It takes ~1:30 to populate array (or arrays, depending on method) of clusters from the DB to be used in the Add or Add Multiple and the best I can bring it in to build the tree is about 9 minutes.  This is unusable - booo.

[Ben]

---

@RWD wrote:

Better late than never.  It's not you!  I have come to the conclusion that there is no (fast!) way to generate tree content programmatically.  I'm populating a tree from a DB (~178K items in tree).  It takes ~1:30 to populate array (or arrays, depending on method) of clusters from the DB to be used in the Add or Add Multiple and the best I can bring it in to build the tree is about 9 minutes.  This is unusable - booo.

---

Not sieeing your code I can only ask about the one silly trick I have used to speed up population of the GUI...

1) Defer FP.updates

2) Hide the control

3)Populate it

4) SHow the control,

5) Undefer the FP-update

Ben

[Hooovahh]

Starting in 2013 the performance of the tree and MCLB were improved greatly.  That being said those tree controls sound a bit excessive.  I wonder if a better UX could be had with a combination of other control types.  Like are there a few common starting point?  Maybe having a drop down of the first level, then the tree populates starting with that indent would be better.  

[RWD]

Thanks for the thought but, I’ve been down that road and it didn’t help much.  Unfortunately I can’t put the code up for a look-see but I’m pretty sure at this point that the tree usage is just slow and alternative methods (list boxes, tabs, etc.) may have to be looked at.  It’s really too bad because our DB application is really nicely represented in a tree view.

The two best methods I came up with were:

• Pull the Parent items from the DB (in an outer loop, relatively small ~150, Add Item To End) and use a nested loop with successive DB queries building a complete Child array (with increasing levels of indentions associated with the Query and Add Multiple Items To End to load all the Children at a pop, potentially hundreds of children.  Update time is pretty variable based on number of children but pretty consistent E-to-E ~13 min.  This does improve with no updates and hiding to the tune of a minute or so.

• A single ‘Parent loop’ that basically builds the entire tree contents, using several successive DB queries again associated with levels of indention in the tree.  This actually builds the Array of clusters pretty quickly with over 178K items in ~1:30 which is workable.  Then pushing all this into the tree with a giant Add Multiple Items To End takes ~9:30.  This method is impacted minimally with no updates and hiding.

One good thing is that our DB isn’t updated too often (we can check for updates) so we may just add a ‘are you really really really sure you want to regenerate the tree and if so go get coffee’ check. ;-)