Case insensitive searching in maps

[Andi_S]

I really like this idea: https://forums.ni.com/t5/LabVIEW-Idea-Exchange/Regular-Expression-search-in-Maps/idi-p/3934993

But as so often there are only a few Kudos - probably because it's a very special feature not needed by many programmers.

I'd like to suggest also to add a function (or option) for a case insensitive search for string keys as first and easy to use implementation. As second I also hope for the regular expression search.

[AristosQueue (NI)]

This is either an impossible request or a bad idea or a serious language extension, depending upon how you look at maps.

TL;DR: See last paragraph for workaround/recommendation since I don't think we'll ever tackle anything like this. Read the intermediate paragraphs for details on why I think it is unlikely.

In general, if you need any kind of searching in a map, you should either not be using a map or you are fine with the linear search through the map (which you do today). You should be using arrays where keys can be individually inspected. Maps have their tremendous lookup speeds because the keys are sorted in memory in various patterns to allow lookup based on how they were originally stored. We don't need to add any ability to search through the map because that already exists in the autoindexing tunnel -- you just need to supply the search criteria, which will vary at every use case (this does form a use case for closures, but that's a whole different feature).

In order to have a map that supports case insensitivity, we would need a brand new data type -- a case insensitive map wherein the keys would care about case sensitivity when they were inserted in the first place. Such a map would treat "a" and "A" as the same key. So we would be looking at some sort of configuration option (not selectable at run time) on the Insert Into Map node and the Build Map node. Such maps would not connect to terminals of today's maps because the internal sort order matters to the actual type of the data, just as you cannot wire a path to a string without a conversion step... they both look like textual data from outside, but they are fundamentally different constructions in memory, and VIs compiled to process one cannot process the other... same would be true of case-sensitive vs case-insensitive maps.

There is a language model that would allow for such type extensions. In LabVIEW parlance, these would be malleable classes where the comparison function used to define the map would be supplied as part of declaring the data type. You would need to attach the static VI reference directly to every block diagram constant and front panel control of the map -- so you would pretty much definitely want to do it only once in a typedef. It gets weird and complicated, but it is feasible. I have doubts that LabVIEW itself will ever go that direction, but I've mapped out what the language terrain would look like just in case someone builds a business case for going there.

Essentially, what you are asking for undermines the advantage of maps as they stand today, and there is little to no gain to you as a developer for NI to supply such operations... they would have very limited scope and be no different than you writing the VIs yourself with the tools you have today, and, arguably (see opening paragraph) if you find yourself writing those VIs, maybe you've chosen the wrong data structure.

TL;DR: So, if you're really doing arbitrary searches through data, use an array instead of a map... arrays are better optimized for linear processing. Or, if you really need a case-insensitive lookup table, you do it by always calling To Lower Case before you insert into/build the map and then by calling To Lower Case when you look in the map later. Both of those are significantly better solutions than NI providing direct VIM support for a case-insensitive map, in my opinion.

[crossrulz]

I use the To Upper Case and a Trim Whitespace before my map insertion and lookups.  I really so no difference between the To Upper Case vs the To Lower Case.  I throw in the Trim Whitespace as well because I have been burnt too many times by non-visible characters (spaces, tabs, end of line characters).

[AristosQueue (NI)]

> I really so no difference between the To Upper Case vs the To Lower Case.

There isn't a difference. I just picked one in my earlier post.

[AristosQueue (NI)]

> I throw in the Trim Whitespace

I hate to see this encouraged to the community. Salting Trim Whitespace on map access is paranoia masquerading as a fix. In cases where the strings are coming from user input or parsing, it might be needed, but it should be part of the parser, not part of the key handling at the point of the map operations. Fixing the functionality of upstream code to meet preconditions is better than wasting performance forcing assumptions downstream to be true, especially when those are already guaranteed true by common upstream operations (e.g., converting enums to strings never leaves extra whitespace at the end).

[Andi_S]

---

01-13-2022 08:28 AM

I use the To Upper Case and a Trim Whitespace before my map insertion and lookups.  I really so no difference between the To Upper Case vs the To Lower Case.  I throw in the Trim Whitespace as well because I have been burnt too many times by non-visible characters (spaces, tabs, end of line characters).

---

Yes, in most situation this a proper solution but there are some situations where it's not so nice.

In my current case I have a function that loads values from a file. Each parameter has a string identifier and a value which is a string, too. The order of the values is unknown and unimportant. My plan was to put everything into a map.

The user shall have the possibility to request a value by giving a key. However, it should not be necessary to know the upper and lower case.

By adding an element into the map, an existing element should be replaced even if the key is different in terms of upper and lower case.

Finally I want to write back the map into a file where the upper and lower case should be used as originally read respectively set be the user.

@AristosQueue:

Yes, of course I can do everything with arrays.

[Andi_S]

I use the To Upper Case and a Trim Whitespace before my map insertion and lookups.  I really so no difference between the To Upper Case vs the To Lower Case.  I throw in the Trim Whitespace as well because I have been burnt too many times by non-visible characters (spaces, tabs, end of line characters).

[AristosQueue (NI)]

Andi: so use the To X Case nodes when inserting or looking. This covers all your use cases and is, in my opinion, a far better solution than NI creating an entirely new data type system and having to educate users about why sometimes a "map from string to path" wire breaks when connected to a terminal also of "map from string to path".

[Andi_S]

ok - I understand that NI will not implement this if the effort is that high.

But I still understand how you want to solve my needs by using a map ...

Lets assume I have just two keys and values in my file:

"First" -> "Hello"

"Second" -> "World"

Solutions:

a) I read the data from the file and put it into the map as it is.

If the user adds "first" -> "Hi" I would get a third element.

if the user requests "second" he would get an element not found error.

So a) this does not work.

b) I read the data from the file, convert it to lower case and put it to the map

Now all user operation (add / get) would work but I lost the case information in my map.

When I write back the data into a file I would see it:

"first" -> "Hello"

"second" -> "World"

The only idea that came up while writing is to use a cluster as data, containing the original key string with upper and lower case and using the same key string but only lower case as map element key.

Before adding a new element I have to put the user-given key into the cluster together with the real value string and convert it to lower case to use it as map key.

When I export everything into a file I can throw away the key string and just use the one out of the cluster.

.... ok, this could work ...

[AristosQueue (NI)]

You would never put "First" into your map. Add a "To X Case" node before the Insert Into Map or Build Map (whichever one you are using to create your map). You would either have "FIRST" or "first" be the map key. Then when you do a lookup, use the same "To X Case" node on the lookup key. If you need to preserve the case information for some other use, you need to store that elsewhere, as you noted.

[AristosQueue (NI)]

Essentially, in all maps, the key needs to be the key, whether that is "To Lower Case" or a hash of the string or whatever it is that makes all your disparate values identical from your lookup table point of view. The specific value that you want to use to represent that key becomes part of the value data if the raw key isn't enough. Does that make sense? 

[Christina_R]

See AristosQueue response for explanation of why this has been declined. If you need case-insensitive keys, then you should convert strings to all lowercase (or all uppercase, if you prefer) before using them as keys for any map operation.