Properly Using Configuration Information/Files

[Nathan_S]

I've read through the previous posts on the forum and have scoured the internet looking for information, but I still don't feel that I know how to properly use configuration information for my program.

Basically, this program is a large scale application that requires a ton of configuration information (as opposed to just specifying constants).  The reason I am doing this is because the scientists we are working for want to be able to adjust almost any parameter at any time.  Right now, I am using a standard .ini file (which is getting unwieldy).  There are 7 main segments for my program (Output HW, Input HW, Network Comms, Calcs, etc.) that need to have configuration information stored.  Currently there are 178 different "sections" with up to 7 "keys" per section.

I have thought about...

1.  Using a single configuration file.  This is getting unwieldy, and being able to change any parameter at any time can lead to a ton of old config files (each old file is time/date stamped and saved so that all old config's can be available).  Creating a parser for this is a bear.

2.  Using multiple configuration files.  Seems better, but this leads to 7 config files and 7 config file parsers to maintain.

3.  Go with an XML format config file.  I'm not too familiar with this, but it seems more scalable than .ini.

4.  Using an Excel file with columns and rows that are parsed for config information.  Doesn't seem to be the way to go... (as there are many different sections and each section has different types of keys)

I am just confused now on the way to approach this problem.  Any help or direction would be appreciated.

[tst]

You can use something like the OpenG variant config VIs or MGI's read-write anything VIs, which will do all the parsing for you and convert a cluster to a human-readable file and back.

NI also has the xCE component, but I never used it - http://zone.ni.com/devzone/cda/epd/p/id/6249

[Nathan_S]

I looked at the OpenG toolkit for variant config files and was hoping to use that, but the other developer I'm working with already has the format established for the config file and the toolkit won't easily port 😞  Great idea though.

I'm working through the weeds even as I type...

[tst]

The MGI VIs use built-in VIs and *might* be easier to convert, although there's already an established format, it's quite possible that porting won't be easy in any case.

[Nathan_S]

Thanks for your help, tst.  I've been looking into these solutions. 

General question:  Is it better to have one config file with a ton of sections (which aren't related or sharing similar keys) or to have, say, 5 config files with more cohesive information?  I'm leaning towards having more files but I'm worried that this might lead to a more complex piece of software.  Any hints or tips would be appreciated.

[tst]

---

@Nathan_S wrote:

General question:  Is it better to have one config file with a ton of sections (which aren't related or sharing similar keys) or to have, say, 5 config files with more cohesive information?

---

I would say it highly depends on your situation. If the sections are completely modular, then it would seem to make sense to have separate files (although you could also have a single file with separate sections). In the end, I don't think it would matter that much either way, so long as you don't try to play with the same file from more than one place at once, so you should do what feels right to you.

[pjr1121]

"being able to change any parameter at any time can lead to a ton of old config files"

This statement is a little confusing to me.  I certainly hope you aren't creating a brand new config file everytime a control is changed on the front panel.  You should have a set of default .ini files (number is up to you) that are read at start-up and written to at close (store most recently used values if you want to).  hide these files somewhere to avoid accidental tampering.  I would then give the user Save/Load Config options.  This will enable a particular user to save the configuration they use to their file locations and load them when they need to.  

When I'm dealing with a large number of controls that need to be initialized at start up, i use "flatten to xml" and "write to xml" (save) and "read from xml" and "unflatten XML" (load).  It's not the same as the .ini files and it is more difficult to read the actual file, but it will accomplish the goal of initializing parameters.  The primary difference here is that you write everything and read everything (no key selection).  Depending on what you want to accomplish, this may be a more managable way to deal with your large number of controls.

[JÞB]

This is one of those rare times I would use menu options.  Save config / Load config are reasonable in these cases and your scientists can handle file naming any way they want to (Sometimes they don't really think the same way "Normal" people do)

Structure of those files can get problematic.  If there are clear boundries where groups of data clearly belong together and have a clear scope branch them into their own file to improve the readability of the file and make that section of code modular.  The overall app that pieces the modules together only needs a config file that knows what the sub- config files are then.  Yes, a config file containing key value pairs of config files!  It does take a small amount of extra time to launch that way but .... well, Launch MAX, users tolerate a bit of launch time when apps open thanks to Microsoft.

[Wayne.C]

Some thoughts on saving config settings.

1. Configuration data stored in database tables.  Table for each major section.
2.  XML and the OpenG Variant Config VI's work very well for .ini files.  Makes it very easy to add or delete entries from config file at later date.  An important consideration.
3. Don't let the end user determine how the config file is formatted.  Let the code decide this.
4. Avoid the user directly editing the .ini file.  Allow the user to make config changes from within the program and allow the program to save the changes.
5. If you use config files, create a functional global to handle each file. 

[Nathan_S]

---

@pjr1121 wrote:

"being able to change any parameter at any time can lead to a ton of old config files"

This statement is a little confusing to me.  I certainly hope you aren't creating a brand new config file everytime a control is changed on the front panel.  

---

Seems I didn't explain myself well.  Every time any one of the configuration options changes, we have to create a new file and move the old file (with time/date stamp) to an "Old Config Files" folder.  That way, in case they need to do any post-processing of the data, they can use the configuration settings that occurred when the specific time-stamped data file was recorded.

The reason why there is so many is because we have a ton of channels to record data from (including network and calculations) which brings us to about 178 different configuration options to be addressed, each option with several keys of its own.