Rif.: network access to INI files

[ElectroLund]

Hello all.

I have an application running on four computers.  They each have access to a local file server.  All five machines are running Windows 7.

The four target computers lookup or create an INI file to log testing data for products under test.  Ocassionally, during the period of testing, we have observed INI related errors.  Specifically, the software is throwing the UIETooManyFilesOpen error around either the Ini_WriteToFile or the Ini_ReadFromFile functions (and rarely the UIEIOError).

The distance between the file server and the taget machines is less than 50 feet.  I have made certain that the file server is not falling to sleep.

The target machines' access to the file server is simply resolved by a mapped path in the form: "X:\folder\folder\".

Should I be doing something differently for this sort of topology?  I've considered restructuring the software such that it does all its log writing locally and only ocassionally writes to the file server.  But I'm unclear why this particular error would happen in the first place.

[RobertoBozzolo]

Well, I have no direct experience on using infiles across a network, but that "too may files open" error reminds me that the inifile instrument makes use of temporary files to wite data to, before actually renaming them to destination file name. This can be seen by looking at Ini_WriteToFile function code in the source file for the instrument.

Could it be that this continuous creation of temporary files gives some problems in a network environment?

However, this situation is specific to Ini_WriteToFIle and should not occurr in Ini_ReadFromFile, which should only return ToolErr_CouldNotOpenFileForReading error.

[ElectroLund]

Yeah, that's my suspicion too, Roberto.  I wasn't sure which of the two functions was generating the error code, so you've helpe me narrow that down.

I have in fact seen these INI temp files.  They look like this: ini$$2.CVI

When I get these errors, those temp files don't get cleaned up afterward.  The reason we chose a network-based topology was that it was possible that we would move a DUT from one test station to another.  If that happened, the DUT's testing log needed to "travel" with it.  Hence, all of my test station computers needed to store and retrieve the test logs from a central location.

So if I'm to change the architecture around a bit, such that the INI work is doen locally to each computer, and then at some time later it moves the INI files to the server, how best to merge that data?  It gets to be overly complicated!

[gdargaud]

Note that there are other INI libraries out there, some open source, using different ways to read/write the files. If you need concurrent write access, something like Devillard's iniparser.c may be more indicated, unless you have lots of accesses.

[ElectroLund]

That's an interesting thought, gdargaud.  Before I reinvent my wheel though, I decided to rule some things out.

Running my app in a debug fashion, I loaded up the maximum number of DUTs and then forced a log file update for each one, as fast as my software would allow, writing to a network location.  Looks like this:

	UnitLoader("0",0);
	UnitLoader("1",1);
	UnitLoader("2",2);
	UnitLoader("3",3);
	UnitLoader("4",4);
	UnitLoader("5",5);
	UnitLoader("6",6);
	UnitLoader("7",7);
	UnitLoader("8",8);
	UnitLoader("9",9);
	UnitLoader("10",10);
	UnitLoader("11",11);
	UnitLoader("12",12);
	UnitLoader("13",13);
	UnitLoader("14",14);
	UnitLoader("15",15);

	while(1)
	{
		for (bank=0; bank<4; bank++)
		{
			for (unit=0; unit<4; unit++)
			{
				LogUnitFile(bank,unit,LOG_RETRY);
			}
		}
		sprintf(stemp,"%i: wrote all 16 INI files\n",count++);
		printf(stemp);
	}

Note that this particular network location is not the same file server I'm having issues with currently.  I'll test that in a minute.  But the above is completing 16 file writes in about 1 second.  I've let it cycle about 100 times through the while() and I didn't once see a crash.

So I suspect that the issue is somewhere else still...

***EDIT

I should add after about 20 of the log write cycles, I do see a considerable slow-down.  At about 75-100 cycles, it takes up to 4 seconds.  But still no UIETooManyFilesOpen crash.

[RobertoBozzolo]

I have observed myself that sometimes writing to a Ini file takes looong time even on local disk: I suspect IniFile is not designed for performance and this way of accessing such a structured set of data can add a considerable overhead.

Have you considered alternative ways to mantain your centralized set of data? Using some lower level file I/O functions may help, the same as moving to a database structure.

[ElectroLund]

I am very open to suggestions at this point, Roberto. 

What sort of "lower level file I/O functions" do you have in mind?

[RobertoBozzolo]

I was simply meaning to use fopen/fwrite (or OpenFile/WriteFile) without the overhead caused by the IniFile instrument and without using the temporary file.

[ElectroLund]

Oh I see.  And then parse the files by hand?   gdargaud made a similar suggestion above, with a different library.

Hmmm

[RobertoBozzolo]

If you only need to access those data from inside your application, you may consider organizing  them in a typedef'd structure and dumping/reading to the file: with a single pass you'll have your data read back in proper fields without need to parsing anything.

Different the case is if you need to share data with other applications.