DIAdem Connectivity Library and Windows Regional Settings

[tstanley]

Hi,

I'm having a bit of a problem with a CVI program that uses the DIAdem connectivity library when deployed to PCs that have their "Language for non-Unicode programs" set to some non-English language (Japanese is one, but I suspect this will happen with others).  The problem comes up with file properties in TDM files that use the µ or 'mu' symbol in their Property name.  When I call the DDC_GetFilePropertyNames(...) function, I get different strings from the same .tdm file depending on whether I have Windows set to English (US) or Japanese.  As far as functions like DDC_GetFileProperty(...) are concerned the Property name string I get when my Windows is set to English is correct and I get an error using the string from when Windows is set to Japanese (-6213 DDC_PropertyDoesNotExist).  The end result is that my program functions just fine so long as the PC is set to English, and fails on PCs where it is set to Japanese.

I've attached a small sample console program.  It will ask whether or not you want to create a .tdm file when run.  If yes, the program will create a test.tdm file with a property named "Test_With_µ_Symbol".  In either case, it will attempt to read the file, calling  DDC_GetFilePropertyNames(..) to read the property names, then DDC_GetFilePropertyType(...) and DDC_GetFileProperty(..) using the property name it gets back.  In order to see the problem I'm having, compile and run the program on a PC set to English.  It will have no problems reading the file property back.  Then take the file program to PC set to Japanese and run it, telling it to not create a new file (i.e. just read the existing file), and when the program attempts to read back the file property back it will fail.  Curiously enough if the program is compiled on the Japanese system it will have no problems running, which suggests that CVI will compile a different program depending on your Windows language setting (!?).

As configuring a computer to use Japanese may be a bit of a pain, here are the bytes in the strings I get back from  DDC_GetFilePropertyNames (...) on the English system (top) and the Japanese system (bottom) which is really the heart of the problem I'm having.  The difference is the -75 on the top string and the -125 and -54 on the bottom string, corresponding to the 'mu' character:
84 101 115 116 95 87 105 116 104 95 -75 95 83 121 109 98 111 108

84 101 115 116 95 87 105 116 104 95 -125 -54 95 83 121 109 98 111 108  

(I didn't post the actual strings as the forum may end up mangling the characters) 

Thoughts?  I guessing I might be doing something I shouldn't by trying to use the 'mu' character in property strings, but on the other hand I don't think that DDC_GetFilePropertyNames (...)  should be giving different results based upon Windows language.  Thanks in advance for any help.  I'm running Windows XP and CVI 2010SP1.

[tstanley]

Any comments on this?  I need to get this to work properly and while I could do a seach and replace to fix the strings I would prefer to have a better solution.  Thanks.

[MilanR]

Hi tstanley,

I have not tried to reproduce this, but do you know exactly what character encoding is being used when you switch the language to Japanese?

It looks like when the code page is switched in windows that the specific character changes to a Multibyte encoding based on the operating system language. LabWindows/CVI has functions that can be used to work with Multibyte encodings which are listed in the help topic Programming for Multibyte Character Sets in LabWindows/CVI.

[tstanley]

I'm not an expert at this stuff, but the Code page conversion table appears to be 10001 MAC - Japanese.  I took a look at the link and I'm not sure how those functions would help because it appears that DDC_GetFilePropertyNames(...) is providing multibyte-encoded strings and other functions like DDC_GetFilePropertyType(...) return an error when I pass them.  Thanks.

[MilanR]

Hi tstanley,

It sounds like the mu character may be coded differently on Windows English and in Japanese. It looks like the Multibyte Character Set packages can be used to determine when multibyte systems are being used. In addition, it looks like the Multibyte Character Set package has functions for scanning a string and checking if a character is single or multibyte. Maybe the functions can be used to determine what the character encoding of the system is and handle it appropriately.

[tstanley]

Hi Milan,

Thanks for the reply.  The idea of checking for multibyte strings and modifying them had crossed my mind. If that's the solution for now I'll go with it.  However, I would like to put the request in that functions like DDC_GetFileProperty(...) and  DDC_GetFilePropertyType(...) accept Multibyte strings for some future release.  Right now it's all too easy to make an application that works fine on an English system and have it fail in strange ways when someone tries to use it with another langauge.

[MilanR]

Hi tstanley,

I think having code to handle the case in different environments is probably the most robust solution as it can be specifically tailored as needed. It may also be possible to force the CVI application to run with a specific code page using the AppLocale application from Microsoft. The AppLocale application is a 3rd party tool from Microsoft that will not be supported by National Instruments and also no longer seems to be updated by Microsoft. Information about the tool can be found at the Microsoft AppLocale page and there is a page that provides some instructions for installing the AppLocale tool on Windows 7.

[MilanR]

Hi tstanely,

Another possible solution is to try and perform the solution that was mentioned in the Unicode forum in LabVIEW. At the end of the forum, Mark Moss mentions "One of our interns recently went through this adapting a program to display Chinese in the user interface.  His solution ended up being surprisingly simple:  Just read the strings from a text file stored using the target encoding.  In our case, there were really only two encodings of interest: Windows code-page 1252 (used on our company's computers located in North America) and UTF-8 (used on our company's computers in China)."

Because it seems that the CVI is encoding strings in the application based on the code page used at the time, it might be possible to have multiple text files saved in the different code pages and to load the strings at run-time. You may still have to use the Multibyte Character Set functionality to determine the platform that is being used, etc.

[D_Biel]

Hi TStanley,

I looked into this a bit deeper and it looks like it is a bug. The user app is writing the string in the system code page, cviusi.dll is converting that to unicode and sending it to usi.dll, then the file is actually written in UTF-8 (at least on the English OS). The opposite is done when reading the file: UTF-8 -> unicode -> system code page (multibyte on the Japanese system). Somewhere in this second stack, the conversion is going wrong. It is a bit of a mess to track down dealing with so many different encodings, so it is going to take a bit of time to get to the bottom of.

From what I have seen so far, I don't think there is going to be a workaround for this. I would recommend avoiding non-ascii characters for now. I will post back when I have more information.

[tstanley]

Thanks for looking into this.  From my limited testing it does seem that the files get written slightly differently depending on the regional settings.  Or in other words it's not just the output of DDC_GetPropertyNames(..) that I have to worry about.  I'm curious as to what you find out but I think for now you're right that the best thing to do is to avoid non-ASCII characters.