Saving JSON file with Notepad as UTF-8 causes error in Unflatten from JSON VI

[BillMe]

After doing a flatten to JSON and write to text file, I opened the file in Notepad and edited one of the fields. Then I Saved As and marked "UTF-8". The Unflatten from JSON function fails with error -375003 "The JSON string is invalid. JSON strings must be encoded in UTF-8 and must conform to the JSON grammer.

All I did was change the numeric value in one of the fields. Does Notepad not save a "true" UTF-8 file? Anyone else had this problem? Makes it kind of hard to let users edit if the software can't read it back in.

[mikeporter]

Can you post the file? I have often edited JSON strings manually.

Mike...

[BillMe]

See attached.

One thing's for sure. The files are different. Look at the hex values of the string output from "Read Text File" in each case....completely different. That tells me Notepad is NOT saving the file in UTF-8 format despite my choosing the encoding.

[rolfk · Accepted solution]

Notepad adds a so called BOM header at the begin of the text file. This indicates to an UTF aware reader the type of UTF as well as the endianess for multibyte UTF encodings (for example UTF-16LE, UTF-16BE, UTF-32). Unfortuntaly Notepad is so "smart" to do that transparently and without any notification.

If you use Notepad++ or some other more intelligent text editor you can avoid that problem of having this BOM autmagically added to the file on saving.

Basically for UTF-8 you get the three hex codes 0xEF, 0xBB, 0xBF at the beginning of the file. You can read the file yourself and check for these three characters in that order and remove them if they exiist, before passing the string to your Unflatten from JSON string and everything should be fine.

See here for more details about the UTF BOM.

[BillMe]

Thank you much. I modified my code to look for the three extended data bytes and strip them off if present before passing to the Unflatten to JSON VI.