10-13-2024 12:20 PM
Alternatively I guess you could use format the number as a string with x amount of places of precision. It seems the goal of the display is to make it user friendly.
10-21-2024 10:33 AM
Thanks. jsontext is what I am using.
The string conversion VIs do provide an optional input for a class conversion routine, but it only gets called for classes, and an array of doubles is not a class, so I guess what I could do is create a class just for the purpose of converting double values to strings, and convert my array of doubles to an array of this class, than wire in another class that knows how to convert the values to strings using reasonable formatting. That just seems like a lot of trouble to have to go to.... but I guess it would work.
10-21-2024 10:38 AM
The problem with pre-converting the doubles to properly-formatted strings, is that then the JSON-to-string conversion puts things like "value": "3.14" rather than "value": 3.14.
In other words, the values become strings in the JSON, rather than numbers. I need the string conversion to format the values without converting them to strings in the JSON.
As I said in another response, I think what I have to do is:
1. Create a special-purpose class that contains a double value, and knows how to properly format the value in JSON, then
2. Create another special-purpose class that implements JSONtext's serializer interface, then
3. When calling JSON-to-string VIs, first convert my arrays of doubles to arrays of the first class, and wire in the second class as the class-to-string serializer (an option input, but it only works for classes).
I think that would work, but it just seems like a lot of trouble to have to go to.
10-22-2024 02:45 AM
@TimBlaisdell wrote:
Thanks. jsontext is what I am using.
Can you show how you are using it? I ask as JSONtext formats 12.1 as 12.1 and 355.47 as 355.47, so you should not be having your problem if you are using JSONtext.
10-22-2024 10:32 AM
If you're concerned about the web service rounding or truncating data, then what happens when you send 9999.12 just as you were going to, or any double-precision number, to the web service? How does that data come back to you? Does it?
It sound like that is the question you want answered based on what I have read.
10-22-2024 01:18 PM - edited 10-22-2024 01:20 PM
Replying to drjdpowell:
Wow. You're right, thank you.
I was comparing the values (which are wavelengths from a hyperspectral camera) to the values in a table in the camera documentation.
It turns out, the actual values being returned by the camera DO have all the extra digits for whatever reason. Probably they are being rounded away in the published table (for the same reason that I want to round them).
So, the extra digits probably ARE really more accurate and not just an artifact of digital representation. However, I'd still like to format them out in the interest of storage and readability. In my line of work, values with many digits of precision are frowned upon because they imply a claim of accuracy that we cannot back up.
10-23-2024 06:00 AM
@TimBlaisdell wrote:It turns out, the actual values being returned by the camera DO have all the extra digits for whatever reason. Probably they are being rounded away in the published table (for the same reason that I want to round them).
The reason being that 1.1 can't be represented correctly in a 1+0/2+0/4+0/8+1/16+1/32+1/64... format.