frac/exp string to number - shifting decimal point, what's going on?

[mes291]

Trying to convert string to number, i.e. 1.234 string to 1.234 dbl. When I use the frac/exp string to number, sometimes the decimal point gets shifted and sometimes it does not. I pulled a small section out and uploaded it here. Can anyone figure out why it shifts the decimal in one case but not the other? I'm sure it's some simple thing I'm missing. 

thanks!

-Matt

[RavensFan]

It seems to work perfectly fine for me.

It would've helped if you had saved as default in the two strings the values that were and weren't giving you problems.  When I type in 1.234, it works fine.  But perhaps your issue is that you typed something in wrong, but not obvious.

[johntrich1971]

I'm not seeing any shift in decimal place. Can you perhaps save some default values that exhibit the problem and upload the vi again?

Edited to add: Looks like RavensFan beat me.

[Frozen]

It works for me.

Did you read the help for the function?

"Interprets the characters 0 through 9, plus, minus, e, E, and the decimal point" Perhaps your string is scientific representation?

EDIT: Guess I am too slow... 2 other people posted before me. LOL

[mes291]

wow, thanks for the quick responses! I've entered in numbers that are giving me problems and made them default. Here is an updated vi. 

thanks everyone!

[johntrich1971 · Accepted solution]

These numbers work fine on mine. The numbers are 1.23456 and 9.70695970695971E-02 (which corresponds to 0.0970695970695971).

Edited to add: You can change the Display Format of the numbers to scientific if you want the numbers to always be x.xxxxxxEy.

[RavensFan · Accepted solution]

Make String #2 wider.  You'll see it says.  9.70695970695971E-02  That would be 0.097....

So no problems!

[mes291]

how embarrassing. Two hours of my life totally wasted on an undersized box.......ugh.

thanks again everyone.