Units and number precision problem

[govindsankar]

Hi,

I have attached a VI that is used for configuring a current source of a spectrometer. This is the Keithley 2600 current source. The vi is used to set the level of source current. But the problem is I am only able to give it whole values that are in Amperes, like 1A, 2A etc. I am unable to give values like 500mA or 0.005A. When I try to do that its only being 0A. How can I make it into mA or a decimal pointed ampere value. Kindly do help me. Thank You.

[GerdW]

Hi govindsankar,

is the resulting command string as expected by you and your device?

Just a wild educated guess: Does your computer use a comma as decimal separator? Nearly all devices expect a point…

[govindsankar]

Yes, my computer uses a comma and also I expect 0.36 A or 360mA. Either of them I want in the device. 0.36A is also fine. 

[billko · Accepted solution]

---

@govindsankar wrote:

Yes, my computer uses a comma and also I expect 0.36 A or 360mA. Either of them I want in the device. 0.36A is also fine. 

---

If your computer uses a comma for the decimal place, it will convert the number to a string using a comma, which the device most likely doesn't understand.  it probably considers everything left of the comma to be valid, and the comma and everything to the right as invalid; therefore, a whole number only.

[GerdW · Accepted solution]

Hi govindsankar,

when your computer uses the comma you need to change this VI to get a point as decimal separator!

Read the help for FormatIntoString and the format strings. Hint: "%.;"…