NI TestStand

cancel
Showing results for 
Search instead for 
Did you mean: 

Convert an Exponential in TestStand

Is there a way to convert an exponential (1.051058-e7) to a floating point number (0.00000051058) in TestStand?
 
The value that I'm getting from my external DLL is as a String, but I need it as a number.  Val() returns the exponential as well.
 
Mike
0 Kudos
Message 1 of 2
(3,116 Views)

Val returns a number. The number can be formatted for display as an exponential, or not, but the actual value is the same either way.

If you store the value in a local or use a numeric limit test, you can set the numeric format of the local or limit test to "%.13f" to display it how you desire. If you need the string version of the number in your desired format, call Str(<number>, "%.13f")

0 Kudos
Message 2 of 2
(3,108 Views)