02-12-2017 02:56 AM
The output from this little VI gives 1.
Should have been 1.000000212346. Why?
Solved! Go to Solution.
02-12-2017 03:51 AM
The value IS actually 1.000000212346 but you are only displaying 6 decimals. Right-click on your control and select Display Format and change Significant Digits from 6 to 13.
You could also have verified the value by subtracting 1.0 from it and see that the result is 2.12346E-7.