Float value???

[plit string]

I wonder what difference between define float f_val = 356.9 (or 356.99) and get the the same float value from CVI run program?

1. When I define a float f_val = 356.9 or f_val = 356.99 in c program  ---> When I verify in debug porgram, it is 356.899999..

2. But if I use CVI GUI in order to enter a float value  f_val = 356.9 or f_val = 356.99 as the following:

        float f_val;

        GetCtrlVal (panelHandle, PANEL_NUMERIC, &f_val);
        SetCtrlVal (panelHandle, PANEL_NUMERIC_SHOW, f_val);

It can show f_val = 356.90 and f_val = 356.99 

I did post the float value problem and have helped (Floating point numbers 32 bits and some fraction float numbers can not be as desired) but it still bother me. Can someone explain the above matter?

[Al_S]

There are two datatypes in CVI for floating point numbers: float (single precision accuracy) and double (double precision accuracy). 

You'll get more accurate results if you use double instead of float.

The numeric controls which you place on your CVI UIR are double by default.

The answer is related to issues raised in your previous post.  It is a limitation of how floating point numbers are represented and stored in binary.  You can spend years searching article written on the subject, like the following google search:

http://www.google.com/search?hl=en&lr=&q=IEEE+floating+point+representation+inaccuracy

[plit string]

Thanks for the explaination, but it still bother me that even I defined f_val in double it is still not much better:

double f_val = 356.9; When I verified f_val, it shows 356.89999999... Meanwhile by CVI program (Set & Get), it can show 356.90.

However, I agree that when I defined double f_val = 356.99; then when I verified f_val it is equal 356.99.  But can you explain the 356.9 case?

[mvr]

This link explains the issue floating point representations of some decimal numbers http://docs.python.org/tut/node16.html

Message Edited by mvr on 01-10-2007 02:55 PM

[mvr]

And this page has links to a calculator you can use to see the resulting conversion of different values along with all the gory details of the floating point format. http://babbage.cs.qc.edu/courses/cs341/IEEE-754references.html

Message Edited by mvr on 01-10-2007 03:00 PM

[plit string]

Thanks for the link, it explains my question