In article <3a71a810@newsgroups.ni.com>, Chris Matthews
writes
>"Klaus" wrote:
>>>hello,>in CVI 5.5.1: If I read out a value from a control which contains
>e.g 2010.1>>(double) CVI put 2010.099999999909 into the variable. With 2010.2
>CVI >works correct !!!>Is this "error" (?) known ??>>Thanks>Klaus >
>
>This is not an error, it's just the nature of programming on a computer.
>The computer has 8 bytes for a double value to accurately represent numbers
>in binary. Not all numbers can be precisely represented in binary. In this
>case 2010.10000000 can't be represented exactly in binary form in 8 bytes,
>so the closest it can get in 2010.09999999. This is always how it has been
>with computers representing numbers. The only thing you can do is increase
>the size of the data type (extended) or deal with the computer's closest
>representation.
>
>Best Regards,
>
>Chris Matthews
>Measurement Studio Support Manager
>
>
The following extract from the new group comp.lang.c FAQ, backs Chris's
point up. For ANSI C issues, comp.lang.c is well worth subscribing to.
Section 14. Floating Point
14.1: When I set a float variable to, say, 3.1, why is printf printing
it as 3.0999999?
A: Most computers use base 2 for floating-point numbers as well as
for integers. In base 2, one divided by ten is an infinitely-
repeating fraction (0.0001100110011...), so fractions such as
3.1 (which look like they can be exactly represented in decimal)
cannot be represented exactly in binary. Depending on how
carefully your compiler's binary/decimal conversion routines
(such as those used by printf) have been written, you may see
discrepancies when numbers (especially low-precision floats) not
exactly representable in base 2 are assigned or read in and then
printed (i.e. converted from base 10 to base 2 and back again).
See also question 14.6.
--
Regards
John Cameron
Senior Design Engineer
Northern Hi-Tec Ltd
Wasco House
Willow Lane
Lancaster
LA1 5NA
United Kingdom
Telephone: +44 (0) 1524 67833
Facsimile: +44 (0) 1524 67544
Email: john.cameron@nht.co.uk
URL: www.nht.co.uk
