11-23-2012 09:39 AM
The problem with equality comparisons on floating point numbers is that due to the finite representation in binary, two numbers which you might expect to be equal actually differ by one bit and the comparisons fail. For example 0.1 added to itself 10 times does not equal 1, it is 0.999999999999999889.
This is true of any digital computer and any programming language. The details may vary sightly due to differing representations but there will always be cases like this.
Lynn
11-23-2012 06:23 PM
Comparing floating point numbers is perfectly legal, you should simply be aware of pitfalls. 😉
If you know where the data comes from (e.g. if you earlier multiplied some values with zero) an equal comparison with zero is perfectly fine. Potential problems arise if you compare the result of some computation with a diagram constant or with a result of some other computation. For example if the desired result is 0.1, we have a number that cannot accurately represented in binary and there are two very close numbers, right above and below, that can. Even if you do the same computation, a slight reordering of operations can gve a different result.
11-23-2012 07:02 PM - edited 11-23-2012 07:08 PM
Thanks for the link. If you had stated that care should be taken as rounding error can cause incorrect results it would have been a lot easier to understand your comment.
Below is an example I recreated in LabVIEW that generates a value of 1 using 2 different methods. As can be seen when the number of digits displayed is extended on the indicators, the number 1 is not what we expect. The snippets of code I posted earlier rounded to the nearest whole number, which would probably have avoided the problem. This has been a good reminder that care has to be taken!
There is also an updated version of the link that was posted earlier: Floating Point Numbers 2012 Edition
11-23-2012 07:07 PM - edited 11-23-2012 07:19 PM
I agree that the "round to nearest" made your code absolutely safe. 😉
Quick puzzle: What's the value of "Numeric" after the loop stops? 😄
11-23-2012 07:09 PM
Remember that the display is not what is in memory. The data (the value on the wire) always exists at the full resolution of the datatype in memory. The display can be formatted in many different ways.
Technically this is not a rounding error so much as a representation issue, which is why altenbach and I did not use that terminology.
Lynn
11-23-2012 07:29 PM
@altenbach wrote:
I agree that the "round to nearest" made your code absolutely safe. 😉
Quick puzzle: What's the value of "Numeric" after the loop stops? 😄
The battery went flat on my laptop before it stopped. Good example.
11-23-2012 09:41 PM
Congratulations on your new found wealth, by the way....
11-25-2012 02:11 PM
Based on the sample code and comments of others here is a potential solution to the OPs question:
Based on what I have read determining zero is not an exact science and very much depends on your application. There may be more sophisticated solutions, but the one presented by Norbet seems robust.