strange behavior with greater?

[Vinc42]

I just noticed something strange when comparing two doubles with greater?.

This code returns true somehow:

I tested it with LabVIEW 2017 and 2019, and also the probes show the same value.

However it doesn't return true for lesser and it only works for 0,1 and 0,3.

I tested a lot  but it doesn't make any sense.

Did I miss something or is this really a bug?

Can anyone reproduce this or explain whats going on?

[RavensFan · Accepted solution]

Create indicators on the result of the subtraction.  Then expand the precision of the display on indicator and all the constants to 15, 16 decimal places.  You'll see that 0.3 is not exactly 0.3.  And when you do a subtraction, those last decimal places might mean you are slightly over or slightly under 0.3.

It is not a bug, it is just a common issue for any program on any computer system related to doing math on floating point numbers.

[Yamaeda]

Since a computer only works in binary number in the end, how would you write 0.3?

Instead of the 1, 2, 4 of the integer side, you'd use 'bimals' of 1/2, 1/4, 1/8 and so on.

So, convert 0.3 and tell us the result. 🙂

[Gregory]

If you do need to check if 2 doubles are equal, you should not use the the equals the function. You can subtract them, take the absolute value, and see if it is less than a very small number, like the machine epsilon constant.

[RavensFan]

---

@Gregory wrote:

If you do need to check if 2 doubles are equal, you should not use the the equals the function. You can subtract them, take the absolute value, and see if it is less than a very small number, like the machine epsilon constant.

---

Technically, the OP isn't using an equal function, just the greater than.

But they are doing A-B to get C, then comparing to see if C is greater than a constant of C on the block diagram.  sometimes it is, sometimes it isn't depending on the value they used for B.

It all relates to the rounding errors in trying to represent decimal values in computer floating point values.

The classic example is using old-fashioned BASIC and doing 1 / 3 * 3 and finding out it is no longer equal to 1, but .999999999.....

[BertMcMahan]

A fun calculator to play around with this:

https://www.h-schmidt.net/FloatConverter/IEEE754.html

Enter "0.3" and see what "Value stored in float" actually is.

[crossrulz]

---

@BertMcMahan wrote:

A fun calculator to play around with this:

https://www.h-schmidt.net/FloatConverter/IEEE754.html

Enter "0.3" and see what "Value stored in float" actually is.

---

That is a fun tool.  But I do have to state that it is using what we refer to as Single Precision Floating Point (SGL).  Most floating point numbers in modern times is DBL (Double Precision), which uses 64 bits instead of 32.  Regardless, we still have the same issue.

[littlesphaeroid]

Another approach might be to multiply the result by ten and convert to an integer before the comparison. 

[thols · Accepted solution]

NI's short explanation.

NI's longer explanation, read about the machine epsilon there, which can be used when comparing floating-point numbers.

Read about this idea. , in the discussion there are some examples how to compare floating point numbers approximately.

[Vinc42]

Ah ok. Interesting that I never noticed this before and I didn't find an explanation after a quick google search.

Thanks for the solution and for the link to NI's explanation!