comparison of 2 double values

[labviewette]

dear community,

please have a look at the simple code attached.

can anyone tell me why the boolean output is false?

the first contant is copied out of a larger code segment.

the value of the second constant is entered directly.

thanks in advance

[KoenR94]

The comparisson pallete doesn't like dbl's.

If you want to compare some doubles who are 0,2 you have to multiply them with 10 infront.

And convert them to an integer.

Integers do work!

[altenbach · Accepted solution]

Maybe you need to display more significant digits to see that they are not equal.

There are many solutions, but you need to decide what is "close enough". 😄

(and, no, the solution given earlier is probably not sufficient unless you only care about one decimal digit)

[Dennis_Knutson]

It's a very basic understanding of computers that equal comparisons of floating point numbers is a big no-no. It is due to the imprecise conversion to binary. This has been discussed numerous times.

[altenbach]

---

@KoenR94 wrote:

The comparisson pallete doesn't like dbl's.

If you want to compare some doubles who are 0,2 you have to multiply them with 10 infront.

And convert them to an integer.

Integers do work!

---

DBLs "work" too, and perfectly at that. This has nothing to do with "does not like". There are no coercion dots, so the comparison operations "likes" DBLs just fine. 😄 (They just might not give you the answer you expect and you need to be aware of pitfalls as mentioned. :D)

What is the point of the while loop? just burn the CPU??? 

[labviewette]

does that mean, that if i enter a constant value of 0,20 that that value is in fact not 0,20 but a value somewhere in the tolerance range of the binary conversion ?

[altenbach]

Many simple decimal values (0.1, etc.) cannot be exactly represented in binary, others (e.g. 0.5) can.

You'll always get the closest possible binary value when entered manually, so if you enter 0.2 twice and compare, they are equal (but both not exactly 0.2). Of course if the value is a result of some computation, it might be slightly different.

[KoenR94]

---

DBLs "work" too, and perfectly at that. This has nothing to do with "does not like". There are no coercion dots, so the comparison operations "likes" DBLs just fine.  (They just might not give you the answer you expect and you need to be aware of pitfalls as mentioned. )

---

Nice to know that! Wasn't aware of this.

---

What is the point of the while loop? just burn the CPU??? 

---

No just to show it was actually running 

Though burning CPU is fun too haha