5.1-5 is not equal to .1 ?

[lordsathish]

Hi all...

I'm using LabVIEW 8.5, When i tried to compare the result of difference of 5.1 and 5 (i.e 5.1-5) with .1 , the result was not equal. I used DBL representation.

Why is this so...?

[TCPlomp]

Your computer is binary, your number system is decimal. There will be rounding problems, using floating numbers for equality will get you into trouble. You should do an in range check of 0.1 +- 0.1 % or so. See wikipedia for more info.

Ton

[mikeporter]

You could also multiply both input nuimbers by 10, convert them into integers, subtract them and test to see if the difference is equal to 1.

Mike...