compare two numbers

[Bart L]

How is this possible? The attached VI shows two while loops where two numbers are compared. I don't understand why the first one gives a "TRUE" result and the second one a "FALSE" result.

[Chaos]

When doing multiplication or division there will be a certain amount of rounding error associated with it.  LabVIEW can sometimes hide this from you depending on the precision you ask for.  You can clearly see the difference if you add an indicator and ask for more decimal places.  This is a common CPU problem and you should manually adjust the result accordingly...

[chilly_charly]

Hmmmm...



Seems puzzling isn't it ?







You have just dicosvered the mysteries and limits of binary operations. You thought the numbers were the same, this is true using the decimal system, and false in binary.



Attached is a simple modification of your vi : It calculate the difference between the compared numbers. That's why you should never rely on an equality test when working with floating point numbers...



And of course somebody else had already given the solution...

Message Edité par chilly charly le 11-15-2005 04:50 PM