A question about comparator

[SCOTTIE]

Hey guys,

I got another question. Now I am trying to make a comparator. Please see the attached vi file. There are two input, num_1 and num_2. Assume num_1= 5.50 and num_2= 5.40. When running this vi, num_2 will not stop at 5.50, but go up to 5.51 and then come back to 5.50 repeatedly. However, if I make num_1 and num_2 100 times larger (num_1=550 and num_2=540, and the both constants of add/subtract become 1), num_2 can stop at 550 as I want. Can anyone kind of explain why it is like that? Thank you!


Regards,
Scottie

[becktho]

Hi Scottie

This has to do with numberrepresentation. Usually you should avoid to compare floating point numbers with equal or not equal.

Although you see on both indicators 5.00 somewhere far behind (e.g. 10th decimal place may not be equal). In this case you would compare 5.00...00 != 5.00...01 whichs is true.

Thomas

[tst]

Hi Scottie. Like Becktho said - this has to do with the way LabVIEW represents numbers. If you change the number of digits of precision (in format & precision) you can see that the number isn't equal to 5.50. what you may want to do is compare to a range and not a single number or work on integers and after you're done divide them and get your fractions.

[becktho]

Hi guys.

This is not just a LabVIEW "problem". You have to be aware of this topic in every programming language, due to the representation of the floating-point number in a computer. (There's some nice theory behind ;-)).

Thomas