Why doesn't this vi work?

[Wiebe@CARYA]

"becktho" <x@no.email> wrote in message news:1139235039510-320742@exchange.ni.com...
Wiebe:
Testing floats for equality is never a good idea, really never.


Yes, it is never a "good" idea. But sometimes it is very convenient to set a value to NaN, so you can check later on if it is a valid value. Or e.g. to test if a calculation was succesfull.


Anyway, the argument against testing for equality is that float values are approximate values. But NaN, -Inf and +Inf are always exact values, as far as I know... (Zero is a bit more difficult). Still it is better to avoid.


Regards,


Wiebe.

[shoneill]

Here's a really silly idea,

Instead of multiplying by 0.2 (Which can't be represented properly in Binary), why don't you instead multiply the other side by 5 (Which CAN be represented properly in Binary).  This removes one rounding error, and it works better.  I've tested it.

Create indicators on both inputs to your comparison functions and set the display to 20 decimal digits.

You'll see the problem.

Shane.

[Marc_A]

Here's another idea.  Before you compare the numbers, use the format into string function and wire %.8f (or whatever precision you need) to the format string.  Then compare the strings.  I just did some testing, and it doesn't add any significant amount of time.  With a loop running 10 million times, it added about 30 ms over simply comparing the floats.  So if you're not running it continuously, you should never see a slow down.

Of course Shane's idea works, too, but if there was a case when it didn't work for you, this might also help.

[MelanieM]

Both are great suggestions!  Thanks!

Melanie