in range and coerce function does not always return a correct result

[MattI]

I passed 20E6 as DBL to "upper limit" input of "in-range and coerce" built-in VI, and also passed 20E6 as DBL to the X input of the VI, yet it says that it is not in range.  (Note: lower limit was hardwired to 0.)

Apparently, there is some kind of roundoff error going on with LabVIEW.

I've seen this before when just wiring 2 doubles of the same value to an "=?" built-in VI.

When I display 30 digits or so, the values then display slightly differently.

How do 2 numbers which were entered by ME (not anything else) precisely as 20E6 manage, over time or by passing thru VIs, or by being copied & pasted -- or whatever -- to become "different"?

[johnsold]

By default the upper limit is not included in the range. If you pop up on the node you can select whether to include the upper or lower limits. In other words you can chose > or >= and < or <= comparisons.

The other issue is a fundamental characteristic of the bianry representation of decimal numbers. This has been discussed before in the Forum and is addressed in the LV Manuals.

Lynn

[Dennis_Knutson]

First, verify that you right clicked on the function and selected "Include Upper Limit". The default does not. Second, there have been problems representing floating point numbers ever since the invention of digital computers. There are simply numbers that cannot be "exactly" represented in binary format. Doing comparisons of floating point numbers is always risky and is something that you as the programmer have to take into account.

[zuolanwang]

This simpliest 'In Range and coerce' does not work on my setup (8.0 and XP). The result should be 0 but it returns -5.

[Darren]

Your lower limit (0) is greater than your upper limit (-5).  The only behavior I would expect in this case is for someone to tell you to fix your code.

-D

[zuolanwang]

Thanks for pointing out.

[DEppAtNGC]

Using 8.6.1, when I place a new in-range-and-coerce.vi on the diagram, it is set to "include lower" but not "include upper". That seems really weird. Why is that? Also, I would swear I had some working code developed in 8.6 that I'm now editing in 8.6.1 and I noticed "include upper" was not checked but should have been... My memory could be wrong on this.. but certainly I think a new instance should not be disparate on the include upper/lower.

Any clues?