in range and coerce not working in EXE

[Mellobuck]

Good Evening,

I am using LabVIEW 8.21, and building an executable of my VIs. When doing so, I seem to get intermittent behavior of the In Range and Coerce function, as seen below:

I am trying to look at an array, and keep the items within the input ranges INCLUSIVE, and get rid of all the others. To that end, I have the double black dots on the In Range and Coercefunction, however it is functioning as inclusive on the lower limit, but exclusive on the upper limit. I copied the block of code over to a separate VI, and it works properly there, but not in my original. As you can see, the top front panel is not letting the high limit "0.269" value through, where the bottom one is. Please note the same code on the block diagrams. These VIs are built into an executable, in the same build specification, but with different behaviors. The non-working code is a sub-vi of a dynamically called VI, but it is included in the specification.

Copying the working code into the non-working VI code to replace it does not result in proper function.

What is happening here, or what am I missing?

Thanks,

Tim

Message Edited by Mellobuck on 10-15-2008 06:59 PM

[RavensFan · Accepted solution]

How are the numbers generated?  Are you sure it's exactly .269 or does the value actually go out to more decimal places and is rounded to that.  Do you have the same problem with other values for the Max and Min.

I wonder if the problem is due to the inherent impreciseness of a computer trying to represent certain numbers.  Basically errors due to trying to perform exact comparisons on doubles.

[Ray.R]

I agree with Ravens Fan.

It does sound like the indicator rounding up due to the number of significant digits being displayed and rounding up.

R

[Mellobuck]

Thank you Ravens Fan,

That seemed to be it. While I was rounding to the 0.001 place, I was doing it by multiplying my 1000, rounding , then dividing by 1000. The 'granularity' of the double precision number meant that either the inputs or the ranges were actually 0.269000000000000017 or 0.268999999999999906, which is probably causing the problem.

-Tim