Floating Point Errors when using "Coerce to nearest"

[Intaris]

Change the coercion of your knob to be multiples of 1 or 0.5 or even 0.25 and it'll work fine.

The problem is that you have defined the valid inputs of your control as multiples of 0.1, which is not exactly representable by floating point.  You (as a human being) realise that -1 is a clear multiple of 0.1, but the IEEE 754 norm doesn't cover that a possibility at all.  This is because we humans typically "think" in integers and we spot immediately that this relationship is there.

Think mathematically how you would force ANY given entry X for the knob to be a multiple of 0.1.  Try to implement the code for this and see how things go.  You would of course need to do this without using any "magic" numbers aside from your entry number and the coerce multiple.  You'll find that you will encounter the exact same problem as you are describing.

[GerdW]

Hi pogo and others,

I too think it is a bug, when the control accepts an input and it does NOT coerce it to the next multiple of the coercion setting…

I tested the behaviour with LV2011SP1 and LV2018: you are able to input a value of 1.00000002 even when you set a coercion value of 0.01 (as described by the OP). I also set the coercion mode to "coerce to nearest" and I expect the control to take my input and to "coerce to nearest" multiple of 0.01 - but it doesn't do so always…

My own test (see screenshot and attachment):

-1.005000000 gets coerced to -1.000000000000000000000000 (as expected)

-1.000050000 gets coerced to -1.000000000000000000000000 (as expected)

-1.000000500 gets coerced to -1.000000000000000000000000 (as expected)

-1.000000005 gets coerced to -1.000000004999999970000000 - WHY? Using the "round(x/y)*y" formula instead still gives the expected "-1.000000000000000000000000"…

So I also think it's a bug when the control value range coercion doesn't behave the same way for all inputs…

[Intaris]

Excel and Mathematica using IEEE 754 notation for representing numbers is not the same as assuming they use the same calculations for any given function.

Of course there are ways of calculating a lot of functions which have more or less probability to create rounding errors.  SOme packages take case of this, others don't even though they may all be using IEEE 754 numeric representation for their individual calculation steps.

[Intaris]

Seems inconsistent.  Could it be that somehwere internally they are using a SGL value instead of DBL for part of the calculation?

The cut-off seems to be 7-8 digits or representation away from the coerce increment of 0.01.  Everything with a larger deviation than this seems to work as expected, but everything which goes beyond the dynamic range of a SGL seems to get lost.....

[Ben]

I do not have time to poke at it myself but...

Qs

Could there be an issue with the pop-up dialog that lets us define the increment being a SGL behind the scenes?

Does using a property node make a difference?

The coercion applies to out of range but not when in-range? 

Ben

[pogo]

At this point, I think we're past trying to guess how the implementation of the floating point values is done.

Regardless of how it is implemented, the two outputs in this new attached example should be exactly the same for all inputs.

As you see, they are not the same for the default case of the knob.

IMHO, the internal code for the knob's coerce property should be changed to match the "direct" code that you see in this example.

I have not yet seen an example where this second case gives an improper result.

Perhaps you'll find one?

[Intaris]

---

@pogo wrote:

At this point, I think we're past trying to guess how the implementation of the floating point values is done.

---

I'll revert to my default pedantic state and agree that we all accept that floating point numbers in LabVIEW are represented by IEEE 754 (I was unaware of this even being a point of contention).  This, however, is not the same as agreeing that we're past the point of finding out how LabVIEW uses / combines / processes these values to come up with the aforementioned inconsistent results for coercing the input values of your control. In the context of trying to identify the exact nature of the problem, it can be very helpful to be able to narrow down to a possible range of issues based on existing knowledge and observed behaviour.

---

@pogo wrote:

IMHO, the internal code for the knob's coerce property should be changed to match the "direct" code that you see in this example.

I have not yet seen an example where this second case gives an improper result.

Perhaps you'll find one?

---

If you change the representation of the datatype of your divisor before converting to Integer to a SGL as opposed to DBL, you will see similar results to your current control (This is what I was referring to with my single-precision comment earlier). So the NI code may use the same basic algorithm but mistakenly impelements the disivor as SGL as opposed to DBL, resulting in deltas smaller than approx 7-8 digits of precision (decimal) being lost. In this case, -0.9949999900000001 will result in -1 with full DBL implementation but -0.99 in part SGL implementation. This is the same result the knob gives for the same input value. Does this correlate with your observations?

It's easier to push a bugfix through which corrects a current algorithm as opposed to switching to a new algorithm I would imagine.

[pogo]

So, taking your advice, I changed the following in the "rounding2.vi" example.

The data type of the numerical constant "0.01" was changed to single.

For the default input case (-1.0000000002), the results of the two methods were still different.

Then, I changed the "round" to I16 instead of I64, which still had no impact on the inconsistency of the two methods for the default input case.

Then, I changed the data type of the output of the "direct" method to single instead of double.

Even with all three changes in place simultaneously, the output of the knob still did not match the output of the direct method.

I don't think I'm at a place where I can guess the internal algorithm being used for the knob's coerce property, so I can't possibly suggest a method to correct this bug.

[Intaris]

What I find interesting is that I can't get a DBL Digital display to set to -1.000000000000000222044605 at all. It only displays its value as -1.00000000000000022000000. This is all without coercing any increment.

Just tried it with a knob, and it also won't.

Using LV 2015 SP1.  Seems like it's not just the coercing which has changed.  I recall AQ mentioning something about a new method to display floating point values in text being used (linked to a research paper which promised significantly faster parsing).  I wonder if they are connected.

[Intaris]

---

@pogo wrote:

So, taking your advice, I changed the following in the "rounding2.vi" example.

The data type of the numerical constant "0.01" was changed to single.

For the default input case (-1.0000000002), the results of the two methods were still different.

Then, I changed the "round" to I16 instead of I64, which still had no impact on the inconsistency of the two methods for the default input case.

Then, I changed the data type of the output of the "direct" method to single instead of double.

Even with all three changes in place simultaneously, the output of the knob still did not match the output of the direct method.

I don't think I'm at a place where I can guess the internal algorithm being used for the knob's coerce property, so I can't possibly suggest a method to correct this bug.

---

None of those were what I suggested. Here's an image of what I meant.

In addition, you are inputting your values according to the coercion of the existing NI control.  Please remove this coeercion in order to be able to test.  In your code, it's impossible to enter certain DBL values due to the inherent coercion of your knob.

Note that the output of the divisor can be set to SGL which is equivalent to this code (but no explicit SGL conversion would be visible), but whose configuration is hidden and a lot harder to find.