Floor (roundind toward -infinity) error

[Nico_Fr]

Hi everybody,

I just found a surprising error in Labview 2013. I want to get the interger part of the division of 2 float numbers: after the division I use the "floor" vi in the numerical palette. In some situations (with numbers below 1) this interger part is lower by 1 to what is should be. Example:

300m / 25m = 12.00 (division output)

floor(12.00) = 11.00

It seems to work only when both initial number are <1 and not in every configuration. I was wondering if it is some epsilon machine error in the calculation of the division

I'll use another solution for my calculation, but I think it may be corrected.

Regards.

[GerdW]

Hi Nico,

I was wondering if it is some epsilon machine error in the calculation of the division

Yes.

For clarity: set the display format of your floats to "%.20g" to see the reason of this "unexpected result"!

You will find a lot of threads handling rounding issues with floating point numbers, here in the forum as well as on Wikipedia!

 Side note: why don't you use Quotient&Remainder function? It gives the same (flaky) result when used with DBLs…

[Nico_Fr]

Didn't went that far in looking at decimals.

Quotient and remainder function has the same problem yes.

I'll try to dig the forums to find solutions, but I think the most obvious is to multiply both number by 10^n with n chosen so that the number I get are intergers (no decimal part). And then I'll use the quotient and remainder function.

[GerdW]

Hi Nico,

the most obvious is to multiply both number by 10^n

Yes.

Didn't went that far in looking at decimals.

You should. With floats: always…

(For background see IEEE754, IEEE854)

[Nico_Fr]

Hum. Finally, I worked on mantissa and exponent, and added epsilon to division of the 2 mantissa. Seems to work without side effect. For the moment....

Many thanks for your help Gerd 🙂