Round to -infinity gives unexpected result in my VI

[Ajay_MV]

In the attached file, the expected resultant after Round to -Infinity is 10 but what LabVIEW gives is 9.  Does anyone know why?  

I have divided 1.66 by 0.166 which is 10 and rounding it to -infinity should give 10.  But LabVIEW gives 9.

[GerdW]

Hi Ajay,

another one with no background on floating point precision... 😄

Please consult Wikipedia on that topic!

When setting your controls to show 20 digits instead just 3 you would see that:

I removed the rounding function to show the full result. I would consider the result correct, after rounding down you get 9...

[mikeporter · Accepted solution]

The problem is rounding error in the numeric representation. The thing to rememeber is that floating point representations are approximations, so while the actual result of the calulation is 10, you can't actually represent 10 exactly as a floating point number. While the indicator may say 10, internally the representation of the answer is something like 9.999999999999999 - which is very, very close to 10, but when you go to round it still gets rounded down.

This is not a problem with LV, it's inherent in floating point representations.

Mike...

[mikeporter]

Wow! I was just guessing and I was only 0.00000000000000122 off!

Mike...

[crossrulz]

This is likely a floating point issue.  Due to a limited amount of accuracy, the result is actually coming out to 9.99999999999999822, which rounds down to 9.