LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

How do you guys deal with floating point errors

Solved!
Go to solution

@JÞB wrote:

@datatechNDT wrote:

Hi guys,

I was wondering how do people deal with floating-point errors? Do you guys always round your answers and numbers before spitting them out? Is there a "conventional way" of dealing with this issue? When precision matters, after X amount of computations you inevitably end up with an answer that is off by a certain amount. For example, if I subtract 0.2 from 2.0 10 times in a row, the answer is not zero. It's close to zero but not zero and when you compare two numbers that's a big issue. If you scale numbers, that's a big issue as well. Any thoughts? 


Over here is @ message 14 is a zipped up project with polymorphic instances for floating point comparisons with a tolerance based on ULP (Units in Last Place).

 

The polymorphic instances could be improved by converting to *.vim (Malleable vis) but work very well.  The ULP method has several advantages over tolerance by %, log10 precision, or significant digit.


The vast majority of people who ask this question are looking at real world numbers - e.g., measurements- where if you go beyond a certain amount of decimal places, it doesn't matter.  But I could see the advantages being real if you are doing "real" math involving pure numbers.

 

As for me, I only need to compare to a tolerance because I work with measurements.

Bill
CLD
(Mid-Level minion.)
My support system ensures that I don't look totally incompetent.
Proud to say that I've progressed beyond knowing just enough to be dangerous. I now know enough to know that I have no clue about anything at all.
Humble author of the CLAD Nugget.
0 Kudos
Message 11 of 13
(720 Views)

Your example of repeatedly adding floats to each other is one that should be but isn't emphasized as something to never do with floats. When I say repeatedly I don't mean x+y+z=c, eg applying a few offsets, but x(i)+x(i+1)+...+x(n), or accumulation. My strategy is to find an alternative in every case, an easy one is when I come across code that says

for (float x = 0.0; x<1000000; x+=.1)

which loses precision every iteration and may introduce rounding errors. It becomes

for (int i = 0; (i*.1)<1000000; i+=1)

in other words I use ints for accumulating and floats for multiplying 

0 Kudos
Message 12 of 13
(717 Views)

@billko wrote:


Yes, that's exactly what I described, except after rounding to integer, I also coerce to integer, even though it's probably not needed because i believe that whole numbers are always represented exactly by FP.


Assuming it's within the significant number of digits, yes. If you write 123456789 as a Single that'll fail.

G# - Award winning reference based OOP for LV, for free! - Qestit VIPM GitHub

Qestit Systems
Certified-LabVIEW-Developer
0 Kudos
Message 13 of 13
(701 Views)