02-08-2022 06:27 PM
@JÞB wrote:
@datatechNDT wrote:
Hi guys,
I was wondering how do people deal with floating-point errors? Do you guys always round your answers and numbers before spitting them out? Is there a "conventional way" of dealing with this issue? When precision matters, after X amount of computations you inevitably end up with an answer that is off by a certain amount. For example, if I subtract 0.2 from 2.0 10 times in a row, the answer is not zero. It's close to zero but not zero and when you compare two numbers that's a big issue. If you scale numbers, that's a big issue as well. Any thoughts?
Over here is @ message 14 is a zipped up project with polymorphic instances for floating point comparisons with a tolerance based on ULP (Units in Last Place).
The polymorphic instances could be improved by converting to *.vim (Malleable vis) but work very well. The ULP method has several advantages over tolerance by %, log10 precision, or significant digit.
The vast majority of people who ask this question are looking at real world numbers - e.g., measurements- where if you go beyond a certain amount of decimal places, it doesn't matter. But I could see the advantages being real if you are doing "real" math involving pure numbers.
As for me, I only need to compare to a tolerance because I work with measurements.
02-08-2022 06:48 PM
Your example of repeatedly adding floats to each other is one that should be but isn't emphasized as something to never do with floats. When I say repeatedly I don't mean x+y+z=c, eg applying a few offsets, but x(i)+x(i+1)+...+x(n), or accumulation. My strategy is to find an alternative in every case, an easy one is when I come across code that says
for (float x = 0.0; x<1000000; x+=.1)
which loses precision every iteration and may introduce rounding errors. It becomes
for (int i = 0; (i*.1)<1000000; i+=1)
in other words I use ints for accumulating and floats for multiplying
02-09-2022 02:32 AM
@billko wrote:
Yes, that's exactly what I described, except after rounding to integer, I also coerce to integer, even though it's probably not needed because i believe that whole numbers are always represented exactly by FP.
Assuming it's within the significant number of digits, yes. If you write 123456789 as a Single that'll fail.