LabVIEW
why is the output of rounding functions a double?
@billko wrote:
@altenbach wrote:
@billko wrote:
So it's always going to return an approximation of an integer unless you actually plug an integer into it?Floating point datatypes represent integers exactly and the output of any rounding operation is thus guaranteed to be an exact whole number, indepedent of datatype. There is no "approximation" and the generic warnings about comparing floating point numbers can be ignored here. No need to convert to integer, which could cause other problems. Ignore the coercion dot!
Comparing it with an integer datatype is safe as long as long as you remain within the valid datarange of each.
That's very interesting. Actually, everything posted was very interesting and it led me to a deeper understading of fractional numbers. I should probably give a round of kudos to everyone for replying.
Actually, that was the first VI I wrote! My children were having isues with basic math so I wrote a vi that generated sums and differences that containned an adjustable percent of "Carries"- Then printed them off Excel!
That was decades ago!
I think both of those children "Get Math" today!-(At least they can ballance a checkbook)
no, the results are not allways positive
@altenbach wrote:
Comparing it with an integer datatype is safe as long as long as you remain within the valid datarange of each.
It's important to note that in this case, "valid" doesn't refer to the just to max and min values you can get to, but also to the fact that the precision of floating point changes over the range, so at some point the precision becomes lower than 1 and you can't represent every integer, which is why this returns false:
The problem here is not with the I32, which can easily represent 21 million. The problem is the lower branch, which can't actually increment by one, because that value doesn't exist. I used single precision here because it happens there earlier (around 2^24, which is 16.7 million), but it happens with double too.
The important thing, I think, is not necessarily to be aware of every single implication of floating point (although it certainly doesn't hurt to have at least a decent understanding of the topic), but to at least be aware that the issues exist and to factor that in when writing code. It might mean you need to code defensively. It might mean you need to do some research on specific operations, etc.
Hi Intaris,
the result of any math operation with a NaN argument is NaN.
That's how it is defined in IEEE754…
