Calculation..

[labmaster]

Dear

I calculaed some value using FOR loop.

Initially, the value is 5 and then deltaX is extracted from the value.

Finally, I can expect the value is 0 but the reuslt is "near" 0.

I got to know the case of 0 ->5 with adding the deltaX is not problem; I obtained 5 exactly.

What is the problem in my calculation?

Thanks in advance.

[becktho]

Hi labmaster

Here some things you should remember:

1. Don't post pictures as *.bmp - use other types which compress the picture better (*.jpg, *.png)

2. If you post pictures, the problem should be "visible" - just by posting the front panel, no one in here will be able to know how your vi works

3. If possible, post your code - so the community can look at it, see how it works and solve any problems

So please post your problem again. If possible with the source code and maybe you also could describe it a bit more precisely.

Thomas

Message Edited by becktho on 09-01-2005 10:17 AM

[labmaster]

Thank you and sorry for my bmp.

I post my simple code.

I tried the code with EXT or DBL.

Please check my code...

Very thank you in advance.

메시지가 09-01-2005 05:27 AM에 labmaster에 의해 편집되었음

[becktho]

As far as I can see your code is ok.

The problem you describe results from the technology (representation of floating-point numbers in the computer). Expecting a floating-point value to be exactly 0 is always tricky. If you really want to have a 0 in a floating-point indicator, then wire a constant to it. Just by calculating I'd say you'll almost never really get 0.

Thomas

[labmaster]

Thank you.

But how did I get the exact number, 5(not 0)? 

Does the same problem exist in C language?

[becktho]

In fact you do not get 5 exactly. You just see 5 because of the settings of your indicator. Just set it once to 20 digits precision and you will get 4.9999....

And you have the same problem in C or any other language, because the origin is the same on every computer. The representation of the number in a binary format.

[falkpl]

to deal with precision problems with real (float) numbers you can always multiply by an integer and then divide by that integer.  Lets you want to have 5 digits of precision, multiply by 10e5, convert to an integer and then divide by 10e5, this will truncate all remaining digits.  For most solutions the slight imprecision in floats or doubles is so insignificant that it only matters in the display and not the calculation (just like $999.99 sales price looks much cheaper than $1000.00)  you can deal with rounding or just ignore the limitations of the precision of the float data type.

Paul