LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

Numerical bug in function "3D Cartesian Coordinate Rotation (Euler)"

Although the function "3D Cartesian Coordinate Rotation (Euler)" is telling that it's double precision, it is for sure only single or maybe worse:

Noticeable rounding errors show up, errors that do not show when one replaces the function by standard sines and cosines.

The rounding errors neither show up in complex calculation.

 

Due to beeing busy, I did not perform an extensive test for numerical correctness of this function, as it is obviously buggy.

 

Herbert

Message 1 of 3
(2,932 Views)

Looking at your demo, I disagree with your assessment. The precision is significantly better than SGL and compatible with DBL. The differences could just be do to a slight reordering of operations in the algorithm.

Message 2 of 3
(2,919 Views)

Check out the value for epsilon (on the numeric palette) and see how it compares to your "non-zero" value.

 

In the world of floating point math 1+"0" = 1, so you can also add 1 to that number (increment if you like) and see what you get.  If you get 1, then you are looking at a double precision representation of zero.

 

If you care about these things, you can coerce to zero by passing values through an increment-decrement cycle.

 

Agree with altenbach here, no bug.

Message 3 of 3
(2,911 Views)