Machine Epsilon must depend on the representation and architecture

[altenbach]

In the real world, machine epsilon is a function of the binary representation of a floating point number.

The labview help describes it as:

---

Machine Epsilon

"Represents the round-off error for a floating-point number with a given precision. Use the machine epsilon constant to compare whether two floating-point numbers are equivalent."

---

From the term "given precision", we would assume that epsilon depends on the representation. In fact, we can right-click on the machine epsilon and select between SGL, DBL, and EXT.

However, if we look at the actual value, we can see that machine epsilon has the identical decimal value for SGL, DBL, and EXT. No matter what representation we chose, we get the value for DBL.

This is not right!

Suggestion: the machine epsilon must depend on the representation. Since the exact representation of EXT depends on the architecture (64, 80, 96, 128 bits total), machine epsilon for EXT needs to adapt accordingly.

Here's one possible way to calculate machine epsilon explicitly. Note the discrepancy for SGL and EXT.

[G-Money]

Functionality already in LabVIEW. Fixed in LabVIEW 2010 if you drop down the primative on the Block Diagram.

[G-Money]

I marked this idea as Declined even though it was a bug fixed in LabVIEW 2010. The Completed idea status is saved for ideas that are actually started here on the Idea Exchange, worked with on by R&D, and then implemented in LabVIEW. We definitely appreciate the feedback but this idea didn't start R&D working on the fix.

P.S. I know that there are some ideas in the Completed section that shouldn't be there and I plan on going through them and cleaning it up when I have time.