A simple puzzler for the long (US at least) weekend...

[Bob_Schor]

Now for something (related, but) completely different!  What if you use the Extended Precision type (whose details are, shall we say, a tad murky)?Altenbach's Floats

You get yet another number!  If you assume that the three lowest bits are thrown away (why?), it's correct, but the Dbl representation is "closer".

Remind me to stick with "reasonable precision" (i.e. Biology, not Physics) ...

Bob Schor

[Alexander_Sobolev]

In LabVIEW 2011 32 bit it was I32 limit, not I64:

[billko]

Thanks, fellow LabVIEW devs - this was fun to follow along!  🙂

[Yamaeda]

---

@Bob_Schor wrote:

Now for something (related, but) completely different!  What if you use the Extended Precision type (whose details are, shall we say, a tad murky)?Altenbach's Floats

You get yet another number!  If you assume that the three lowest bits are thrown away (why?), it's correct, but the Dbl representation is "closer".

Remind me to stick with "reasonable precision" (i.e. Biology, not Physics) ...

Bob Schor

---

Hmm, shouldn't Extended precision have more ... precision?

/Y

[alexderjuengere]

---

@Yamaeda wrote:

---

---

Hmm, shouldn't Extended precision have more ... precision?

/Y

---

I'd say in this case it is "as good as"

Bob's comment about the obscurity of the EXT number format is based on this:

https://zone.ni.com/reference/en-XX/help/371361R-01/lvhowto/floating_point_numbers/

https://zone.ni.com/reference/en-XX/help/371361R-01/lvconcepts/how_labview_stores_data_in_memory/

Extended-precision floating-point numbers have an 80-bit IEEE extended-precision format.

Note  In some cases, extended-precision floating-point numbers can have a 64-, 96-, and 128-bit IEEE extended-precision format depending on the computer processor. 80-bit is the most common.