U32 to DBL converstion

[Madan]

I am just trying to understand this conversion operation. My understanding was that the conversion to higher data type would be loss less.

Can anyone explain why "8389388" of type U32 gets converted to "8389390" DBL.

Thanks

[PhillipBrooks]

I think because your indicator is set to show 6 significant digits. The last digit rounds up and you see 8389388 as 8389390

Change your indicator's properties to show 7 (or more significant digits and you will see 'the correct value'.

[John_P1]

Hi Madan,

By defult LabVIEW only displays 6 significant digits on the DBL and EXT indicators.  Try right-clicking the indicator, select "Display Format", then raise the digits of precision to 7 or higher.

Best Regards,

[TCPlomp]

Spoiler

---

@Madan wrote:

I am just trying to understand this conversion operation. My understanding was that the conversion to higher data type would be loss less.

Can anyone explain why "8389388" of type U32 gets converted to "8389390" DBL.

Thanks

---

What exactly were you expecting?

A conversion converts the data from one type into another type and keeps the information at the same value.

If you are interested into a binary transer (meaning that the binary data is the same and only the datatype is converted) have a look at typecast:

Ton

[Cory_K]

---

@Phillip Brooks wrote:

I think because your indicator is set to show 6 significant digits. The last digit rounds up and you see 8389388 as 8389390

Change your indicator's properties to show 7 (or more significant digits and you will see 'the correct value'.

---

To elaborate on Philip's (correct) post, there is a difference between how data is displayed and how data is stored.

When you use U32, that is the internal data type,which explicitly dictates the amount of memory LabVIEW should allocate for that object.

Displaying it is just a matter of formatting the number for the user, in this case, set to 6 significant digits.
But again, this does not mean internally LabVIEW threw out that precision, it just isn't displaying it.