setings for numeric constant

[ku_ku]

Hello, I understand it's trivial question but I can'tsolve it for an two hours. I'm creating numeric constant type EXT, displey format floating point. Constant I created is 1.333 and I need only three digits after point. But when I run my VI LabView ouns it up from 1.333 to 1.33   .Where to define to take in acount three digits afer point?

[smercurio_fc]

In the Format & Precision tab you can set the number of digits and the Precision Type. You probable have "Digits" set to 3, and "Precision Type" set to "Significant Digits". You want "3" and "Digits of Precision". If you click on the Help button on this dialog page it will explain the meaning of the controls on that page.

[ku_ku]

OUUUU.... I still can not get further with setings for numeric constant.... Where to mark that i need constant with 3 significant digits. Insted of 2 significant digits as seen in pic. digit1.....

[MikeS81]

Hi ku ku,

why don't you use the functions of the "format into string" function?

You already use %.3f. Where do you have problems?

Mike

[ku_ku]

...ok I'll do so.... but I only wanted to do it without extra functions...

Thank you......

[Ben]

---

ku ku wrote:

...ok I'll do so.... but I only wanted to do it without extra functions...

Thank you......

---

I think you can.

Click on that Advanced button then enter the format code Mike posed.

Ben

[smercurio_fc]

I think you're confusing the display with the value. Are you talking about the value that gets displayed in the little bubble that appears when you have highlight on? If so, that's irrelevant. That only displays 2 digits, regardless of what you do. The value of the wire doesn't change.

[ku_ku]

It is clear with constants representation and presision...

but arises some kind of  strange problem, equal function is not working as I suppose, it should mean that the input arguments are not equal... but how I can diagnose where the problem? 

[smercurio_fc]

You cannot use the Equal function on floating point values. This is not a bug, but a fact of life. Floating point numbers are not stored in a computer's memory the same way they are written on paper. See here. You have to perform a "equal up to a point" comparison. Search this forum and you will come across many threads on this topic.

[LabViewGuruWannabe]

You are trying to compare two values that are not exactly equal. Search the message board for "compare floating point" for more information.

Basically, you might need to multiply your values by a factor of 1000, convert to integer, then compare for equality.