LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

how to control the number precision???

Hi guys,

 

I have a number 350.00

I need to make 350.0

 

How to do that on LV8.6?

thanks in advance.

Message 1 of 11
(10,501 Views)
Right click on the control and go to display format.  Set it for digits of precision and 1.  Floating point and uncheck Hide trailing zeroes.
Message 2 of 11
(10,491 Views)

This is probably just a purely cosmetic formatting issue. If you want to e.g. truncate a number to 1 significant digit, you would need to d a bit more code.

 

Ravens solution applies to front panel controls and indicators, but you have not told us the details of the problem so we can only guess.

 

(Other possible interpertations would be for example if you want to write such a number to a file or to a text indicator or if this "number" is actually not a numeric datatype but a formatted string to begin with).

 

What is the datatype of the "number"? Where does it come from?

Message 3 of 11
(10,478 Views)

As altenbach said it depends on your problem.

 

If you just want the front panel to display it differently, right click and change the format.

 

If you want the number to be written to somewhere (a file/function) in a different format,

you could use a numeric conversion. 

 

Functions >> Numeric >> Conversion 


From there, theres a bunch of functions that can convert for you.

Cory K
Message 4 of 11
(10,460 Views)

I guess to elaborate on your question:

 

Right now your number is formatted (it appears) as

%.2f 

 

You want

%.1f

 

the number before the 'f' signifies the number of decimal places 

Cory K
Message 5 of 11
(10,459 Views)

Hi again,

 

Couldn't solve it yet!

Could not find really clear help for "Fixed-Point Function", unclear what is "fixed-point type" and no examples in LabVIEW 8.6 help. DISAPPOINTING!

 

 I include the.vi in the attachments.

 

 

 

 

 

Hi guys,

 

I have a number 350.00

I need to make 350.0

 

How to do that on LV8.6?

thanks in advance.

0 Kudos
Message 6 of 11
(10,420 Views)

Problem solved,

 

turns out subvi does the conversion itself.

 

However it is very bad that LabVIEW does not have NUMBER precision control (decimal point)!!! shame!

 

 

 

0 Kudos
Message 7 of 11
(10,400 Views)

testas123 wrote:

However it is very bad that LabVIEW does not have NUMBER precision control (decimal point)!!! shame!


Well, you can always write your own subVI with a few strokes of the mouse. 🙂

 

If you are talking about truncation (a data operation) and not formatting (a cosmetic issues), be aware that many nice and round decimal digits cannot be correctly represented in binary, so you might still be slightly off when you would display e.g. 16 decimal digits.

Message 8 of 11
(10,368 Views)
in LV 8.6.1, I'm following the steps you describe to set the Display Format on a slider contol, but I always get 4 digits of precision. So in Slider Properties, I set Type: Floating point, Digits: 0, Precision Type: Digits of precision, the slider digital display indicates: 74.9226.  It seems to always revert to the default of 4 digits.  How do I override this default to get, say 75??
Greg R.
0 Kudos
Message 9 of 11
(10,115 Views)
In the upper left corner of the "Display Format" tab, there is a ring where you need to select where to apply the formatting. Make sure to choose "digital display".
Message 10 of 11
(10,106 Views)