LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

defining precision in block diagram

I want to define 2 digit single precision in block digram is there any way to do it. i used sgl conversion function but it uses 6 digit precision how can i redefine it to 2 digit
0 Kudos
Message 1 of 2
(2,522 Views)
"Precision" is either
  • (1) A cosmetic term, showing a certain number of decimal digits.
  • (2) An indication of the underlying precision of a given value, irrespecitve of the representation.

    For (1), just right-click on your diagram constant or indicator and change the formatting.
    For (2), you need to possibly truncate your numbers, e.g. to approximately truncate to 2 decimal digits, multiply by 100, take the integer part (or round to the nearest integer), and divide by 100. Keep in mind that many decimal fractions don't have an exact binary representation, so you might introduce other problems by doing this. I would definitely stay with DBL, things are worse in SGL representation.

    What is your motivation to change the precision?
  • Message 2 of 2
    (2,518 Views)