LabWindows/CVI

cancel
Showing results for 
Search instead for 
Did you mean: 

Formating Function Fmt(); Limiting precision digits

 

Hello; 

 

I Would like to limit the number of precision digits to 3 in a double variable but I can´t.

I represent a double variable in an Excel Report and it shows as 1.12345678 and I´d like to be 1.123

 

I have to format an ASCII variable to double variable: Fmt(&double,"%f[p3]<%s",ascii);

I use the formatting modifier [p3] but it doesn´t runs as I expected.

 

ascii variable : "1.123456789"  --> double variable: 1.123456789000000 --> Excel Report: 1.12345678

 

any ideas? 

 

 

Thanks a lot 

 

 

 

 

 

 

 

 

 

0 Kudos
Message 1 of 2
(4,662 Views)

You cannot limit in such a way the number of bytes scanned from the string. There are several alternative methods to accomplish what you are trying to do, I can suggest you the following alternatives but you could find other ways. All of them use Scan instead of Fmt.

 

1) Limit the number of bytes scanned from the string to a stated number of digits:

r = FindPattern (a, 0, -1, ".", 0, 0);
Scan (a, "%f[w*]", r + 4, &d):

 

2) Post-process the scanned real number:

Scan (a, "%f", &d);
d = (int)(d * 1000.0) / 1000.0;

 

3) Pre-process the string before scanning:

r = FindPattern (a, 0, -1, ".", 0, 0); a[r + 4] = 0;
Scan (a, "%f", &d);
 

Beware of the fact that all these alternatives don't round the numbers: they all truncate them to the stated number of digits.

 

A supplementary alternative could be to customize the destination cell in Excel sheet to display only the required numner of digits, while maintaining the whole content inside the cell (it could be better in case you  need to perform other calculations inside Excel).



Proud to use LW/CVI from 3.1 on.

My contributions to the Developer Community
________________________________________
If I have helped you, why not giving me a kudos?
0 Kudos
Message 2 of 2
(4,659 Views)