01-25-2010 02:23 AM
Hello;
I Would like to limit the number of precision digits to 3 in a double variable but I can´t.
I represent a double variable in an Excel Report and it shows as 1.12345678 and I´d like to be 1.123
I have to format an ASCII variable to double variable: Fmt(&double,"%f[p3]<%s",ascii);
I use the formatting modifier [p3] but it doesn´t runs as I expected.
ascii variable : "1.123456789" --> double variable: 1.123456789000000 --> Excel Report: 1.12345678
any ideas?
Thanks a lot
01-25-2010 03:13 AM
You cannot limit in such a way the number of bytes scanned from the string. There are several alternative methods to accomplish what you are trying to do, I can suggest you the following alternatives but you could find other ways. All of them use Scan instead of Fmt.
1) Limit the number of bytes scanned from the string to a stated number of digits:
r = FindPattern (a, 0, -1, ".", 0, 0);
Scan (a, "%f[w*]", r + 4, &d):
2) Post-process the scanned real number:
Scan (a, "%f", &d);
d = (int)(d * 1000.0) / 1000.0;
3) Pre-process the string before scanning:
r = FindPattern (a, 0, -1, ".", 0, 0); a[r + 4] = 0;
Scan (a, "%f", &d);
Beware of the fact that all these alternatives don't round the numbers: they all truncate them to the stated number of digits.
A supplementary alternative could be to customize the destination cell in Excel sheet to display only the required numner of digits, while maintaining the whole content inside the cell (it could be better in case you need to perform other calculations inside Excel).