LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

data to excel limited to 3 digits of precision

I can't write a floating point number smaller than 0.001 to excel. If I try to write 0.0037, 0.004 is written to excel. Writing 0.0003 results in 0.0000 being written to excel. If I write a very small number directly in excel, I can correctly read it from LV. I am using LV Report Generation Toolkit for MS Office V1.1. I expect this is a configuration problem in either LV or excel but can't find it. Any suggestions?

Example code is attached.
0 Kudos
Message 1 of 3
(2,927 Views)
Hi Bob,

If you look into the "Write to spreadsheet file", there's an input for the format specifier. This defaults to %.3f, which will save binary data in ASCII with only 3 decimal places. If you need more (say 10) then wire a string with %.10f and it should work fine.

Hope this helps

Shane.
Using LV 6.1 and 8.2.1 on W2k (SP4) and WXP (SP2)
Message 2 of 3
(2,914 Views)
Shane,
Thanks for the pointer. I figured this would be something easy. I didn't think to open Excel Easy Table.vi and snoop around.

Bob
0 Kudos
Message 3 of 3
(2,904 Views)