LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

Controlling the precision of a double precision number

Solved!
Go to solution
Solution
Accepted by topic author GDthomas

As has been already stated, the TDMS file stores the data in a flat binary format.  A DBL is 8 bytes.  How it is displayed is completely up to the program that read the data.  In this case, it is a formatting issue in Excel.


GCentral
There are only two ways to tell somebody thanks: Kudos and Marked Solutions
Unofficial Forum Rules and Guidelines
"Not that we are sufficient in ourselves to claim anything as coming from us, but our sufficiency is from God" - 2 Corinthians 3:5
Message 11 of 12
(784 Views)

Wow, a lot more answers than I expected. From what I've gleaned from your answers it's like crossrulz says it's how Excel is reading it and there's nothing to be done in LV.  The reason I asked in the first place was the powers that be didn't like the way it looked when he read the data. I'll just tell him he'll has to go through the excruciating process of clicking his mouse a couple times to change the formatting. He'll live.

Thanks all for the input, it never ceases to amaze me how willing people are to help out on this forum. 

0 Kudos
Message 12 of 12
(769 Views)