TDMS file size

[united kingdom]

Dear all,

I am using PCI-6251 card to acquisitate data from a sensor, the sample rate sample number are both 1,000,000. I save the data into TDMS file in every 10 second, the file size is about 80 MB. I think it is very big, How much file size is in saving 1,000,000 data?

Many thanks

[shoneill]

I don't understand the significance of the 10 seconds you mention.  Are the data coming at a rate of 1 million data points per second for 10 seconds, or is the total number of data points 1 million?

Given a 4-Byte floating point number, 2 values @ 1 million numbers yields 4*2*1000000 giving 8 MB.  Thus, if your TOTAL size is 1 million points (x2) then 80MB is too big.  If it's 1 million per second over 10 seconds, then the file size is correct.

Shane.

PS Given the fact that I've never once used a TDMS file, I might be talking rubbish.....

[united kingdom]

it is 1 million data per second over 10 seconds. Many thanks for telling me it is correct. 

 I have tried by using  DAQmx Channel compressed function to reduce the file size, but I did not work out. Is there any ways to compress the waveform data before saving?

Lyn

[AnalogKid2DigitalMan]

What is the frequency range of the signal you are sampling? You may be able to lower your sampling rate if it is under 500KHz (Nyquist Theorem- sample at least 2 times faster than highest frequency component of the signal).

For example, if you signal only on has frequency component of up to 100KHz, then you can sample at 200Ks/sec and still have an adequate reconstruction of your signal.

[united kingdom]

Hi,

The question is not about the freqency. It is about the savign data file size.

I am using PIC-6251 card to acquire from a sensor which is only generate signal occassionally. In most time, the data is only noise, I do not need record all of them, so the file size is not very big. But, in some suituation, I need to record all the data including noise in very high sample rate. The file size is to big store in PC.

I just found an example how to Zip file before saving, I will give a try.  Could anybody tell me how to unzip data file in LabVIEW? Thanks

[AnalogKid2DigitalMan]

It is directly related to frequency- if you sample at a lower rate, that generates less data, thus file size is smaller.

Don't know how to unzip, maybe through a system application call.

If 80MB is too big for your hard drive, suggest you get a larger one. Windows will eventually stall out with limited system resources.

Alternative- save to a USB memory stick?

[shoneill]

Analogkid's question is a valid one.

If you have problems with file size, examining how much data you really NEED to acquire is a valid question.  If you sample only half as fast, your files will only be half as big.  I'm assuming you're fixed at your current rate for one reason or another.

I'm also assuming you're doing many measurements.  I can imagine that many 80MB files would soon make a big difference in the free capacity of your hard drive.

Regarding Zipping, I believe the OpenG toolkit has some Zipping and unzipping functions.  I think LV 8.20 also has such functions built-in if I'm not mistaken.  If none of these are useful, ckeck out 7-zip.  It's a free, open source (more or less) zip program which can process standard zip files in both directions and has a command-line interface meaning you can call it from LV (over System exec).

Shane.

[Herbert Engels]

One way of storing your data with a smaller disc footprint is to store unscaled integer data plus scaling coefficients. That will ususally reduce the file size by a factor of 4 (8 bytes for a double precision floating point number vs. 2 bytes for the 16bit integer you get from the hardware). LabVIEW ships with examples that show how to implement this with TDMS files (writing and reading).

Herbert