Logging large data using NI Daq

[Easwar]

Hi,

I have an USB 6289 and using labview to collect data and log. I have a few doubts regarding this. Check the below code.

I'm using 4 AI channels and logging data continously with sampling rate -100K Hz ; samples per loop - 50K. and i need to collect the data for a minimum of 30 minutes.

1) When i save it as TDMS file, it occupies 2 GB of memory for 20 minutes of data collection although I do not care about memory. When I open it in excel, the length of each AI channels reads some 150M samples but the last cell index reads 1048576. Where are the other samples? Has it been overwritten?

2) I want to use the data for post processsing in Matlab. What is the efficient way of loading it in Matlab since it takes a long time to load the data in Matlab ( I did this by saving the data in .xlsx and used xlxs read function).

Please help. Thanks in advance !

[Lukin]

Hi Easwar,

 1.  The reason you only see 1,048,576 elements is that is the maxium number of rows Excel can display. See this link from Microsoft

2.  I have never tried to read a TDMS file in Matlab but it looks like there are some .m files that will do it.  Found this on there website but I have never used it so I can't vouch for it. ConvertTDMS 

Lukin

[Easwar]

Hi Lukin, Thanks for the information. I tried the program you mentioned and it worked. But it took some time to load. processing the data also is taking a lot of time and makes the system slow. Is there any efficient way of doing it like saving the data with minimum memory usage?

[Lukin]

Can you define what you mean by it takes a long time.  Its a large amount of data and it will take some amount of time to work with and will depend on how powerful your computer is.  I am not an expert in file types but as I understand it TDMS is a binary file and thus should be pretty efficnet.  Perhaps someone else can recmened a faster method. 

Alternatively you could do some preprocessing on the data in LabVIEW before you save it to par the data down to something more managble.

Lukin