DIAdem not saving all samples from TDMS file

[Jean_Baptiste]

Hello to all,

I have several large TDMS files (1.8Gb each) containing approx. 120 million sampels. When I try to open these fiels in Diadem it previews the correct amount of samples (120 Million), but when I try to save tha data in Matlab format it saves me only half the samples, 60 Million. Also, I have a Matlab script that opens the TDMS files and does exactly the same thing, it is not saving all samples. 

Can you please advise me on this issue? Is there a way to automatically fragment large TDMS files into smaller ones?

Best regards,

Ion 

[benoitsimon]

Hello Jean Baptiste,

It seems your TDMS file is fine since you have the correct number of data in DIAdem.

It may comes from the conversion to the MatLab format. Maybe MatLab can't accept more than 60 millions data. You should ask that to the MatLab expert on their forum.

However, if you want to fragment your TDMS file, you have to do it manually in DIAdem, when you load the file into the Data Portal, or you can automate the function by using VBScript. (Last panel in DIAdem)

An other solution would be to fragment your TDMS file directly where you write it. (In LabVIEW maybe?)

Regards,

[Jean_Baptiste]

Hello,

Thank you for your reply. Matlab can handle signals over 60 million points. 

The problem is that I have used also a Matlab application that that does the same thing. I thnk that maybe I need more memory in order to store the entire data. 

Best regards,

Jean

[benoitsimon]

Ok, so you should try to fragment your TDMS file then.

[Jean_Baptiste]

Is there an automatic procedure that does the fragmenting? I have a large number of TDMS files that I need to fragment. 

Best regards,

Jean

[benoitsimon]

No, you have to do the script yourself.

You can use the Enable Recording Mode button in the script panel to register the action you make in DIAdem as a start for your VBScript.

[winner]

This command may help you TdmsFileDefrag see http://zone.ni.com/reference/en-XX/help/370858K-01/comoff/tdmsfiledefrag/

Hope this help

Winfried

[Brad_Turpin]

Hi Jean,

I think we may be struggling with terminology.  When you talk about "fragmenting", I suspect you mean "segmenting", to take your huge TDMS file and turn it into N smaller TDMS files.  How many channels are typically in each of these TDMS files?  Would it be sufficient to create a smaller TDMS file for each channel?  If you need instead N TDMS files with all channels in them, how many samples would make sense in each file?  I can send you a script that does either in batch mode, once I'm sure I understand what you need.

By the way, a "fragmented" TDMS file is one that could store the same number of samples and channels in a much smaller TDMS file-- that's what Winner was talking about.  The easiest way to check if you have significant TDMS fragmentation is to compare the *.tdms file with its *.tdms_index file.  The *.tdms_index file should always be very small and tiny compared with its *.tdms file.  If it's large, that means you've got fragmentation that could be removed.

Brad Turpin

DIAdem Product Support Engineer
National Instruments

[Jean_Baptiste]

Hello Brad,

Indeed, I need to segment my tdms files and there are many (24 TDMS files x 1.8Gb x 10folders). The thing is that in each file there are up to 10 channels x 120 million samples, and segmenting each file into smaller ones will not only solve my problem, but also reduce the computation time.

Best regards,

Jean

[Brad_Turpin]

Hi Jean,

I adapted a script I had to suit your needs.  Let me know if you have questions using it.

Brad Turpin

DIAdem Product Support Engineer

National Instruments