TDMS & Diadem best practices: what if my signal has pauses/breaks?

[josborne]

I built a LV2011 datalogging application that stores lots of data in TDMS files.  The basic architecture is like this:

Every channel has these properties:

     To = Start time

     dt =  Sampling interval

Channel values:

     1D array of DBL values

After datalogging starts, I just keep appending to the channel values.  And if the TDMS file size goes over 1 GB, I create a new file and start over.  The application runs continuously for days/weeks, so I get a lot of TDMS files.

It works fine.  But now I need to change my system to allow the data acquisition to pause/restart.  This means there will be breaks in the signal (probably between 30 sec and 10 mins).  I had originally considered recording two values for every datapoint (value & timestamp) like an XY graph.  But I am opposed to this in principal because I feel it fills up your hard disk unnecessarily (twice us much disk footprint for the same data?).

Also, I have never used Diadem, but I want to ensure that my data can be easily opened and analyzed using Diadem.

My question:  Are there some best practices for storing signals that pause/break like this?  I would like to just start a new recording with a new start time (To) and have Diadem be able to somehow "link" these signals ... eg have it know that its a continuation of the same signal.

Obviously, I should install Diadem and play with it.  But I thought I would ask the experts about best practice first, since I have zero Diadem knowledge.

[Brad_Turpin · Accepted solution]

Hi josborne,

Are you planning on creating a new TDMS file each time the acquisition stops and starts again, or were you wanting to store multiple start/stop sections withing the same TDMS file?  The easiest way to handle the date/time offset is to store one waveform per Channel per start/stop section and use the "wf_start_time" Channel property that is native to TDMS waveform data-- whether you're wiring an orange floating point array or a brown waveform to the TDMS Write.vi.  DIAdem 2011 has the ability to easily access the date/time offset when it is stored in this Channel property (assuming that it is stored as a date/time and not as a DBL or a string).  If you only have one start/stop section per TDMS file, I would definitely also add a "DateTime" property to File level.  If you want to store multiple start/stop sections in a single TDMS file, I would recommend using a separate Group for each start/stop section.  Make sure you're storing the following Channel properties in the TDMS file if you want that information to flow naturally into DIAdem:

"wf_xname"
"wf_xunit_string"
"wf_start_time"
"wf_start_offset"
"wf_increment"

Brad Turpin

DIAdem Product Support Engineer

National Instruments

[josborne]

Thank you very very much.  Kudos!  This was very helpful! 

I haven't decided exactly how I will manage the breaks (eg should I create a new file for every break).  At first, I thought it would be convenient to have multiple files, but I now see there are some tradeoffs.

I think I will install Diadem and use this info to figure it out.