Speed of reading header section in TDMS

[labmaster]

I am designing my software and I just ordered LV2009 and PXI DAQ module.

During studying database, I got to know the Datafinder for searching TDMS (DAQ data).

Using Datafinder, I can search the TDMS files easily but there seems to have a limitation in my demo version.

Can I apply several conditions (Date, Model, Serial) in finding TDMS with Datafinder? 

In my case, I will have so many TDMS files in different subfolders and trace the specific data with header info.  

Anyway, I am wondering how long does it take the time to get the header when I try to read the properties (header information file) only.

Of course, the DAQ data will be drawn one-by-one later after getting the info.

Can you estimate the time to read the header info only for 300MB vs. 30MB TDMS?

I am thinking how to realize the database to find specific data with static mdb file or real-time search.

Labmaster.

Message Edited by labmaster on 04-21-2010 07:41 PM

[StefanR]

Hi Labmaster,

using the LabVIEW DataFinder Toolkit you can apply several conditions finding TDMS files.

To do so, the files you want to search for must be in a so called Search Area of DataFinder. Only files within a Search Area are indexed by DataFinder.
You can add any folder on your computer or a computer on the network you have access to as Search Area. You can do this by eg. right-clicking the DataFinder icon in your tray and selecting 'Configure...'. Within the configuration dialog you can amongst others also define your Search Areas. You can also do this programatically within LabVIEW (see DataFinder_Configure.vi in LabVIEWExample Finder).

A good example to apply several conditions searching with DataFinder is the DataFinder_Pass_Fail_Analysis.vi (see LabVIEW Example Finder).

Greetings from sunny Aachen

Stefan

---

Please note that in case you want to search for date/time properties those needs to be 'optimized for searching'. You can do this by selecting 'Optimize Custom Properties...' by right-clicking the DataFinder tray icon. You can also do this programatically (a good example is in the DataFinder_Pass_Fail_Analysis.vi if you open the Prepare DF(DataFinder_PrepareDataFinder.vi)).

Optimization is needed for two reasons. First consider a custom property 'Date' which for most of your TDMS files is a date/time value and for some TDMS files of type text/string. In this case DataFinder does not know which data type you are referring to.
The second reason for optimization is performance. Since there could be several hundreds of different custom properties non of them is treated as 'optimized proprety' by default - since this will slow down indexing speed and increase index size. Optimizing your most important custom properties will improve query performance and allow you to search for eg. date/time properties.

[Herbert Engels]

The time needed to read header information does not relate to the overall size of the file. It could take a lot of time on a small file, but be very fast on a huge file, depending on how data has been written to the file. However, if you are using DAQmx to log data to TDMS files (as opposed to the "TDM Streaming" API in LabVIEW), reading the header will be very fast for all of your files.

Performance also depends on what kind of hard drive you're using, whether you're accessing the file via network and so forth. That's why any estimates on time consumption might not really reflect the performance you're seeing. If you want a ballpark number, you could use a VI that does about the same thing DataFinder does, see below.

Hope that helps,

Herbert