Can you create custom index files for custom plugin types?

[AnscombesQuartet]

I work with DIAdem for data analysis, and have inherited an in-house file format and associated data plugin that reads the file into DIAdem's data portal. Because of how the plugin was designed, each file in this format is read afresh each time it is loaded, which causes each file to require 30-40 seconds from opening to being able to use it.

I've noticed there are file formats (e.g., .TDMS) which generate index files after the first opening, and I've noticed these speed up the opening of even large files by a great deal. I assume these index files contain metadata, header information, offset records for where channel data begins, etc. - it may be all of these, or none of these, but I was hoping to find any information or resources about generating these files for custom data plugins. Does such a thing exist?

[OscarCardenasNI]

Hello, 

Some clarification about the TDMS Index File can be found in this document under Index Files: TDMS File Format Internal Structure

The index file is an exact copy of the *.tdms file, except that it does not contain any raw data and every segment starts with a TDSh tag instead of a TDSm tag. The index file contains all the information to precisely locate any value of any channel within the *.tdms file.

To be honest, I have not seen so far other Dataplugins that create and use Index Files but you could edit/optimize the DataPlugin using the following resources: 

Resources for Writing Your Own DataPlugin

Debugging DIAdem User Dialogs (SUD) and DataPlugins

As an alternative, you can check the loading settings from DIAdem > DIAdem Settings > NAVIGATOR > Bulk data: Always load bulk data 

Is the loading of the data also slow when using the option Register Data? 

Regards,