What is the format of Datalog Files?

[chavezb]

I'm trying to find the format for Datalog files.  I would like to create files in other applications that I will read in LabView, and it would be nice if I could create Datalog files.  If anyone knows where I could find this format I would greatly appreciate it.  Thanks

[Dennis_Knutson]

Depends on what you mean by datalog file. There's datalogging done from the front panel (Operate>Log at Completion) and datalogging using any number of file write functions (Write to Spreadsheet, Write LVM File, Write to SGL File, etc.). In most cases, the format of the data you write is something that you select and will depend on what you want to read it. Most other applications can import text files (i.e. Excel) though you create bigger files and many can import binary formatted files (smaller and faster to read/write). To understand some of the different binary formats, read the application note called LabVIEW Data Storage that's part of LabVIEW Bookshelf.

[chavezb]

Thanks for the quick response.

The type of datalog file I am talking about is the kind when you drop an "Open File.vi" on the block diagram and you wire something to the "datalog type" input.  LabVIEW automatically creates "records" in the file, making it easier to read the file back out.  I am interested in what the format of each "record" is?  I assume there must be some type of header information that would include a type descriptor, record number, etc.  I have used these types of files before, but I always created them in LabVIEW, now I am trying to create the file in C.

Thanks again for the help

[Dennis_Knutson]

The format of the file is the "something" you wire to the datalog type. If you wire a cluster that has a string and a DBL, then each record is a cluster with a string and a double. If you look at any of the file I/O examples, the read functions include the type of data to be expected. There is nothing automatic.

[chavezb]

Maybe "format" is the wrong word; I'm looking for the structure of the file.  If I open the file in a hex editor, what is the structure?  There seems to be a code (44 54 4C 47(hex) = DTLG(ascii)) that I vaguely remember indicating that it is a datalog file.  Then there is about 12 bytes of stuff, followed by a type descriptor.  I think I've seen the file structure somewhere, but I can't find it now. 

Thanks

[Dennis_Knutson]

Again, the structure is going to depend on what you are writing. The binary structure of a file to which you write integers (i.e. I32, U8, etc.) is going to be different than the structure when you write a DBL. The document I mentioned, describes all of the different file formats. A big difference between LabVIEW and for example, C++, is that LabVIEW writes in big endian format. Most programs in windows/Intel use little endian. This is partially a legacy from the Mac origins of LabVIEW and to maintain compatability between LabVIEW running on different platforms. I think part of the confusion here is that there is no standard for "datalog" files. Some examples use one type and others use a different type so to know the format of the file you looked at, you would need to know the VI that created it.

[chavezb]

I do understand what you are saying and I appreciate your help (as well as patience) with me.  However, I do believe that there is some common information at the beginning of the file that must be there.  If I want to create this type of file in C, I need to know exactly what bytes to write to the file so that LabVIEW will be able to open it.  I've looked at the LabVIEW bookshelf and I can't find anything in the Data Storage section about datalog files.

[TylerS]

chavezb,

Generally, it is not recommended to use datalog file types when the files will be accessed (or in this case written) from another language. Datalog files are basically files of multiple datatypes (using clusters), so to read them in LabVIEW, you would have to know the exact structure of the cluster (i.e. the datatypes included) and wire a dummy cluster with the same datatypes to the "datalog type" input of the File Dialog.vi. If you're looking to write files in another language, I would recommend using a regular binary file type or a text file, like Dennis mentioned. That way you wouldn't have to worry about how LabVIEW stores the binary data for a cluster.

I hope this helps some. Have a great day!

Tyler S

[CoastalMaineBird]

There is a header there, which you can prove by storing a boolean into a file, and observing that the file is over 100 bytes long.

Notice that if you FLATTEN a structure (any structure), you get a flattend data output and you also get a TYPE DESCRIPTOR array. This information describes the 'format' of the data in the structure. There are type codes for I32, STR, etc., etc.
I believe, from my own experimentation, that this data is in the file somewhere. This is how the FILE DIALOG functions can show you files of ONLY the type you asked for. The dialog reads the headers of all files, and compares them to the datalog type you want.

There is also a 'DTLG' code at the beginning, as a first-level turkey filter.

Beyond that, I don't know. I would join the others in recommending that you NOT do this, as it will be A) difficult, and B) fragile. NI has chosen to change how they store booleans (for example) in the past, and might do so again. They will likely not notify you when they do so. Notice that you will have to deal with endian issues, as datalog files use a standard (big-endian) format for multibyte ints and floats, which differs from Intel storage practices.

I say the best solution is to store your data in a bytestream file for maximum space efficiency, or a text file for maximum readability.

[Dennis_Knutson]

Tyler,

Thanks for the additional help,

chavezb,

Look at the two shipping examples called Simple Temp Datalogger and Write Datalog File. Both are writing what can be considered datalog files but the format of each is different. In one, each record is a cluster with a string and DBL array. In the other, each record is a cluster with a DBL and two strings. Both are valid and be easily written and read with LabVIEW but the header and the data in the file will be completely different. You will not find any information in the documentation specifically on datalog files because there is not such thing as a single "datalog" file type. It is a highley generic description only. What you will find is information on how LabVIEW stores arrays, DBLs, I32, clusters, etc. because these are precisely defined data types.