Reading a C structure from file

[BowenM]

So... I have two separate monstrosities of a C structure (100 ish entries) that I want to read into LabVIEW.  It contains a mix of char[], char, int[], int, etc.

I'm fairly sure there isn't a way to read the entire structure with one read into a LV cluster because clusters contain extra "size" bytes for each array or string.  However, writing the VI to read these one entry at a time is going to be ... tedious.  So, I'm posting in hopes that I'm terribly wrong and that someone has a simple way to read the entire structure at one time as a cluster.

(though this might be a good excuse to get better at VI scripting...)

Edit:  Hmm... if anyone knows the exact way LabVIEW stores arrays / clusters in memory, I could write a VI that takes the C struct string, parses it and makes the correct reads.  For arrays it would just add on those extra bytes, store every read as a concatenating byte array in a shift register then type cast it to cluster when done. 

[pincpanter]

I'm stuck to LV2011, but I believe you can't do this in any version.

However, if you create a "matching" cluster and a constant array containing the byte sizes of the structure entries, you can:

1) read the whole structure as a U8 array

2) split the array into individual sub-arrays, one for each entry

3) in a for loop, cast each sub-array to the proper LV datatype and assign the value to the corresponding cluster element

Implementing a trial-error method, you don't need to determine in advance the correct datatype of each element. This is quite inefficient, though.

Furthermore, of course, you can simply cast from U8 array to value only if the endianness of numeric datatypes in the saved structure matches LV.

[rolfk]

It depends!

char[n] elementName;

is in fact a fixed size element that is equivalent to a LabVIEW cluster with n * u8 elements. 

But

char *elementName;

is a pointer, which would only occupy 4 or 8 bytes.

So if your cluster only contains fixed size arrays or scalars, you could theoretically create a cluster containing the necessary clusters inside with the fixed amount of u8 elements, (or u16 or u32, etc). However this will result in a monstrosity (your works) of a LabVIEW cluster that is pretty difficult to deal with in the rest of your application. You most likely do want to have a String for char[] elements to work with them as strings, rather than a cluster with 100 or more bytes, from which only a few are used.

While char[] variables also can be used for byte arrays in C, they are generally representing zero terminated strings, and for fixed sized char arrays inside structures, the reminder behind the first zero byte is usually unused, but still allocated in memory.

Scripting while a fun thing to do, is most likely not going to be of much help in this case, unless you envision to have to translate many dozen of such structures in the future. The problem is not so much the scripting in itself but the fact that you need some input description of your structure for the scripting function. You could of course use the C declaration of the structure as input but then you need to write first a C syntax parser that at least understands enough to translate everything inside those structures to a valid intermediate form that your scripting function can use to create the final LabVIEW datatype. And writing text parsers is always a pretty tricky and definitely work intense adventure.

Personally I would bite the sour apple and write a VI that reads the individual elements and then translates that properly into a suitable LabVIEW datatype to be put inside the somewhat more manageable LabVIEW cluster. So not a cluster of many clusters with x integer elements inside each, but instead a cluster with strings for text fields, arrays for numeric arrays, etc.

While text strings are usually zero terminated, numeric arrays are usually bounded by an extra element in the structure that defines the number of valid numbers inside the fixed size array. The translation VI then would read in that many numbers from the file and after that advance the file pointer to the end of the fixed array size.

There is no use in knowing how LabVIEW stores arrays in memory for this specific problem. You simply read in the values from disk using the Read from Binary File and then assemble the array or string accordingly and insert it into the cluster.

[BowenM]

Thanks - that is something else to try if my above idea doesn't work... though after some Google searching I'm pretty sure it will.

http://zone.ni.com/reference/en-XX/help/371361H-01/lvconcepts/how_labview_stores_data_in_memory/

http://zone.ni.com/reference/en-XX/help/371361H-01/lvexcodeconcepts/array_and_string_options/

[nathand]

If you can post your C structure, we'll be able to give you more specific advice. Do you need to parse out all the elements of the structure, or just a few values? If the latter, it might be easier to treat the entire struct as an array of bytes, and index out only the ones you need. The C struct will have a fixed size; any variable-sized elements will actually be pointers to data somewhere else, so the underlying struct remains the same number of bytes.

[Ben]

If the data is coming from file... I would not expect to find pointers in the file.

I have never found a good way to tackle this challenge aside from walk through the data one field at a time and populating clusters that mimic the original data structures. I did those to be able to read legacy files saved by application that my code replaced.

War story time!

Spoiler

In about LV 7.0 there was very nasty bug where if you closed and opened a lot of files at the same time, LV would get the file references mixed up! Datalog entries of one file type would get appended to the end of files using a difference datalog type. Opening a file once it was corrupted as one datalog type would eventually crash LV when the data file type changed. Very nasty!

I wrote code to decipher the mixed up data log files and put the data in the right files.

Not long after the customer replaced the File I/O sub-system with an SQL data base and washed their hands of that problem without looking back.

Ben

[BowenM]

Thanks everyone for the responses.  I started a couple VIs to automate this (played with scripting some, played with my above idea some) but in the end decided to go with Rolf's advice and bite the bullet.  This isn't something I likely will ever need to do again - I was just being lazy 🙂

[rolfk]

This should give you an idea of quick and dirty and a bit more elaborate but truly useful. 

However, please note that this sample assumes byte packing of the data structure in the C application. This means every element inside the structure follows immediately the previous element.

With default compile options, most C compilers nowadays would align each element inside the structure to the smaller of the two values:

1) its own element size

2) usually 8 byte

For this example this would mean that there are additional 4 bytes between the number and FloatingPoint element, as well as 3 extra bytes between the isDevice and dataLen element. No extra bytes between the dataLen and the data array though, as the doubles inside the data array already align on 8 bytes naturally.

[BowenM]

I feel like the resulting code belongs in the Rube Goldberg thread...

[rolfk]

What about the idea to write subVIs for common data elements?