Good way to initialize clusters and parse files into them

[Hummer1]

I'm writing a small application to open some sonar .xtf files and look at their data.  The format is published and I have done a crude job of building a cluster and initializing it...then reading the file header and parsing it into the cluster so I can use the data in the headers to read the data packets that are stored next.  Here is an example of how I have done one of the parsing jobs...very crudely,

Read the header binary in (length defined)

Build the cluster by creating variables from the input file...and then loading the file data in the appropriate cluster variables after casting and ordering the bytes properly...

Surely this is not the best way to do this...perhaps loops controlled by the number of bytes in parameter to be parsed...???

I'm looking for a smart strategy for looking at the definition document, which defines the name of the variable, where the variable appears in the header (byte offset), how many bytes it is made up of....and translating that information into the cluster definition efficiently...

Ideas would be appreciated....

Thanks in advance for your help.

Attached are a couple of examples of parsing part of the header information.

Hummer1

[RavensFan]

Look at the OpenG toolkits such as OpenG Variant Configuration File VI's.

When writing, you feed it the cluster, and it creates an .ini file of the data in the cluster.

When reading, you feed it a cluster constant which defines the data structure, get a variant, then you can use Variant to Data using the same cluster constant to get the output cluster.

Of course it is a good idea to TypeDef this cluster so that if you need to change anything, all areas of your program are updated.

[Hummer1]

Didn't find descriptions of how to ... just vi's that do ... so would like to understand better how to...

So, I should type def the cluster for use anywhere...(should be able to do that.)

Is there a good way to help with building the type def...or is it pretty much as I have done...make a cluster with the variable constants of the right type loaded into it?

Thanks.

Hummer1

[Hummer1]

Also, note that the order of the bytes needs to be reversed ... If I build that into the type definition, will it work...?  for example...a single...with 1d array reversal...into the variable in the cluster...?

Seems like you would have to do that when you read the data the first time (or every time you load data into a parameter that needs it) and load the header into the cluster...assuming that the data presented to the cluster elements is in the right byte order....

Hummer1

[Hummer1]

I don't understand what you do to the input byte array to get it into the structured/type defed variant to data....

For example

The byte array comes in from the left...The type def cluster difinition (thank you very much for that idea)...comes into the "Type" terminal on the variant...and the output is the cluster just as I would like to have it...BUT...it doesn't run...says the type variant is not compatible with the data type input...(I used the To Variant) on the byte array to try to fix that...?

Any thoughts...

Thanks.

Hummer1

[RavensFan]

You missed the whole part about the OpenG toolkit.

Here is an example of one I am using in my app right now where I want to save and later recall a cluster to a file that consists of two arrays.

I'm now not so sure this is exactly what you are looking for because now I see you mention an ".xtf" file which is probably a file you get from somewhere else that already has a defined structure.  What I'm showing is a good way to save cluster data and read it from a file.  But if your file is not coming from LabVIEW, you probably will have to figure out how to manually parse it and load it into your desired data cluster.

[Hummer1]

Ahhhh!  Now you have it.  I'm looking for a way to read a binary file (of known structure) and move the byte oriented data into the cluster the first time...I understand how to read and write after that.  I have built the structure in the type def...and that works fine.  I bring the type def to the "Type" input on the Variant to Data function...that works fine.  Then when I try to wire the byte oriented data from the array to the Variant to Data function...I get an error...so I inserted the "To Variant" function and it allows me to make the connection, however I get the previously mentioned error at run time.

There is a note about Parent/Child relationships between the data which was completely incomprehensible to me in the help for the Variant to Data...that may have something to do with the problem...but I don't know what.

Thanks for your help.

Hummer1

[RavensFan]

There may be a way to do this using "Flatten to String" and "Unflatten from String".

One issue is that it looks like your strings are fixed length, while LabVIEW uses variable length strings where the string length is preprended to the string bytes when the data is flattened.   But I think if you yourself insert the bytes 16 and 53 into the string data you read from the file at the appropriate places, you can get LabVIEW to read the data and convert it to your cluster.

1. Read the data as a string.
2. Insert the bytes listed above at the correct places.
3. Pass that through the Unflatten from string to convert that string data to a cluster.  You define the cluster by feeding a constant that defines the datatype into the function.

I think if your binary files as a not too complicated structure, and you define your data types for each element of the cluster precisely, you can get this to work.

Play with the attached snippet.

[Hummer1]

Excellent...HOWEVER...how do you do the calculations and then save a version as the definition for the format...(I'm not asking this very well...)  If I were making a Custom Control and saving it as a type def...there is no way to do the calculations in that kind of definition screen...(only get to view the front panel...) Do you save the "value" in your example as a custom control type def2 to make it the input to the Variant to Data definition...It would then include all the correct length ... (and thanks for reminding me about the endian stuff...knew there was a way to do that, just forgot ... and the search for endian didn't turn up a function.)

So, save the value as a class or something ...?

Thanks.

Hummer1 (feeling like we are almost there.) 

[Hummer1]

The problem is that the data stream from file comes in as a byte oriented array...not a cluster as shown in the example you provided....I can successfully parse the data using the cluster type def, and then loading it with the parsed data as shown in the first example programs I sent...that works fine.  What I can't do is to find a way to reduce all that to use the Variant to Data conversion.  Converting the input file (byte binary) into a character (by flattening) doesn't seem to work...as it contains control characters and different type numeric representations that you have to tell it before it can unscramble the bytes....

bin->Variant to Data->Cluster would be ideal...if there was a way around the variable length character strings in the type def...

Interesting.

Hummer1

Thanks.