Parsing binary file bit-wise?

[poemanji]

Dear forum members and mods,

I'm trying to modify/regenerate a configuration file that is in binary.  So far I can parse a few pieces of data, but there appears to be a 1 bit offset that disrupts byte-wise parsing.  That is, a two-byte unsigned long number is offset by a bit so it doesn't fall neatly on two blocks of bytes.  I think booleans stored as single bits are causing the offset.

For example, what should be "00 5e" in hex binary shows up in the file as "00 05 e0" (the bordering zeros belong to some other pieces of data).

The file was generated in labVIEW as a dump of nine controls of various data types, and the data types and binary file format were chosen to minimize the number of bits used to describe the information.  Normally I would contact the LV programmer, but that is not currently an option because the code used to generate the config file is proprietary and the programmer is not available.  We are licensed to use the software and create our own config files, but this is currently done by row-by-row editing of an array of clusters in a fully-built LV application, and that takes forever.  Plus I'll have to do it many times as we change parameters.

The configuration file consists of 100+ rows of 9 items each.  If I can "crack the code" of how each control is represented (double, boolean, unsigned long, etc) then I could write my own config files to drastically speed up our workflow.

Binary file I/O certainly isn't my area of expertise, so I apologize if what I'm asking is obviously flawed somehow.  I've searched the forums but haven't been able to get it to work for LabVIEW 8.2.  Any help would be much appreciated!!

Thanks,

Brian

[nathand]

---

Poemanji wrote:

I'm trying to modify/regenerate a configuration file that is in binary.  So far I can parse a few pieces of data, but there appears to be a 1 bit offset that disrupts byte-wise parsing.  That is, a two-byte unsigned long number is offset by a bit so it doesn't fall neatly on two blocks of bytes.  I think booleans stored as single bits are causing the offset.

For example, what should be "00 5e" in hex binary shows up in the file as "00 05 e0" (the bordering zeros belong to some other pieces of data).

---

Booleans are represented by bytes, not bits, so that's not the problem.  Also the "shift" that you're showing is 4 bits, because each pair of characters represents a single byte.

Can you share any code, or at least a copy of the cluster?  What data type do think is encoded in that hex example?

[poemanji]

Thanks for your reply, and the insight on the bit offset!  Is there a way to shift the file position by 4 bits?

Unfortunately, I won't have access to the LabVIEW code, or even the front panel of the vi.   Also, the context help for the built application doesn't reveal the representation type of the control.

[nathand]

I'm a bit confused.  What information about the data do you have?  How do you know it's off by 4 bits?  Or, how do you know that what you're seeing is not correct?  What are you using to read the binary file?  When you wrote that "what should be 00 5e..." how are you determing what it should be, and in what program does it show up as "00 05 e0"?  If that program is your own LabVIEW program that you're writing to edit the file, can you share that code, along with a copy of the file you're attempting to modify?

[Darin.K]

---

@poemanji wrote:

Is there a way to shift the file position by 4 bits?

---

On the surface this looks like a job for Type Cast Man.

Part of me wonders if that is really what is going on since it would be pretty tricky to do that type of shift in the original data.  But if it works, it works.

[Ben]

---

@poemanji wrote:

Thanks for your reply, and the insight on the bit offset!  Is there a way to shift the file position by 4 bits?

Unfortunately, I won't have access to the LabVIEW code, or even the front panel of the vi.   Also, the context help for the built application doesn't reveal the representation type of the control.

---

can you get a screen shot of the FP where the edits are supposed to be done?

WIth the right cluster (mathes that used in the app) you should be able to open the file and just read the array of clusters.

Ben

[poemanji]

Interesting idea about reading directly to a cluster.  If I had the cluster of the right variables, is it straightforward to code reading from the file into the cluster?

Thanks for your help!

[Ben]

open as a data log file with an array of the cluster as the prototype.

of course if it isn't the cluster type you specify LV will probably crash with an out of memory error.

This link re memory and how data is stored may help.

Ben

[poemanji]

Thanks for all the great suggestions everyone

Here are screenshots from a program called HexEditor that I used to view the configuration files.  The first image shows one value highlighted as a double that should be 60000, and then the 4 bit shifted unsigned integer that should be 94.  My hands are tied in that I can't post the config file itself and I don't have (and couldn't post) the LV application itself.  I could post my LV app to decipher it, but honestly it's a mess and work in progress!  (It's funny how quick I've gotten used to the clean up feature in LV 2010 versus LV 8.2)

It doesn't appear to be a datalog because it's missing the "DTLG" string at the beginning, and the structure seems different.  The configurations are meant to be deployed to microcontroller hardware, so maybe the programmer took extra steps to encode the info in as few bits as possible. That link on how LV stores data should be very helpful.  I'll try my luck with the typecast and boolean array approach.

I won't be working with the built LV application and these files until sometime next week, but when I do I think you guys have prepared me to decipher and regenerate the config file.  Thanks again!

-Brian

[Ben]

When you "crack the code" please post an update for those that follow in your path. I'm not asking for a formal presentation. Just notes that you wish we had offered. Thank you for the update, Ben