Read and Write BLF Files

[Hooovahh]

Description

Description-Separate-1

Overview

Read and write BLF binary files using the Vector provided DLL, or a more Raw API using the zLib Inflate/Deflate. 

Description

Vector has a custom binary file format used primarily for logging raw CAN data.  When using a Vector product such as CANalyzer, a log can be done in ASCII which baloons the file size but has the most compatibility, or you can log in BLF which is a small binary file, which also has some level of object compression built in.  With this API you can now read these BLF files and get the raw CAN frames in a form that is similar to the XNet frame structure, allowing for easy conversion using the XNet API.  You can also use this API to perform the reverse function.  Where you may want to log into a BLF file, so that Vector tools like CANalyzer can analyze and view your data.

All the core functions are implemented by calling the Vector provided DLL, which is documented online and has a shipped example installed with CANalyzer.  There is an alternate API that also allows for reading and writing BLF files by not using the Vector DLL and instead uses the OpenG zip utility.

Hardware and Software Requirements

Back saved into LabVIEW 2014 (version 2) or LabVIEW 2017 for version 3, and requires the Windows platform for executing the binlog.dll.  Version 8 is saved in 2018 and has an example of both the DLL version and the Raw version which requires the OpenG zip package.  The OpenG zip method works on all platforms that API supports, and is much faster than the DLL method.

Steps to Implement or Execute Code

The external dependencies are referenced with the Package Dependency.vipc file.  Once those packages are installed through VIPM, open either of the examples and run them to write and read a file.

Additional Information or References

Version History

Version 8

• Raw Open has Overwrite input
• Raw added Polymorphic Write CAN Data, that can write a single frame, multiple frames to the same channel, or multiple frames to different channels.
• Raw bug fix with ID mask of standard CAN frames when reading.

Version 7

• Bug fix with uncommon object types
• Added IDs To Return filtering on the Raw Read function

Version 6

• Made reading more robust supporting other objects like System Variables in the BLF.

Version 5

• Added a secondary API for reading and writing using the OpenG zip utilities.
• Added reading and writing CAN FD frames

Version 4

• When writing to a BLF the day light savings flag is used and will offset the write by an hour if necessary
• Object Offset was removed and instead before every read the Object Offset is read from the session.  This is used instead of tracking the current offset since it had issues with newer CANalyzer files

Version 3

• Bug fix with the Read CAN Frames.vi which now accepts reading CAN_MESSAGE2 which newer Vector tools may write to
• Fixed Read CAN Frame.vi to return an ID masking out the 29 bits for extended, not 27

Version 2

• Back saved to 2014
• Added CAN Channel input and output to the Read and Write functions

Description-Separate-2

[HNCW]

Hi Hooovahh,

I have been using your API to create blf files from my tests.

Recently I am having trouble opening some blf files on CANoe 10 SP7, the issue is pretty erratic. No major software change has been done to the software or the unit being tested.

When I compared the blf files of a readable unit vs a corrupt file, I am able to find out that the corrupt file had the header info missing. 
Could you help me read these corrupt files and also solve this issue from happening the future.

Thank you.

[Hooovahh]

A BLF file can store more than just CAN Frames.  In the code when we read CAN frames you see what we really are reading are objects.  And to read an object you must know the expected size of the object.  A normal CAN 2.0 frame is so large, and a CAN FD frame is larger.  But you can also store variables, and event triggers, and all kinds of random things and I couldn't know how big those other objects would be.  So in an older version of the code I just had a constant in there.  If you come across an object that is of an unknown type, just hope it is X amount of bytes in size, then try to read the next object.  This has obvious limitations.  So in a private edit of the code that I haven't released yet it looks at the object and tries to find the next instance of the header, which should be the start of the next object.  I'll be posting an update to this some time that hopefully fixes that limitation.

[Hooovahh]

Actually looking at the code posted in version 8 it should handle this already.  You can see this in the Get CAN Frames From Uncompressed Data.  We read the Object type, and a type of 72 is a System Variable, 1 is CAN 2.0 frame, 101 is FD frame, and the default should work as I described looking for the next header.  Are you having issues with the newest post I made?  Are you able to share the BLF?

[luckyasd12]

Hi Hooovahh,

I want to convert the TMDS file of the LIN message into a blf file. I think I need to modify the binlog. dll to adapt to LIN message. CAN you give me some guidance？

Thank you very much!

[Hooovahh]

I don't have any experience with this, and I didn't know that LIN data could actually be stored in a BLF.  I also don't have any LIN hardware to test with, or any sample TDMS files with LIN data in it.  Sorry.

[JoePerkins]

Is there a LV2017-compatible version 8?

[Hooovahh]

I've uploaded a file for 2017 that is version 8 which was back saved from 2018.  I don't have 2017 on any of my development machines at the moment so I can't actually test it.

[yshiv]

Hi Hooovahh,

I started to write application to parse the BLF file which shall contain CAN and ETH messages.

For CAN messages, it works like a charm.. Kudos to you!

when it comes to ETH messages, there is no implementation and started to define the message structure; but LabVIEW crashes the moment it tries to parse the ETH object. attached the type def that I created based on the .h file snippet here. 

Do you think I should create a cluster with 1600 u8 numeric? or how to handle the "BL_LPBYTE" data type?

typedef struct VBLEthernetFrame_t
{
    VBLObjectHeader mHeader;                     /* object header - NOTE! set the object size to*/
                                                 /* mHeader.mObjectSize = sizeof( VBLEthernetFrame) + mPayLoadLength; */
    BYTE            mSourceAddress[6];
    WORD            mChannel;
    BYTE            mDestinationAddress[6];
    WORD            mDir;                        /* Direction flag: 0=Rx, 1=Tx, 2=TxRq */
    WORD            mType;
    WORD            mTPID;
    WORD            mTCI;
    WORD            mPayLoadLength;              /* Number of valid mPayLoad bytes */
    BL_LPBYTE       mPayLoad;                    /* Max 1582 (1600 packet length - 18 header) data bytes per frame  */
} VBLEthernetFrame;

Please share any feedbacks or ideas. 

Thanks,

yshiv

[yshiv]

I am not able to attache the ctl.. so here is the snapshot of the same,

thanks,

yshiv

[Hooovahh]

Sorry I don't have any experience with this data type, or Ethernet IP in general.  Getting DLL definitions to be correct is always a pain and usually involves lots of crashing until it works.  I can't say for sure but aren't you missing the last piece of data the BL_LPBYTE mPayload?  When I wrote this API it was a lot of guessing and checking until it worked.  As for the actual data isn't that just an array of bytes?

[yshiv]

Sure, I agree with you about the datatypes creation for DLL, unless the DLL creator has a very good documentation. 

However, i was able to create the typedef for the ETH frame. The update made was creating a cluster of 1600 elements of u8 for the payload part of the class. 

The blow is the code snippet and the ETH type def.

Now no more crashes and still validating the packets populated. I shall keep this thread updated about any further development. 

[AGJ_CR]

I am using the 2018 version 8 attached in here. There seems to be a problem with the fractional seconds read from BLF files.

The issue can be easily reproduced with the Example VI. Just write one frame and read it back, there is an error in the fractional portion of the timestamp (100ms to 600ms).

The problem seems to be in the VI below. It incorrectly starts reading the bytes one position earlier than expected, versus how it was written.

Original code:

Fixed code:

[Hooovahh]

Great catch, thank you.  I won't be publishing a new release here for this minor fix.  It is pretty inconsequential, since this is reading the header, and the time between frames is still the same.  This is just a start offset issue.  Also I don't have access to LabVIEW 2018, so the earliest the fix would be available in is 2020.  Still I will include the fix in future releases of the CAN File IO package available online.

[Racim1875]

Hi Hooovahh,

I’m facing an issue with a similar application to analyse the CAN frames caming frome a bench When. I use the generated BLF file in CANalyzer to analyze several hours of data, the file only contains a few milliseconds of data and has no end time.

It seems like the file might be getting corrupted during generation. I suspect that something is going wrong during the writing or flushing process.

Do you have any suggestions on what might be causing this, or how I could ensure the BLF file is properly finalized? have you an idea?

[Hooovahh]

Are you able to write, and read a small amount of data?  Is there a size that it breaks on?  I suspect it should be fine for quite a large size using the OpenG zip functions but I don't know. I've always tried breaking up my files into smaller ones around 100MB or so. Do you have some example code that creates a bad file?

[Racim1875]

Hello Hooovah,

Yes, I am able to read and write LIN data without any issues. However, as soon as I start the acquisition of the CAN FD network, no further data appears to be saved in the .blf file — even though the file still grows to around 100 MB.

When I convert the blf to ASCII format, the resulting file contains only about 127 KB of data. This leads me to believe that the file may be getting corrupted after the CAN FD acquisition starts, possibly due to an issue in the code.

Unfortunately, I'm not sure how to verify or resolve this. Do you have any suggestions or troubleshooting steps I could follow to diagnose the problem?

The Code is the same as yours,  Can you tell me which part of code i must check or vérify?
If you’d like, I can share the entire project VI with you by email.

[Michele__]

Hello Hooovahh,

I inherited an old labview code where CAN frame was logged into BLF file using your API

While I was debugging the code I notice that CAN frame with echo were not logged, despite these frame were passed to Write CAN Frames.vi

Opening BLF file with external toolkit, or reading BLF File using your API Read CAN Frames.vi, I can not find raw frames with echo? = True

        

While I was debugging this issue, going inside your API, I have thought to download the latest version of your APIs from this NI Example page and this solve the issue. With the latest version, raw frame with Echo? = True are logged

Now I'm asking if this is something that you know, and you fixed.

Thanks for your support

Best regards

Michele

[Hooovahh]

Hey Michele, sorry I don't know anything specifically about the echo feature. I downloaded version 2, and version 8 from this post here, and both do take the Echo value and store it, but I do see that the raw method writes it as a byte flag. I also see for FD it is written as a Direction, and not just a flag.  I guess what I'm trying to say with this is that I don't have any thing special in regards to the echo.  If you do discover I'm doing something wrong I'll be glad to update it.  But if the newer release does what you want then that's good.