ncWriteMulti Issue

[Aid]

Hi,

     I have successfully developed a multi-threaded application using CVI 7.0 and four Series 2 High Speed PCI CAN cards running at 500K Baud that can flash via CAN upto 4 modules at the same time. When the application was first developed I employed two CAN cards to flash 2 modules each (one Module per Port), but experienced a number of timeout issues due to module apparently not responding. At the time I assumed that it was caused by a thread interaction issue in the driver and hence I decided to increase the number of cards from 2 to 4. This eliminated the majority of the timeout issues, however I still occasionally see an issue that causes the download to be aborted, when the module being flashed fails to acknowledge a data block. After some investigation I think that I can reproduce the issue on a more frequent basis and I would like to share this with you / get your comments.

Overview

The flash data is downloaded to the module in blocks of 2048 bytes.

Each CAN frame contains a sequential number followed by 7 bytes of data.

The CAN frames are constructed and passed to the CAN card using the ncWriteMulti function for immediate transmission (No Timestamp).

After every block, the module transmits back an acknowledgement frame.

When the issue occurs, the module fails to acknowledge the block.

Observations

If I run the application on a single module (CAN Card #1, Port 1), the issue will occur approximately 1 in 200 runs.

If I run the application on a single module (CAN Card #1, Port 1) and monitor it with NI MAX Bus Monitor (CAN Card #1, Port 2 (Listen Only)), the issue occurs approximately 1 in 7 runs.

If I run the application on a single module (CAN Card #1, Port 1) and monitor it with NI MAX Bus Monitor (CAN Card #2, Port 1 (Listen Only)), the issue occurs approximately 8 in 1 run.

After looking at the resulting monitor trace files I have found that when the module fails to respond, one or two of the CAN frames contain corrupted data. The frames after the corruption are correct. If the frames are passed again to the CAN card via the ncWriteMulti function the data is correct, implying that the corruption occurs withing the driver or on the CAN bus.

From the trials I have conducted it appears that this corruption is some how influenced by the use of the Bus Monitor and the CAN card being used. Since the monitor is not normally used I am also wondering if the issue can also be influenced by the use of multiple CAN Cards ?

Does anyone have any experience of this / know of a solution ?

Regards

[Tom Haggerty]

I have not seen this before but do not flash with NI Card.  I use Canalyzer -- although I would like to use NI in the future -- hence my interest!  When I flash it will be with one arbitration ID so I do not see the need for a write mult.  Also I will be flashing using ISO/DIS 15765-2 standard,  which requires a first frame followed by a flow control response and then send a batch of consecutive frames based on the directions (Block size and seperation time) given by the flow control frame.  Is your requirement similar?

Tom

[Aid]

Hi Tom,

The protocol I am using here is very similar, here's the basic sequence...

1). Create an array of NCTYPE_CAN_STRUCT * 512. (Note: 512 is the maximum number of frames)

2). Using the NCTYPE_CAN_STRUCT, create each frame in turn (Frame Type, Arbitration ID, Date Length, Data Bytes etc) and copy the structure into the next position in the buffer. If you have more than 512 frames, you will have to repeat this process again after the buffer has been transferred to the CAN card until all the frames have been processed.

At this point the buffer should contain x frames, ready for transmission.

3). Transmit the first frame to the module using ncWriteMulti, set the data size to (sizeof(NCTYPE_CAN_STRUCT)).

4). Wait for / receive the flow control frame.

5). Increment the buffer pointer to the second frame. (i.e. &TransmitBuffer[1])

6).Transmit the remaining frames to the module using ncWriteMulti, set the data size to (sizeof(NCTYPE_CAN_STRUCT)*Remaining Frames)

This method appears to have two major advantages over a For-Next loop using ncWrite to transmit a single frame at a time.

1). The PC processor usage is much lower, which is great for a multi threaded application.

2). The frames are transmitted much faster, resulting in a lower cycle time.

I hope this helps.

Aid 

[Aid]

Hi Tom,

             Apologies, after running some trials the following should read...

This method appears to have two major advantages over a For-Next loop using ncWrite to transmit a single frame at a time.

1). The PC processor usage is slightly lower, which is great for a multi threaded application.

2). The frames are transmitted slightly faster, resulting in a lower cycle time.

Of course this will also depend upon the application.

Aid

[Tom Haggerty]

The data gets "corrupted" -- does this happen on the same frames each time or is it a purely random event.  If it is random, I would check that there is a solid ground or reference and that the ECU that you are flashing is not trying to perform other tasks or that the bus voltage is not being loaded by something.  If it is not random but occurs at about the same time or frame sequence, then check your program thoroughly to ensure it is not caused by a buffer problem or frame "building" part of the program.  Other then that it sounds like maybe a bug when you try to fill the buffer to full.  Maybe try to reduce the amount of messages stored for writing -- this may slow the flash sequence but if it gets rid of the problem it may be worth it!

Sorry I don't have a definitive answer, but, again, I've not seen this problem before!

Tom

[Aid]

Thanks Tom,

                     I've performed some more investigations this morning, with the following results....

1). The corruption appears in random frames, typically a single frame in a block, but I have seen two consecutive frames in a block.

2).  If I send the buffer again without rebuilding it, the second attempt always works. Therefore it appears that the source buffer in my application does not contain the corruption.

3). I replaced the ncWriteMulti with a loop and ncWrite call to send single frames and observed a similar failure rate when the CAN Bus Monitor is active. Again if I resend the buffer without rebuilding it, the second attempt always works.

The really interesting bit is the influence of the Bus Monitor on the number of corrupted frames. The rate is significantly higher if I monitor the active CAN Port  with a CAN Port on another card in the same machine. Since the Bus Monitor is in "Listen Only" mode and the data doesn't appear to be corrupted on my side I am currently working on the assumption that the issue lies in the driver.

Aid

[Tom Haggerty]

I hate to bring this up but its worth checking...  since just connecting the monitor seems to add to the problem, it may be a termination issue.  So verify that you have the proper termination.  If the cable run is a short run, and all the nodes are terminated that may be a problem - only terminate the ends of the CAN node.  If the ECU side has a termination resistor and you have a short cable run try removing your termination resistor on the NI side, if the ECU side does not have a termination resistor try adding one to that side. 

You could be borderline with your termination and that could be causing the voltage level on the cable to be close to the ambiguous range.

[Aid]

Good point, however in my setup the monitor port is always physically connected. Furthermore the issue only seems to occur when I run the Bus Monitor software from the Measurement & Automation explorer.

Aid

[Tom Haggerty]

I think you've narrowed it down to a problem in the CAN driver for the CAN card -- but you knew this already!!!  Sounds like a bug.

Tom

[Aid]

Hi Tom

              It's taken a while for me to come to this conclusion, but I don't think that there's anything left that's in my control. Thanks for the help, your input helped me to clarify my thoughts.

Do you know how I can get NI to look into this issue ?

Aid