PCIe-1429 randome timeout errors

[ChrisAtKodak]

I'm using a National Instruments PCIe-1429 Camera Link frame grabber that gives timeout errors about 20% of the time whenever the transmit clock is stopped and restarted.

The problem I see occurs for both a 1-tap 16bit and 8-tap 8-bit camera configuration.  Whenever the camera changes modes the camera link transmit clock is momentarily interrupted and the PCIe-1429 fails to acquire an image 20% of the time.  

From the Measurement & Automation explorer it returns the error: "A timeout error occurred while waiting for the specified event.  If waiting for an image, verify that all video data is acquired within the timeout period.  If waiting for a signal, verify that the signal assertion occurs within the timeout period".

Now, I know my timing is correct because if I simply interrupt the transmit clock again the PCIe-1429 will begin to acquire images.  It will then always correctly acquire images until the next time the transmit clock is interrupted.  There is about a 20% probability of a timeout error if the transmit clock is interrupted.  It behaves like the PCIe-1429 phase lock loop on the camera link receiver has problems re-syncing.

Our camera (an internal Kodak built camera) uses the National Semiconductor DS90CR287 channel link transmitter.  When the PCIe-1429 gives a timeout error I check the DS90CR287 transmit clock on a oscilloscope and it always looks good.

Is this a known problem of the PCIe-1429 or a known problem of the DS90CR287 transmitter?

[Tejinder_G.]

Hi Chris,

I did not find any known issues documented regarding this.  A few questions:

1) What driver version are you using?

2) When you mention changing camera modes, are you referring changing the bit allocation configurations or something else?

[ChrisAtKodak]

Thankyou for your reply Tejinder G.

I am using NI-IMAQ 4.3

The camera mode change involves downloading a new program to an Altera FPGA.  It takes about 0.5 seconds to reprogram and start producing new transmit, line, and frame clocks to the camera link transmitter.  During download, the transmit, line, and frame clocks all stop in the low state.

The problem is occurs on all of our CCD camera test systems using the PCIe-1429.  It occurs using our software to acquire images and using NI software.  On the same cameras we can swap out the camera link connector board and use parallel LVDS with the NI-1424 frame grabber.  The NI-1424 always worked correctly.

Thanks,

Chris

[Daniel_W]

Do you know if the frame/line clocks complete before the FPGA is reprogrammed?  If LVAL or FVAL fell prematurely (before all of the line/frame was received), and the PCIe-1429 only saw a runt frame, I would expect the PCIe-1429 to respond with this timeout error.

Can you control where in the frame the FPGA is reprogrammed (such as programming the camera to only output frames when triggered, and then not sending a trigger for some time before changing modes)?

[ChrisAtKodak]

Daniel,

I altered my timing program to listen for a request from the computer and stop all timing only at the end of an image readout and then wait to be reprogrammed.  That prevents an interruption in the middle of an image.  I still get the timeout errors.

Note that once I get a timeout error they will keep occurring until the cameralink transmit clock is interrupted again. I would hope when issuing a grab command after a timeout, the NI device drivers reset internal line and frame clock counters.

Chris

[ChrisAtKodak · Accepted solution]

Apparently there is some bug in the DS90CR287 channel link serializer.  The phase lock loop sometimes fails to produce an output serial clock to the frame grabber with the proper duty cycle if the input clocks are interrupted.  A phase lock loop should be able to recover from that.  We noticed National Semiconductor changed their spec sheets to require power cycling the DS90CR287 after any of the input clocks are interrupted.  We implemented the power cycle procedure and now the timeout errors appear to have stopped.

All fixed!

[Daniel_W]

Chris,

It's good to hear you found a workaround.  Thanks for sharing your solution.

-Daniel