cRIO 9040 crash

[aVoil]

Hello everyone,

I experienced crashes with several cRIO 9040

I'm using (almost) the same RT application on 4 cRIOs, all running on Linux, to acquire and process data

One of the cRIO works fine, the other 3 crash after some random time (10min to 40hrs +), leaving the cRIOs completely frozen.

In my Labview program, I have set up an error log but when the crashes occurs, nothing is written on it, even after restarting the cRIO

Is there a way to have an error log that's linked directly to the cRIO instead of my program?

Thank you in advance for your time

[ZYOng]

From my experience, the most common cause of the cRIO crashing is the memory leak, usually from File I/O.

I would recommend Monitoring CPU and Memory Usage on Real-Time Embedded Targets and see if the memory is growing. Then try disabling some parts of the codes to locate the source of the memory leak.

[aVoil]

Hi ZYOng,

Thanks for your answer, we've been monitoring our CPU/Memory usage without any success : 

When the cRIO crashes, the HMI shows that both CPUs are below 60% usage and Memory usage is below 70%.

On one of our cRIO, we've activated FPGA in order to check the cRIO's temperature. With this activated CPUs usage were below 5% on the time of the crash, physical and virtual memory was below 60 % and the cRIO was arount 39°C.

We've isolated the problem to the loop controlling the shown data in the HMI, which happens every 250ms. These datas are identical between cRIOs, and so is the loop. But the problem is that one of our cRIO works fine, while 3 others don't.

[AndyClegg]

Hi - is possible/easy to physically swap the working cRIO with one of the others, just as a check to see if it is the actual cRIO that is problem, or if it is more to do with the location (power, interference, vibration, etc). Might not be anything, but might give a clue.

[aVoil]

Hello Andy,

We finally found an old working cRIO to replace one that has been crashing, investigation is still going.

All of our cRIOs are :

 - In the same room, less than 5m apart from one another

 - Connected to the same network & electrical cabinet

 - In an "vibration free" zone (our machinery does not induce any notable vibration)