cRIO 9035 reboots without visible reason

[Gerd_Bauer]

Hi to all, 

We use several cRIO 9035 systems in a laboratory - which are running fine for weeks, but sporadically do a reeboot without any visible reason. 

There is nothing suspicious in the system log viewer (web frontend) and in the error logs (MAX), the only entries found are related to the boot process - no hint why the system was restarted. 

The systems are based on LabVIEW 2018 / RT 18.0. The application is running a scanned 128 channel data acquisition with 10ms loop time, range checking, alarming, control loops, datalogging to a SD card (EXT4) and a HMI. The HMI is accessed via remote panel - without connection to the DP video output.

I am logging the cpu load which is constantly about 50% user / 20% system / 0% nic / 30% idle

and also the RAM status which is somewhat like 765 MB used / 138 MB free / 28MB shrd / 172 MB buff / 305 MB cache

The reboots occured totally independently on all three systems we have... any proposal where to start searching next?

Thank you for your ideas...

Gerd

[GuilleCad]

Have you checked the temperature of the cRIO?

[Gerd_Bauer]

Hi GuilleCad,

I did - but just by touching the cooler - around 40°C I would say... 

Anyway this is a good hint, I will also add the internal temperature to my logdata then.

Regards

Gerd 

[Gerd_Bauer]

Hi there,

We still see out cRIO 9035 reeboting from time to time without any obvious reason.
Meanwhile I added logging the cpu load and memory usage (via linux shell command) and also the backplane temperature (FPGA node). 

The system cpu load varies between 60-80%, sometimes goig up to 90%, shortly before the last reboots it was at 55..63%.
The memory usage was 620k..720k "used" with at least "120k" free.
The system temperature is constant around 38°C.

Any idea where RTLinux is storing additional debugging information?
The system logs just show the messages that are generated during the reboot - but no message for the time short before...

would be happy for any hint

Gerd