sbrio 9651 boot problem

[hepguven]

Hi,

We have an issue about sbrio 9651 boot unstability. When we power up the system approx. 1 of 4 boot hangs at different levels. 3 of 4 are successful.

We checked the "console out" and boot hanging levels mostly related on

1. "Started NI Started Variable Engine: done".

2. "Daemonizing... Writing process ID to "/var/run/runlvrt.pid"

Our configuration as follows:

- Labview 2020 SP1

- Labview 2020 SP1 Real-time

- NI Compact RIO 2022 Q3 drivers

- sbrio9651 firmware version: 22.8.0

- Startup application with Ethercat and network shared variables deployed.

- socketed clip is used.

Thank you.

[hepguven]

Hi again,

We narrow down the problem to EtherCAT slaves. When we unplug the EtherCAT connection, no boot problem occurs. After plugging, "i2c" timeout frequently pop-up, and occasionally boot hangs. Any help would be nice. Thank you.

Below is console out related with pluging the EtherCAT slaves (at this attempt, boot is finished):

Booting LabVIEW RT...
[ 2.047136] rtc-ds3232 0-0068: hctosys: unable to read the hardware clock
INIT: version 2.88 booting
Starting udev
[ 5.851158] ubi0 error: ubi_open_volume: cannot open device 0, volume 2, error -16
[ 5.901078] ubi1 error: ubi_open_volume: cannot open device 1, volume 0, error -16
hwclock: ioctl(RTC_RD_TIME) to /dev/rtc0 to read the time failed: Invalid argument
Next boot mode is runtime
vm.overcommit_ratio = 95
Starting rtctld: done
INIT: Entering runlevel: 5
Starting system message bus: dbus.
Starting NIAuth: done
Starting OpenBSD Secure Shell server: sshd
/etc/ssh/sshd_config line 93: Deprecated option UsePrivilegeSeparation
done.
Starting NI Service Locator: done
Starting nirioserver: done
* Starting Avahi mDNS/DNS-SD Daemon: avahi-daemon
Starting systemWebServer: done
doneting nimxs: [ OK ]
Starting lkads: done
Starting applicationWebServer: [ 25.430861] cdns-i2c e0004000.i2c: timeout waiting on completion
CC▒ _ _ ___ _ _ ____ _____
| \ | |_ _| | | (_)_ __ _ ___ __ | _ \_ _|
| \| || | | | | | '_ \| | | \ \/ / | |_) || |
| |\ || | | |___| | | | | |_| |> < | _ < | |
|_| \_|___| |_____|_|_| |_|\__,_/_/\_\ |_| \_\|_|

NI Linux Real-Time (run mode on NI-sbRIO-9651-0213ea7c)

Log in with your NI-Auth credentials.

Interface eth0 - MAC addr: 00:80:2f:36:67:77 - obtaining address (primary - auto)
Interface eth1 - MAC addr: 00:80:2f:36:67:78 - EtherCAT
Interface usb0 - MAC addr: 00:80:2f:36:67:79 - down (auto)

[ 82.470735] INFO: rcu_preempt self-detected stall on CPU
[ 82.470754] 0-...: (6000 ticks this GP) idle=b86/140000000000001/0 softirq=0/0 fqs=0
[ 82.470755] (t=6000 jiffies g=437 c=436 q=0)
[ 82.470773] rcu_preempt kthread starved for 6000 jiffies! g437 c436 f0x0 RCU_GP_WAIT_FQS(3) ->state=0x402 ->cpu=0
NI-sbRIO-9651-0213ea7c login: [ 87.910843] cdns-i2c e0004000.i2c: timeout waiting on completion
NI-Industrial Communications for EtherCAT 20.0 started successfully.
NI Scan Engine initialized.

Welcome to LabVIEW Real-Time 20.0

[hepguven]

Still facing the issue, any help would be great.

Thanks.

[hepguven · Accepted solution]

Hi again,

So we talked this issue about our local representative and they forward this issue to NI R&D. So right now, we solved the issue in other way. 2 messages ago we mention that if we unplug the etherCAT cable out, sbRIO boots normally. We go over this phenomena, and power up our related slaves after sbRIO boots up. This action, solved the booting and manage to connect all slaves as normal.

Beyond this solution, we will upgrade our all softwares, drivers and etc. to 2023 and we will check again. 

Thanks.