Compact RIO Crashing On Boot Up

[impala454]

Hello, I have a Compact RIO 9025 which is crashing on boot up.  The cRIO reboots a couple of times to try again but ends up in safe mode.  If we "catch" the cRIO at just the right time during the boot up, either via the web server or connect with a PC based LabVIEW application, it does not crash.  Version details in the log below.  Any thoughts?  I've searched "DataSpace.cpp" and the first call of the stack trace "ThSleep" extensively to no avail.

                            cRIO-9025 Boot


Copyright 2008-2009 National Instruments Corporation





Bootrom version: 2.1.2f0
Creation date: Nov 25 2009, 08:11:35




Press any key to stop auto-boot...
 0
auto-booting...


boot device          : sata=0,0
unit number          : 0
processor number     : 0
host name            : lvrt
file name            : /c/ni-rt/system/vxWorks
flags (f)            : 0x8

Attaching to SATA disk device...
SATA0: SST NANDrive
Reliance File System Driver
Datalight Reliance v3.2.2 Build 1376BV
VxWorks Edition for ppc603
Copyright (c) 2003-2008 Datalight, Inc.  All Rights Reserved Worldwide.
Loading /c/ni-rt/system/vxWorks...3066656
Starting at 0x10000...

SATA0: SST NANDrive
Reliance File System Driver
Datalight Reliance v3.2.2 Build 1376BV
VxWorks Edition for ppc603
Copyright (c) 2003-2008 Datalight, Inc.  All Rights Reserved Worldwide.

Adding 8002 symbols for standalone.

-> * Loading vxfpsup.out: vxfpsup
lvusEngine: PPC603 CPU detected...
CPU tick frequency: 98.997174 MHz [Using: 1000 MHz]
MAX system identification name: Subscale-Spare2
Initializing network...
Device 1 - MAC addr: 00:80:2F:15:7B:9C - 169.254.17.41 /24 (primary - static)
Device 2 - MAC addr: 00:80:2F:15:7B:9D - 169.254.17.42 /24 (static)
* Loading nimdnsResponder.out: nimdnsResponder
Starting mDNSResponder task tMDNS.
* Loading nirtdm.out: nirtdm

Loading LVRT...
* Loading nimdnsResponder.out: nimdnsResponder
* Loading nirtdm.out: nirtdm
* Loading vxfpsup.out: vxfpsup
* Loading tsengine.out: tsengine
Time sync source: ptp now active
* Loading nisvcloc.out: nisvcloc
* Loading mxssvr.out: mxssvr
* Loading taggerrt.out: taggerrt
* Loading nisysapirpc.out: nisysapirpc
* Loading NiViSrvr.out: NiViSrvr
* Loading niserial.out: niserial
* Loading nidp_target_end.out: nidp_target_end

NI-VISA Server 5.0 started successfully.
NI Scan Engine initialized.
* Loading NiRioRpc.out: NiRioRpc
* Loading niorbs.out: niorbs
* Loading sysstatepublisher.out: sysstatepublisher

Startup Application: /c/ni-rt/startup/startup.rtexe
NI-RIO Server 3.6 started successfully.

Welcome to LabVIEW Real-Time 9.0.1
Error Message =Internal Error : "DataSpace.cpp", line 368
LabVIEW version 9.0.1
For assistance in resolving this problem, please exit and relaunch LabVIEW, or contact National Instruments.
*** BEGIN SYSTEM EXCEPTION LOG ***

Target type: cRIO-9025
Target code: 7459

System time (UTC): 2016-06-27 14:28:28
System tick count: 20351 ms

Exception code: 0x00000700

Register contents:
DAR = 0x00000000  DSISR = 0x00000000
MSR = 0x0008B032  FPCSR = 0x82008000
 LR = 0x0191A49C    CTR = 0x00000000
 CR = 0x00000000    XER = 0x2000AF48

GPR  0 = 0x00000000    GPR  1 = 0x03616ab0
GPR  2 = 0x00000000    GPR  3 = 0x00000000
GPR  4 = 0xFFFFFFFF    GPR  5 = 0x00000000
GPR  6 = 0x00000000    GPR  7 = 0x03617750
GPR  8 = 0x074C0FDC    GPR  9 = 0xfc45f868
GPR 10 = 0x03616360    GPR 11 = 0x001f0000
GPR 12 = 0x24000442    GPR 13 = 0x00000000
GPR 14 = 0x00000000    GPR 15 = 0x00010288
GPR 16 = 0x036AE1C0    GPR 17 = 0x00000000
GPR 18 = 0x00029248    GPR 19 = 0x036d7408
GPR 20 = 0x00000000    GPR 21 = 0x00000000
GPR 22 = 0x036BCA80    GPR 23 = 0x02010000
GPR 24 = 0x02010000    GPR 25 = 0x036b5d64
GPR 26 = 0x00000404    GPR 27 = 0x02000000
GPR 28 = 0x02000000    GPR 29 = 0x00000001
GPR 30 = 0x03616AC0    GPR 31 = 0x03616cf8
 PC = 0x00000000 in module 0x0

Thread ID: 0x03617750   Thread name: LV TL[100]: L46144300
Thread stack base: 0x03617750  stack size: 131072

Call Stack:
0x1bd9484+0x74c: ThSleep () in module lvrt.out
0x1807c28+0x8dc: OnOccurrenceAndOccurAtTimeForExec () in module lvrt.out
0x1807600+0x2b4: OnOccurrenceAndOccurAtTimeForExec () in module lvrt.out
0x1728060+0x18784: Typeinfo name for MaDirHandler () in module lvrt.out
0x17281a0+0x188c4: Typeinfo name for MaDirHandler () in module lvrt.out
0x17fcf6c+0x0: WriteFlatHandleTransferData () in module lvrt.out
0x1b8ae4c+0xd878: CfgGetDefaultFrom () in module lvrt.out
0x1b8ac04+0xd630: CfgGetDefaultFrom () in module lvrt.out
0x191a4c4+0x4b0: GetDbgCache () in module lvrt.out
0x191a364+0x350: GetDbgCache () in module lvrt.out

All Loaded Modules:
        MODULE NAME     MODULE ID  TEXT START DATA START  BSS START
        --------------- ---------- ---------- ---------- ----------
        RawEthernet.out 0x045b9d48 0x045ba2d0 0000000000 0x045ba738
        HsPacketBuilder.out 0x04124900 0x0485b020 0x048b5a50 0x048b5ec0
              TgrDD.out 0x03057058 0x032ef0c0 0x032f9340 0x032f9920
        sysstatepublisher.out 0x010db318 0x032ae188 0x032d7200 0x032d9f00
             niorbs.out 0x02f75168 0x0301ac28 0x0304e610 0x030541b8
           NiRioRpc.out 0x010dd900 0x02f68ff8 0x02f74028 0x02f741e8
                mxs.out 0x030a8e40 0x02ed4240 0x02eeec40 0x02ef0ad0
        ni_tagger_plugin_mxs.out 0x010ea7c0 0x032fdf28 0x0337f0c8 0x03385ff8
             NiFpga.out 0x030ac2c8 0x02f07258 0x02f50380 0x02f50950
           lvalarms.out 0x02ea58c0 0x0305afc8 0x0308a900 0x0308ac88
             lvuste.out 0x010b1c08 0x0301a1f0 0x0301a448 0x0301a478
        nNIBlueBus_nCrioFixed_nRefnum.out 0x010ec1c0 0x02f75fd8 0x03009f78 0x03013160
             modbus.mnd 0x02c31db8 0x02d48b90 0x02de6a18 0x02def840
           epicssrv.mnd 0x02c105f0 0x030ad178 0x03239d68 0x03244f78
        sysadminsvc.out 0x02bd0610 0x02c36688 0x02c43ac8 0x02c44b40
            crio_ce.out 0x02a0c5e0 0x02df51c0 0x02e74c18 0x02e79780
        lvceconfig_handler.out 0x02a0da98 0x02bb1410 0x02bce5f0 0x02bcfdb8
           NiViAsrl.out 0x029f2770 0x029f5178 0x02a09d18 0x02a0a2d8
            NiViRpc.out 0x029d0d90 0x02bd4700 0x02c06f38 0x02c07fb0
        nilvicv_deployhandler.out 0x029d0800 0x02a0e3b8 0x02a1bb00 0x02a1c9e0
        nilvicv_runtime.out 0x029cd4c8 0x02c89ca0 0x02d3e818 0x02d489a8
            NiViUsb.out 0x029cc7e8 0x029a2c60 0x029b7e50 0x029b8678
           NiEnAsrl.out 0x029a26d8 0x029bf288 0x029cb3b0 0x029cba18
            NiViPxi.out 0x026ebb48 0x029d2ef0 0x029f1220 0x029f1d00
             nipals.out 0x027e82b8 0x02856d60 0x0290c328 0x0290c6d0
           NiViEnet.out 0x027e6980 0x02848b90 0x02856a40 0x02856ca8
        crio_expert.out 0x027050b8 0x027799e8 0x027deb10 0x027e4928
            NiViRio.out 0x02704400 0x026d52b0 0x026e0c30 0x026e3628
             bb_lib.out 0x02508590 0x02a28490 0x02b783e8 0x02b909f0
           nipspxts.out 0x02468fd0 0x027eefb8 0x02844008 0x02848968
             nilvce.out 0x02506a30 0x026827c8 0x026b5930 0x026b8638
           csserial.out 0x024672b8 0x027093c0 0x0275fa90 0x02765c48
        nidp_target_end.out 0x02244e80 0x023aea08 0x0243d090 0x02443a40
             visa32.out 0x02235010 0x02509798 0x025228b0 0x02522a20
           niserial.out 0x0233d8b0 0x0246ad58 0x024e9278 0x024e9328
           NiViSv32.out 0x02346310 0x02374f78 0x023842c8 0x02384e50
         ws_runtime.out 0x011e5770 0x0253ab58 0x02659d50 0x02663088
           NiViSrvr.out 0x011e5168 0x0234f5f0 0x02373d88 0x02374b70
            nilwpce.out 0x0212c890 0x021d4f88 0x02220040 0x02224538
        nisysapirpc.out 0x01146f60 0x0218a138 0x021b6d18 0x021b8b70
        libcopyModule.out 0x01155170 0x01155fb8 0000000000 0000000000
        ni_traceengine.out 0x011435f8 0x0212d050 0x02147028 0x02148d78
           nisysapi.out 0x0170bdc0 0x011650b0 0x011cdd08 0x011d1318
        libdirModule.out 0x0170b838 0x0170c4b0 0000000000 0000000000
           nilxtcor.out 0x01726208 0x02246298 0x02314e28 0x0231c9a0
        libespModule.out 0x01727920 0x01158040 0000000000 0x01164ff8
           taggerrt.out 0x0112b6b8 0x015414b8 0x016e5798 0x016f80f8
           niwebsrv.out 0x01118148 0x01118710 0x01119020 0x01119048
            logosrt.out 0x010fde90 0x0137ad18 0x0152d358 0x0153ef80
        lvrfpmodule.out 0x010fd908 0x01109d98 0x0110aea8 0x0110aed0
        lvauthmodule.out 0x010fc460 0x010fcd50 0x010fd8c0 0x010fd8e8
             mxssvr.out 0x00bbcc28 0x00be3228 0x00c5f848 0x00c645b0
              mxsin.out 0x00bbb388 0x011e7bf0 0x0136ec50 0x01379098
              mxsdd.out 0x00af9378 0x00af9af0 0x00b08f20 0x00b09768
              mxsdb.out 0x020e8330 0x00ee2900 0x00fc8348 0x00fcea28
           mxsutils.out 0x020499c0 0x00b5bd00 0x00bab600 0x00baec08
           nisvcloc.out 0x00a2c920 0x00add1a0 0x00ae32a0 0x00ae3330
             ts_rtc.out 0x020ec390 0x020356e8 0x02048168 0x02048990
             ts_ptp.out 0x00a4abf8 0x02076610 0x020e60d8 0x020e6a20
           tsengine.out 0x00a44550 0x0204c710 0x020755f8 0x02076518
           NiRioSrv.out 0x00a0e5a0 0x00c6c410 0x00d74e10 0x00d75af8
             nirpcs.out 0x00a0d568 0x00a309c0 0x00a433d8 0x00a437b0
               lvrt.out 0x009d8e00 0x01728060 0x01f9aab0 0x01ff7320
          libappweb.out 0x009d8878 0x00a8b9a0 0x00ad3880 0x00ad3d78
           niCPULib.out 0x009b82a0 0x009bd358 0x009d83c8 0x009d8670
              nipci.out 0x009b7b38 0x009bac48 0x009bd178 0x009bd210
           libexpat.out 0x009b7110 0x009ef000 0000000000 0x00a0ce90
             nirtdm.out 0x0086fbe0 0x00870160 0x00879880 0x008799a8
           libiconv.out 0x008837e8 0x007c4c18 0x007d0cf0 0x007d0d18
           ftpserve.out 0x00855748 0x00972c38 0x00982e90 0x009839c0
▒       nimdnsResponder.out 0x007f5238 0x007f57c8 0x00824918 0x00824980

[Call.Me.Wilbur]

Unfortunatly, that log doesn't narrow things down too much. Has this behavior happened consistently before or is this a new occurance? If it worked fine in the past were there any changes in the interim that could have caused this?

Have you tried formatting the RIO and reinstalling software? If the issue is a software corruption this may clear it.