cRIO disconnects from Ethernet network and stops responding to pings

[Maurice_01]

I have a similar issue. I thought I fixed it, but unfortunately I didn't. I have some TCP/IP communication between a Crio and a Win2000 FTP server at a rate of 25kb/200 ms. My data transfer is between 100 kb and 7Mb per file. I placed my TCP/IP functions in a normal while loop and NOT in a i timed loop. I notice the following behaviour:

NI-Rio 3.1.0

- With the Crio's only transferring the 100 kB files- no problems

- With the Crio's transferring both the 100 kb files- and the 7Mb files I first get problems. The crios start with an error 56 and endup with an error 42, (while all other Crio's can connect with the FTP server)

- A crio with this error is ping-able, but otherwise not approachable in the project window.

- A crio with this error does however still service the Shared Variables.

After upgrading to NI-RIO 3.2.1 I noticed the following:

- A Crio with this error became totally unaproachable. Shared Variables also unresponsive

My best guess is that it got something to do with the load of the Crio.  I recently reduced the datarate, but increased the looptime from 100 kB/500 ms to 25kB/250ms. The problem now shows itself more quickly. (day range instead of week range). Unfortunately I still have no clue what the problem could be and at NI they also don't have any clue. Anybody have a good suggestion?

[RichardJennings]

I realize this is an old topic, but it seems the best place to post. We went through a lot of the same issues when using tcp/ip on RT platforms. After a while the system would become unresponsive at the ethernet port even though the rest of the app would continue running. The cRIO or RT PXI chassis needed rebooting in order to recover.

After a lot of pain and talks with NI Platinum support we found that the issue was caused by running out of tcp/ip sockets. The RT platforms, both Pharlap and VxWorks, have a limit of 64 simultaneous tcp/ip sockets. The problem is compounded by the tcp/ip stack holding on to any closed socket for 60 sec after closure.

In our application we had an RT PXI chassis with 14 simultaneous tcp/ip connections. We worked around the issue by limiting reconnect attempts only after a 60 sec delay. This solved the issue in the short term, but for the long term we had to move to UDP socket-less communication to our distributed cRIO targets.
Hope this helps,

Richard

[Maurice_01]

Related to this problem and Crio disconnecting from the network:

- Do NOT use any TCP/IP functions inside a timed loop on an RT platform

- Do NOT use any FILE/IO functions inside a timed loop on an RT platform

(using file IO functions will result in the same kind of behaviour that will crash your application)

Be carefull when using shared variables in timed loops. If you need to exchange data between loops, preferably use FIFO que's or global variables. This is however not based on anything, but my experience is that using FIFO's and globals is very stable.

Kind regards,

Maurice

[pseudocode]

Hello, Maurice_01

I have same problems on my sbRIO9602 and cRIO9004. thank you for your kindness.

In my design I put some file logger in Timed loop and TCP connection in normal while loop.  While loop transfers data to timed loop through FIFO. In this case, TCP communication crashes after some time with error 42.

Here's some question about your advice.

1. You mentioned that do not place file IO in timed loop not to cause crash. 

  Does that crash TCP open function?   or it causes another kind of 42 error?

[pseudocode]

Hello, Richard

Your advice is very helpful to me. That problem has given deep pain to my team.

Would you mind explain more about 60s delayed TCP connection? What does mean "short term"?

I think 60s delayed TCP connection might be nice solution to RT if your explanation is right. Is there another problem in it?

I want to find TCP solution because I cannot choose UDP as an alternative.

[Thør]

It seems like the 60s delay allows you to recoup sockets that were being held onto.  Off the top of my head I can't think of any other problems introduced by this method, especially if you require TCP.

[craige]

RichardJennings, would you be willing to post your Application Support Reference Number for so that other AEs can locate the solution to this problem? I'm experiencing identical problems with PXI-1031, 8108 controller running LV-RT2010sp1, NI-RIO 12.0, Industrial EtherCAT 2.0.

It amazing how rampant this issue is. I'm wondering if this is the cause of all my headaches for the past 2 years.