cRIO-9012 crashes by continuesly DMA.Read() with no data after some long time

[unterberg@greylogix.de]

(german)

Guten Tag,

unser cRIO-9012 stürzt nach einigen Studen/Tagen ab. Software: LabView 8.6, NI-RIO 3.0.0 NI-VISA 4.4

Unsere Anwendung betriebt in einer Endlosschleife einen DMA-FIFO.Read Aufruf.

Wenn für sehr lange Zeit keine Daten im DMA-FIFO zum auslesen zur Verfügung stehen, stürzt die Software ab und zwar um so früher, je kleiner der Wert für Timeout gesetzt ist.

Der Absturz macht sich in sofern bemerkar, dass die angesteuerte Status-LED aufhört zu blinken.

Wird das System im "Beobachtungs-Modus" (also durch Ausführen direkt aus der Entwicklungsumgebung) gestartet,konnte noch kein Absturz erkannt werden.

Meine Frage:

Unter Verwendung eiens DMA-FIFOs zum Datenaustausch zwischen FPGA und cRIO, kann es da zu einem Fehlverhalten kommen, wenn DMA-FIFO.Read in einer Endlosschleife aufgerufen wird, ohne das Daten im FOFO zur Verfügung stehen?

Spielt hier das Timeout eine große Rolle?

Vielen Dank im voraus.

 Torsten Unterberg

****** ENGLISH ********

Hi,

 have a problem running the DMA-FIFO.Read mehtod inside an endless loop, when the DMA-FIFO has no data to read out.

According to the Timeout setting the cRIO will just freeze. The lower the timeout is, that faster the cRIO will freez. (2 hours ... 15 days)

Any ideas?

Thanks in advance

Torsten Unterberg

[J.Harv]

Torsten--

        I haven't seen any behavior like you described.  I'd be interested to see what your code looks like. Is it possible for you to make a small example that shows the problem, or at least post your code?   Thanks.

Regards,

John

[unterberg@greylogix.de]

Hi,

sorry for the German. I thought this would turn out as a standard email request, and not get posted in an English forum.

 Since my cRIO now is in an longtime test (checking out the freeze) I'm not able to build a example vi for demonstration. Further, maybe the hole project is mandatory for the freeze. So I attached the hole project.

Have a look at The "cRIO Read FPGA Write RS485.vi". There are 3 continuously loops.

1) Reading the DMA-FIFO from the FPGA. Waiting For Data...

2) Exchanging parameters with the FPGA.

3) Sending Data via RS232 (and to DMA-FPGA via RS485)

In the loop 1) there is a FPGA.Read which I was talking about before. Changing its Timeout parameter to a low number (10) lets the cRIO freeze within 2 hours. Changing it to 5000, lets the cRIO freeze in 8 hours. At the moment I'm running a test with -1 (infinite) since 48 hours without freezing the cRIO.

When I talk about freezing the cRIO, I mean that there is no traffic anymore on the RS232. In the beginning I had the status led on during transition. Now to check if cRIO is working, I let the status led flash inside the loop 3)

so when the cRIO "freezes" I mean that I do not see the led falls anymore.

Hope u get my point.

Thanks

Torsten

[dittman]

Until yesterday I have had the same problem (lastly polling of two 9853 CAN recievers stopped to work out of the blue after ~11 minutes 40 seconds) ... Some strange behaviour a customer detected several months ago (which I workarounded by daily power cycling) is 'fixable', now.

LV 8.6.1 FPGA (I didn't try LV 2009) seems to not support infinite loops 😉 - after finding out this I consulted the LV help. I would have misunderstood "... saturates on reaching its maximum value ..." if I had read this ealrier, because I didn't need the loop index at all.

The working workaround is to nest the 'infinite' loops. You have to stop the nested loops at cycle (2^31)-2 so the surrounding one restarts it.

@NI:

There are some places within this forum where people do not see the requirement for loops that are:

- real infinite loops (every main loop on embedded devices)

- loops that stop _only_ on user determined conditions (for example for polling devices, especially more than one per loop)

Is a LV 8.6.1 patch available that prevents loop index counting code from beeing generated to the FPGA if it is not used in a partikular while loop struckture? (The nested loop aproach unfortunatelly costs execution time and FPGA space without any benefit.)

I didn't try it out but there are some NI FPGA examples shipped with the NI example finder that would work for a few minutes until reaching 2^31-1 cycles (at 40 MHz count rate it would be ~53.7 s). May be they should be tested/fixed.

Kind regards

Thank you very much

Manuel

[J.Harv]

Manuel-

    Thank you for posting and letting us know.  Do you think you can file a CAR about this?  That's the best way to make sure it's tracked and we do investigate this behavior with our examples. 

Regards,

John Harvey

NI-RIO PSE

[dittman]

Hello John,

could you declare what "CAR" means, please? In that case google was not my freind and searching this forum didn't help much, too.

A "CAR" seems to be something like RMA, I guess, but I was not able to find the place where it could be created.

Best regards

Manuel

[muks]

A car is a corrective action request generated/created by a ni rep after they are able to reproduce the behaviour.

[J.Harv]

It's just a request to have something done.  If you aren't familiar with how to create one, post back, I'm sure the NIC AE that gets this assigned to them would be happy to file it for you.

[Roy_F]

Manuel,

Sorry for the confusion. CARs are what we use within R&D to track bug reports. Users can't file these directly (although it is on our radar to get there someday). To report a bug to NI, go to NI.com, click Contact NI, then click Request Support and you are on your way.

A less direct way to report a bug is to start a discussion thread on the problem. The issue will be seen by an Applications Engineer (AE) monitoring the thread and if it is obvious to the AE that the thread is discussing a bug, they should file a CAR and post a response to the thread with the CAR number. When you see a CAR number for the issue, you then have assurance the problem was reported to R&D. If you don't see a CAR number posted to the thread, ask for one (or an explanation).

We appreciate the time it takes to report problems. The benefit  to users who report problems are numerous, not the least of which is helping others become aware of or avoid the problem and hopefully getting the problem corrected in a future release. Your immediate benefit might be that we can provide a workaround if you don't have one already or a better workaround if the one have is painful.

Thanks,

Roy

[dittman]

Hello Roy,

your reply is the first _really_ helpfull one, great. Thank you very much for not introducing further unknown (and unguessable!!! like 'CAR') abbreviations.

I wrote a very detailed problem report using the way you mentioned.

I will reply if anything relevant happens (mail by NI, a new thread, ...).

Best regards

Manuel