timed loop finishes late on RT

[soeren]

I have a CompactRIO controlling my system in the following manner:

-FPGA VI setup to configure inputs and outputs to different modules (9481 9401 9211 etc.)

-RT VI running on cRIO-9004. 

                --Timed Loop 1 : using inputs and outpus to FPGA to control system, communication with host VI with shared variables

                --Timed Loop 2 : Logging data from shared variables 

-Host VI running on Host computer exchanging shared variables with RT VI.

My problem is that the loop rate of my two timed loops on the cRIO only can be very low 10-20 Hz before they start finishing late and my shared variables loose connection the RT VI. What could be the reason for this. Could it be the communication to the FPGA that should be changed to DMA ?

Regards Soeren

[Christian_L]

Hello Soeren,

Please also attach your Shared Variables file which is needed to open the project.

Based on your description I don't think that the problem is related to the communication between LabVIEW RT and the FPGA, but related to the use of the shared variables. The configuration that you have used for each of the variables will affect the performance of the accesses to the shared variables.

[soeren]

Ok, i forgot, here are they, I attached the project file also.

[Christian_L]

Hello Soeren,

The short answer is that you are trying to do too much in each of those loops which is loading up the processor to 100% load.

I did a quick benchmark of the network shared variables with RT FIFO and they take about 60-70 us (microseconds) per read or write operation on the cRIO controller. This also depends on your network configuration between the cRIO system and the machine that is hosting the Network Shared Variables.

Local (single process) shared variables with RT FIFOs take about the same amount of time, while local shared variables without the RT FIFO take about 20 us.

In summary you should reduce the amount of shared variables that you use in your loops. One alternative to using so many individual shared variables to transfer data to the host, is to collect all the individual data points into one array and pass the array using a single array type shared variable. On the host you can separate the array back into the individual data points. This will significantly reduce the overhead. another option would be to collecte the data in an array and use TCP/IP communication to transfer the data to the host.

In your bottom loop where you are logging data to disc, you are opening and closing the file for every write operation. This will take up significant amounts of time. You should open the file once at the beginning of the

In the top loop as part of your FPGA communication your are reading and writing several values that never change during the execution of the application. These include AI Channel Settings, AI Calibration, and Loop Rate TC (ms). You should only access these values once on the FPGA before you start the loop and then use the local values inside the loop rather than reading/writing them from the FPGA every time.

One of the terminals on the inside left side of the Timed Loop allows you to retrieve the duration of the previous run of the Timed Loop. Use this information to further identify which of your two loops is taking up more time and needs to be further optimized.

[soeren]

Hi again

Thanks for the feedback, it was helpfull.

I tried the "use less shared variables, and put them in arrays" strategy and the reduced the number of shared variables significantly, although there are still room for improvement. Although things are working a bit better than before, I don't see a significant improvement, and I can't raise the Deterministic loop frequency very much :(, maeby I'm just trying to do too much at once. Is there anyway to find out which steps in the loops are the time critical ones?

I have attached the changes, the FPGA vi used is the same.

Hope you can help me

Soeren

[soeren]

Lol....I could only attach 3 items, so here's the project file

[Christian_L]

Soeren,

I did some more looking at your code, and noticed the Configure Formula Express VIs that you are using. These VIs are not as efficient as the Formula Node or implementing the same code using LabVIEW math functions.

Please replace these Express VIs with the Formula Node, which is located on the Structures functions palette, and you should see a significant improvement. I have attached a basic example comparing the performance (benchmark) of the ExpressVI against the Formula Node and basic LV math functions.

In addition to using basic benchmark techniques like I have shown in the attached example to determine how fast different functions run in LabVIEW, you can use the LabVIEW Execution Trace Toolkit  to look at the detailed timing performance of LabVIEW code. This tool will give you a graphical representation of different pieces in your loop and how long they take to run.

[soeren]

Hi, thanks for the fast reply

I sort of expected the Formulas being slow, so I have already started removing them. Looking forward to see if the changes you suggest increase the program speed.

Regards

Soeren

[soeren]

Hi again

I finally got the VIs running. The time critical loop on the RT runs with 100Hz and the datalogger with 10Hz without finishing late. Other problems are occuring though. Now the RT suddently freezes and losses connection ad has to be rebooted to work again. I had the same problem before I switched to shared variables, and I thought they solved it.

I tried to find out where the problem lies, i put in error indicators in loops and shared variables, but I didn't find anything. I don't know if it is the RT, FPGA or shared variables. During operation it seems to occur when using some of the PWM generating code in the FPGA, but I'm not sure how to check if this is the problem. In my FPGA code there are 4 PWM generators running at a very low frequency I don't think they should be any problem for the FPGA VI.

Any ideas for solutions?

[soeren]

here are the last two files for the project