Real-Time Measurement and Control
DMA fifo read high cpu usage
Solved!
Hi al, this is a long one!
I have noticed some strange behaviour when I read from a DMA FIFO between FPGA and RT host on my crio (9014 controller and 9104 backplane). The FPGA is writing 2 data points to the FIFO every millisecond and the RT loop is reading 500 data points every 250 ms. The RT loop period is controlled using the wait untill next ms multiple function.
The use of the wait untill next ms means that in the first iteration the wait will not occur so I gave the DMA FIFO read method a 750ms timeout to allow the data points to accumulate on that first iteration.
I then ran the VI and it read data as expected and there was always 0 elements in the FIFO. Then using the the system monitor i observed the cpu usage on the RIO and was surprised to see it was around 30% 
!
After much head scratching and many other attempts I decided to set the timeout to zero so the read just timeouts untill there are enough data points and then runs a little bit behind so that there is always a constant (non zero) number of data points in the FIFO. Now the VI runs and uses only around 3% CPU.
I then thought that in the first case that as I was reading exactly the number of points in the FIFO that I was tripping some sort of polling behaviour in the read function which was hogging the cpu.
So then I ran a case with a zero timeout where I was reading with 0 data points left in the FIFO as in the first case and expected to see the read function timeout but it never does and the CPU usage is normal.
So what is happening? I am stumped!
Thanks,
Steve.
Hi again,
Yes, I expected exactly what you said to happen. So when observing the CPU usage you would expect a brief spike in cpu usage followed by low cpu usage as the loop timing function makes the thread sleep.
I realised that the resource monitor is a crude way to monitor this as it updates slowly (around 1Hz) so I ran the DMA read in a loop with timing forcing it to wait 1500ms between reads. At the rate the FPGA was writing to the FIFO the data should always be present when the function tries to read it (only just hence always running with zero elements remaining).
Supposing that the read on the RT has executed 1ms early it should only have to poll for 1ms for all the data to be present. So on the resource monitor we should see a brief spike in CPU activity followed by 1500ms of CPU idle. In fact we would be lucky to see that CPU spike at all at the update rate of the resource monitor! So the CPU usage would appear continually low if anything!
As you say if this brief polling behaviour is occurring then a read function with zero timeout should immediately timeout (and return an error code-50400). As you can see in the code I posted I check for this error, clear it and set a latch if it does occur.
To summarise:
The expected behaviour of a loop with a timeout enabled is:
Iteration 0: Use of wait until next multiple function results in short 'wait' first iteration so read function polls for the data to be written by the FPGA and does not timeout and reads all of the data as soon as it is present.
Iteration 1...N: As the data is being written as fast as it is being read we expect to see zero elements remaining (we do). Brief polling may occur but only for a maximum period of time equal to any jitter in the loop execution period.
Expected CPU usage: Brief initial spike followed by possible intermittant spikes caused by polling. This is unlikely to be visible on the resolution of the resource monitor so overall a low CPU usage should be observed.
Observed CPU usage: Continually high.
The behaviour of the loop with zero timeout is as follows:
Iteration 0: Use of wait until next multiple function results in short 'wait' first iteration so read function times out (as expected) and sets the latch.
Iteration 1...N: Read function is now reading happily with x elements reaming in the FIFO due to the timeout in iteration 0 so press the catchup to flush the FIFO and simulate the behavior of when a timeout was enabled.
Iteration N...: Loop is now reading with zero elements remaining. We are expecting to see the function possibly timeout every so often due to small jitter in loop period. However it never does!
Expected CPU usage: Low
Expected timeout behaviour: Intermittant if at all
Observed CPU usage: Low
Observed timeout behaviour: Never
It seems that when a timeout is enabled on the first (short) iteration of the loop it polls and then never stops polling - even though we have shown it doesn't need to!
I am really just interested to explain what is happening as I can repeatably make it occur. It may well be that I have made a mistake somewhere!
Regards,
Steve.
