Consumer loop very slow, queue size grows. How to speed up my consumer loop? As the array grows the slower it gets.

[Wauters]

LabView version: 2012

OS: Windows XP SP3

Hardware: USB NI cDAQ-9174 NI-9221

Application: Oscilloscope

I'm relatively new to LabView and I'm currently experiencing some performance issues with my project. I guess it has something to do with the way it's programmed. I'm hoping to get some tips from you guys.

This is my producer loop. It should be capable of acquiring 100 000 Samples/s. I guess I've done this the right way and can't really be improved. Or am I mistaking?

This is (part of) my consumer loop. It's too slow, the number of elements in the queue keeps getting bigger. I'm doing two types of calculations on the queue data. One for changing the scalling (a multiplication), one for changing the y-position (a summation). Because each time there are 10 000 samples acquired I have added an extra loop (inner loop). This loop splits the array into x samples when the requested samples are smaller than 10 000 or adds them when the requested samples are bigger than 10 000. This depends on the user input (time/dev).

For example: when the user request 100 seconds of data to be plotted on a graph we get an array of 10 000 000 x 8. Is this considered big? Enlarging the array to that size is very very slow, the queue builds up rapidly.

I don't know what really slows it down or how to 'debug' this properly. Transposing the array twice seems avoidable?

Maybe I'm doing this in an inefficient way? Any thoughts that might help me?

The VI's are attached.

Thanks for your input.

[LV_Pro]

I don't have 2012, so I can't really look at your code, but if you are trying to "display" 10E6X8" it will slow you down, as the user interface thread wil be stressed. You really can't see that many points, your display will be lucky to show 2000 points horizontally, unless you have a really high res monitor and special board. I can't tell too much from the images, and as I said, don't have 2012 installed to look at the rest of your code. I built a test a while back, and it would slow down due to the image display processing time. As I can't see what is going on with your queue I can't determine why it would fall behind. Also, how many data points are you capturing in the master loop, it has a 100mS loop timer, which should be a lot of time for the consumer loop to do simple scaling and shifting (mult and addition).

[tst]

I'm also going based on the image, but I would add two additional points:

1. Your pause loop has no wait. This will cause it to use the CPU as much as it can. I'm assuming most of the time this isn't an issue because it's false, so the loop only runs once.
2. You're building the array inside the loop, which can be very inefficient because arrays have to be contiguous in memory and this requires constantly reallocating memory for the array. I would suggest that you either allocate memory for the array yourself in chunks and then use Replace Array Subset to put the data into the array or that you keep the data in the queue until the end and then process it when you know exactly how large the array is going to be.

[Wauters]

Thank you for your replies so far!

LV_Pro,

I agree it is a bit silly to plot more than 2000 points. I will change this. But even without a graph, the consumer loop seems to be unable to handle the speed.

tst,

1. Ok, I will change this

2. Thank you for pointing this out. I implemented your technique and have some increased performance.

Still not the speed I would expect from LabView. The system is 'stable' with a sample rate up to 100 Hz, increasing this makes my queue overflow.

I must be doing something else wrong... Anymore ideas?

Latest version of program in attachment.

[tst]

Well, you have some additional things which are probably hurting your performance. For example, you are continually writing to the same file, which would require the OS to work on it. You're still resizing arrays in loops in other places. You're using value property nodes, which most likely force the loop to execute in the UI thread (although like the pause loop, I think it's less likely this is your issue).

In general, if you want to find out what the source of your problem is, it is helpful to start disabling parts of your code and seeing how that affects the result. For instance, your logging loop can be easily disabled by not enabling the recording. If you do that and see that the loop now does work, you know that's the source of the problem and you look into that (using the same method to disable internal parts).