background acquisition buffer

[joubiyann]

Hello,

I am acquiring a signal and I recieve this error meesage after 4 or 5 seconds :

"Your application was unable to retrieve data from the background acquisition buffer fast enough so the unretrieved data was overwritten with new data. To prevent this error, you might increase the size of the background acquisition buffer, increase the amount of data you read from it per call to the read function/VI, slow down your acquisition rate, or reduce the number of tasks your computer is performing."

I have understood the problem but I don't know how to increase the size of the background acquisition buffer and increase the amount of data you read from it.

I want to see my signal until I stop the VI and I want to switch between many acquisition rates (8000, 11025, 22050, 44100).

Is there someone who can help me ?
Thanks.

joubiyann

[joubiyann]

There is the VI

[TylerS]

joubiyann,

There are a couple of things you can try here. One is to increase the buffer size input to the SI config.vi. It's currently set that the default, but the help file recommends increasing this if you get a buffer overwrite error. Another option would be to decrease the overhead in your code by not using Express VIs. Express VIs are fantastic for quick and easy configuration without having to code, but if you start having problems with execution timing or memory space, it would be better to use regular VIs because Express VIs tend to be slow and cumbersome (relatively).

I hope this helps. Have a good one!

Tyler S

[joubiyann]

Tyler S

Thank you for your reply.
I have try to increase the size of the buffer but it doesn't work.
For the second option, I don't think using an Express VI. How can I see the type of the VI ?

joubiyann.

[TylerS]

joubiyann,

The FFT, Trigger and Gate, and Amplitude and Level Measurement VIs in your program are all Express VIs. You can tell by the blue appearance an the fact that they include a configuration window. Basically, these VIs differ from regular VIs because they create custom code behind the scenes based on the configuration settings. This all requires additional overhead. What I think would be the best idea for your program would be to put the processing code in a different loop from the data acquisition code. You can accomplish this by using a Parallel Loop architecture, a Master/slave architecture or a Producer/Consumer architecture (there are templates available for each of these). By doing this, the acquisition loop would be free to acquire all the data points without having to wait on the 5 express VIs to complete their operations. This would allow you to read the points off the buffer much more quickly.

Tyler S

[joubiyann]

Tyler S

I have tried to use parallel loops structure in order to acquire data on one loop, and to make measurement on the other.
The problem is, I can't have my data on the second loop.
My data is available only at the end of the first loop.
Is it possible to have the data before the end of it ?

joubiyann

[TylerS]

joubiyann,

I would recommend using the Producer/consumer (data) architecture rather than the parallel loops architecture for what you are trying to do. The way this would work is that the data you acquire would be queued in the producer loop and then the consumer loop (in parallel) would obtain that data from the queue and process it. This way each loop can run at it's own speed but the data is still processed in the order that it's acquired. The template for this architecture is located under VI from Template>>Frameworks>>Design Patterns in the template dialog window. There are two versions, data and events, but I recommend using the data version.

Have a good one!

[falkpl]

Producer consumer is the way to go, I have done this many times, in the producer loop put your daq and in the consumer loop consume the data shared between them using a queue.  Often the buffer over run is not caused by a buffer that is small but an unbalanced read rate to data acquisition rate.  Lets say you are acquiring 20K samples per second, and you are reading it in a single loop at 10K samples at a time, if processing is going on in this read loop you will start to fall behind if the processing takes more than 0.5sec.  So the unread portion of the buffer will grow,  If you set the buffer size to 50K you will over write the buffer in 5-10seconds.  If you increase the buffer size you will only delay the inevitable overrun, but if you remove the processing from the loop your problem will go away (most likely).  To send the data over a queue to the other loop has very little overhead so your daq thread is unaffected by processing overhead. One other thing to keep in mind is that changing the read size can improve the efficiency of the daq and the processing and depending on the application the size of the read does have a "sweet spot" where the buffer is happy and will not overrun independent of the processing on the other loop.  Imagine the buffer as a barrel which collects water from a hose (incoming data), there is a hole in the bottom of the bucket to release the water.  To keep the barrel from over flowing you can either get a bigger barrel (larger buffer) or increase the size of the hole (speed of the daq loop)  if you are filling the bucket faster than you are draining it it will overflow over some finite time.  Since the finite time is usually long in most data acquisition and barrels can be only so large (RAM limitations: economic and technical)  you will most likely drain faster (small efficient daq read thread) or fill slower (lower rate of acquisition).  I hope this helps and does not just confuse you.

-Paul

[Sheldon_Stokes]

This scheme is a great way to avoid buffer overflow because the reads are more deterministic and can be scheduled to prevent buffer overflow.  However it should probably be noted that this scheme may add complexity to the processing.  The queue acts like a flexible array of data.  This data can be anything.  Most likely if you are reading from your soundcard (a guess), you’ll want a queue of 1D or 2D arrays.  So the queue acts like a 2D or 3D array of sorts.  In that rather than one long continuous stream of 1D or 2D data, the queue contains array elements that are <number of values read> long.  This is not a big deal if you are treating each read as a separate entity; like taking the mean or RMS of each reading.  But if you want to treat your data as a continuous stream; like you want to take 2048 point running fft’s and are reading 10,000 points at a time.  Then you have to de-queue your data into a ring buffer or at the very least grab your data in the appropriate sized chunks.  

For your sample rates and real-time nature of your processing, you really want to use a ring buffer if you want to reconstruct the data stream, “insert into array” and “delete from array” will kill you. 

I’d love to hear that I’m wrong about queues, but that’s the way I’ve always used them (as “hold anything” arrays of sorts).

Sheldon