Double Buffer I/O Balancing

[km4hr]

I have some questions about how data buffering runs continously without collisions between the data provider and the data consumer.

What mechanism keeps the input (data written into the buffer) and output (data read from the buffer) balanced? Obviously input and output must be equal or the buffer either overflows or "goes dry". 

I'm thinking of a tub full of water. One person uses a scoop to continously put water into the tub. Another person scoops water out of the tub. If the inputs and outputs aren't balanced the tub either overflows or eventually goes dry. If the person removing the water has a small scoop then he has to bail faster than the guy putting the water in. How do they stay in sync so that the tub stays full at all times?

The double buffering explanations I've read show the data provider writting data into the buffer non-stop. He neither stops nor slows down for anything. He doesn't even know the the data consumer exists. It's left to the data consumer to remove the data at just the right moment to avoid getting rear-ended or either over-running the provider in front. How does the data consumer manage to stay out of trouble?

[OVR_CZ]

Solution 1:{function DAQ_DB_HalfRead+DAQ_DB_Transfer}

second person periodicaly control level of water in tube, and when tube is Half full(or more) then remove exactly half of the capacityof tube.

but there is some time limit:

Tchecking+Tremove+Treserv<TputHalfData

Solution 2:{function DAQ_Check/DAQ_Monitor+DAQ_DB_Transfer)

   the second person, periodicaly get level of water (count of acquired data)and get this measured capacity from tube

   there is time limitation similar to first solution:

   Tchecking+Tremove+Treserv<(2*TputHalfData)-Tchecking

Tchecking=time period of checking data(for example: every 100ms)

Tremove =time for remove exactly half buffer data and process it

TputHalfData=time for fill data with size of half buffer to buffer

Treserv=some reserv time (beacuse of  windows multitasking,...)

On windows ,be aware that timing bellow 100ms is like magic, works well about 95%(if user not click to anything:) )

This depend on faktors like CPU utilization,Disk activity, Os type, hardware drivers(specialy notebooks),... .

For this very fast timing(50..200ms), is sometimes need to tune up your application acquire threads with HIGH or critical priority, which is hard for develop and debug.

So i recomend use slow timing i.e. TputHalfData>1000ms

//Sory for my english 🙂

{Development mantra:Windows is not Realtime operation system, windows is not ...} 

[dummy_decoy]

a data buffer is not a bath tub: the consumer does not care if the buffer is half full. its goal is to process every data in the buffer, emptying the bath tub, so to speak. in most application, we design a consumer so that it processes data faster than the producer in order to never overflow the buffer. an underflow (buffer empty) condition is not critical, the consumer simply waits for enough data to become available: if you empty the bath tub faster than it fills, you only have to wait for water. eventually you use the available time to clean the water spills/take a nap/smoke a cigarette: a program will use the available time to perform secondary tasks like processing user events, displaying results, ...

double buffering is quite different: its like having 2 bath tubs. the producer fills one tub, then proceed to the second tub while the consumer empties the first, so the producer and the consumer are never working on the same tub. double buffering can be extended to triple, quadruple, ... n-buffering. note that double buffering generally applies to imaging applications.

however, the organisation of buffers is totally different between a producer-consumer scheme and double-buffering, due to naming conventions. in a producer-consumer scheme, the name "buffer" represents a store which can hold multiple "data point". on the other hand, double-buffering is generally used for imaging application where, historically, a buffer means an image. so a buffer in imaging application is one "data point" for the producer-consumer scheme. thus, double buffering is a producer-consumer scheme where the buffer is 2 data points in length, each data point holding a image.

now regarding the consumer staying out of trouble,  it depends on your application: some application absolutely needs data to be acquired and does not care if the consumer can handle the load, other applications absolutely needs the data to be processed and slows down acquisition so that the consumer can follow. it's a question of compromise, and every application is unique in this respect. now think back of your bath tub: if your friend is filling the tub faster than you can empty it and you feel it will overflow, you just yell for him to stop. applications work the same: the buffer generally comes with additional data (available space, total length) and/or signals (ready/working) which allows to avoid overflowing, acting like yelling at each other or measuring the height of water in the tub.

[mvr]

Additionally, the double buffer implementation for data flow originated in hardware designs where the memory configuration did not support simultaneous read/writes.  For example where there was a need to flow the data for an incoming image into a different bank of memory than the current outflowing image data from a second bank of memory.  For simple producer/consumer data flow a queue or ring buffer is far more common.  CVI contains a very nice queue, the Thread Safe Queue (TSQ), that is very suitable to producer/consumer data flow, even for images.

Message Edited by mvr on 08-26-2009 07:09 AM