Releasing Queue in Producer Consumer model

[RPKumar]

I am having a VI which reads data from ethernet,writes to disk,processes and displays. Since lot of processing and displays are there, I am converting it to a Producer- Consumer model so that no data is lost.
I am using 2 loops. IN the Producer I am doing the UDP read and Write to disk functions.and use the enqueing also.In the Consumer loop I am dequeing it and do all the processing and displays.My idea is to avoid any data loss in UDP reception and file writing.
My confusion- where to use the release queue?from the Producer or Consumer ? In a third loop ? Should it be in a sequence structure ?
How to set the loops stop conditions ?
My requirements for stopping - when I press the stop button the program should stop completely(the Consumer loop should stop only after dequeing and processing all the data in the queue).  If i don't press the stop button the whole program should run continuously (if data does not arrive for some time it should wait without time out-i am using -1 for the time out condition in UDP read.)

[Norbert_B]

RPKumar,

i believe that your question should already be answered if you take a closer look into the producer/consumer supplied with LabVIEW. You can find it in the template browser which you can open by select "New" on either any VI or from getting started dialog. The producer/consumer can be found in the frameworks-area...

hope this helps,
Norbert B.

[tbob]

The example Producer Consumer vi won't quite work for RPKumar.  In this case, when the stop button is pressed, the entire vi will stop even though there are still elements in the queue waiting to be processed.  I have modified the example to stop only when the stop button is pressed and the queue is empty.  When stop is pressed, the remainaing elements in the queue will still be processed until there are none left.  If the queue becomes empty before the stop button is pressed, the dequeue function will wait until an element appears.  One bug here.  If the queue is empty before the stop button is pressed, the dequeue is waiting.  If the stop button is pressed while dequeue is waiting, the process will hang forever because there is no timeout in waiting.  You may want to set some large timeout value, or put in some code to queue in a dummy element when the stop button is pressed so that dequeue doesn't wait forever.

[johnsold]

I have waited until both loops have competed before releasing the queue. In simple cases just wire the error clusters (or any data line) from each of the loops to Release Queue (error cluster from one loop and queue reference from the other works nicely). In other programs the Producer loop sends a "Halt" command to the other loop. The last thing the other loop does before it stops is to sned "Stopped" status back to the Producer. The Producer stops after receiving the "Stopped" status and then releases the queue. This method is easily expanded to multiple loops and can indicate to the user the status of things as they shut down if that may take a few seconds or longer.

Lynn

[RPKumar]

I have tried the model shown in the template earlier.That will stop the consumer loop without waiting for the que elements to be processed.I will try with the other methods suggested and see.Thanx for all the suggestions

RPK

[GReaper]

I just want to back this up after learning the hard way.  I think it is better to release the queue in the consumer loop versus the producer loop.

I just spent 2 days trying to find out why all of a sudden my vi wasn't saving the setup data to the INI.  It started after I added two queued events to process on exit for the consumer loop.  Only 1 was processed before the producer loop released the queue.  So my second pushed event was not getting processed.

I think the template for this in LabVIEW should be modified to release the queue in the consumer loop unless there is some benefit to doing it in the producer?  Logically it makes more sense, not to release it until everything in it is processed.

Cheers,

[tbob]

Here is the ultimate Producer-Consumer loop.  It queues a stop command when the stop button is pressed.  This ensures that every element is dequeued and processed before the bottom loop stops.  The last queue data to get processed is the stop command because after you press the stop button, the top loop stops enqueueing more elements. 

If your data type is other than a string, you can create something to represent stop, like NaN for a numeric. 

Notice the shift registers.  This is good practice.  Every loop starts with an error out of the previous loop.  If an error occurs, it is passed on to the next loop iteration.  Also, if an error occurs in the consumer loop, the loop stops.  It is good practice to put shift registers on the queue reference also, but not necessary in this case.  If you use a For loop, and the loop iterates 0 times, the output of the queue reference would be a null reference, and the Release Queue would not release the queue.  With While loops, they always iterate at least once, so it isn't necessary.  Its just good practice to make a habit of using shift registers so you don't forget when using For loops. 

Notice also that the error in to the Release Queue is not wired.  If there were an error, the Release would not take place.  All errors are merged at the end to report exactly where an error occured.  A timeout is included in case an error prevents the stop command from being enqueued or dequeued.

[GReaper]

Notice also that the error in to the Release Queue is not wired.  If there were an error, the Release would not take place.  All errors are merged at the end to report exactly where an error occured.  A timeout is included in case an error prevents the stop command from being enqueued or dequeued.

---

Now THAT is an interesting hunk of code.  I was hanging that queue release off the error out in the consumer loop for data dependency and didn't think if I was crashing my way out (what, my code?) that an error would prevent the queue from being released and the exit isn't clean.

Having shift regs for the error and queue will add more wiring traffic to the consumer loop but the pay off is a clean exit and assurance that all queued events will be processed.

I'm not sure I see what the timeout on the dequeue gets you.  There is no error out set on a dequeue timeout but that output flag is.  I assume you could OR that in to the stop condition.  If your producer loop is handling events for the user interface you may want to keep the timeout at -1.

Cheers,

[johnsold]

tbob,

Nice code!

Greg,

One use of the timeout is to allow some action to take place even if no new data is available. For example update a status indicator or elapsed time to let the operator know the program is still alive and happy. I have used it also to implement an interuptible wait. Suppose some process needs a 10 second wait, but you want the "stop" command to be handled immediately if it occurs. This will handle that situation nicely. Just use the Timed Out boolean to select which case to execute.

The wait function could be eliminated and just use the timeout (you need to check the Timed Out output if you do this).

Lynn

[tbob]

I thought the Timeout on Dequeue Element caused an error out.  Thanx for pointing that out.  I will modify my template accordingly.