What if the consumer loop produces an error?

[1984]

Hi community,

The question I have about the producer-consumer pattern is how to handle if an error occured in the consumer loop? I mean the data in this pattern travels one way only, from the producer to the consumer, but I guess there could be many scenarios when if the consumer throws an error the app should stop safely and this includes stopping the producer loop as well.

In the template shipped with LV if the producer stops then it destroys the queue reference which forces the consumer to stop, but there is nothing in the consumer which can stop the producer.

Its possible that I misunderstand the main concept, but I have developed several applications based on this model and I always had to find a way to communicate stop (and other) events from the consumer to the producer. Sometimes I managed it with a local variable, or an FGV sometimes I used user events and so on. These worked relatively well for me, but they always felt a bit hacky.

(I think this question can be generalized as how to do two directional communication between loops)

Would be nice to get an input from others.

Thx.

[AeroSoul]

You could use the error from producer to exit it's loop.

If consumer closed the queue, then the next time producer tried to write to it, it'd throw an error.

[crossrulz]

---

@1984 wrote:

In the template shipped with LV if the producer stops then it destroys the queue reference which forces the consumer to stop, but there is nothing in the consumer which can stop the producer.

---

The thing I have learned over the years is that the consumer needs to be in charge of the queue.  The producer should not be the one creating or destroying the queue.  The queue's lifetime should be directly related to the consumer.  So if the consumer errors, it closes the queue and the producer will then have an error when it tries to enqueue.  When the producer is done, it should send some sentinel or message stating that all of the data has been sent. The consumer can then detect that sentinel or message and shutdown properly, including destroying the queue.

[RTSLVU]

One thing you have to consider when using the built in design patterns and program templates is...

They are just a basic framework to start your program. You are going to have to modify them to suit your needs.

Although I have studied the templates and frequently use a Producer/Consumer architecture and have recently become a huge fan of the CMH architecture. I find starting a new program with one of the built in templates more work than it's worth due to all the changes required to make them functional. I pretty much end up deleting or changing everything anyway. So it actually takes me less time to start with an empty block diagram.

[FireFist-Redhawk]

I made my own version of the producer-consumer template with extra functionality, and what I added to stop the producer from within the consumer is User Events. Add dynamic event terminals to your event structure, and generate an "Exit" user event from the consumer whenever you need to do so:

Spoiler

[crossrulz]

---

@RTSLVU wrote:

Although I have studied the templates and frequently use a Producer/Consumer architecture and have recently become a huge fan of the CMH architecture.

---

I have not had to use a Producer/Consumer for a long while now.  But I have given a good amount of thought to using the Streaming Channel.  What I like most about it is the "Last Sample" input which is a clear message telling the consumer to stop.  The other thing I'm curious to see is seeing the actual flow of data with the producer on the left and the consumer on the right.

More of an aside, I don't think I have needed the Producer/Consumer since the DAQmx Logging came out ~12 years ago.  Letting the DAQmx driver handle the logging for me is a lot simpler and faster than my Producer/Consumer setup.

[billko]

---

@crossrulz wrote:

---

@RTSLVU wrote:

Although I have studied the templates and frequently use a Producer/Consumer architecture and have recently become a huge fan of the CMH architecture.

---

I have not had to use a Producer/Consumer for a long while now.  But I have given a good amount of thought to using the Streaming Channel.  What I like most about it is the "Last Sample" input which is a clear message telling the consumer to stop.  The other thing I'm curious to see is seeing the actual flow of data with the producer on the left and the consumer on the right.

More of an aside, I don't think I have needed the Producer/Consumer since the DAQmx Logging came out ~12 years ago.  Letting the DAQmx driver handle the logging for me is a lot simpler and faster than my Producer/Consumer setup.

---

I like the Netflix streaming channel because I can watch stuff any time I want to.  😄

[drjdpowell]

Could someone explain to me the point of the "Producer-Consumer" templates?   To me they seem incredibly clunky.  What is the point of the separate Event Structure "Producer" loop which does nothing but repackage events onto a queue to the second loop?

[RTSLVU]

The Producer/Consumer is intended for high speed data collection to decouple the DAQ from any processing delays.

For instance say you are running a DAQ at 1Mhz (Collecting data 1,000,000 times a second), performing a Fourier analysis and saving it to disk.

If the Fourier analysis and saving to disk were in the same loop as the DAQ chances are you would never reach the 1Mhz sampling rate.

The Prod/Con allows the Producer loop to run full speed and put data in a queue. The Queue allows the Consumer loop to take all the time it needs to process the data.

[drjdpowell]

I was asking about the templates that come with LabVIEW, where the "Producer" has a Event Structure.    UI events happen at low rates and delays of 50 ms are unnoticed by the User.  I had assumed that was what the OP was referring to.