Producer Consumer Loop for my Project?

[billko]

---

@drjdpowell wrote:

---

@billko wrote:

---

@drjdpowell wrote:

Whenever someone asks about "Producer-Consumer" I recommend they instead look into the JKI "State Machine" Template, a single-loop event handler.  Use one JKI loop for each piece of hardware, get them to work independantly at first, and then have a "Main" loop that communicates to/from all of them via User Events.  Do NOT use "Continuous Measurement and Logging" as a guide.  

---

I would actually encourage the use of LabVIEW best practices first - i.e., the Producer/Consumer design pattern - for two reasons:

---

That's just it: the "Producer Consumer" NI template is very poor practice.  It takes a lot of experience to build an application based on that without introducing race conditions.  Its apparent simplicity is a weakness, since it is inherently more complex than a single-loop design, such as the JKI.  It should be eliminated as a standard design pattern.

Here's a talk I gave on this last year.

---

Maybe we're looking at it from different perspectives.  In the real world, if a prospective employer asks you what a Queued State Machine is and you can't describe it to them, you lose.  I agree that it is simpler to use one loop than two, but until NI does retire the creaky Producer/Consumer design pattern, it is here to stay, and I believe you should learn that first.

[drjdpowell]

---

@billko wrote:

---

Maybe we're looking at it from different perspectives.  In the real world, if a prospective employer asks you what a Queued State Machine is and you can't describe it to them, you lose.  I agree that it is simpler to use one loop than two, but until NI does retire the creaky Producer/Consumer design pattern, it is here to stay, and I believe you should learn that first.

---

What happend to "LabVIEW best practices"?  Your not doing new LabVIEW programmers a favour by pushing poor programming templates that lead them to write buggy code, just because they are included in a standard LabVIEW install.  The JKI state machine is a "Queued State Machine" that has existed for many years and is widely used.   Sure, one needs to know the NI template because of all the buggy code out there written in it, but one will never be able to understand the flaws of the NI template if it is presented as a good way to program.   

[billko]

---

@drjdpowell wrote:

---

@billko wrote:

---

Maybe we're looking at it from different perspectives.  In the real world, if a prospective employer asks you what a Queued State Machine is and you can't describe it to them, you lose.  I agree that it is simpler to use one loop than two, but until NI does retire the creaky Producer/Consumer design pattern, it is here to stay, and I believe you should learn that first.

---

What happend to "LabVIEW best practices"?  Your not doing new LabVIEW programmers a favour by pushing poor programming templates that lead them to write buggy code, just because they are included in a standard LabVIEW install.  The JKI state machine is a "Queued State Machine" that has existed for many years and is widely used.   Sure, one needs to know the NI template because of all the buggy code out there written in it, but one will never be able to understand the flaws of the NI template if it is presented as a good way to program.   

---

Well, okay, you have your opinion, I have mine.

Your work is to convince NI that the JKI State Machine is a better way to do things.  (Believe, me, you don't have to convince me that the Producer/Consumer design pattern is a clunky way of doing things!)  And I'm not quite sure how "widely used" it is; I've probably worked ten different places in ten years, some of them "LabVIEW houses" and I have yet to encounter this in the wild.  In contrast, everywhere I've been that treated LabVIEW seriously were using the antiquated Producer/Consumer design pattern.  But I'm also aware that ten data points is still a very small sample size.

My job is to convince new users to program in a way that is the accepted "Standard" so they may achieve a common ground with other developers.  They can be talking apples to apples and oranges to oranges.  Once you get that down, you can develop your own - or adapt - more efficient methods of programming.

All this could be moot with the advent of channels.  This may render the conventional Producer/Consumer officially obsolete.

[bockdoug]

I am glad that there seems to be no definite answer to my question. At the moment I am using a simple state machine (not the JKI one), but this is already getting very cluttered.So drjdpowell's recommendation to use JKI is the closest other option. The other suggestions using queues (either QMH or notifiers, they both seem related) seem the cleanest choice in terms of coding, but I find them very confusing without much varied help around for free (which I would need, I've only been using LabVIEW for about 4  months).
Maybe one last question, directed to the supporters of QMH: If my application were to use QMH, would I need five queues (one each for each device and another one for logging) or is one queue all it takes? If one queue is enough, how do the device loops "know" which loop is meant to execute the message? 
Cheers
Doug

[Blokk]

I still keep my opinion that you could just go with the simplified QMH as I explained already in this thread. Why to have separate loops for temperature and pressure? They change rather slow. You can have one producer loop which will generate the data with higher rate and lossless manner. You have a loop where you measure everything else, temperature, etc. In your consumer loop you can log and display your data, using the Data queue from the high rate producer loop plus requesting latest values using "Get Notifier Status" from the slow lossy producer loop. This means you only need 3 loops and one Queue one Notifier. Of course, it is nicer if you have a GUI loop too to handle user commands in a better way, in this case you get one more Queue and a loop with an Event structure...

I cannot imagine simpler solution for your project...

[Blokk]

And regarding to your last sentence about using a single queue. Do not forget one important thing with a queue: if you dequeue an element, it is lost from the queue. This means never dequeue a single queue at multiple places, in case of parallel loop it is not determined which loop will "win" and get the element...

If you think about an efficient "one to many" communication, you might use Dynamic User Events... But I do not recommend this if you are still at the beginning of you learning curve 🙂

[mcduff]

I agree with the doctor; learn to use the JKI State Machine. I took to the time to learn it and it is an extremely useful tool. Like you, I started with the LabVIEW State Machine template but that became rather clunky fast. I even gave a talk about the JKI State Machine to my local user group and convinced some to use it. (Bilko I'm looking in your direction. )

The advantages for using it are:

1. Built in Event loop
2. Built in Error Handling
3. Built in organization. (When you see how the states are organized, you will be "forced" to organize your states also.
4. Flexibility - Strings are much more flexible than enums and you have the ability to comment out states for testing

If every nanosecond and byte is important, than enums are slightly faster and more efficient than strings. But for 99.99% of the applications, you will not notice it.

For your application I think it can be done in one loop. Here is a "JKI Macro" for your application

1. Go to Wavelength XX with the OSA
2. Measure Temperature
3. Measure Humidity
4. Measure Pressue
5. Repeat

This macro is done when the user hits the Start Button.

I am going to assume your instruments are GPIB based, if so, commands over that bus cannot be done in parallel. If you have DAQ devices that monitor continously, then multiple loops would work. FYI, I do use multiple JKI State machines connected by user events.

Cheers,

mcduff

[bockdoug]

I think I will give the JKI a shot. It seems the easiest to implement from where I am right now. The OSA I am using is a GPIB connection. At this stage I was thinking of taking temp/humid/press measurements first, then scanning the OSA spectrum (which can take anything from 0.1s to 10s), followed by another temp/humid/press measurement.

[mcduff]

At this stage I was thinking of taking temp/humid/press measurements first, then scanning the OSA spectrum (which can take anything from 0.1s to 10s), followed by another temp/humid/press measurement.

At these kinds of speeds, one loop is fine.

The keys, at least for me, in using the JKI State machine are:

1. Keep your States small and atomic, ie, a state should just do one thing
2. Put your States together using the "Macro" Functionality. For an example, see how the "Exit Macro" is called; it calls multiple states.
3. Make the cluster that carries all of the data through the State Machine a Type Definition. This should be the first thing you do.
4. Don't worry how big the Data Clusterasarous becomes.

I typically do stuff like you are asking all the time. One loop should work. As you get more experienced you can add mutliple loops together.

Cheers,

mcduff

[Blokk]

Guys, you made me interested in the JKI state machine, couple of years ago I had a look at it, but did not play enough to start to use it.

Regarding to the single loop approach: this is even more simple than my solution, but do not forget in this way you get "gaps" in your OSA spectrum data. If this is ok for you, I would just go also this single loop way...