Queue: one producer vs multiple consumers

[RavensFan]

I have an idea, whether this is feasible or not, I don't know. 

Since you are looking at potentially a hundred queues,  why not make a hundred queues by creating an array of queues?  One queue for each consumer.  Name the first queue 0 and put it in index 0 of the array.  Next queue name 1 and put in index 1 of the array and so on.

If you have a message for consumer 5, select the element 5 of the array, and put the message into that queue.

Each consumer loop would select the appropriate element of the queue array for its dequeue functions.

Looping structures at the beginning of the programming should make it easy to create any number of queues.  I don't know if there is any structural reason within LabVIEW this can't be done.  Or an array of queues, if possible, would cause some memory management nightmare for the operating system.

Maybe it would not even have to be an array, just dynamically create all of the queues in an initialization loop and refer to them by an ID number for its name.

Message Edited by Ravens Fan on 10-27-2006 05:13 PM

[unclebump]

Completely from left field, how about using memory tags inside the DSC engine??

[Kevin_Price]

---

JoeCAE wrote:
Kevin P.,

Actually, if I should preview, I would only preview the first element in queue. Since one of the consumer, being the destinator, will eventually pop out the element, it is okay.  So that avoids using semaphore and simplifies the thing.

---

Yeah, I see your point, but I was guarding against a degenerate case.  If one of the consumers gets busy or ceases execution, then as soon as its message moves to the front of the queue, the entire system grinds to none-too-graceful halt.  It seems that just previewing one element could make all the consumers too interdependent.


Anyhow, it looks like some better ideas have started to come in...

-Kevin P.

[Underflow]

Arrays of queues do work, but (at least in LV7.1) they need to be of the same data type.

Keep in mind that, with named queues, you don't have to keep the queue reference wires around, you can just keep a list of the names that have been created so far.  Then, when you need one, you just obtain that queue by name.  This avoids some of the data type issues with the queues, and any odd behavior with reference arrays.  It is, perhaps, stylistically painful to some.

Just make sure you close all of your named queues gracefully when you're done!

Joe Z.

Edit: Not sure if I'm really just restating what Ravens Fan said.  Apologies if so.  Ah, well, must be time to go home, have a good weekend folks 🙂

Message Edited by Underflow on 10-27-2006 07:23 PM

[RavensFan]

Thanks Underflow,

I think we are on the same page.  I just recently had LV Intermediate 1&2, where I first learned about queues.  This thread really got me thinking about how to use queues cleanly, and also make the concept expandable.

I had to try out the concepts.  I am attaching two VI's I created based on the Producer/Consumer DesignPattern (Data) example VI.  They are showing just 2 consumer loops, but it could be expanded to any number.

In "ProducerConsumerData (Multiple Queues) Folder\ProducerConsumerData (Multiple Queue Array).vi" I created an array of queue references.  So if you want a particular queue, you just use the array functions to index the array to that reference.  The array was built using auto-indexing.  And just one wire representing the array of queue refs is passed around the VI.

In "ProducerConsumerData (Multiple Queues) Folder\ProducerConsumerData (Multiple Queues).vi" I created multiple named queues.  In one case, I used string functions to create "Loop0" "Loop1" .... "Loop(n-1)" as the names.  In the other instance shown in disabled code, I used autoindexing to feed in any list (as string array) of arbitrary queue names.  I think the advantage to either of these is that by using named queues, you don't have wires running to numerous loops (though I quess you should still have error wires to keep the order of execution).  The advantage to the string array of queue names is that the queue names can be meaningful.

One thing I did learn by doing this is that using the force destroy option of the Release Queue VI was necessary for the VI to stop running since I obtained the queues a number of times by name.

[JoeCAE]

Thanks to all of you,

I decided to go for the array of queue, where each element has a queue in and at least a queue out. The master has one queue for inputs that the slaves use to send data to it, and the master holds an array of queue names to output data to each slave. In summary, the master has one queue in an array of queues out, and each slave has one queue in and one queue out. That way, the slaves are not interdependant as each of them has their own data bus input.