Dequeue with wait on multiple queues similar to "wait on notification from multiple"

[Tushar_Jambhekar]

Current Scenario:

for dequeue element we can pass one reference only. And the function waits till timeout if no element is present in queue. If element appears before completion of timeout then function returns immediately with data.

However if I want to wait for data from multiple queues then I need to dequeue from them one by one. This approach has following problem.

Lets say I Have two queues Q1 and Q2 and I am Dequeuing element from them one by one with 10 Sec timeout. Now lets assume there is no data in Q1 but there is data available in Q2. Now when I dequeue element from Q1 it waits for 10 sec and returns and then I dequeue element immediately. If situation is opposite then Q1 returns immediately but Q2 waits for 10 Sec.

However what I need that if Q1 or Q2 has data then there should not be any wait/

In Notifier palette there is a VI called "wait on notification from multiple" this VI accepts array of notifiers and return immediately if any of the notifier receives notification.

Similar functionality will be highly useful in queues

[RavensFan]

Why wouldn't you just put two dequeue functions in parallel in your code?  Whatever function, VI, or structure that would follow them wouldn't execute until both dequeue functions had returned an element.

[JackDunaway]

Tushar - What are the disadvantages of using User Events and the Event Handler Structure as a workaround/replacement? Granted, you lose a few queue abilities (a couple ideas to make this better: Option to discard stale entries and Event queue size). Finally, an alternative implementation for this suggestion might be Expand Event Structure Functionality.

[AristosQueue (NI)]

It's a pretty rare use case. Usually if you have two producers that both need to feed the same consumer, you just give both of them the same queue refnum. Or you use a named queue so that they each have a refnum for the same underlying queue. The only case I've seen where you actually need two separate queues and then merge them is when you have to do intermediate processing on the data for each queue. If that's the case, then you just set up your code like the code I've shown below (I have deliberately minimized my error handling in this diagram so you can focus on the other wires). And in that case, the Wait Multiple function wouldn't help you. Does the setup below work for you? What situation are you in where the Wait On Multiple Queues is desirable, compared to simply having one queue used in both enqueue locations?

[AristosQueue (NI)]

Ravens Fan wrote:

> Why wouldn't you just put two dequeue functions in parallel in your code?  Whatever

> function, VI, or structure that would follow them wouldn't execute until both

> dequeue functions had returned an element.

The original idea explains why this doesn't work.

[RavensFan]

 

---

@Aristos Queue wrote:

@Ravens Fan wrote:

> Why wouldn't you just put two dequeue functions in parallel in your code?  Whatever

> function, VI, or structure that would follow them wouldn't execute until both

> dequeue functions had returned an element.

The original idea explains why this doesn't work.

---

The only way I can see the original idea explains how this wouldn't work is that if you get an item in Queue 1 and it dequeues it, but queue 2 times out, it that you've now pulled the element from queue 1.  But the idea is that the usere doesn't want any element dequeued from either queue until they both get their next element.  And the work around for that is if one queue times out, the other queue pops its element back in at the front of its queue.

Here is what I envision he is asking for and how to make it work.  Of course, this would become more unwieldy as the number of parallel queues would grow.  But if I had a need to do this, I'd probably try to design an architecture that wasn't relying on waiting for multiple queues.

[AristosQueue (NI)]

> But the idea is that the usere doesn't want any element

> dequeued from either queue until they both get their next element.

Nope. The idea is to get data from one or the other queue. Specifically to *not* wait for both. That's both from the title of the idea (it's how the notifier prim behaves), and explicit in this comment

> However what I need that if Q1 or Q2 has data then there should not be any wait/

[Tushar_Jambhekar]

JackDunaway

>>What are the disadvantages of using User Events and the Event Handler Structure as a workaround/replacement?

i deviced a simple example where two dynamic VI communicate with User events and Queues and i found that Queues are around 15-20% faster on my machine. also as number of dynamic(parallel) VIs trying to communicate with each other increases(Many to many) then this difference also increases by certain amount.

I cant see any way to attach my test code here

Secondly if i want to get ref to a perticular queue in some other place I can obtain the ref by specifying name.which is habdy in some cases. this can not be done with user events

This gives me one more idea: Can we have user events with "Name" and then have function like "Get User Event Ref from Name". I will post it separately

I have deviced a framework which allows complete "loose coupling" of msgs. (see video here: http://www.youtube.com/watch?v=wgoUdb441Ek)

this kind of thing is certainly not possible with event structure, when i use event structure it is mandatory that i define all my events and their data types at the time of writing code.  which limits the ease of extending application.

>>a couple ideas to make this better: Option to discard stale entries and Event queue size). Finally, an alternative implementation for this suggestion might be Expand Event Structure Functionality.

yes these are really good ideas and i have already gone through that. and certainly make life better. but my thoughts are written with some different use cases in mind.

Aristos Queue

>>It's a pretty rare use case.

i agree but so is the case with "wait on notification from multiple"

and i also know that workaround can be applied. and so can be done with "wait on notification from multiple"

after all notification is a some kind of variation of queue.

and consistancy in palettes is important. (I remeber the discussion with merge error vi should be xnode and various other points which talk about user interface consistancy)

secondly this feature can help me to develop a framework, see the existing framework video in the link given above. however current framework can have only two priorities. i want to develop much superior framework with very flexible priority system.

once framework is developed user of framework wont even see the queues most of the implementation will be hidden inside frame work. so development of framework is "pretty rare use case" but use of it is in almost every application and that makes it "pretty important use case"

and when i am planning to make framework which will automate the queue management and creation process. the workaround will completely defeat the intention as user has to do everything himself

Ravens Fan

I think Aristos Queue has already answered your posts

[RavensFan]

Okay.  I think I now understand what Tushar is looking for once I went back and reviewed the help on Wait on Notification (Multiple) which says "Waits until at least one of the notifiers you specify receives a message".

I was trying to rely on my memory about this VI (which wasn't good because I have not needed to use this one).  I think the title for this function is actually kind of misleading.  I interpret as waiting on notification from multiple which means you are waiting for multiple notifications to occur before moving on.

Now I don't see why you'd need this in a queue format.  Queues are either a one to one or many to one messaging format.  If you have multiple consumers, then you should have multiple queues.  But since Tushar's description is just multiple producers to a single consumer, I see no reason you need to create two queues.  I would just have each producer put the data in the same queue.

I'm sorry, but you have yet to sell me on why a Queue (Multiple) function is necessary.

[Tushar_Jambhekar]

>>I'm sorry, but you have yet to sell me on why a Queue (Multiple) function is necessary.

Lets say i want to implement messenging system with 5 levels of priorities(Level 1 Low Priority and Level 5 High priority)

Now consider following situation.

• I have single queue which delivers the message.
• The queue already has 2 messages with Leve3 Priority (and code dont know the priority unless queue previewes or dequeues element)
• now lets say i want to add another message then there might be following errors

• if i add level 4 message in the end then at consumer end low priority message will be processed first
• similarly if i add Level 2 message in the front of queue the at consumer end low priority message will be processed first

Now consider if i create 5 Queues each one is specifically for perticular priority then problem gets solved automatically.

[RavensFan]

That is an interesting use case.  And to make your idea even work for that, you'd have to have a requirement added to your idea that there is a queue priority among the multiple queues that queue#5 (i.e. highest priority) always gives out its element on the dequeue ahead of any lower queues.  Only if queue#5 is empty, does queue #4 get to return its next element if it has one.

Right now, there is no such thing as a notifier priority on the Wait on notifier (multiple) function.  It just returns an array of elements from whatever notifiers have been set (assuming that you have the ignore previous set to false).

I think the idea of having a queue priority is just too specific to ever get implemented on a Wait on multiple queue function if it was ever developed.

I think you'd have to go with some sort of queue manager concept like Aristos queue proposed with the intermediate queues, and even that would be complicated and require previewing, dequeueing, storing, sorting, re-enqueueing.

I think to make your idea work, you could use 5 queues, but also use a common queue that gets a simple message whenever a new element is placed in any of the 5 queues.

You first wait on the common queue with a timeout.  Once it sees an element, you now now there is at least one message in one or more of the other queues.  Then you can check the status of the other queues to see if any messages exist in them starting with the highest priority working to the lowest.  Once you find one, you dequeue its next message and the loop reiterates.

This way if you have 3 messages come in before the dequeue function gets hit again, priority #3, #1, then #5 in that order, you've essentially thrown 3 "chips" into your common queue.  It pulls one, then searches the queues in order and allows the highest priority message to be handled first.  The common queue will still hold two chips so that the loop knows there are more messages to process.

Beyond all of this, I would possibly just forego queues and create an action engine inherently stores what would be the queue elements as an array that it can sort everytime it gets a new message and puts it in its proper place and returns the most important message off the stack.