Dequeue with wait on multiple queues similar to "wait on notification from multiple"

[Tushar_Jambhekar]

Current Scenario:

for dequeue element we can pass one reference only. And the function waits till timeout if no element is present in queue. If element appears before completion of timeout then function returns immediately with data.

However if I want to wait for data from multiple queues then I need to dequeue from them one by one. This approach has following problem.

Lets say I Have two queues Q1 and Q2 and I am Dequeuing element from them one by one with 10 Sec timeout. Now lets assume there is no data in Q1 but there is data available in Q2. Now when I dequeue element from Q1 it waits for 10 sec and returns and then I dequeue element immediately. If situation is opposite then Q1 returns immediately but Q2 waits for 10 Sec.

However what I need that if Q1 or Q2 has data then there should not be any wait/

In Notifier palette there is a VI called "wait on notification from multiple" this VI accepts array of notifiers and return immediately if any of the notifier receives notification.

Similar functionality will be highly useful in queues

[AristosQueue (NI)]

> and so can be done with "wait on notification from multiple"

> after all notification is a some kind of variation of queue.

There is no workaround for Notifiers. They are a variation of the queue, but the particular variant makes it impossible to write a correct, non-polling subVI that has the same behaviors as the Wait Multiple primitive. Regardless, the waiting on multiple notifiers is a lot more common than waiting on multiple queues -- it comes close to a primary use case for notifiers.

> Lets say i want to implement messenging system with 5 levels of priorities(Level 1 Low Priority and Level 5 High priority)

Creation of a priority queue is a very different request than a "wait on multiple". Nothing in your original request gives any priority to one queue over the other when checking for data. Phrased that way, there might be more utility, but I suspect the right answer would be to ask for a priority queue -- where a sorting routine is supplied as part of the queue's creation -- rather than blending multiple queues.

[AristosQueue (NI)]

Also, if you want to write a priority queue, you can do it pretty quickly as an action engine (aka functional global) with an Enqueue and a Dequeue action and an array in the shift register. A little bit of smarts for manipulation of the array to cut down on data allocations, and you should hit the performance of the queue prims themselves. And you could apply whatever sorting behavior you want, from a fixed comparison op, to a passed in VI Ref to a dynamic dispatch call if your data is LV classes.

[gregoryng]

I wanted to post my use case that led me to find this idea. I am implementing a consumer that manages writing data to disk. It receives packets to be written to multiple files on disk. Packets for each file should be written in FIFO order, but write ordering among the files is not important. To improve disk streaming performance, it is desirable for one thread to handle all writes to disk, and to process as many available packets for one file as possible, before switching to writing to the other file.

So the consumer (file manager) would "wait on multiple" from one of several producer queues, then flush one of the queues' data to disk as they became nonempty.

In my case, the latency of polling the producers is fine, so for now I will probably roll my own.

[AristosQueue (NI)]

gregoryng: I have seen a system like this written by another user. They did this:

Have N queues, one for each of your content generators.

Have a "Someone has data" notifier.

Whenever a content generator enqueues to a queue, it fires the notifier. The content writer waits on the notifier. When the notifier fires, the content writer calls Flush Queue on each of the queues -- any that come back non-empty get written to the appropriate file.

In this way, you avoid continuous polling if you have gaps where no content producer is producing any content. If you don't have such gaps then continuously calling Flush Queue is just as efficient as waiting on multiple queues and waking when any has data.

[Darren]

Any idea that has not received any kudos within a year after posting will be automatically declined.