Data handling between VIs for idiots

[MancPablo]

Wibble....

Message Edited by Manc Pablo on 07-31-2008 10:08 AM

[tst]

I won't answer all the question for Damien, but I can answer some of them.

---

1. Do you create the queue with size=1 originally?  If so, does this allow LabVIEW to do some further optimizations?

---

If you're diligent about the way you implement this (and I believe you have to be for it to work correctly), you don't actually have to set the queue size, but in any case, setting the size on a queue has no optimization value (at least according to the current documentation).

---

2. How are "writes" handled exactly?  I first picture doing a Deque (and discard) followed by an Enqueue of new data.  Is this guaranteed to act "atomically"?

---

That's exactly the concept, and it also has to do with your next question. The idea is that you always set the dequeue timeout to -1 and that you start every operation with a dequeue. That means that if the queue is empty, the entire read-write segment has to wait until the segment which currently has the element puts it back into the queue. This basically guarantees an atomic operation.

Regarding the data copies, I suppose that depends on the exact nature of the diagram. Remember that even if you want to re-enqueue quickly, LabVIEW probably does not "know" that you're using the queue to lock other sections of the code and will probably prefer to optimize the data copies. Of course, that's a guess, but not a wild one.

[DFGray]

A bit of code is worth a thousand words.  I have attached the templates I use to create a standard reference object.  These are written in LabVIEW 7.1.  Use the In Place element to access the data elements if you have LabVIEW 8.5+.  The code does not answer all your questions, however.

1. Setting the size does give you a bit of an optimization due to all element allocation occurring at queue initialization.  However, in practice, this is meaningless if you always use only one element.  I set the size to one element to make the object as robust as possible.  I write infrastructure code that is used by programmers of many experience levels, so try to make things as easy as I can.  It also helps keep me honest
2. See the attached code for a write (I call it a "set").  It is guaranteed to be atomic because nothing else can access the data since it is temporarily out of the queue.
3. I have always used infinite timeouts for the queues.  I can easily envision situations where you would not want to do that, but have never run into one.  Note that the destroy VI does a force destroy, causing all get/set VIs which are waiting on data to immediately run with an error.  If your program is designed well, either this will not happen or the program will expect the error and use it to take appropriate action (e.g. exit, abort, &c.).
4. If you fail to enqueue after a dequeue, it will cause a hang/timeout the next time an enqueue is requested.  The easiest way to deal with this problem is to directly connect the error wire from the dequeue into the enqueue.  Placing anything else between them will cause problems.  You can combine errors at the end of the VI.  I often isolate the error in/out of the enqueue/dequeue from the rest of the error flow until the end of the containing VI.
5. If the dequeue produces an error, it did not dequeue data (you get a default data set).  I can not think of any situation where you would want to enqueue data under that situation.  A bad queue reference will simply produce another error.  An input error to the dequeue means the data was not dequeued, so an enqueue will hang/timeout or cause a pending operation elsewhere to hang/timeout.  A force quit will cause the dequeue to error, but there is no queue any more, so an enqueue is pointless.  Run the error wire directly from the dequeue to the enqueue and it will automatically take care of most circumstances.  Note you should only enqueue if the dequeue did not time out.
6. Notifiers always make a copy for every reader and do not guarantee delivery.  As implemented, with infinite timeouts, once the queue is empty, all other readers will wait until the enqueue occurs.  No matter how long it takes, the process is atomic.  Data copies are up to the programmer.  Once the data is dequeued, it is possible to make many copies before passing one back to the enqueue.  As mentioned above, the In Place element helps, but users of pre-8.5 LabVIEW can use the cluster "magic pattern" to reduce copies.  Despite what the buffer viewer says, there really is no copy made when dequeueing the data.  This is why the queue is such a good reference object.

Hopefully I answered your questions.  If not, let me know...

[tst]

DFGray wrote:

Setting the size does give you a bit of an optimization due to all element allocation occurring at queue initialization.

---

As you mentioned, it's meaningless in this case, but: "max queue size only limits the number of elements in the queue. It does not preallocate memory" (from the LabVIEW 8.5 help). It is important in queues which hold more data.

[MancPablo]

As you say, Mr DFGray, a bit of code speaks a 1000 words.

I have generated a small simulation of what I need to do, which I have attached. Is this the sort of setup I should be using and the correct implementation of queues?

And finally (I hope) I will be generating multiple elements. Is it better to: a) have multiple single element queues with different names? b) A single multi-element queue? or c) have a single queue with a single array of multiple elements? (assuming I have all the same data types)

thanks in advance for yours (and everyone elses) advice.

Paul


Message Edited by Manc Pablo on 08-05-2008 07:56 AM

[RavensFan]

First, make sure you have a way of stopping your program besides using the Abort button.

I wouldn't create one master queue.  It would be putting all your eggs in one basket.  And you don't want too many queues as that would be a lot to handle.

I would recommend looking at all of your variables.  Group them into ones that work together.  Create a cluster that contains them and let that be the datatype for 1 queue.  A group of related variables that are unrelated to the first would be a 2nd cluster and the element type for a 2nd queue.  And so on.  Use a handful of queues where the datatype is a cluster of related variables.

[MancPablo]

Thanks for the info.

So what is the advantage of a cluster rather than an array?

Paul

[RavensFan]

I can see 2 advantages.

1.  You can name each element of the cluster so that you can unbundle by name within your code and that makes the code self-documenting.  With an array, you will have to remember or make notes as to what element 0 means, element 1,  .....

2.  You can mix datatypes.  For example, a boolean that indicates run status and a numeric that indicates speed could be put in the same cluster and remain bundled together.  In an array, all elements have to be the same datatype, so you would have to translate the boolean to a numeric value such as 1 or 0 to store the data.

One disadvantage is that you cannot scale you data structures at run time.  For example, if you have a cluster of 5 different numerics representing speed, you can not change it while the code is running to add a 6th speed channel unless you've already accounted for that in your programming.  But with an array, the user could define 1 to an infinite number of channels (within reason of course) during runtime and the program could iterate through the arrays to extract or replace values.

There would be ways around this such as arrays of clusters, or clusters of arrays, or arrays of clusters of arrays,  clusters, of arrays of clusters, .....  and so on.  Just a matter of which arrangement of structures gives you the flexibility you need for handling your data.

Message Edited by Ravens Fan on 08-05-2008 11:44 AM

[DFGray]

I see Ravens Fan has given you all the info you need.  I would like to add a couple of other points.  When I start a LabVIEW project, I design it from an object-oriented perspective (a remnant of the days a decade ago when I programmed in C++).  Each object is a collection of logically related collection of data and methods.  The data goes into a strict typedef cluster which is stored in a single-element queue (I usually make the queue reference a strict typedef, too, as in the example code above).  This determines how much data goes into each cluster.  The methods are stored next to the data and namespaced with it.  In LabVIEW 7.1.1 and before, namespacing was a prefix on the VI and control filenames.  Now, namespacing is more easily done with libraries and objects.  I would suggest using objects and making the object data your queue reference.  That way you get inheritance too.

If you are unfamiliar with object oriented programming, I would highly recommend you learn it.  Like the shift from C++ to LabVIEW, there is a large context switch going from non-object to object oriented programming.  However, it is worth the effort and will reward you with easier modularization and more reuseable code (but only if you actually stick with your pardigms).

Let us know if you need more help.