Build Array and Avoiding Buffered Allocations ... a little help here??

[jonathanupr]

I have been using a present DAQ system that has multiple while loops, there is 1 main DATA collection loop.  To get the data from the other loops that are running simultaneously I've been using Local Variables to pass into the main DATA loop and then building these into 1 main array (concatenating them). 

Is there a better way to do this?

Thanks a bunch

[DianeS]

Good God, yes.  What on earth do you have all those parallel loops for?  You only need two!  One to collect the data and one to write it to a file.  Use a queue to pass the data between loops, and use a notifier to stop both loops with one button. 

See attached.

The broken "Run" arrow is because you didn't include your sub-VIs in your post, therefore I couldn't include them in my modification.

Message Edited by DianeS on 11-10-2009 06:54 PM

[JÞB]

---

DianeS wrote:

Good God, yes.  What on earth do you have all those parallel loops for?  You only need two!  One to collect the data and one to write it to a file.  Use a queue to pass the data between loops, and use a notifier to stop both loops with one button. 

See attached.

The broken "Run" arrow is because you didn't include your sub-VIs in your post, therefore I couldn't include them in my modification.

Message Edited by DianeS on 11-10-2009 06:54 PM

---

jon-

You should do ten "Hail kNIghts of NI" for pennance!

Dianne -  Great Work!  'nuff said- Just posting to wathch the exchange he's all yours!

[Harold_Timmis]

Look up in the example finder notifiers and than look up master slave architecture and that should get you started.

[jonathanupr]

Diane thanks for the input.  I've attached the main VI architecture I am using (I went ahead and stripped it down to allow you to see what I am working with).  I've got a Queue system already set up, though more so for state machine execution and not passing data via a Queue.

So the Queue method would work much better?  I never really thought about putting all the device driver sub-vis into one while loop, is this the most efficient way to go?

Sad to say I've not dealt with passing data through Queues other than what I'd been doing with this particular VI.   

Take a look at the block diagram (the VI is broken, which is ok ... as I'm just searching for solutions on what I could do with this state machine setup).  Does your solution still work with this setup?  I'll start digging into info on passing data from one loop to the next (I've always sufficed for Globals and Locals).  

What about using Shared Variables?  Or a Functional Global?

Thanks

[DianeS]

You appear to be someone who loves a loop.

This is not in and of itself bad...it's perfectly ok to love a loop.  However, I think your fondness for them is verging on the unhealthy.  It might even be heading down that slippery slope.  🙂

You don't need a separate loop to handle each individual thing you want to do, as I pointed out in my previous post.  I looked at your code (by the way, it's difficult to tell exactly what's going on since I don't have any of your custom controls, subVIs, etc.) and there are six -- count 'em, SIX! -- parallel loops.

Yikes.

As nearly as I can tell, this is what you are trying to do in your code:

1.  Collect data (continuously, I assume) from some instruments

2.  When the user selects an item from the drop-down menu, perform some action on the data

3.  When the user presses stop, perform some kind of controlled shutdown

At the very most, you need three loops to do this.  (I can think of several ways that you could do it with two, but we'll leave that for now)  You need one loop for your event structure.  You need one loop for your DAQ, assuming you want it to be collected all the time regardless of what the user is doing.  And you need one loop for your state machine.

You're on the right track...you just need to put some more thought into this design and make it both more readable and more efficient, which is what I think you're trying to do.  What I showed you in my previous post can easily be incorporated into your VI.

So.  Put all of your DAQ in one loop and queue it out to the other loops as needed, the way I showed you before.  I must ask...in your NI-DAQ loop, if you encounter an error, you appear to handle it by closing down your CAN module and then trying to restart?  Why not put an error handler there instead and actually handle the error?  And what is that feedback node for?  You already have a shift register to pass the data between loop iterations.

I have absolutely no idea what your third loop is doing.  I have none of your subVIs or custom controls, etc. so it's difficult to determine what's going on there.  There are -- what -- five nested structures in it?  The while loop, then a case structure, then a sequence structure, then another while loop, then another case structure.  Ouch.  Why can't whatever it is you're doing be done in your state machine?

Queues are generally the best way to pass data between parallel structures, as they preserve dataflow (unlike locals and globals).  You can see exactly where the data is coming from and where it's going.  You can also use them between VIs, which is very handy.  Meaning, you could conceivably put your DAQ in a subVI, have the subVI running in your state machine loop (outside the state machine!), and still queue the data to your main program.  (Ladies and gentlemen, feel free to chime in as to why this is not a good idea -- it's worked well for me but I would be happy to hear arguments against it.)

Please shrink down your block diagram (get rid of all that empty space) to fit on one screen (or, at worst, only sprawling off the screen in one dimension) so I don't have to scroll all over the place trying to figure out what you're doing and what data is going where.  And tidy up your wiring.  And why are you bringing two references to the same queue into the loop containing your event structure?  Oh, and what is that sequence structure in your second loop for?  And when you terminate the program, release your queue.  If you want to flush it too, that's fine, but remember to release it.

Hope at least a little of this helps.

Oh, and thanks, Jeff.  I'm adding some more "Hail KNIghts of NI!" penances, too.

Message Edited by DianeS on 11-11-2009 03:42 PM

[jonathanupr]

In regards to implementing the queue for the DAQ data, how do you actually go about implementing 2 queues within 1 VI? 

I'm trying to wrap my mind around this, the queue that I am presently using is constantly calling the "Data" case within the state machine loop (this is how I am allowing the user to select the time duration to log data to spreadsheet).  It seems that if I were to add the DAQ data directly into this queue that I would be directly causing a traffic jam.  Is this correct?  If so, how do you go about adding another queue within the VI?

Also, just a quick question:  why is the queue the best way to pass data to another loop within the VI?  I can see the synchronization of data flow, but could this be done via a Functional Global?  Are there less buffer allocations with a queue setup?

Thanks.

[DianeS]

Just add another queue to your VI.  You'll have two...one for your state machine and one for your DAQ transfer.  You can have as many queues as you need in a VI.  You aren't limited to just one!

When you use a queue, you aren't going to lose any data, unless of course you choose to flush the queue before you have dequeued the last element.  If your DAQ loop executes faster than your file write loop, the data will just pile up in the queue and be stored there, in order, until your file write loop has a chance to dequeue it.  Likewise, if your DAQ loop executes more slowly than your file write loop, your file write loop will wait patiently until something shows up in the queue.  The execution of one loop is not dependent on the execution of the other -- which is the whole point of having parallel loops, right?  They can run at different rates, as needed, and not screw each other up.

To understand why a functional global is not a good idea here, think through the way a functional global works.  Now think through how you would have to write your code in order to use one.  A functional global is a sub-VI containing a single-iteration while loop (there's another loop for you, no wonder you keep fighting the idea of using a queue! ) and an uninitialized shift register.  Right?  Ok.  So you'd need that sub-VI in your DAQ loop so you could write to it.  You'd need another instance of that sub-VI in your file-writing loop, do you could read from it.

So, suppose for some reason your DAQ loop iterated two or three times for a single execution of your file write loop.  Now you've lost two or three iterations' worth of data.

Say your file write loop executed two or three times during a single execution of the DAQ loop.  Now you've written the same data to your file three times.

And then there's the problem that the sub-VI containing your functional global can't execute simultaneously in both loops unless you make it reentrant.  Making it reentrant completely defeats the point of the functional global, since now you have one instance where you write to it, and a totally separate instance where you read to it, each of which contains its own SEPARATE data.  That's the whole point of reentrant execution -- the clones are completely independent and don't talk to each other.

Now, since the functional global can't execute simultaneously in both of your loops, but each of your loops contains that sub-VI, your loops are going to be dependent on each other in an unpredictable fashion.  So much for independent operation...and have a good time hunting down any odd behavior that might (will) occur.

I know this is kind of a verbose response, but I really don't feel that you have a good understanding of a lot of this stuff.  You seem very concerned with buffer allocations, but the real key here is understanding how your code will behave for each of these different approaches, and then choosing the best approach.  Leave the buffer allocations for now and try to think more about effectively architecting your code so that it behaves consistently and properly.  Good coding techniques will get you there, but you need to understand them first.

Message Edited by DianeS on 11-14-2009 08:04 PM

[jonathanupr]

Again thanks for the response.  Definitely on key with my lack of experience with these issues...thus it is very helpful to get info on the subject matter at hand.  The VI I am co-editing right now was an inherited VI, have done some pretty decent things with it thus far..but wanted to start tucking corners a bit in going back and looking to make it as efficient as possible. 

I totally missed the screw ups and difficulties that the functional global would've caused...would've found it out, but a waste of time.  

I take it queues are pretty simple?  If not, then the lack of subject matter on them (I haven't been able to find much) could use a bit of a push.

So queues can function in parallel just like the while loop?  

So in what areas would the current VI I have without the queue really suffer (areas such as memory performance, loss of data, lack of data synchronization,etc...) ?

[DianeS]

If you haven't been able to find much information on queues, then I have to ask:  where have you been looking?

First of all, there's the LabVIEW help which ships with LabVIEW.  It contains information on how to use queues, plus examples of how to use them.

Then there's this forum.  If you search "queue", you'll get over 6000 hits (I just did it, so I know this for a fact).  If you search "multiple queues", you'll get over 300 hits.  If you search "parallel queues", you'll get about 400 hits. 

Then there are books...such as "The LabVIEW Style Book" and "LabVIEW Advanced Programming Techniques".  They're both available on Amazon.

I am definitely gaining the impression that, in spite of having one in your program, you don't really understand queues, what they do and how they're used.  You should really understand your own program, particularly if you want it to work.  You say you inherited the code...was it already set up as a queued state machine?  If not, what led you to that particular architecture?  Someone told you it was the best one for your application?  It's great to follow advice, but not to skip understanding.  So let's work on developing your understanding. 

My usual approach to learning something is to sit down and write myself a little VI that illustrates what I'm trying to do.   Then I fool around with that VI until I get it to do what I want.  This allows me to narrow down a problem to its essence, then try various things and watch how they work.  That's how I develop understanding.  "If I set it up like this, it behaves like this."  Very valuable! 

If I get totally stuck, I log on to these forums and do a search for information on what I'm trying to do.  I am seldom, if ever, the first person to ever attempt whatever it is I'm attempting -- as is the case with you in this instance.  99% of the time, I'll find a solution which I can tweak to meet my needs.

If none of that works, I post the question. 

So.  My point is that there is TONS of information, readily available, on the subject at hand.  I'm going to ask you to look for it on your own.  Once you've done that and gained a better understanding of what you're doing, you will be able to answer the questions in your most recent post by yourself.  In fact, you should be able to answer them right now, with the information I've given you in this thread.

Go over your options, think your way through them, write some code for yourself to illustrate how each of the possible approaches would work, and then come to your conclusions.