How to Best Implement DAQmx DO, AO, and AI with TDMS & Producer-Consumer Design Pattern

[qtism3]

Hi,

I started my program using simply state machines to control the flow of my DAQ operations, but I am beginning to find that my application is running much slower than intended. In addition, the program is becoming harder to re-use in terms of its modularity. I have read extensively on possible design patterns that could better improve the performance of the program. Thus far, I've looked over producer and consumer, master and slaves, queue message handler, and JKI state machine. I understand the basic high-level application for each of the patterns, but I am still having some trouble understanding how to implement the finer details. Out of the previous options, I am leaning more towards producer and consumer. Please advise.

Specifications

- Hardware - USB6008

- Hardware - Stepper Motor (ActiveX control)

- Hardware - Robotic Arm (Serial control)

- Six DO for Relay Control

- Six DI for Relay Read

- One DO for On/Off Operation

- One AO for Signal Generation Operation

- One to two AI for Data Acquisition

Timing of the operations is crucial, but not to the bits/s or byte/s. The program needs to make sure that one operation is done before the other one starts, especially for the stepper motor and robotic arm control. The signal generation and data acquisition needs to be able to occur simultaneously. I have already taken care of how to get the program to communicate with the stepper motor and robotic arm.

I have been following Paolo's example on Producer and Consumer (see snippet and link).

Some Questions (For Now)

1) Best practices for creating DAQmx Input and Output in Consumer Loops? If SubVi is suggested, how to best create one? If not, how to break the DAQmx operations efficiently in the state-machines?

----- In my previous attempt, I have over 40+ states. I have also broken the start, read/write, and stop of each task in the states. I created and cleared the task outside. However, it became hard for me to loop through different settings (i.e. different amplitudes/frequencies/numeric controls) if I, for example, were to input in a text files with over 10+ settings sets. It was my intention to keep the program simple but still robust and efficient, but due to my lack of experience... the program just grew!

2) What other SubVis should I consider implementing?

3) How to best address the synchronous (signal generation, data acquisition, and TDMS data logging) and asynchronous (relay control, stepper motor control, and robotic arm control) timings?

Looking forward to the responses. Thanks in advance!

[Kevin_Price]

I won't claim these to be *best* practices, just sharing some thoughts that I've found to work out fairly well.

I largely gravitate toward a Queued Msg Handler structure with one "main brain" loop that coordinates several other dependent Msg Handler loops while receiving data from them.  These auxillary msg handling loops tend to be built around on a specific instrument or logical grouping of devices.  I try to design them to have a relatively simple messaging API and give them internal smarts & independence.

My starting point would imagine one msg handling loop for the Stepper Motor API and a separate one for the Robotic Arm API.  That's not a hard rule though, you may find it easier to coordinate their sequential operations by putting them in the same loop and carefully designing both your messaging API and the internal "state variables" to work closely together.   A similar idea holds for the buffered AO and AI DAQmx tasks.  I initially imagine them as separate loops, but it may be easier to manage their coordination if you put them in the same loop.

Now, on to your specific questions:

1. 40 states is not trivial to organize and manage.  Consider whether there are cases where you can combine a sequence of states into a single subvi instead.   If you want to work through many distinct config sets, you should have "states" or "message cases" inside the main loop that can handle the needed task config and de-config.

2.  I make subvi's either to package functionality for reuse, or to save block diagram space (possibly to help clarify a sequence of higher-level operations).

3. I make data acq loops that are responsible for publishing their data back to the "main brain" loop.  The main brain can process, display, or repackage it for a data logging loop.  I wouldn't argue against sending data direct from the data acq loop to the data logging loop though.

   As to the async stuff, that all works very nicely with the QMH structure.   The "main brain" can receive any info whenever it happens to arrive, then uses it to decide what control messages it may now need to send to the auxillary loops.

There are times that a QMH loop and a parallel state machine loop on the same block diagram make a very useful combo.  You can get the responsiveness of a QMH along with the sequencing/logic/structure of a state machine.

-Kevin P

[qtism3]

Hey Kevin P.!

Lots of thanks for the push toward QMH. I've spent the past few days making the changes and played around with some of the functionalities that the QMH Continuous Measurement & Logging (CML) example had. It is much easier and cleaner to program with.

I'm still working out some kinks, but thus far I am satisfied. Currently, I have all the functions in one loop, but I might take your advice and separate them into multiples. I also have the issue with the waveform not displaying correctly (see image).

Some more questions:

Do you have any advice on iterating through a specific task a number of cycle? I think I can re-use my EnqueueMessageArray.vi and change it to iterate through one specific task and add them to the queue, but how would that affect my queue buffer? At the moment, it is set at 10 (I think I will end up increasing this if I want to automate through 1000+ cycles).

Also, do you happen to have any examples (other than LabViews's CML) that I may refer to?

[Kevin_Price · Accepted solution]

A QMH and the need to march through a repeating sequence of tasks isn't the most natural fit.  Some of my QMH goals are:

- the vast majority of execution time should be spent at the Dequeue function, idly waiting for the next message.  An implication of this is that the queue must usually be empty, allowing the QMH to be almost instantly responsive to any new message.

- each message must be handled quickly, i.e., each message case executes quickly.  

- expect that any message could arrive at any time, so code for robust behavior regardless of the message sequence

Pre-queueing a long list of messages to try to implement a repetitive sequence violates the first goal and tends to complicate the third (due to shortcuts caused by coding with an expectation about msg sequence in mind).

If you like the QMH overall, there are ways to work this in.  One is to wedge it right into the QMH loop.  You'd build something like a sequencer action engine.  At appropriate times, you'd load up your long repeating sequence into it.  You'd need to call it other places too to update the sequencing progress it'd track internally.  And you'll have to be careful how you treat conflicts between what the sequencer wants you to do and any different new message sent in on the queue.

Another is to add a dedicated task sequencing state machine in a loop parallel to the existing QMH.  In this approach, most of the work and code is in the state machine.  The QMH is primarily there to handle interactions with the other QMH modules and to have a very high level supervisory control over the state machine (when to start, pause, stop, etc.)

Food for thought.  Meanwhile, a couple detailed comments on your posted code:

- when you find yourself enqueuing an array of messages, consider whether you should implement the sequence as a subvi instead

- QMH queues would typically not be limited in size

- QMH dequeues would typically have an infinite timeout.  Not always though, so if you have a good reason to make an exception, just be sure to handle the expected timeout condition gracefully.

- Wait functions rarely belong in a QMH main loop, and especially not outside the msg handling case structure.  If you need enforced delays, I'd strongly recommend you approach it a different way.  One example would be to put the delays into a state machine loop that's parallel to the QMH.  Also consider implementing long delays with a short delay that repeats until the full long delay elapses.  Between those repetitions, it's then possible to be responsive to important events like errors or shutdowns.

Tidbit: I found the following thought helpful as a summary of what distinguishes a QMH from a queued state machine.  In a QMH, state information is held in a cluster and state transitions occur when cluster data is modified inside particular message cases.   A QMH lingers at the Dequeue function outside any message cases and is usually waiting to be told what to do next.  In a state machine, state is implied by the case being executed so state transitions occur when you leave one case, iterate the loop, and enter another case.   A state machine lingers inside a particular state case and usually makes its own decisions about what to do next.

-Kevin P

[qtism3]

Thanks a lot for your help, Kevin! I will definitely try to address some of the concerns that you've mentioned.