state machin and daqmx aquisition problem

[coralie38]

Hi,

I am developping a program using a state machin.

One of the state of the program is the acquisition part. 2 orbit displacement sensors and 1 analog input differential channel is acquired.

The while loop around the state machin is clocked to 1ms.

I encounter a problem after adding the acquisition of the analog imput channel using Daqmx.

When I use Analog DBL 1 chan 1 sample option in the  read Daqmx VI, I get the erreur

"200 279 The application is not able to keep up with the hardware acquisition.
Increasing the buffer size, reading the data more frequently, or specifying a fixed number of samples to read instead of reading all available samples might correct the problem"

When I use Analog DBL 1 chan N samples option in the  read Daqmx VI and give a fixed number of samples to read, the program is slowed down very much, the sampling frequency going from 700 Hz to 100Hz.

Do you have any suggestion?

Here enclosed, the program main.vi.

Thanks,

Coralie

[Bob_Schor]

Are your VIs in LabVIEW Project?  [If not, they should be -- learn about LabVIEW Projects to "logically organize" your VIs].  If so, you can compress the folder that includes the Project (and its files) and attach the resulting .zip file, which will give us access to your sub-VIs so we can open your Main without "Missing VI" errors.

You are off to a good start with a State Machine.  Now you need to learn about the Producer/Consumer Pattern (one way is to open LabVIEW, click File, New ... (the dots are important) and do "New from Template, Producer/Consumer Design Pattern, Data").

What you want to do is to have two loops, your State Machine, which includes the Data Acqusition part, and needs to acquire data at a fixed, hardware-driven rate, producing data, and a Data Processing loop, running in parallel that "consumes" the data, saving it, looking for a Start Signal, etc.

One thing about State Machines -- the "variables" that it handles are generally carried in Shift Registers, with as little use for Local Variables (you're pretty good about this, except for Points -- put this on a Shift Register, initialize it to 0 in your Initialization state, and wire it to the Points indicator on the right edge of the While Loop).

So here's a question -- how "tight" is your timing for initialization?  It looks like you are reading data, waiting for it to reach some stable state, then starting your collection.  Suppose you took data at 1KHz, but collected 1000 points at a time (so you'd get a new set of data 1/sec, implying your "start time" resolution would be 1 second).

You'd set up your State Machine to have an "Acquire 1000 Samples" State.  Let's start by saying that we are only going to Acquire, but are not going to do anything with the data (pretty silly, I know, but I'm trying to make a "teaching point" here).  So you have a DAQmx Read function to Read 1000 points (at 1KHz).  Oh, you also have the DAQmx Timing Function set up for 1000 points, Continuous sampling.  So you do your first DAQmx Read, and it takes 1 second for it to finish.  The "Next State" is (for this example) "Acquire 1000 Samples".  Do you see that this State will be executed exactly at 1 second intervals?

OK, but now you want to use this data.  So after acquiring the 1000 Samples (which you put on a Shift Register), your next state is "Decide if Ready to Save".  I'm assuming that any computations that you need to make take much less than 1 second, so won't interfere with the ongoing Sampling.  Here's a nifty trick -- you have another Shift Register with the State you need to do after "Acquire 1000 Samples", initialized to "Decide if Ready to Save".  The output of "Decide" is to either leave this Shift Register alone, or to place "Process Acquired Data" on the Shift Register.  And, of course, the "Next State" used by Acquire 1000 Samples is exactly the value saved in this Shift Register (it sounds complicated, but if you make a little VI and build this, it should be obvious).

OK, so what about "Process Acquired Data"?  If you are sure you can accomplish all of the processing within 1 second, you can just make this another State in your State Machine.  However, if you think that it might occasionally (like, perhaps, the first time you call it) take a little more time, (because maybe you need to open an output file, re-initialize a Chart/Graph, set Property Nodes, etc.), you can simply put the Acquired Data onto a Data Queue and let the parallel Consumer Loop "consume" (= process) the data.

The point here is that the critical Acquire 1000 samples case does nothing else than acquire 1000 samples, so its timing is fixed by the hardware clock in your DAQ device to be exactly 1 second.  Note that the structure of the State Machine is "do everything else you need to do, then do an Acquire to use up the remaining time until the next "tick" of the 1-second clock" -- the logic in this particular Case is "Wait until the data are ready, then use almost no time, a few microseconds perhaps, and give the data to the user".  If, however, you try to do something else within this state, you have "Wait, Acquire, use time in processing", so the loop can run slower than its 1-second period.

If you need a finer-grain control of your time (i.e. if you need to make decisions faster than 1/sec), you can change your sample size to 100 (in which case your Acquire loop runs at 10 Hz, or 0.1 sec) or 10 (100Hz, 0.01 sec).  You could even process every point, but then you'd really need a quick "Decide if Ready" case and your "Process" case would almost surely require a Producer/Consumer pattern (you can easily "export to Consumer" data in 1 msec, and since its in a Queue and out of the State Machine loop, the processing won't "block" the on-going acquisition, but it makes things a little more complicated).

Hope this is helpful.

Bob Schor

[cora38]

Hello,

Thanks for all your adivces.

I followed the NI CORE 1 training last year and I wanted to implement state machin for this project because I thought it would be the best for what I wanted to do.

Yes, VIs are inside a project. I attached it in a zip file this time. I translated most of my comments in english as well.

Let me explain more in details what is the purpose. The objective of this program is to move a calibration bench vertically by generating a pulse and to acquire data from 2 displacement sensors and an analog input during the duration of the pulse. The data acquisition rate should be around 700Hz (the best would be 1KHz but it seems to be difficult). Several pulses can be generated (defined by the user). A new file is create after each pulse to store data in different files.

The state machin I developed do the following:

1- Test on the number of pulse already generated

2- Open a file n°n

3- Generation of the pulse n°n

4- Acquisition and data stored in arrays during the pulse duration

5- Data copied in file

6- Generation of a new pulse + opennew file + acquisition .... or data process if number of pulse attempted

7- Stop program

I did not know about the Producer/consumer design pattern but it seems to be a good solution to acquire data at the maximum rate and store data and display them in the consumer loop without interferring with the acquisition part.

However, I am wondering how to implement this and keep the state machin.

After searching on the forum, I found 2 possibilities:

- Keeping the state machin as it is and build the P/C pattern in the acquisition state

- Build the P/C pattern and create the state machin inside the producer loop. Once the acquisition state is reached, data is produced, queued and transfered to consumer loop which store data and display them.

Can you suggest me which would be the best?

Moreover I have another question regarding the acquisition frequency. How to fix it?

Is it fixed by the DAQmx Timing + "samples to read" for the whole loop (acquisition frequency for both analog input and displacement sensors) or do I have to had a clock inside the loop?

Thanks for your help,

Regards,

Coralie

[RavensFan]

"Machine".  Not "machin".

[Bob_Schor]

Here are some suggestions and things to try to change in your State Machine implementation:

• Variables (such as Pulse Number) should live on a Shift Register.  Initialize them in an Initialize State or by wiring from outside the While Loop to the left Shift Register.  Place the Indicator near the wire (which runs through all your States, including the States that change it) outside the Case Statement and just before the While Loop ends.
• You do not need the Sequence Frame here -- it only gets in the way.  All your sequencing (in Main) is handled by Data Flow (everything with a wire going into the While loop has to run before the While Loop is entered, and nothing wired to the output of the While loop can run until the While loop exits).
• Try to keep all wire straight going across the While loop.  I like putting "Variables" (wires connected to Shift Registers) in closely-spaced parallel Shift Registers near the top of the While, and bringing them down when you need to use them.  This includes the "Reference" wires coming out of DAQmx and your Instrument VIs.
• It is OK to have a lot of little States and to call them sequentially (unless you need them to run simultaneously).  I always put one (or several) "Initialize" states in my SMs, and thereby avoid having initialization code "outside" the While Loop (the SM).  
• There should be a single Error Line running through the SM.
• Each State should do (only) one thing (though I often do several common Initializations).  For example, "Initialize Instruments", "Initialize AI Samples", "Open File".  Each (simple) State has its function on the (single) Error Line, "pulling down" its required Inputs and Outputs from the "Variable Wires" (and, in the case of the Initialization States, actually defining these initial values).
• This method of creating State Machines leads to many states having lots of Shift Registers that need wires connecting the left and right terminals.  Do you know about the "Auto-wiring" trick?  If you look at the output Tunnel on your Case Statement, right-click the tunnel and choose "Linked Input Tunnel, Create and Wire Unwired Cases".  A second later, all your Cases have wires.  You'll need to "break" a few wires, of course, to connect them to a function in the Case that uses them, but it saves a lot of time and effort.
• You should have no trouble achieving 1KHz throughput.  Your State Machine should have a single State called "Produce Data" that has the DAQmx Read in it (your current "Acquisition" State).  What you do when you have the data is to immediately put it into your Data Queue and send it to your Consumer Loop.  I'm going to come back to the Consumer, which may also turn out to be a State machine ...
• Instead of using Elapsed Time or other complicated calculations to decide whether to continue "Producing", consider using a counter (I want to acquire for 10 seconds, 1000 points at 1 KHz = 1 second per loop, or for 10 loops) and simply "Count" to decide whether the next state should be Produce Data or "Decide what to do next".
• If your Consumer is going to be doing File Handling, then it, too, might want to be a State Machine, having a State called "Open Files" (and a "Variable" in a Shift Register being the File Refnum), a second state called "Save Data" where you wait and dequeue the data, writing it to the file, and a final "Close Files" that closes the file and exits the Consumer loop.
• But now you have a problem -- how does the Consumer know to stop?  One method, which I've used, is known as the "Sentinel" method.  The Producer normally sends an array of 1000 points to the Consumer.  The Producer also knows when you are finished taking data (either because you pushed a button saying Stop, or it took "enough" data, or some other criterion).  So when the Producer is ready to exit, it sends one last set of data, an empty array, to the Consumer.  The Producer does not release the Queue, as the Consumer still needs it -- it lets the Consumer release the Queue.  
• The Consumer, when it dequeues the data from the Producer, checks to see if it got an empty array (the "Stop" Sentinel).  If not, it saves it to a file as normal, and stays in its "Save File" state.  Otherwise, it goes to the Close File State, which closes the File.
• Most of my State Machines, in addition to having an Initialize State, also have an Exit State, which is the final State to be called.  This is the State that has the Stop constant to stop the While loop.  It is also the State where (in the Consumer) the Data Queue is released.
• Another State that is good to put into every State Machine is an "Error" State.  I often have a VI I call "Error Handler" sitting outside the Case Statement on the "output" side.  If it detects an Error on the Error Line, it (a) saves the Error on a Shift Register, (b) intercepts the "Next State" line and substitutes the Error State, possibly saving the Next State if you want to try to recover from the error, and (c) lets the State Machine proceed, but now to the Error State.
• The Error State can pop up a Dialog Box informing you of the Error and letting you Quit or Recover (you'll need States to handle both cases).  If you are going to Quit, you'll want to close things "gracefully" (so the Producer should send the Consumer the Sentinel), and then do "Exit" next.

Whew!  There's a lot of stuff there, but take it one step at a time and you'll end up with a much more robust State Machine and (I think) a good appreciation of the Producer/Consumer design.  I'll also bet that your Block Diagram will be much smaller, possibly even able to fit on a single Laptop Screen (which is the Design Criterion I always try to meet for my own code).

Bob Schor

[cora38]

Hello,

I really appreciate your help and I would like to thank you for that.

I am trying to understand each point of your last message and try to apply them to my program but i am still confused about some of them:

- First about the acquisition state (Produce data) and the consumer state (data storage): do I have to create a new state in the machine for the consumer part or do I have to make a consumer/producer pattern with a parallel loop outside the state machine for the consumer part? I understood that the consumer dequeue the data, store them in a file and release the queue when the producer finised using the sentinel method to close the file.

- Then, I acquire data with the Daqmx but also other sensors (serial link) with "read_Orbit" VI. I would like to acquire them at the same time. I should include read_Orbit VI + Daqmx in the same acquisition state, don't I? And maybe sequence them with the error line?

- Regarding the single error line, I do not understand how to achieve it as I have many error lines in and out for initialization and close part (relays, orbit, DAQmx etc...). Should I gather all the init sub VIs in a single sub VI and link the error out of each to a single error out (same for the close part)?

- Finally, I do not really understand your third point explaining the "variables" closed to the shift register...

I think it would be easier for me to see an example as I am not very much familiar with it. I am sorry to ask this much but I don't find some examples on the web that could show me how to process to do that.

Thanks,

Coralie