producer -consumer code with several variables (types)

[Blokk]

Bob you just gave me the last push, I just have to start playing with these Channels on the incoming weekend 🙂 However one question regarind to this part: "Notice the Messenger Channel coming out of the State Machine (specifically coming out of the Error Handler), which allows the State Machine to send itself State Information.  Such "looping" behavior is not that evident in this part of the code, but is important in the Producer."

Is there no such danger as in the case of QSM explained before (self enqueue-ing consumer), I feel the logic is the same (similar?) here?

Flo-w:

Sorry, I feel we hijacked a bit your post with this discussion, and you need some guide with your very actual problem.

First I would suggest to clean up your VI and put it into a project (if it was not done yet, also put all your subVIs/typdef controls in the same folder/subfolders, and in the project). Also, create the required typedef clusters for the needed data types for messaging (like a typdef enum plus a variant inside a typedef cluster). Your consumer loop is huge, you should shrink it and use some subVIs to encapsulate some of the functions. So the idea is the following:

• GUI loop with Event structure, Enqueu-ing commands to be send to the DAQ loop where you have your state machine
• Second loop or DAQ loop: since you want a continuous acquisition, just use a 0 timeout for the Dequeue, and you can check if it was timed-out: in this case sample your data (but do not forget a timing function here: for example use the "Wait Until Next ms Multiple" with the required sampling interval to slow down and make the sampling equidistant). If there is incoming command from GUI, process it with your state machine, and do the required action. In the sampling case, create a Time Stamp too, and alongside with your new data send these to the third loop.
• Third loop (display and data logging): Just Dequeue the sampled data and the time stamps, display them on a Chart for example, and log them into a TDMS file.

If you have the above done, or at least up to a certain level, you could post your project zipped up here, so we can have a better look. I see also some strange things in your "READ" case, like some VISA Flush I/O Buffer, why do you need these? Do you have a manual how to communicate with these hardware?

Finally, I see you created a few global variables for some parameters. I did not check deeper, but you might get in the danger to create some race conditions with these variables... Anyway, if you have a cleaned up code, it will be easier to see what you try to do.

[Blokk]

"Edit time has expired"...So as a new post here, sorry for the "noise":

Let me suggest something! I think it is more efficient and less confusing if we keep this place dedicated only to the actual task Flo-w tries to program. I created a separate post here for the more general discussion regarding to benefits/disadvantages of different design decisions:

http://forums.ni.com/t5/BreakPoint/Queued-State-Machine-pro-and-contra-snakes-eating-themself/td-p/3...

[Flo-w]

It's far from being finished, but I am now reaching a point where I am a bit confused, and sorry for asking it again but I am lost about the timeout:

1) is that ok to use a timeout event in the first loop ? if not, what should I do

2) my second loop is about aquiring the data from two devices + 1 DAQ. The DAQ run at 1 KHz. The previous waiting time was set at 100ms, which may be too rapid for the  2 other devices. So, what should I do with that second loop ? should I aquire every half a second? every second?

3) the third loop is where I display my data. I am here confused again about the refresh rate... I am adding the data to a queue in the second loop, but how are they displayed in the third one? does the refresh rate depends on the second loop (say the second loop is adding elements to the queue every 1/2s; are those data displayed every 1/2s in the third loop?) ??

Also I would like to record my data every minute. between every minute I'd like to save every data in an array, and average that after a minute elapsed, for finaly saving that averaged variable. is tha tunclear?

Sorry, many questoins, but I am a bit lost 😞

My code is messy but that the least messy I can do, and beleive me I've spent a lot of time trying to make it as clear as I could

Thanks for your help !

Flo

[Blokk]

---

@Flo-w wrote:

It's far from being finished, but I am now reaching a point where I am a bit confused, and sorry for asking it again but I am lost about the timeout:

1) is that ok to use a timeout event in the first loop ? if not, what should I do

No, you do not need a timeout in the top GUI Event loop. Do not trigger the "Read" actions from your top loop. Use a timeout for the Dequeue element at the second (VISA) DAQ loop, do your Read action when timeout happens (there is a "timed out?" output from this function) as it was adviced by crossrulz. You only do trigger other actions from that GUI loop. So no need for a timeout case at all in your Event structure!

2) my second loop is about aquiring the data from two devices + 1 DAQ. The DAQ run at 1 KHz. The previous waiting time was set at 100ms, which may be too rapid for the  2 other devices. So, what should I do with that second loop ? should I aquire every half a second? every second?

So you have 2 serial connections, and one DAQmx one? One option to take the two serial read in series, and in parallel you acquire the data from your DAQmx with 1 kHz. The question is, do you really need that 1 kHz sampling? Do you need all that data, or you just average them up? Do you need continuous sampling, or you do not mind if you have some "jitter" or "gap" between two bunches of sampled data array? I imagine you could just read out 500 values, and in parallel you read the other values from the 2 VISA sessions. Using software timing, you could get more or the less good sync (up to a certain level...)

3) the third loop is where I display my data. I am here confused again about the refresh rate... I am adding the data to a queue in the second loop, but how are they displayed in the third one? does the refresh rate depends on the second loop (say the second loop is adding elements to the queue every 1/2s; are those data displayed every 1/2s in the third loop?) ??

Exactly! But of course you do not need to display all values, using a module devision with a case structure, you can for example only display every second incoming value pack...That is why it is a good idea to generate a Time Stamp in your DAQ loop: you send this time stamp info to the Display/Data logging loop, so when you plot the values, they will represent the proper timing info, and not when they were dequeued in your third loop. Same applied for file logging: you log the time stamp into the file, so if sometimes your logging loop slows down a bit, it does not cause any problem: the timing info comes from the DAQ loop.

Also I would like to record my data every minute. between every minute I'd like to save every data in an array, and average that after a minute elapsed, for finaly saving that averaged variable. is tha tunclear?

This is just very easy using a case structure and a shift register (also have a look at the Point by Point averageing function)...

Sorry, many questoins, but I am a bit lost 😞

My code is messy but that the least messy I can do, and beleive me I've spent a lot of time trying to make it as clear as I could

Thanks for your help !

Flo

---

See some replies above with different colours. Sorry right now I have just not enough time to give more help, but I might be able to do so in a few days...

[Flo-w]

---

My replies in blue :

@Blokk wrote:

---

@Flo-w wrote:

It's far from being finished, but I am now reaching a point where I am a bit confused, and sorry for asking it again but I am lost about the timeout:

1) is that ok to use a timeout event in the first loop ? if not, what should I do

No, you do not need a timeout in the top GUI Event loop. Do not trigger the "Read" actions from your top loop. Use a timeout for the Dequeue element at the second (VISA) DAQ loop, do your Read action when timeout happens (there is a "timed out?" output from this function) as it was adviced by crossrulz. You only do trigger other actions from that GUI loop. So no need for a timeout case at all in your Event structure!

Ok, so I add an event structure in my second loop, with two cases: one timeout and one second event, but what do trigger that second event? the other actions are triggered by the first loop, but I dont really get what does trigger that "non-timeout event" of the second loop.

2) my second loop is about aquiring the data from two devices + 1 DAQ. The DAQ run at 1 KHz. The previous waiting time was set at 100ms, which may be too rapid for the  2 other devices. So, what should I do with that second loop ? should I aquire every half a second? every second?

So you have 2 serial connections, and one DAQmx one? One option to take the two serial read in series, and in parallel you acquire the data from your DAQmx with 1 kHz. The question is, do you really need that 1 kHz sampling? Do you need all that data, or you just average them up? Do you need continuous sampling, or you do not mind if you have some "jitter" or "gap" between two bunches of sampled data array? I imagine you could just read out 500 values, and in parallel you read the other values from the 2 VISA sessions. Using software timing, you could get more or the less good sync (up to a certain level...)

That is exact, 1 DAQ, 2 Serials. Ideally, I should sample every 500ms (so the average flowrate/pressures/etc are quite close to the reallity).
I've read this paragraph again and again, and few parts are unclear to m:

- will the loop be able to loop every 500ms with that much code to interpret in the (futur) timeout event ? my not P/C orientated version of that code wasn't able to loop every 500ms, hence my question.

- I want to display my data in the 3rd loop as often as possible, so that's why I wanted a 500ms loop in the 2nd one:to get a high refresh rate. 

- logging-wise, I'd say, I can average everything up and record that in a text file every minute (I guess I would have ~120 log per minute, so I would just have to do a very simple averaging of my arrays)

- you wrote: "One option to take the two serial read in series, and in parallel you acquire the data from your DAQmx with 1 kHz". I have looked on the forum for trying to find an example of that kind of architecture, and didn't find anything. could you please be more specific on how to do that ?

- "Using software timing, you could get more or the less good sync (up to a certain level...)" : so unsing the function wait until next ms right? but I don't see how that would sync my DAQ and the two serials...

3) the third loop is where I display my data. I am here confused again about the refresh rate... I am adding the data to a queue in the second loop, but how are they displayed in the third one? does the refresh rate depends on the second loop (say the second loop is adding elements to the queue every 1/2s; are those data displayed every 1/2s in the third loop?) ??

Exactly! But of course you do not need to display all values, using a module devision with a case structure, you can for example only display every second incoming value pack...That is why it is a good idea to generate a Time Stamp in your DAQ loop: you send this time stamp info to the Display/Data logging loop, so when you plot the values, they will represent the proper timing info, and not when they were dequeued in your third loop. Same applied for file logging: you log the time stamp into the file, so if sometimes your logging loop slows down a bit, it does not cause any problem: the timing info comes from the DAQ loop.

Here you basically state that the time of sampling has to be logged in the DAQ loop? so that's what I am doing already.

Also I would like to record my data every minute. between every minute I'd like to save every data in an array, and average that after a minute elapsed, for finaly saving that averaged variable. is tha tunclear?

This is just very easy using a case structure and a shift register (also have a look at the Point by Point averageing function)...

Sorry, many questoins, but I am a bit lost 😞

My code is messy but that the least messy I can do, and beleive me I've spent a lot of time trying to make it as clear as I could

Thanks for your help !

Flo

---

See some replies above with different colours. Sorry right now I have just not enough time to give more help, but I might be able to do so in a few days...

---

Thank you for your massive help!

[Blokk]

No, do not add another event structure to your code! You do not need. I already told you, you can use the timeout of the Dequeue in your DAQ loop to do the sampling! Do NOT use two event structures in your vi.

EDIT (timeout value was missing)

[Blokk]

Now answering some of your remaining questions:

That is exact, 1 DAQ, 2 Serials. Ideally, I should sample every 500ms (so the average flowrate/pressures/etc are quite close to the reallity).
I've read this paragraph again and again, and few parts are unclear to m:

- will the loop be able to loop every 500ms with that much code to interpret in the (futur) timeout event ? my not P/C orientated version of that code wasn't able to loop every 500ms, hence my question.

- I want to display my data in the 3rd loop as often as possible, so that's why I wanted a 500ms loop in the 2nd one:to get a high refresh rate. 

- logging-wise, I'd say, I can average everything up and record that in a text file every minute (I guess I would have ~120 log per minute, so I would just have to do a very simple averaging of my arrays)

- you wrote: "One option to take the two serial read in series, and in parallel you acquire the data from your DAQmx with 1 kHz". I have looked on the forum for trying to find an example of that kind of architecture, and didn't find anything. could you please be more specific on how to do that ?

- "Using software timing, you could get more or the less good sync (up to a certain level...)" : so unsing the function wait until next ms right? but I don't see how that would sync my DAQ and the two serials...

I am not familiar with your serial VISA code part, but you could explain your 2 serial HW components, and how you programmed the communication. Using two separate serial ports for these two devices should be fast enough to finish a read-out in half second (you can run them in parallel), unless you make some programming mistakes or these devices take unusally long time to respond to a "read" request.

Besides, please specify your 3rd device: is it an NI hardware? What kind of signal you need to read, parameters, settings? Required number of samples? Why do you need lots of samples to average it? Does this value fluctuate much? Do you program it properly using low level DAQmx functions (do not use the DAQ Assistant express VI!)?

You should make a single while loop test-VI only for testing the speed of these 3 DAQ units, before trying to integrate the DAQ part into the Producer/consumer project.

I imagine you do not need tight sync between the 3 measured values, you just want to get measurement values from your 3 devices in every 500 msec, yes? If so, something like this simple in my mind (this is just a possible, and might not the optimal solution!):

[Flo-w]

Thank you for your help. I am going to try and answer your questions as precisely as I can.

1) I tried to simplify my code (can't test it yet), but I expect it to be quicker. So maybe half a sec is doable. 

2) third device is indeed a NI hardware. they are two NI 92XX on a chassis. Should I set it to sample every 2 Hz, if such a thing is possible through the daq assistant ?

3) "Do you program it properly using low level DAQmx functions". I am unsure about what you mean here ?

I have modified my code accordingly, and I've got few more questions:

a) should the DAQ assistant be placed within the case structure containing my reading code ? or outside? or somewhere else ?

b)should I use a wait or a wait until next ms function? I don't really get the difference between the two

c)where should I put this timer? within the second loop? within the first case struct ?

d) the third loop will loop at 500ms (concidering I'll set the second one at 500ms), hence:

     - Do I stlil need a timer there ? (dumb qustion here, but I prefer to triple ask rather than struggling. I am still feeling unfamialliar with the P/C architecture)

    - I am using the first cell of my cluster (2nd loop - case structure true) to tell the case structure of the third loop what should happen. basically, if "1" -> display and record. but that brings me to the point where I actually never send any other information rather than "1". What I mean is that if the dequeue is timed out, that will trigger the true case structure, hence the reading of my serial (2nd loop), but should I send an empty bundle containing just "0" in the first cell to tell the case structure of the third loop what to do in case the dequeue isn't timed out (means I clicked a button somewhere on my VI)? which would be bassically to keep displaying the last variable received... is that clear?

     - which leads me to ask: should I have this case structure in the third loop ? should I remove it / use shift register for every variable displayed there?

I have attached my code. feel free to critize it. You guys are amazingly helpfull !

Flo

[Blokk]

I give some replies with different colour. I do not check your project yet, I beleive you will make soon changes again, and I do not want to work twice 😉

---

@Flo-w wrote:

Thank you for your help. I am going to try and answer your questions as precisely as I can.

1) I tried to simplify my code (can't test it yet), but I expect it to be quicker. So maybe half a sec is doable. 

Well, I suggested to make a test VI with a single While loop, and put your two VISA reads inside to benchmark the required iteration time...

2) third device is indeed a NI hardware. they are two NI 92XX on a chassis. Should I set it to sample every 2 Hz, if such a thing is possible through the daq assistant ?

I already wrote you NOT to use DAQ Assistant, it is not intended to be used in real applications. It initializes/closes resources at EVERY iteration, thus slow! Learn how to program DAQmx properly, see the many example VIs via Help menu --> Find Examples --> Hardware input/output --> DAQmx. What kind of signal you sample, and how many channels? Voltage signal?

3) "Do you program it properly using low level DAQmx functions". I am unsure about what you mean here ? See at 2).

I have modified my code accordingly, and I've got few more questions:

a) should the DAQ assistant be placed within the case structure containing my reading code ? or outside? or somewhere else ?

Use the DAQmx Read function, but yes, it should be inside the Read case, in parallel with the two VISA Reads...

b)should I use a wait or a wait until next ms function? I don't really get the difference between the two

Read the help info of the functions! (click on the "?" mark at the top right of the window)

c)where should I put this timer? within the second loop? within the first case struct ?

Your DAQmx task should time the Read case. So no need for additional timing function.

d) the third loop will loop at 500ms (concidering I'll set the second one at 500ms), hence:

     - Do I stlil need a timer there ? (dumb qustion here, but I prefer to triple ask rather than struggling. I am still feeling unfamialliar with the P/C architecture)

No need for timeing! The 2nd (DAQ) loop will time it via the Queue.

    - I am using the first cell of my cluster (2nd loop - case structure true) to tell the case structure of the third loop what should happen. basically, if "1" -> display and record. but that brings me to the point where I actually never send any other information rather than "1". What I mean is that if the dequeue is timed out, that will trigger the true case structure, hence the reading of my serial (2nd loop), but should I send an empty bundle containing just "0" in the first cell to tell the case structure of the third loop what to do in case the dequeue isn't timed out (means I clicked a button somewhere on my VI)? which would be bassically to keep displaying the last variable received... is that clear?

     - which leads me to ask: should I have this case structure in the third loop ? should I remove it / use shift register for every variable displayed there?

I have attached my code. feel free to critize it. You guys are amazingly helpfull !

Flo

---

[JÞB]

OK, 

This thread is getting hard to follow.  Just really long and rambling Perhaps it is time for a restatement by the OP in a new thread.  Blokk, you have done a great piece of work bringing this poster through some tough problems! Kudos

Now, for some free advice that will greatly reduce the code complexity

1. You asked about the wait "b)should I use a wait or a wait until next ms function? I don't really get the difference between the two"  You have exactly one Wait in tha controller v5.vi.  It should not be there at all! Delete it and the "0" constant wired to the Dequeue Element it is right above EDIT: After re-re reading the thread use a 500ms Timeout wired to the dequeue.NOTE: I did a code clean-up with Ctrl+U and waited a while- that original code is ugly and some other things will help too 
2. You asked "c)where should I put this timer? within the second loop? within the first case struct ?" The answer is "In the trash"  Let the loop timing be set by the dequeue element function!  the loop only needs to respond to enqueued commands so let it be timed by the enqueuer
3. You Stated "  - I am using the first cell of my cluster (2nd loop - case structure true) to tell the case structure of the third loop what should happen. basically, if "1" -> display and record."

Here is you basic problem Right there in that timing and dequeue.  You have a timeout of 0 wired to the dequeue element so, the Timed Out? Output is always true!  Everything in the false case that output is wired to is "Unreachable Code" You WANT to have a way to execute that code.  However, as you currently have that loop structured it is either:

• Do Something (which is unreachable!)
• Or Acquire + Display and Log in the next loop

Following my edit above will resolve quite a bit of that.

Now lets look at the logging loop.  My friend, you have a mess!  Unbundle the values you want logged, Scale them, build those values into an array and use Write to Delimited File.vi.  That will reduce your BD real-estate by %50 and vastly improve our chances of finding actual bugs in the code by eliminating a huge Rube-Goldberg construct that is straining our eyes.

FINALL Edit:  Somebody Please help the OP to learn what a Type def'ed cluster control is  Hint Flo-w, ask that question in a new thread