Producer-Consumer loop with queue doesnt start

[MaKIT]

Hello,

ich want to sample data with 500000kS/s for aprox. 10 minutes. For that i set two while loops (producer and consumer) to write this continously in a tdms file. I do not know what I did wrong but the consumer loop doesnt start while the producer loop seems to run perfect. Only when I set the number of samples in the DAQmxRead manually then the producer and the consumer loop runs 1 time. But then the limit is the size of the puffer storage.
Can anyone help me with that. Im quite new in the Labview community.

Thanks a lot
Martin

[GerdW]

Hi MaKIT,

---

@MaKIT wrote:

ich want to sample data with 500000kS/s for aprox. 10 minutes. 

---

500000kS/s = 500MS/s = 0.5TS/s…

Really?

---

@MaKIT wrote:

For that i set two while loops (producer and consumer) to write this continously in a tdms file. 

---

Why don't you use the TDMS writing feature in DAQmx directly (as explained in the example VIs)?

[MaKIT]

Ah sorry, I wanted to write 500kS/s!

I found online to do it with this both loops. Thank you for your hint. I will try it with this TDMS writing feature. Can you explain where I can find this example VIs?

Best regards
MaKIT

[PalanivelThiruvenkadam]

In the LabVIEW Panel go to Help ->Find Examples and Search For TDMS Write based on your data to write.

[GerdW · Accepted solution]

Hi,

---

@PalanivelThiruvenkadam wrote:

In the LabVIEW Panel go to Help ->Find Examples and Search For TDMS Write based on your data to write.

---

Or look for the DAQmx example VIs, there are several that explain the TDMS logging for DAQmx tasks…

[Yamaeda]

The way you've wired your Stop button the Writing loop will not start until you press Stop and it'll stop automatically ... The Queues shouldn't have timeouts.

This simple change should solve it, though it's kind of a hack and if you have lots of data still in queue it'll be ditched. A better variant is to send an empty array when you want to quit and check for that in the consumer loop and have the Release queue on the Consumer output.

Something like this (the waveform cluster can be changed to Icon to have a cleaner diagram)

[MaKIT]

---

@Yamaeda wrote:

The way you've wired your Stop button the Writing loop will not start until you press Stop and it'll stop automatically ... The Queues shouldn't have timeouts.

This simple change should solve it, though it's kind of a hack and if you have lots of data still in queue it'll be ditched. A better variant is to send an empty array when you want to quit and check for that in the consumer loop and have the Release queue on the Consumer output.

Something like this (the waveform cluster can be changed to Icon to have a cleaner diagram)

---

Thank you,
I deleted the time outs and made the changes as you suggested. Now both loops runs and sample data and write it to the tdms file if the sample time is approx 30s. If I increase the time to 60s for example, I get a pop up window: the size of the memory is too low I cant finished the operation.

With the indicators of the loop counters I can see that the producer loop is almost twice fast as the consumer loop. 

Im not sure if I have the correct settings for the DAQmx Timing. I choose "continouus Samples" and for the DAQmx Read I set "analog, multiple channels, multiple samples, 1d"?

[Yamaeda · Accepted solution]

Since you're going for a high sample rate i'd try Finite samples, buffer size same as sample rate and read 50% of those per loop (so you read and queue 2 times/second)

I see a flaw in my design, change it like this. (i ditched the last samples, now it sends the last sample and then the empty array to signal end)

[Bob_Schor]

OK.  I took a look at the code you originally posted, and have some comments and suggestions.

• For your code to work, you must be capable of writing the data faster than you acquire it.  You "know" the acquisition speed (though I'm not sure you are actually getting it -- see later bullet points).  You should write a little demo program for yourself that generates, say, 1 mega-byte of data in whatever format you want to save it and writes 500 copies of it to a disk file (TDMS or binary, whichever you prefer) and times how long it takes.  
• If you want to take a large amount of data in a continuous manner (without having unknown gaps in the data record), you must use "Continuous Samples" (not "Finite Samples") in setting up your DAQmx Sample Clock.  Furthermore, there should be no timing routines inside your Producer Loop -- the DAQmx Read functions as your clock (and the timing of the DAQ hardware is much more "predictable" than the Elapsed Time function as it doesn't depend on your PC's hardware and software).
• Once you have determined that your file writing speed is at least as fast as your data-generating speed, you need a better way of stopping the Producer/Consumer pattern.  I strongly recommend you use a "sentinel", a "signal" that the Producer (who "knows when to stop") sends to the Consumer (who, when it sees it, stops and destroys the Queue).  Here's how it works:
  1. The Consumer should be a While loop inside of which is a Case Statement with the Stop button connected to its selector.
  2. The "False" case is the "Do a DAQmx Read and put the data on the Queue (you should not need a timeout).  That's all you want in the Producer -- "get the data and send it to the Consumer".
  3. The "magic" comes in the "True" case, when you want to stop the loop.  [Note -- a perhaps "better" way to stop the P/C pair is when you collect "enough" data, i.e. when the Loop index "i" reaches, say, 1000].  In this case, send an empty Array (a legal element for the Queue) to the Consumer and exit the Producer Loop, leaving the Queue intact.
  4. In the Consumer, you also put a Case Statement.  When you enter the Consumer, dequeue (without needing a Timeout) a sample, and test it to see if it is an empty Array, which you wire to the Case selector.  If it is False, then the Queue returned Data that you want to write (so go ahead and write it), which finishes this Case.  If it is True, then you know the following:
     • The Producer has finished and sent you a message to that effect, so the Consumer can also finish.
     • The Consumer closes the Data File.
     • The Consumer does the Release Queue (since both Producer and Consumer are done with it).
     • The Consumer exits.
  5. Now the Producer and Consumer have both exited, the File has been closed, and the Queue has been destroyed.
• Always (well, almost always) use the Error line!  In particular, your (unnecessary) "Elapsed Time" function happens at an "unknown time" in the Producer loop -- it is not necessary, and without knowing whether it runs before or after the DAQmx Read, doesn't tell you anything useful!

Bob Schor

[Bob_Schor · Accepted solution]

I'm ashamed of myself.  Possibly because I was "tea-deficient", I gave the "LabVIEW 2015" description of the "better way" to structure a Producer/Consumer design.  And I consider myself a "Champion of Channel Wires" ...

What you really want to do for a Producer/Consumer is to use a Stream Channel Wire.  The Stream Channel Wire is precisely what you want for a Producer/Consumer design.  In addition to having a "Write Stream" VI where you put "whatever you want to send" to the "Read Stream", it has two additional Boolean inputs, "element valid?" (which defaults to True) and "last element?" which defaults to False.  It's partner, the "Read Stream" VI, also has a pair of Boolean outputs with the same name.  Look in the Examples that ship with LabVIEW -- the first two examples under "Channel Basics" show the Stream Channel in action.  I haven't used a LabVIEW Queue since 2016 ...

Bob "Asynchronous" Schor