Data Acquisition: Gaps in Data

[TopNotch]

Hello All,

I am new to labview. I am reading data from a pressure transducer and a laser displacement sensor using a USB-DAQ from omega engineering.

There are gaps/jumps in Data despite implementing the Producer/consumer architecture, and making as efficient as possible the VI design. I have tried different combinations of sample rate, samples per channel and number of samples. For instance, I have tried sampling rates of 10kHz and 10k sample per channel, and varied the samples to read (based on the recommended maximum of 1/10th of sampling rate). I thought data overwrite in the buffer could be responsible, and I tried wiring a -1 to the samples to read i.e. read all samples in the buffer. Still did not work

I think the problem may lie in the code, but my newbie mind cannot pick it out.

I have attached my code and would appreciate any help as I am working against a deadline

[Kevin_Price]

I have no knowledge of the 3rd-party DAQ device you're using, so can offer only very limited thoughts based in part on how NI DAQ devices would work.

1. No real need for 2 queues and 2 consumer loops.  You can do both file writing and UI display in 1 shared loop.

2. Probably a mistake to ignore (i.e., not react to) errors in your producer loop.

3. Your producer loop has both a fixed wait and is also specifying a # samples to read from your clocked DAQ task.  This can lead to an overconstraint of your loop timing.

    The default value for # samples on your front panel is -1, which is actually the correct magic # to use for NI DAQ devices.  It means "give me whatever's available right now" and adds no further timing constraint to the fixed wait.

    I have no idea what it means or what behavior to expect when you feed that -1 to the API function for your 3rd party device.

4. Please do yourself a favor and get rid of the unhelpful detour through "Convert to Dynamic Data."   Simply index individual rows (or maybe columns, depending on the behavior of the 3rd party driver) from the 2D array and take the Means of *these*.

The most important insights will need to come from someone else with detailed understanding of the 3rd party device and driver.

-Kevin P

[TopNotch]

Thank you Kevin_Price for your message.

1) I have faced no problems so far using 2 queues and 2 loops.

2) How do I react or actively engage with errors in the producer loop? Do you mean wiring an indicator to the "message" terminal on the simple error handler VI?

3) How do I "deconstrain" the loop?

4) I have read in many forums here to avoid dynamic data wires. Initially, I tried to index array, but I cannot pass a 1D array into MeanPtbyPt VI. I want to perform a moving average of the data hence this seems to be the option I could think of. I will appreciate, better, more efficient design suggestions

Also, I am using  OM-USB-1608FS-PLUS data acquisition device from omega engineering

Thank you

[Kevin_Price]

The quoting mechanism here is kinda clunky, so I'll respond inline in red.

---

1) I have faced no problems so far using 2 queues and 2 loops.

    Not a great plan nonetheless.  Branching the data wire to 2 destinations in the producer will likely require a data copy.  If you sent direct to 1 queue, no data needs to be copied.  The queue would simply take sole ownership of a pointer to the data.  This ownership would then be transferred to the consumer upon dequeue.  All without copying the array data.

    It may not matter much now for this app, but it's never too soon to start learning good practices.  Your use of 2 distinct queues and loops also leads to quite a bit of duplicated code in the 2 consumer loops.  This is another generally bad habit because changes to this duplicate code are more likely to get out of sync during future changes and maintenance.

2) How do I react or actively engage with errors in the producer loop? Do you mean wiring an indicator to the "message" terminal on the simple error handler VI?

    Most simply, I would just allow any errors from data acq or enqueuing to terminate the producer loop early.  The error dialog after the loop will then suffice to inform you.

    Again, this isn't totally crucial, but it's generally best to learn about errors as soon as they're going to cause unintended behavior.  Here, the loop would have iterated infinitely until you happen to decide to manually stop it.

3) How do I "deconstrain" the loop?

    This pretty much depends on your 3rd party device & driver.  As I (kinda) said before, your approach would have worked fine with an NI device and the DAQmx driver.

    It's only a guess, but I'd tend to expect that other DAQ devices would support *some* method for returning "whatever new data is available now."  Whether the magic # -1 is the *mechanism* for that kind of request is not something I'd count on.

4) I have read in many forums here to avoid dynamic data wires. Initially, I tried to index array, but I cannot pass a 1D array into MeanPtbyPt VI. I want to perform a moving average of the data hence this seems to be the option I could think of. I will appreciate, better, more efficient design suggestions

    I'm not aware of a fully "certified awesome" alternative, but would prefer one of the following:

1. Feed the 1D array into an auto-indexing For loop that contains "MeanPtbyPt".  Output to a NON-indexing tunnel.  (Right-click and choose "last value" mode.)

2. Knowing that you'll average 500 samples at a time, setup your producer to read and enqueue 500 samples at a time.  Then your consumer would dequeue that chunk of 500 samples and could use the regular Mean function.

    Note: with an NI device and DAQmx, you'd request 500 samples per read and *remove* the fixed time delay.  DAQmx would manage loop timing by waiting for each set of 500 samples to arrive and be available.

    I have no idea whether something analogous is feasible with your 3rd party device.

Also, I am using  OM-USB-1608FS-PLUS data acquisition device from omega engineering

    Yeah, afraid I know nothing about making it do what you want.

- Kevin P

[Bob_Schor]

I believe the problem is in your Producer Loop.  I'm going to assume that the ULx devices work somewhat similar to NI's DAQ devices, say the USB 600x, which include devices that can sample at 1 kHz for 1000 samples.

• The DAQ Device is your "clock" -- it has the hardware timer, never needs to pause for Virus Checking or to upgrade Windows 10, etc.
• Your DAQ device should be set to "Continuous Samples", which means "Just as soon as you finish one acquisition of 1000 samples at 1 kHz, which should take you precisely 1.00000 seconds, output the data as the User directs (say as a 2D Array of Dbl) and start taking the next set of samples".
• Do not put any "Wait" function inside this loop -- the DAQmx Read is your clock, let it do its job.
• Now your Producer will run one loop per second, missing no samples (since the next Read does not wait for the loop to repeat, but starts as soon as the first set of data have been acquired.  No missed points!

Bob Schor

[RTSLVU]

There is absolutely no reason to use multiple queues and multiple producers and consumers.

If you need to put  more than one data set or type in a queue. Make that queue a cluster (better yet a Type Def Cluster) that contains all the data types you need.

Then use ONE producer to en-queue the data and one consumer to de-queue the data.

Here's an example of a Producer enqueueing a waveform and several arrays of measurements into a cluster to be saved directly to disk in the consumer loop.

I am using a NI USB Oscilloscope here and continually streaming live waveforms directly to disk at up to 50Mhz sample rate without any data loss or gaps.

When "Record" is True I load the queue in my Producer loop

When "Record" changes to True I open temporary file in my Consumer loop

And start streaming data to the file

When "Record" changes to False, I empty the queue and popup a File Dialog asking the user where to save the file.

Then I copy the temporary file to the new file save location and delete the temporary file