Drop of sample rate during data acquisition with CompactDAQ and LabVIEW

[cpon]

Hello everyone,

I'm hoping there is someone who can help me with my problem.

I'm having a problem regarding my data acquisition and the desired sample rate of my measurements.

I'm working with the cDAQ-9189 chassis and the modules NI-9205, NI-9217 and NI-9219. The software I'm using for the acquisition is LabVIEW 2021 (Professional). 

I want to read voltages from 16 channels in RSE mode from NI-9205, 3 resistances (4-wire) from NI-9219 and 2 temperatures from NI-9217. All modules have different maximum sample rates: NI-9205 (max. 250kS/s aggregate), NI-9219 (max. 100S/s/channel) and NI-9217 (max. 400S/s aggregate).

To acquire some data above 100S/s I obviously need to pass different sample rates to each of the timing-VIs. However, even if I pass a value below 100 S/s to all of the modules, the desired samplerate is only reached for a few minutes maximum and then drops exponentially. For example, if I pass a value of 50 S/s for all acquisition VIs/modules, the sample rate drops to 2-5S/s after a few minutes/hours. 

I want to acquire my data for at least one hour up to 8 hours.

I started my acquisition using the Read-VI with the option "Analog 1D double N Channels 1 Sample" and saved the values in a .csv file. That's where I first detected the error (similar to the attached file, independent of the value of the desired sample rate, in the attached image i selected 200S/s but the diagram looks similar if i just pick 50S/s, it just drops lower then).

After that I adjusted the code regarding multi-tasking (seperate tasks for each NI-module), pipelining (read values in the first loop, save them in the next loop while already acquiring new values using shift registers). Still no difference in sample rates.  

(If at this point someone knows, what I was doing wrong, please let me know.)

Right now, I'm using the waveform N Channels N Samples-form of the Read-VI. The actual status of my code is attached. I am now able to acquire data from equidistant timesteps with different sample rates for each NI-module. BUT after a few seconds the while loop duration is longer than usual and in my .csv-file i get a gap between my measurements. (see image sample-break). The task that has a samplerate of 500 S/s has a gap of 1,376s (between the acquisition time of 23.8 and 25.2 seconds); the one with 400 S/s has a gap of 1,438s (between the acquisition time of 24.5 and 25.9 seconds) and the one with 100S/s has no gap.

I am suspecting that this is the same problem as i had in the first version of my code described before. Can someone explain my errors or why these errors occur?

I would really appreciate any suggestions as i really need this high speed acquisition of data. Thanks in advance!

Regards,

cpon

[santo_13]

Please share the VI in an older version (like 2019)

While you get that, without even looking at the VI, here are some pointers to understand the issue at hand,

• Do all your instruments support your requested sampling rate?
  • Not all instruments support all values of sampling rate, refer to instrument specifications to know the list of discrete sampling rates supported
• Did you try to read back the actual sampling rate using the DAQmx task?
• Have you tried using DAQmx datalogging capabilities instead of logging to file separately?
• Always use DAQmx N sample read - do not use 1 sample read for higher sampling rates (anything above 100S/s)

[crossrulz]

You definitely should not be logging your data in the same loop as the acquisition.  Look into the Producer/Consumer to pass the data to another loop for logging.  You might want to also avoid using the Export Waveforms To File as it opens the file, writes to the file, and closes the file each time it is called.  The opening and closing of the file is terribly slow.  You may be able to take stuff out of the VI to use for writing to the file inside of the loop and then closing the file after the loop.

[Kevin_Price]

[Edit: queued this up but didn't get around to answering for a while.  Largely redundant now after other replies, but here it is anyhow FWIW.]

I have several suspicions about the code, but can't open version 2021 to look.  Please save back to ~2018 or earlier to get a bigger audience.

Meanwhile:

1.  The data acq sample rates are set in hardware and won't vary like that.  It appears that you're seeing a change in the *iteration rate* of your reading loop.

2. It appears that you may be reading only 1 sample per iteration, hence the initial fast iteration rates.  That's usually a bad approach.

3. Managing multiple tasks with different sample rates in one common loop requires some care regarding the # samples you read per iteration.  You won't generally be able to specify constant integer #'s for each task because the rates won't generally have ratios that correspond to small integers.

   At least 2 of the tasks should specify (-1), which means (for continuous tasks) to read all samples currently available.  The 3rd task could be a constant integer, preferably representing about 1/10 sec worth of samples.  Or it could also be (-1), and you add a Wait timer to pace your loop at about 10 iterations/sec.

-Kevin P

[GerdW]

Hi cpon,

---

@cpon wrote:

I started my acquisition using the Read-VI with the option "Analog 1D double N Channels 1 Sample" and saved the values in a .csv file. That's where I first detected the error (similar to the attached file, independent of the value of the desired sample rate, in the attached image i selected 200S/s but the diagram looks similar if i just pick 50S/s, it just drops lower then).

After that I adjusted the code regarding multi-tasking (seperate tasks for each NI-module), pipelining (read values in the first loop, save them in the next loop while already acquiring new values using shift registers). Still no difference in sample rates.  

(If at this point someone knows, what I was doing wrong, please let me know.)

---

Using the "1 sample" mode for anything >100S/s is (usually) plain silly. Don't do that!

---

@cpon wrote:

Right now, I'm using the waveform N Channels N Samples-form of the Read-VI. The actual status of my code is attached. I am now able to acquire data from equidistant timesteps with different sample rates for each NI-module. BUT after a few seconds the while loop duration is longer than usual and in my .csv-file i get a gap between my measurements. (see image sample-break). The task that has a samplerate of 500 S/s has a gap of 1,376s (between the acquisition time of 23.8 and 25.2 seconds); the one with 400 S/s has a gap of 1,438s (between the acquisition time of 24.5 and 25.9 seconds) and the one with 100S/s has no gap.

I am suspecting that this is the same problem as i had in the first version of my code described before. Can someone explain my errors or why these errors occur?

I would really appreciate any suggestions as i really need this high speed acquisition of data.

---

It's better this way!

(Why do you need those local variables to set sample rates? And using <1kS/s is usually not considered "highspeed"…)

Your pipelining does not help here because the slow stuff are those 3 FileWrite functions! Have you looked inside of those functions and wondered how much they need to do to convert your waveforms into CSV data? There are better ways to save waveforms to file(s)…

Simple solution: let DAQmx write your data to TDMS files! This feature is built into DAQmx, you just need to configure the filepath!

[cpon · Accepted solution]

---

You might want to also avoid using the Export Waveforms To File as it opens the file, writes to the file, and closes the file each time it is called.  The opening and closing of the file is terribly slow.  

---

I just checked the Export Waveforms to File VI and its really a huge VI. As some have stated here, using TDMS logging might be a solution, but as we usually had difficulties in postprocessing the data written in TDMS files, I wanted to avoid that one.

---

@crossrulz wrote:

You definitely should not be logging your data in the same loop as the acquisition.  Look into the Producer/Consumer to pass the data to another loop for logging.  

---

This one seems to be a suitable option as well, I think I might try this for the next adjustments in my code. 

For now I think even with the errors in my code i found a solution. It was a very stupid error to think, that the buffer is cleared after each Read process. I found somewhere here on this page the hint, that it's not cleared but the read index is increased. That means, by adjusting the Samples per channel at the timing VI i could save all data with the desired sampling rate without any drop of the rate.

Thanks to you all guys for your hints how to improve my code!

I'll attach my actual code (with the programming "errors", but with working sampling rate) 

[Kevin_Price · Accepted solution]

I made a few minimal mods to illustrate a couple other points.

1. Query for *actual* sample rate as suggested by santhosh.  

2. Change from Finite sampling with 1 hour buffer to Continuous sampling with much smaller auto-sized buffer.  Because you're servicing the buffer regularly in your reading loop, there's no need to make it so big.  Honest.

   (I left it as an exercise for the reader to stop the loop after 1 hour.)

3. Because all tasks are being read in the same loop, I made the changes I hinted at in point #3 of my earlier post.  First read ~0.1 sec worth of samples from one task, then read everything available in the buffers of the other tasks.

I didn't do anything about TDMS logging or Producer/Consumer since you already seem to be on track to try those out.

-Kevin P