Acquiring data simultaneously from 9 sensors at high frequencies and extended periods of time

[KewQiu]

Hello everyone,

I want to start explaining that I'm really new to NI products and LabVIEW, and I am currently reading through the Core 1 Materials that a friend lent me. I am working in a project that is looking to acquire the pavement response in a currently in-service highway. To do this, a test section was instrumented with 3 earth pressure cells, 4 strain gages and 2 thermocuples. I have availble a NI cDAQ-9174 and the following modules: NI 9235 (strain gages), NI 9213 (thermocuples) and a NI 9201 (pressure cells). The sensors would go connected to the mentioned modules in the cDAQ, and everything would be controlled through a LabVIEW VI.

As part of some initial testing, I connected all 9 sensores to the DAQ and used LabVIEW's DAQ Assistant to get some preliminary data measurements and save them into separate TDMS files. Since I was looking to continually get 1 sample every 0.001 seconds (Acquisiton Mode: Continuous Samples, Samples to read: 1, Rate (Hz): 1000), I could not get any data due to buffer size issues (LabVIEW displayed an error due to insufficient buffer size). In the end, I could only get decent measures with only the 3 earth pressure sensors connected and acquiring data at a rate of 1 sample every 0,004 s (250 Hz).

Due to this, I'm guessing that I have to use NI DAQmx functions and assign a buffersize for each sensor individually, with this I could also use different sample rates for each (Thermocuples need to only get 1 sample every second). But, my main worry is that even if I do this, due to the high frequency of acquisiton and my planned mesurement time (4 hours for now), what I'm trying to do might not be reallistically possible. 

So, besides the needed storage for the generated TDMS files, is it possible to achieve what I want with my current equipment (DAQ, modules, etc.) and using LabVIEW, while also taking into account that in the future I might need to up the frequency to 2000 Hz and mesaure for a straight 24 hours.

I appreciate any help on the matter. If you can also throw in some advice o anything I should have to consider with my project, you are welcome to do so and I will greatly appreciate it.

Finally, apologies if I posted this in the wrong section of the board, but it seemed like the right place regarding my inquiries.

Thanks in advance!

[RavensFan]

The problem is you are only reading one sample at a time for something coming in 1000 samples per second (x however many channels.)  There is no way for the PC to keep up only grabbing a sample at a time.

Try grabbing 100 samples at a time or 1000 samples at a time.  Even a DAQ Assistant should be able to handle that.

What do you consider an "extended period of time"?  For a road monitoring project, it sounds like it would be on the order of days.  1000 samples per second and 86,400 seconds per day winds up being a lot of data.  Can you store all of that?

[yea_likethecity]

Hello Kew,

From the specifications of your modules, you should be able to measure everything except the thermocouples at 1 kHz. (Thermocouples are spec'd at 75 samples per second.) As RavensFan mentioned, sampling one sample at a time for all 9 channels at 1 kHz is inefficient and puts far more work than necessary on your computer processor. By setting the Samples to Read to 1000 in DAQ Assistant, you'll send a sample set of 1000 samples (acquired at 1 kHz) to your computer once per second, which puts the burden of sampling on the DAQ device while your computer just has to receive and process the data.

Here's a good knowledgebase article on buffer issues with DAQmx:

Understanding and Avoiding Overwrite and Overflow Errors with NI-DAQmx

https://knowledge.ni.com/KnowledgeArticleDetails?id=kA00Z000000PAraSAG&l=en-US

Good luck on your project!

[Bob_Schor]

NI has a whole series of tutorials on DAQmx.  You can find them by searching for NI-DAQmx Tutorials.  My favorite (particularly when I want to wean someone away from the Dreaded DAQ Assistant) is "Learn 10 Functions in NI-DAQmx and Handle 80% of your Data Acquisition Applications".

Bob Schor

P.S. Match your sampling rate to the frequency content of the phenomenon you are studying.  For "road work", how much data is really above 100 Hz?  Do you need a continuous record of it all?  Might, say, a 1 second sample at 1KHz be taken once a minute?  Is there an "Event" (say, a collision) for which you want high-frequency data, but (again) only for a short period of time?

All of these things are easily doable with LabVIEW and reasonable sensors.  But do consider the size of the data files and how much is "real" data (meaning you want to analyze it) and how much is "filler" (irrelevant "noise" between interesting "events").

Bob Schor

[KewQiu]

---

@RavensFan wrote:

The problem is you are only reading one sample at a time for something coming in 1000 samples per second (x however many channels.)  There is no way for the PC to keep up only grabbing a sample at a time.

Try grabbing 100 samples at a time or 1000 samples at a time.  Even a DAQ Assistant should be able to handle that.

What do you consider an "extended period of time"?  For a road monitoring project, it sounds like it would be on the order of days.  1000 samples per second and 86,400 seconds per day winds up being a lot of data.  Can you store all of that?

---

Thanks for your input RavenFan!

Sorry for the late answer, but I couldn't give a proper response until now. Thanks to your and Austin's explanation, I know have a clear understanding on what modifying "Samples to read" does. I will be visiting the test track next week, so I will be testing this out and seeing for how long I can acquire data from the pressure cells and strain gages before I get the buffer error. I will be leaving out the thermocuples for now. 

Regarding your question about what I consider extended periods of time. You are completely right, a road monitoring project, the periods of time is in the order of days, even months for really big projects. But for now, what I considered a extended period of time was getting at least 4 straight hours of data from all sensors, and in the future, if possible with my current equipment, get this to at least 24 hours. About the storage, right now I'm saving all the data to my computer's hard drive, but I have the possibility to get and external hard drive just to save this project's data. 

Also, this project that I'm doing, is just the beginning of a larger scale project of road monitoring, so that's why maybe my periods of time don't seem that big. Right now, what we want is to verify that all the instrumentation works in the instrumented tracks and we are getting the correct readings, while trying to get as much useful data as possible to see how to analyze it. In the future, I may have to give a propossal on how to obtain this data remotely, if possible. 

Again thank you very much, and any other recommendations are welcomed!

[KewQiu]

---

@yea_likethecity wrote:

Hello Kew,

From the specifications of your modules, you should be able to measure everything except the thermocouples at 1 kHz. (Thermocouples are spec'd at 75 samples per second.) As RavensFan mentioned, sampling one sample at a time for all 9 channels at 1 kHz is inefficient and puts far more work than necessary on your computer processor. By setting the Samples to Read to 1000 in DAQ Assistant, you'll send a sample set of 1000 samples (acquired at 1 kHz) to your computer once per second, which puts the burden of sampling on the DAQ device while your computer just has to receive and process the data.

Here's a good knowledgebase article on buffer issues with DAQmx:

Understanding and Avoiding Overwrite and Overflow Errors with NI-DAQmx

https://knowledge.ni.com/KnowledgeArticleDetails?id=kA00Z000000PAraSAG&l=en-US

Good luck on your project!

---

Hello Austin,

Thank you very much for your input and for confirming that my equipment will do for now. With your and RavensFan's explanation finally made me understand how sampling works with the DAQ Assistant, and your recommendations will help a lot. I did came across the article you linked, but didn't quite get the hang of it at the time, thank you for posting it, since now it makes a lot more of sense!

Again thank you, and any other recommendations are welcomed!

[KewQiu]

---

@Bob_Schor wrote:

NI has a whole series of tutorials on DAQmx.  You can find them by searching for NI-DAQmx Tutorials.  My favorite (particularly when I want to wean someone away from the Dreaded DAQ Assistant) is "Learn 10 Functions in NI-DAQmx and Handle 80% of your Data Acquisition Applications".

Bob Schor

P.S. Match your sampling rate to the frequency content of the phenomenon you are studying.  For "road work", how much data is really above 100 Hz?  Do you need a continuous record of it all?  Might, say, a 1 second sample at 1KHz be taken once a minute?  Is there an "Event" (say, a collision) for which you want high-frequency data, but (again) only for a short period of time?

All of these things are easily doable with LabVIEW and reasonable sensors.  But do consider the size of the data files and how much is "real" data (meaning you want to analyze it) and how much is "filler" (irrelevant "noise" between interesting "events").

Bob Schor

---

Thank you for replying Bob!

The tutorial you recommended has been very helpful regarding how NI-DAQmx works. 

You made a really good point regarding how much data I really need, and thank you for pointing this out. I would like to explain that I do need to get data a 1kHz, this is because I'm trying to get the complete response of the pavement when a car's wheel goes through the instrumented track, cars will pass approximately at like 80 km/h, because the instrumented track is on a in-service highway. Due to the high speeds, I need high sampling frequencies.

The kind of response I'm looking to obtain is similar to the uploaded picture, this was done reading 1 sample at 250 Hz and the speed of the vehicle going through the test track was like 10 km/h (on purpose). The raw data had a lot of noise, so I used a Savitzky-Golay Filter to clean it a bit. Theorically, most response curves are in the order of miliseconds, thus the need of sampling at 1 kHz.

In this case, I do have an event, and it would be when the vehicles goes through the test track, making the values in strain and pressure change. Being this my event, is it still possible to filter out the filler data: all the data that is recorded when there are no vehicles passing through; the time between a vehicle and the next one, when the instrumented track is empty. Can it only write the data when a vehicle is passing? If this is possible, then it would really optimize a lot of space and time when it comes to analyzing and managing all the measured data. Do you have any recommendations as to where should I begin reading to program this kind of stuff?

Again thank you very much!

PS: I am sorry for triple posting, I couldn't find the way to multi-reply and multi-quote, so that all my answers were in one post. How do I do this? Also, sorry for answering this late, as I stated before, only until now, I could properly give a reply.

[yea_likethecity]

Hi KewQiu,

It sounds like what you want is triggering. See if this tutorial has some useful information:

Tips and Techniques in Data Acquisition Triggering - NI-DAQmx

https://www.ni.com/en/support/documentation/supplemental/21/ni-daqmx-data-acquisition-triggering-tec...

[KewQiu]

---

@yea_likethecity wrote:

Hi KewQiu,

It sounds like what you want is triggering. See if this tutorial has some useful information:

Tips and Techniques in Data Acquisition Triggering - NI-DAQmx

https://www.ni.com/en/support/documentation/supplemental/21/ni-daqmx-data-acquisition-triggering-tec...

---

Thank you! That's really helpful!