Delayed or overflowed data acquisition with python and nidaqmx

[RobertRondo]

Hey guys,

i'm having trouble getting my code running the way I want it to. The code is supposed to acquire data from sensors and store it in a file with time stamps. A linear motor is controlled using one of the sensor values. I use an NI cDAQ 9184 and Python with the nidaqmx package. The first problem that has occurred is delayed transmission of the data. As a result, the control process of the engine is very slow. I created a nidaqmx.task and in a loop the current sensor value is requested via task.read(). This method has a delay of several seconds. It seems to me that the cDAQ and the task.read() method are constantly recording data, caching it and ejecting the cached data on request (as in a queue). However, I would like to see a value only on request and no data to be stored as it will be saved to a csv file. If I recreate the task each time I call the task method, I get the appropriate and non-delayed values. However, creating a new task costs about 20 ms and does not seem to me to be the intended solution for my request. Of course, this also slows down the data recording, which i would like to avoid.

See the two following code snippets which describe the procedure i wrote for DataAcquisition: 

def DAQ_Monitoring(c,condition,ExChange,sample_rate):        
    
    with nidaqmx.Task() as task:
        task.ai_channels.add_ai_current_chan("cDAQ9184-1B901BFMod1/ai0")
        task.ai_channels.add_ai_current_chan("cDAQ9184-1B901BFMod1/ai1")
        task.ai_channels.add_ai_current_chan("cDAQ9184-1B901BFMod1/ai2")
        task.timing.cfg_samp_clk_timing(1000)
        
        sample_RateSeconds = (sample_rate)*(10**(-3))    #convert sampleRate from [ms] to [s]
        startTime = datetime.now().timestamp()
        oldTime = 0
        while condition.value == 3:
            while ((datetime.now().timestamp()-startTime)-oldTime) < sample_RateSeconds:
                pass
            timeData = datetime.now().timestamp()-startTime
            oldTime = timeData
            pressureIn,pressureVessel,volumeFlow = task.read()
            position, force = pullData(c)
            accousticEmission1 = 999
            vibration = 999
            pressureIn = (1000000/16)*(pressureIn-0.004)
            pressureVessel = (200000/16)*(pressureVessel-0.004)
            volumeFlow = 3250*volumeFlow            
            ExChange[:] = [timeData,pressureIn,pressureVessel,volumeFlow,accousticEmission1,vibration,position,force]
        

def DAQ_Monitoring(c,condition,ExChange,sample_rate):        
        
    sample_RateSeconds = (sample_rate)*(10**(-3))    #convert sampleRate from [ms] to [s] 
    startTime = datetime.now().timestamp()
    oldTime = 0
    while condition.value == 3:
        while ((datetime.now().timestamp()-startTime)-oldTime) < sample_RateSeconds:
            pass
        timeData = datetime.now().timestamp()-startTime
        oldTime = timeData
        with nidaqmx.Task() as task:
            task.ai_channels.add_ai_current_chan("cDAQ9184-1B901BFMod1/ai0")
            task.ai_channels.add_ai_current_chan("cDAQ9184-1B901BFMod1/ai1")
            task.ai_channels.add_ai_current_chan("cDAQ9184-1B901BFMod1/ai2")
            task.timing.cfg_samp_clk_timing(1000)
            pressureIn,pressureVessel,volumeFlow = task.read()
        position, force = pullData(c)
        accousticEmission1 = 999
        vibration = 999
        pressureIn = (1000000/16)*(pressureIn-0.004)
        pressureVessel = (200000/16)*(pressureVessel-0.004)
        volumeFlow = 3250*volumeFlow            
        ExChange[:] = [timeData,pressureIn,pressureVessel,volumeFlow,accousticEmission1,vibration,position,force]
    

I have read many forum posts and tried to dig through the documentation of nidaqmx. I came across the Stream method. I found very little about this, but I also tried to implement this method. The sensor values are transmitted correctly and without delay. But here the buffer runs over and, depending on the sample rate, sooner or later ejects the following error:

(base) PS C:\Users\Jetting\Desktop\Robert\SCT_TryOut_DifferentAcqTypes> python SCT_18.02.2021_v1.0_stream.py
SCT_18.02.2021_v1.0_stream.py:290: UserWarning: Attempting to set identical bottom == top == 0.0 results in singular transformations; automatically expanding.
ax.set_ylim(np.amin(dataPlot),np.amax(dataPlot))
SCT_18.02.2021_v1.0_stream.py:291: UserWarning: Attempting to set identical left == right == 0.0 results in singular transformations; automatically expanding.
ax.set_xlim(np.amin(timePlot),np.amax(timePlot))
SCT_18.02.2021_v1.0_stream.py:290: UserWarning: Attempting to set identical bottom == top == 0.16962570625000006 results in singular transformations; automatically expanding.
ax.set_ylim(np.amin(dataPlot),np.amax(dataPlot))
SCT_18.02.2021_v1.0_stream.py:291: UserWarning: Attempting to set identical left == right == 1.4031758308410645 results in singular transformations; automatically expanding.
ax.set_xlim(np.amin(timePlot),np.amax(timePlot))
Process Process-1:
Traceback (most recent call last):
File "C:\Users\Jetting\anaconda3\lib\multiprocessing\process.py", line 315, in _bootstrap
self.run()
File "C:\Users\Jetting\anaconda3\lib\multiprocessing\process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\Jetting\Desktop\Robert\SCT_TryOut_DifferentAcqTypes\SCT_18.02.2021_v1.0_stream.py", line 105, in DAQ_Monitoring
reader.read_one_sample(values_read)
File "C:\Users\Jetting\anaconda3\lib\site-packages\nidaqmx\stream_readers.py", line 370, in read_one_sample
_read_analog_f_64(self._handle, data, 1, timeout)
File "C:\Users\Jetting\anaconda3\lib\site-packages\nidaqmx\_task_modules\read_functions.py", line 35, in _read_analog_f_64
check_for_error(error_code)
File "C:\Users\Jetting\anaconda3\lib\site-packages\nidaqmx\errors.py", line 127, in check_for_error
raise DaqError(error_buffer.value.decode("utf-8"), error_code)
nidaqmx.errors.DaqError: The application is not able to keep up with the hardware acquisition.
Increasing the buffer size, reading the data more frequently, or specifying a fixed number of samples to read instead of reading all available samples might correct the problem.
Property: DAQmx_Read_RelativeTo
Corresponding Value: DAQmx_Val_CurrReadPos
Property: DAQmx_Read_Offset
Corresponding Value: 0

Task Name: _unnamedTask<0>

Status Code: -200279

Here it seems to me that the stream is constantly storing data to the memory which leads to an overflow (Error -200279: Unable to Keep Up with Acquisition in DAQmx - National Instruments). But i have no idea how to overwrite the data instead of appending it to the memory. I honestly have to admit, that i did not fully understand how task.read() and the stream method do their job. 

Following is the code with the streamer:

def DAQ_Monitoring(c,condition,ExChange,sample_rate):        
    with nidaqmx.Task() as task:
        task.ai_channels.add_ai_current_chan("cDAQ9184-1B901BFMod1/ai0")
        task.ai_channels.add_ai_current_chan("cDAQ9184-1B901BFMod1/ai1")
        task.ai_channels.add_ai_current_chan("cDAQ9184-1B901BFMod1/ai2")
        task.timing.cfg_samp_clk_timing(rate=1000,sample_mode=nidaqmx.constants.AcquisitionType.CONTINUOUS,samps_per_chan=2)
        reader = AnalogMultiChannelReader(task.in_stream)
        task.start()
        values_read = np.zeros((3,), dtype=np.float)
        sample_RateSeconds = (sample_rate)*(10**(-3))    #convert sampleRate from [ms] to [s]
        startTime = datetime.now().timestamp()
        oldTime = 0
        while condition.value == 3:
            while ((datetime.now().timestamp()-startTime)-oldTime) < sample_RateSeconds:
                pass
            timeData = datetime.now().timestamp()-startTime
            oldTime = timeData
            reader.read_one_sample(values_read)
            pressureIn,pressureVessel,volumeFlow = values_read
            position, force = pullData(c)
            accousticEmission1 = 999
            vibration = 999
            pressureIn = (1000000/16)*(pressureIn-0.004)
            pressureVessel = (200000/16)*(pressureVessel-0.004)
            volumeFlow = 3250*volumeFlow            
            ExChange[:] = [timeData,pressureIn,pressureVessel,volumeFlow,accousticEmission1,vibration,position,force]
        

Any advice or help and improvement of my code is appreciated. If any information is missing i am sorry (this is my first post), in case just ask. 

I also added two python files as an attachement. 

Thanks!

[IDASYAN]

Hello,

Have you checked this documentation?  

Also for timestamp passing part this one.

I believe links upper will help you.

Luck!

[RobertRondo]

Hello IDASYAN,

first of all thank you for your reply. I found the solution by myself. The feeling, that the data is stored in a queue, was correct. The task.read() method starts a continuous task and saves data to the buffer, which is what i didn't knew. The solution was to simply add an arguement to the task.read() method to make sure all the data from the buffer is read at once. The argument is: "number_of_samples_per_channel = nidaqmx.constants.READ_ALL_AVAILABLE". Now the process works perfect! 

The links you posted in your replay might have been a good hint anyways, so thank you again and have a nice week!