How to use DAQmxRegisterEveryNSamplesEvent while avoiding data being overwritten

[David_Lin]

I am using DAQmxRegisterEveryNSamplesEvent to continuously sample and display data, but I had one big problem. Because the data plotting could take up some time, the DAQmxReadAnalogF64 called in the callback function will, after running for some time, run into the error "Attempted to read samples that are no longer available... -200279". I tried to increase the buffer size, but it only delayed the error, not remove it. So would anyone please suggest how I can solve this problem without missing any data? Thanks a lot in advance.

[Mcdan]

It sounds to me like you're not removing data from the buffer as fast as hardware is producing it.  What is your application doing with data read?  One other thing to be aware of, is that if your data handling code takes longer than the time between Every N Sample Events, additional events will be dropped.  If this happens, and your event handling code is only reading 'N' samples at a time, it will fall behind.  Again, this would be a symptom of the code which reads data consuming data slower than hardware is producing it.  Without seeing your application, I can't offer any more specific advice than that.

Hope that helps,

Dan

[David_Lin]

It is actually a producer/consumer issue, the producer is the DAQ card, and consumer the chart plotting. The only problem is that the producer is producing too fast while consumer is consumingtoo slowly.  When plotting data, newly acquired data has to be appended to the old ones, resulting slower and slower refresh as total data number increases. Without using Labview, any suggestions how to solve the issue?

[Mcdan]

How fast are you acquiring data?  How long are you running for?

A few things to keep in mind:

1) If you are continuously appending new data to your collection of previously read data, you will constantly be allocating memory.  This process is slow, and will get slower as your data collection gets bigger.  It sounds like you are seeing the effects of this.

2) If you are plotting your data on a chart, you have a fixed number of pixels that you can display.  Therefore, displaying a chart with millions of samples probably isn't going to be a lot more useful than displaying decimating that data and displaying a more smaller subset.  I suspect that this would be what the chart implementation does.  As your data set gets bigger, this process probably takes longer and longer as well.

I think the better questions are:

1) What data needs to be displayed at any given time?

2) Can this be implemented in such a way that uses a fixed amount of memory (ie... Can you log data to file(s), and only display a certain subset of the data)?

I realize this isn't a cut and dry answer, but hopefully it'll point you in the right direction.

Dan

[David_Lin]

Hi, Dan, thanks a lot for your answers.

The sampling rate is 100KS/s each channel for 3 channels, and the total sampling time is less than 80 seconds. I set callback for every 0.5second sampling, i.e., 50K samples each channel (150K in total for 3 channels). After reading the 150K data, I push them into a buffer, and the plotting function will then read from the buffer to plot. Now I have a few questions.

1. I have used DAQmxSetBufInputBufSize to set the input buffer to be 150K*50, which equals to 25second of sampling. How come it is not big enough? The overwrite error happened in less than 25 seconds.

2. Should the callback happen every 0.5 second? Will it depend on what DAQ card (USB, PCI, PXI) I use?

[Mcdan]

David,

In answer to your questions:

1. I have used DAQmxSetBufInputBufSize to set the input buffer to be 150K*50, which equals to 25second of sampling. How come it is not big enough? The overwrite error happened in less than 25 seconds.

Which DAQ device are you using?  What is the exact error code you are seeing?  How long does it usually take to get the error?

If you're seeing the error in less than 25 seconds, it sounds like an issue transferring data between the device and your computer's memory.  This can be an issue on USB, if your device is not attached to a USB 2.0 High Speed host controller.

2. Should the callback happen every 0.5 second? Will it depend on what DAQ card (USB, PCI, PXI) I use?

Yes, I would expect that you would get a callback every 0.5 seconds.  However, if your callback takes longer than this to execute you will miss some callbacks.  DAQmx does not queue up these callbacks.  Have you tried profiling your callback to see 1) how often it occurs and 2) how long it takes to execute?  This should not depend on the DAQ card.

If you can post your code, or a simplified version of your code which demonstrates the issue, I'd be happy to take a look at it to see if I can spot any issues.

Dan

[David_Lin]

I am testing with PXI-6133 and interface card is ExpressCard 8360. I did a test without plotting data by calling AppendData() listed below, but only push data to buffer, and no error occured. After adding in data plotting function AppendData(), the error did happen within 25 seconds. I suspect the situation would be even worse if USB card is used.

The callback function only set an event so that another thread can continue with data reading, so the callback function itself won't take up much time.

//Task config
DAQmxErrChk (DAQmxCreateTask("",&task));
DAQmxErrChk (DAQmxCreateAIVoltageChan(task,ch,"",DAQmx_Val_Cfg_Default,-10.0,10.0,DAQmx_Val_Volts,NULL));
DAQmxErrChk (DAQmxCfgSampClkTiming(task,"",srate,DAQmx_Val_Rising,DAQmx_Val_ContSamps,dataperchan));
DAQmxErrChk (DAQmxRegisterEveryNSamplesEvent(task,DAQmx_Val_Acquired_Into_Buffer,numCallback,0,cbfunc,NULL));
DAQmxErrChk (DAQmxRegisterDoneEvent(task,0,donefunc,NULL));
DAQmxErrChk(DAQmxSetBufInputBufSize(task,buffsize));

//Callback func
static CCallbackTestDlg* pt = (CCallbackTestDlg*)pt2Object;
SetEvent(pt->m_hEventNsample);
return 0;

//thread to read and plot data

WaitForSingleObject(m_hEventNsample, INFINITE);

int32 TotalSampleNum = m_nSampPerCh*3;
Read_ContSample(AIHandle,&m_vAIdata[0],m_nSampPerCh,TotalSampleNum,read,0.0))

EnterCriticalSection(&m_csBuffAccess);
m_qvBuff.push(m_vAIdata);
LeaveCriticalSection(&m_csBuffAccess);

AppendData();    //the function to plot data

[Mcdan]

Can you provide the exact error code you receive?  I don't see any obvious issues with the code you posted.

You've set srate to 100 k, buffsize to be 7.5 M, and numCallback to 50 k?

Can you provide the relevant DAQmx call in Read_ContSample?

Like I mentioned, I certainly don't see any obvious issues.  If you let me know what error code you're seeing, I'll see if I can work back from where the error is set to figure out what's gone wrong.

Dan

[David_Lin]

The error is

"Attempted to read samples that are no longer available. The requested sample was previously available, but has since been overwritten. Increasing the buffer size, reading the data more frequently, or specifying a fixed number of samples to read instead of reading all available samples might correct the problem.
Property: DAQmx_Read_RelativeTo
Corresponding Value: DAQmx_Val_CurrReadPos
Property: DAQmx_Read_Offset
Corresponding Value: 0
Task Name: _unnamedTask<0>
Status Code: -200279"

The Read_ContSample is 

DAQmxErrChk (DAQmxReadAnalogF64(task,dataperchan,tout,DAQmx_Val_GroupByChannel,data,bufsize,&rd,NULL));

 I set numCallback to be 150K, not 50K. Since I sampled 3 channels together and each channel is 50K when callback is triggered.

[Mcdan]

The error you're receiving would be due to a buffer overwrite on the host side, rather than the device FIFO.  This would be consistent with your reads not occurring fast enough.  One thing to consider in reading your comments is that all DAQmx functions take parameters in terms of samples, regardless of the number of channels in your task.  DAQmx will do the necessary math to ensure that your buffers are big enough to accommodate the number of channels in your task.

DAQmx also allows you to query certain information about the task, so I think the next step would be to verify that things are being set up the way we think they are.  Can you call the following functions right before you start your task to ensure that DAQmx is configured the way we think it should be:

DAQmxGetBufInputBufSize -  we would expect this to be 7.5 M

DAQmxGetSampClkRate - we would expect this to be 100 k

When you call read, can you first call DAQmxGetReadAvailSampPerChan?  This should tell you how many samples are currently in your buffer.  If this number catches up with your buffer size I would expect to see the -200279 error you're seeing.

Dan