data acquisition: data was overwritten.

[Monique497]

Dear Labview helpers,

I am searching through the discussion forum for a question/problem that I have. Although I read some suggestions, I would like to have something confirmed, since I also  have read things that contradicted with one another. I am simply not that confident with labview yet and I hope you folks can help me out with a (hopefully) straightforward question.

General remark:

I have a DAQ card (PCI-6024E) installed. Then I code the following:

1. I created a pulse train (using counter 1 on device).

2. A second counter (called 'counter 0') is configured to read the time stamps between incoming pulses (from the pulse train).

see attached VI.

Problem:

Up to 100 kHz the program can follow the amount of data coming in and there is no error.

However, at around 200 kHz I get error -200141: 'Data was overwritten before it could be read by the system.'

Their suggestion: 'If Data Transfer Mechanism is Interrupts, try using DMA or USB Bulk. Otherwise, divide the input signal before taking the measurement.'

My questions:

1. Labview suggests to make sure DMA is used when data is transferred. I know how this can be programmed, using the 'channel property node' under DAQmx subpallete. However,

I have read in several topics that the driver takes care of this and by default, DMA is used. So I do not need to program this explicitly, I would say.

2. In other topics I have read, people suggest 'producer-consumer loops'. This is what I would like to confirm. Is this the only solution for the error I get?

I was hoping that someone could check my VI to see if I could do something else to prevent the error. 

Further remarks:

1.When I choose the pulse train frequency to be a certain value, I always make sure that the sampling rate of the DAQmx Timing function is twice as high, to prevent undersampling. PCI-6024E has a max. sampling rate of 200 kS/s. I am aware of the fact that, choosing a pulse train frequency higher than the sampling rate, leads to undersampling. However, 

I would say that this does not create the error described above. 

2. I know that, by choosing a certain sampling rate, Labview automatically creates a pre-allocated buffer. However, one can also tell the size of the buffer explicitly in the DAQmx Timing function.

3. In the DAQmx read function is set 'Number of samples' to -1, meaning that whatever data is available in the buffer, it will read it. 

I hope my question is clear.

Thank you in advance, 

Monique

[Kevin_Price · Accepted solution]

Here are several comments, roughly in order of your questions:

- Your older board has 1 DMA channel available for counter tasks.  DAQmx will try to use it automatically for your counter input task.  You don't need to configure it explcitly, though it also wouldn't hurt to do so.  (Note: if you had 2 such input tasks, you'd likely get an error *unless* you explicitly requested one to use interrupts.)

- producer-consumer loops often help with a very similar-sounding problem.  They're a good technique to get familiar with.  But they aren't likely to help you with your immediate problem.

    Separating into distinct producer-consumer loops helps a problem where the app doesn't retrieve data out of the task buffer fast enough, often because too much other processing is happening in the reading loop that prevents the loop from iterating fast enough.

    Your error likely means that the driver can't move data from the board into your task buffer fast enough to avoid overwriting the FIFO on the board itself.  You can't really do anything in your app to help that problem, it's pretty much a hardware/driver restriction.  See this thread I was in recently for some more info.   Note: your board uses the same counter chip as the one referenced in that thread.  A newer X-series board would support much faster rates than your older E-series board. 

- your thoughts about undersampling & oversampling don't really apply to these kinds of counter tasks.  It's a common belief carried over from regular analog and digital tasks where the 2x minimum rule *IS* crucial.  But counters are different.  A counter task doesn't sample a *signal*, it samples an on-board hardware *count register*.  But the count *value* in the register gets incremented by hardware signals regardless of how often you sample it.   Since the count changes get "caught" by the hardware regardless of how often you sample, you don't need to be sure to sample fast enough to catch every count value.

    Counter tasks are often used with high speed signals and relatively low sample rates where the sample rate may be < 1/1000th of the count rate. 

- you're correct about how DAQmx will size a buffer based on the requested sample rate.  However, I don't think you can solve your current problem by changing the size of the task buffer, I'm pretty sure it's a hardware FIFO issue as mentioned earlier.

- you're right that specifying -1 samples to read means to read "whatever # is available right now".  Your reading loop is presently very CPU-greedy.  It's possible the CPU greed is interfering with the driver's ability to move data out from the board into the task buffer.  Try adding something like a 10 msec wait in the loop.  There's a chance that freeing up some CPU may let you support slightly higher sample rates.

- if you really need more reliable and faster sample rates, you should strongly consider a newer board based on the DAQ-STC3 timer chip such as the X-series devices.

- you might want to consider using dataflow to sequence your task starts.  I would typically start the edge counting task before starting the pulse train that acts as the sample clock

-Kevin P

P.S.  For someone who considers themselves "not that confident with LabVIEW",   you've done quite a good job with both the code and with researching a variety of info to troubleshoot your problem.

[Monique497]

Dear Kevin_Price,

I have seen many of your comments, to other people before (you seem very active!), and every time I appreciated your clear and explaining replies. This time it is not different. Thank you for your helpful reply! I really appreciate it. 

Indeed, my board is rather old. Luckily, the final program will be running on a PCIe-6323 board. I checked the FIFO size of that board (see attached manual/specifications sheet).

It is noted to be 4,095 samples. 

The PCI-6024E used was chosen to test already some things. 

Eventually, I will do Fluorescence Correlation Spectroscopy (FCS) measurements (of which quite some threads already exist on the NI forum). For that I expect a photon rate averaging around 100 kHz, although at some instances of time the rate can be much higher  (as expected from a typical photon arrival distribution).

Again thank you for your help. I will definitely consider and implement your suggestions.

Kind regards,

Monique