Increasing fetch performance

[jim_monte]

Hi,

I have found that the PXI-5122 digitizer that we are using requires substantially less
CPU time as the fetch size is increased up to about 1000000 points. However, this is
causing a larger latency between acquiring the data and the current time than is desired.
An idea I had to resolve this problem would be to partition the buffer into subregions
by assigning sentinel values before the fetch at various offsets. These addresses
would be assigned values that could not be obtained from a 14-bit card so that
changes in these values could be used to determine the progress of a (binary) fetch.
This solution requires that the card transfers data to the application buffer in a
sequential manner, or at least in a known sequence. Is the data moved sequentially?
If not, what is the sequence that data is moved? If you have other alternative
ways that this could be done, those would be appreciated also. Thanks for any
information.

Jim Monte

[MLopez]

Hi,
The data is moved in one big copy of up to a megabyte, that would be 500 points for the 5122.
Another option would be to fetch data in a continuous mode using the read pointer mechanism.
You can set the "Fetch Relative To" property to the read pointer and fetch in small chunks.
An example for this is installed and it is called "niScope EX Fetch In Chunks", and another one
is "niScope Fetch Forever".

Thanks,
Manuel

[jim_monte]

I am currently fetching in continuous mode using the read pointer. The
fetch is similar to the Fetch Forever example except that the fetches are
raw binary values. However, there is an enormous improvment in performance
as the fetch size is increased. For example, with 500 kpoints being fetched
at once (I am assuming you meant 500 000), I can achieve sustained fetching
of 50 MSamples/sec with about 15% CPU available (not doing anything else but
fetching.) Using 500 points per fetch an overrun will occur at
4 MSamples/second. However, 500 kpoints is a substantial delay which I would
like to reduce. So is that "one big copy" from low address up, or something
else?

[MLopez]

Hi,
Yes, I meant 500k points. The copy is done from low address up so your method might work today.
However, this is not guaranteed to be done the same for every device and it might change in the future as optimizations are done to the fetching performance. Also, polling the buffer to see if it has changed might end up decreasing performance again.

Maybe you can find an optimal spot not so small that it hurts performance but not so large that it increases the latency too much?
It usually varies from system to system but there is always a "sweet spot" were performance is best.

Thanks,
Manuel

[jim_monte]

Hi,

Unfortunately, for this system that "sweet spot" in terms of memory transfer rate
is around 1 million points. If done right, the polling would probably increase
performance regardless of any latency considerations. Comparing two values is only
a few machine instructions, and it would allow a smaller working set and better
locality of reference. Considering that Microsoft claims that a page fault results
in the loss of about 1 million instructions, even saving a few faults would make
the comparisons worthwhile. Maybe NI will add a read-only attribute to describe the
transfer scheme being used if they become creative with the transfer sequence.