question about reduced sampling rates

[jim_monte]

Hi,

Based on tests I have done, it seems that when the PXI-5122 card
is acquiring data, it always samples at full speed and writes all
data to memory. It then creates a reduced sampling rate in the
driver by selecting every nth point. As a result, the effective
number of points that a given amount of memory can hold will become
lower as the sampling rate decreases. It also implies that if
memory size is not changed, the amount of time required to fill it
is independent of the sampling rate. I would appreciate if people
would confirm this. Otherwise I have to see what is wrong with
my tests! Thanks for any information.

Jim Monte

[psisterhen]

Although the device samples at full speed, it does not write every sample to on-board memory. The decimation (every nth point) is done in hardware, so the device only stores samples corresponding to the actual sample rate you set. So, while the number of samples that the device can store DOES NOT depend on the sample rate, the time to fill the on-board memory DOES depend on the sample rate.

[jim_monte]

Thanks for the reply. That is how I thought it would work until I
programmed it. Maybe you can explain what I am seeing...

With a program that only fetched, I could get a sustained fetch rate of
50 MSamples/sec without any overflows, and about 85% CPU utilization,
with a continuous binary fetch of 500 000 points per fetch. Actually,
this program did just a little bit more. It calculated statistics on how
many points were received per fetch. The average per fetch was only
slightly under the maximum fetch requested, just over 499 999.

Adding another thread to process this data caused performance to get
MUCH worse. When I requested a sampling rate anywhere near what I was
using before, I got nothing but overrun errors and 0 points processed.
When I reduced the sampling rate to 12 000 samples/sec, things got interesting.
In 30 seconds, I got 15 overruns (after which I aborted and restarted the
acqusition) and the maximum number of points fetched was 128. This card
has 16 MB of RAM and each fetch was immediate, that is, there was a timeout
of 0 specified when miScope_FetchBinary16() was called. (BTW, other values
for timeout did not help.)

Now for some math with the numbers I gave. To assume the worst, suppose
that the RAM is channel-dependent, so that each has only 8 MB (I do not know
if this is true or not but think it is). Each sample requires 2 bytes
for this card, so there would be 4 million points that could be stored.
Then with a sampling rate of 12 000 (the actual rate was given as 12 000.480019), it would take 4E6 MB /12E3 = 333 seconds for memory to fill.
But I overran memory 15 times in 30 seconds! If all of memory were available
this should be impossible. Also, those times when I did not overrun, I would
have expected more data since the card was on the verge of overrunning.

Below are two runs I made that have some trace information printed to show
what was happening.

Jim Monte

NOTE: I got a message that this post was too long, so I will add the
runs as separate post(s).

[jim_monte]

Here are the runs that go with the previous post

*** FETCH ONLY ***
Scope init OK
Card passed self-test. Message: Scope Self Tests PASSED.
Driver revision: Driver: NI-SCOPE 2.70, Model: NI PXI-5122, Compiler: MSVC 6.00
, Engine: IVI 2.03, Compiler: MSVC 6.00, Components: VISA-Spec 3.00
Firmware revision: Product code: x28E0, product revision: 2, total memory bytes:
16777216, product serial: x00DD632B, calibration map: 2, DFC FPGA: xDFCA00F3, m
em 0 FPGA: x50419, mem 1 FPGA: x50419.

Serial number: DD632B
Device temp is 37.000000 degrees C
Task id is 31653952
Memory size is 16777216 bytes
Maximum sample rate is 100000000.000000 bytes
Card resolution is 14 bits
Card sample size is 16 bits
Channel 0 configured OK
Horizontal configured OK
Actual record length is 1 points
Actual sample rate is 50000000.000000 Hz
Trigger configured OK
Fetch set relative to read pointer
Buffer of size 1000000 allocated
999998264 points fetched:
min: 0
max: 500000
ave: 499999.132000
std dev: 38.808434
Press any key to continue


==========================================================================

*** FETCH WITH PROCESSING ***
Parameters being used:
min sample rate = 12000.000000 samples per second
number of points per fetch = 2000000
number of sets of stats = 1
run time = 30 seconds
clean interval time = 10 seconds
backlog recovery time = 60000 milliseconds
max number of buffer in use = 7
number of buffers per block = 4
number of entries in initial block list = 1
number of avail entries in initial free buffer list = 4
card name = "PXI1SLOT2"
Scope init OK
Card passed self-test. Message: Scope Self Tests PASSED.
Driver revision: Driver: NI-SCOPE 2.70, Model: NI PXI-5122, Compiler: MSVC 6.00
, Engine: IVI 2.03, Compiler: MSVC 6.00, Components: VISA-Spec 3.00
Firmware revision: Product code: x28E0, product revision: 2, total memory bytes
: 16777216, product serial: x00DD632B, calibration map: 2, DFC FPGA: xDFCA00F3,
mem 0 FPGA: x50419, mem 1 FPGA: x50419.

Serial number: DD632B
Device temp is 37.250000 degrees C
Task id is 31653952
Memory size is 16777216 bytes
Maximum sample rate is 100000000.000000 bytes
Card resolution is 14 bits
Card sample size is 16 bits
Channel 0 configured OK
Horizontal configuration set to allow more records than available memory
Horizontal configured OK
Actual record length is 2 points
Actual sample rate is 12000.480019 Hz
Trigger configured OK
Fetch set relative to read pointer
overrun error while fetching
overrun error while fetching
overrun error while fetching
overrun error while fetching
overrun error while fetching
overrun error while fetching
overrun error while fetching
overrun error while fetching
overrun error while fetching
overrun error while fetching
overrun error while fetching
overrun error while fetching
overrun error while fetching
overrun error while fetching
overrun error while fetching


Total fetches: 2978
Total points processed: 356736
Min points fetched: 16
Max points fetched: 128
Ave points fetched: 119.790463
Std Dev points fetched: 5.505637

V min: 0.000234
V max: 4.403112
V ave: 0.800538
V RMS: 1.876157
Processed 356736 points in 30.004077 seconds -- 11889.584295 points/second
Press any key to continue

[psisterhen]

In the second example, it looks like you're setting the Allow More Records Than Available Memory flag. Unless you're actually doing a multi-record acquisition and will be fetching more records than can fit in on-board memory, you should not set that flag.

Without being clear on exactly how you've configured the acquisition, this is what I believe is happening:

Normally, the driver will allocate as much on-board memory as it can to each record. If you have 8MB and one record, it will allocate the entire 8MB to that record, even if the record size is only one sample. When the acquisition starts, the device will start storing samples (at the actual sample rate) in that on-board memory. When the memory allocated for the record is exhausted, it simply wraps around to the beginning, overwriting samples already stored. Normally, you don't care about these samples, because they're not part of the actual record. The actual start of the record is determined by when the trigger comes in. Yet you can fetch all of the samples that were stored, even the ones that aren't part of the record. As long as the acquisition engine has not wrapped around (overwriting the oldest samples) you are fine. If you try to fetch samples that have been overwritten, the driver will report an error.

However, when you set the Allow More Records Than Available Memory flag, the driver will try to conserve space, and allocate the minimum on-board memory for each record (so it can pack in as many records as possible). It looks from the printout that your records are tiny. So the acquisition engine will fill up that allocated space very rapidly (even at slow sample rates). When you fetch relative to read pointer, the first fetch will try to get samples from the beginning of the acquisition (not the beginning of the record), but those first samples have probably already been overwritten. That's why you get the overrun error.

So, the quick answer is: don't set the Allow More Records Than Available Memory flag. If that doesn't work or is not acceptable, we'll need to know more about your application (number of records, record sizes, trigger configuration, reference position, etc.).

[jim_monte]

The second program continuously fetches data without ever
triggering. It configures the acquisition much like the
"Fetch Forever" example does. The setting of the Allow
More Records Than Available Memory flag was done in some
intermediate version of the program when I was testing how
a triggered approach would work. It turned out that the
rearming was not fast enough, so I had to abandon the use
of triggering. But in case I wanted to reconsider
triggering later, I left the flag setting in the program.
The documentation did not indicate that setting it would
cause any harm. (Or at least I did not see anything in it
that said it would.)

Removing the call to set the flag and making no other change
allowed the program to run at 11.1 Msamples/sec without any
overrun errors. That is an improvement of a factor of 1000,
which is not too bad for a minute's work.

Jim Monte

[Teigen]

Jim,

I'm happy to clear up any confusion as to when the decimation takes place. The help documentation for the 5122 contains a schematic which shows the architecture of the board and the order of operation for analog-to-digital conversion, decimation, etc. This schematic can be found under NI High-Speed Digitizers Help>>Devices>>NI 5122 Overview>>Block Diagrams>>NI PXI-5122 Block Diagram. This schematic confirms that decimation step precedes the loading of samples into onboard memory. The onboard memory can only hold a fixed number of samples, therefore the rate at which samples are transfered to the onboard memory is dependent upon the type of decimation that is being performed. For more information about decimation and sample clock concepts for high-speed digitizers, please see NI High-Speed Digitizers Help>>Fundamentals>>Clocking>>Sample Clock

Scott Teigen
Applications Engineer
National Instruments