Buffer problems while reading at 10MHz rate

[pavanbabut]

Hi,
I am trying to use the C#.NET example of 'Continuous Analog Acquisition using Internal Clock to File' for reading input from 3 channels at the same time at a rate of 10MHz (the maximum limit on PCI-6115). But the program stops as soon as I hit 'start' button. I am running it in ContinuousSamples mode. It runs fine at lower rates, but I need to run at 10MHz. What is the best optimised buffer value to read in the values? Has anyone tried at 10MHz?? someone help me out .....

thanks,
-Pavan

[Jesse_O]

Hello Pavan,

You do not mention the error you receive when you try to run the application at 10MHz.  I expect that you are running into buffer overflows.  This typically happens when you try to sample at rates that high and save the data to disk. This is because of the slow transfer rates of hard drives.  You would require transfer rates of approx 20MB/s per channel (10M samples/second * 2Bytes/sample = 20MB/s), or 60MB/s for 3 channels.

If you stop writing the data to a file, you should be able to read the data at that rate.

Regards,

Jesse O.
Applications Engineering
National Instruments

[pavanbabut]

Hi Jesse,
Thanks for your reply. I am using 'ContAcqVoltageSamples_IntClk_ToFile' program with parameters Dev1/ai0:2, 1MS/s. 10MHz. I am able to run the program and record successfully, if I use FiniteSamples mode, but I need to run it in ContinuousSamples mode as I am recording live video data at 10MHz. The error is like a Message dialog with OK button but without any description, just a blank Message Box.
Thanks,
-Pavan

[Jesse_O]

Pavan,

When using the parameters "Dev1/ai0:2, 1MS/s", you would only require 6MB/s which is within most hard drive data transfer rates.  When running at 10MHz, you require 60MB/s which is most likely faster than your hard drive an handle.

Regards,

Jesse O.
Applications Engineering
National Instruments

[pavanbabut]

Hi Jesse,
We are using Dell Precision 530 Dual Processor system. Yeh, I too agree that it may be a bottleneck for HDD transfer rates. But what do you think is the best way/setup to acquire data at 10MHz from 3 AI channels in ContinuousSamples mode?
thanks,
-Pavan.

[Jesse_O]

Pavan,

The typical way to achieve a faster transfer rate to disk is to setup a raid system (usually raid 0 with 4 drives).  There are some things to keep in mind when setting up a raid system.  First, when from you DAQ card you are transferring approximately 60MB/s from your PCI card to your RAM.  You then read that buffer with c# and store that data on a hard drive.  If you purchase a PCI raid controller, you are sending that data back along your PCI bus requiring 120MB/s total along your PCI bus. The theoretical maximum burst rate for a PCI bus is 132 MB/s.  Most systems are not capable of transferring this fast, and you system might not be able to keep up with 120 MB/s. 

The other option is to have a system with two PCI buses, or one with a PCI bus and PCI express bus.  We also have an alliance partner that specializes in high speed stream to disk solutions.  Please see the following Knowledgebase here.  If you are interesting in one of their solutions, please verify with them that their driver supports your device in .NET.

Finally, when streaming to disk make sure that you have binary file selected or you will require much more than 60MB/s when streaming to disk.

Regards,

Jesse O.
Applications Engineering
National Instruments

[pavanbabut]

Hi Jesse,
Thanks for your reply. Lemme get clear over here. Using a dual processor system with 1GB of physical memory, 18GB HDD (with 15k rpm, 16MB cache) and the board being installed on 32-bit PCI, if I want to acheive recording 3 AI at a rate of 10MHz using 1MS/channel/sec, is it not possible?? Actually the c# program does exactly what you said, copying all the data onto the RAM and then write it into a file after I stop acquisition. Can you give me details about the requirements and limitations of both s/w and h/w setup for recording 3AI at 10MHz and 1MS/sec/channel? Meanwhile I will go through your options to see which one works good for us.
thanks,
-Pavan.

[Jesse_O]

Pavan,

I apologize for the mix-up.  When I was looking at the code I assumed that LogData function actually saved the data to file.  In this example, all the data is saved to memory then saved to file.

Now if you have 1GB of memory, and you are only saving data to RAM, it will only take a couple seconds (approx 16) for 1GB of memory to be filled with this acquisition.  This is assuming that windows and all other applications are taking up no RAM.  The more space each application is taking the less time before you fill your memory.  If it was my system I would get approx 3-4 seconds of data before I ran out of memory.  I assume the problem you are having now is associated with this.  As soon as you run out of RAM and start to use your hard drives swap file you run into the hard drive speed limitation.  For 1 minute of data, if you saved it all to RAM you would require 3.6GB of memory.  You should be able to run this application for very short periods of time, or change the acquisition mode to finite and only acquire a small amount of data (a couple seconds).

If you comment out the the LogData line in the AnalogInputCallback ( LogData(data); ) method and the line CloseFile(); in the stop button callback I would expect that your application would run successfully.  This would allow the acquisition to take place without putting all that data into memory or saving any data to a file.

If you change the application to save to disk during the acquisition, you will encounter the problems I mentioned before.  Streaming directly to disk, or using a raid controller on a separate bus, is probably your best options for acquiring that much data at 10MHz.

Regards,

Jesse O.
Applications Engineering
National Instruments

[pavanbabut]

Hi Jesse,
I managed to run th program in Conituous mode atlast by playing with the sample rate and on-board sample size, but again its the same problem of ram. It is runnin out of space after 5 secs of acquisition. So, we are thinking of increasing the ram to 2GB so that it can accomodate atleast 10 sec of data.
Actually we have mentioned that we need atleast 10MHz recording and playback while ordering the board, but they have sent this card instead. Since this board is not capable of outputting analog data at 10MHz on 3AO, we are thinking of different approach of using single AI and dual DI. I would really appreciate if by any chance, you have any example programs which are capable of reading in and reading out 1AI/2DI and 1AO/2DO into and form a data file??
thanks,
-Pavan

[pavanbabut]

By 1AI/2DI and 1AO/2DO i mean recording at 10MHz on all channels and generating output at 4MHz on all channels using PCI-6115

-Pavan