How could I speed up when I'm saving the acquired data from FlexRIO 7962 to PC

[ybdsyb]

I am new in LabVIEW + FPGA. I am dealing with this project: Using 5781 as ADC and DAC, the acquired data from ADC was sent to 7962 for processing, and the processed data was then transferred to PC through PCIE. The problem is: If the data was set to sent every 1 s, the data could be stored quickly; but if the interval time was set to be 1 ms, the size of data would be sufficient large, and the time of reading data from the PCIE buffer would be extremely long. So could you please help me dealing with it?

[Terry_ALE]

What's the data rate going to disk? 

I assume you are using a DMA FIFO to get data to the PC, can you confirm? 

[ybdsyb]

Thanks for your reply Terry!

I didn't configure the data rate, so it might run as default settings.

It's true that the data was transferred through DMA FIFO from FPGA to PC. The total sampling time was set to be 4 s with a interval time of 1 ms, so I got 4000 periods. The HOST VI was programmed to read the buffer every 8192 points and stored into an array. After reading out all the data (4000 periods * 8192 points), the array was saved to a .txt file. The saving process was fast enough, but the reading procedure was relatively slow, especially during the latter parts. 

Have you ever met this phenomenon before?

[Terry_ALE]

Hard to tell without seeing the project and code.

Did you start from an example?  The examples are good to get a baseline and then go from there.

Do you need to read interim data based on a trigger based or is there a duty cycle?  More generally, what are you looking to do?

[ybdsyb]

It seems that the issue was caused by my mis-setting on the Invoke Method of DMA FIFO Read in HOST vi. 

I tried to read the FIFO data of 8192 points from buffer in each while loop, but the write speed of FIFO was way faster than this process. So it took a longer time to read out all the data.

I had updated the code by reading all the remaining data in FIFO every loop, but another problem occured to me. I saved these data in a 2-D array, with 8192 points in each row and 1000 rows (as reading 8192 points every time), and write the array into a .txt file as long as the sampling process was finished. But when I amended the procedure, the reading process was unpredicable (the amount of read data in one loop). So I'm wondering if there is a path to read tha data and save the data concurrently without impact on saving process performance?

[Terry_ALE]

You could log in the same process that you read from the DMA-FIFO.  Right now, do you collect the data and then record it?  8000 x 1000 is not that much but it could be stored in one shot or in row by row.

[ybdsyb]

In the beginning I read 8192 every loop and stored in a 2-D array, and right now I read all the remaining data of FIFO and stored in an 1-D array. I've tried before that wrote data into the txt every loop, but the high frequency of the writing process made the reading process became significantly slower than the former loops. 

The file attached is he present program I used (all 2019 version, but in Chinese, although I think other versions of language could recognize those functions and controls). If I planned to collect 1 s of data (1000*8192 points), it worked well; but when I tried to collect 60 s of data (60000*8192 points), error will appear that out of memory.

Could you please give me some instructions about how to log while reading? And  if this out of memory error could be solved by adding memory of PC ( right now with 8GB in total and nearly 2GB are available)?

[Terry_ALE · Accepted solution]

Check out the High Throughput Streaming.lvproj example project for streaming to disk.

[ybdsyb]

OK I'll try it. Thank!