Labview is slow to process large data

[detech]

Hi, I use labview and a PXi card to acquire data and then to do FFT. In frequency domain, I do some math and then do IFFT. Acquisition is fast enough, but FFT and iFFT are very slow. My data size is 2*10^6 (2 million).

I am not sure if there is a way in labview to process large data array. Please advice? Thanks

[johnsold]

What do you consider slow? How much time does the processing take?  Are you making extra copies of the data?

On my computer a 2 million point FFT takes about 345 ms and the IFFT about 276 ms.  The times increase rather quickly as the data set size increases.  For 2^21 (2097152) points the FFt takes 494 ms and the IFFT takes 439 ms.  A 5 % increase in the size of the dataset increased FFT time by 43% and IFFT time by 59%.

Some kinds of processing can be done in segments and the segment results combined for the final result.  Is that feasible for your process?

Lynn 

[JÞB]

Here is a KB Article on managing large data sets in LabVIEW.

2M points can certainly chew up tons of memory as DBLs this yields 8M per copy! and copys happen every time you branch a wire or pass it to a front panel object. consider placing your data in a queue and then you only need to pass a pointer to the queue for each function that will process the data- and get rid of any graph that displays every point (decimate the data for display- the user can't process 2Mpts)

[Cory_K]

---

detech wrote: 

Hi, I use labview and a PXi card to acquire data and then to do FFT. In frequency domain, I do some math and then do IFFT. Acquisition is fast enough, but FFT and iFFT are very slow. My data size is 2*10^6 (2 million).

I am not sure if there is a way in labview to process large data array. Please advice? Thanks

---

Much of the processing speed depends on how you coded it.
The more copies you make of the data, the more memory you are using.
Try to avoid unnecessary coercions, indicators, branching, etc 

[detech]

Thank you for help.

I do need to display data in FFT and IFFT to see some specific peaks. Does it mean the display make it very slow and take too much memery?

Thanks

[Cory_K]

When you have large data sets, it doesn't make it impossible to display them, you just have to be very careful about how you do so.
What are you trying to display? Just the peak values?

Or by 'display' do you mean you are trying to plot your data on a graph/chart? 

Message Edited by Cory K on 06-02-2010 12:33 PM

[detech]

Yes, plot data in a graph and look how many peaks. What I did is, save acquired data in a txt file first and then read it back for FFT.  Seems soneone said txt file take more memory.

Instead, I use mathcad, it is very fast. I am not sure what labview is so different in processing data.

[johnsold]

Please post your program.  Some of the people who participate in this Forum are very good at finding bottlenecks in code.

Lynn 

[JÞB]

---

johnsold wrote:

Please post your program.  Some of the people who participate in this Forum are very good at finding bottlenecks in code.

Lynn 

---

Ditto:

Also- a brief discription of the end use of the data.  It would help us eliminate unnecessary overhead while preserving the information you really are interested in having.  LabVIEW doesn't eat memory- poor data structring does.  We can show you additional tips and tricks (if Christian doesn't beat us all by an order of magnitude)