Writing Data in a huge Data Array slows program down

[johanneshoer]

Hello,

I have a problem, please help!

I need to record measurement data, approx. 50 Data per second. (5 chanels, 10 measurements/second)

So do this, by simply writing the data into a data array by adding new data at the end of the data array. 

That works, but after a while the program slows down, probably due to the huge array size. After some minutes it becomes so slow, that it takes more than 100ms to add the data to the array, which is not acceptable.

How can I modify or improve my program, so that the handling of the huge array does not slow the program down?

I allready run LabVIEW with higher windows priority (abovenormal).  

The array could have up to 15000 rows and 8 coloumns.

I use LabVIEW 7.1 (!)

Johannes

Message Edited by johanneshoer on 09-11-2009 10:21 AM

[Harold_Timmis]

you may want to post your code so that we may edit your code to help you more

[johanneshoer]

ok, in 5 minutes

[Darin.K]

Locals, stacked sequences, and memory thrashing.  I anticipate some fun comments here. 

My advice is to do a search of the forums on Memory Thrashing, and among other things initialize a large array and replace elements instead of constantly appending. 

[Harold_Timmis]

yes there are quite a bit of problems with your code as darin has pointed out...

sequence structures for the most part are 'not to be used', as well as local variables

you may also have race conditions within this code because of your local variable use.

[Ben]

---

Darin.K wrote:

Locals, stacked sequences, and memory thrashing.  I anticipate some fun comments here. 

My advice is to do a search of the forums on Memory Thrashing, and among other things initialize a large array and replace elements instead of constantly appending. 

---

Yes, do what Darin suggested.

The tag "LabVIEW_Performance" and most of its related links should cover most of the issue you are seeing.

and one more VERY IMPORTANT THING!

Set all of the priorities back to normal and fix the code.

Trying to fix a perfomance problem by changing the priority is like trying to a get a slow worker to work faster by yelling louder and more often. You may think he will work faster but in actuality you are just distracting him and slowing him down".

Your app is thrashing and bumping the priority only makes it thrash more!

Ben

[johanneshoer]

here is the code.

it's just a part of a huge vi .

i appreciate your comments.

Message Edited by johanneshoer on 09-11-2009 10:53 AM

[johanneshoer]

---

Harold Timmis wrote:

...

sequence structures for the most part are 'not to be used', as well as local variables

you may also have race conditions within this code because of your local variable use.

---

Thank you for the hint. I did not know that sequence structures are 'not to be used'. That is new for me. Ok, i have to admit, that I never attended LabVIEW classes, but learned LabVIEW all by my own. I will check out that LabVIEW Performance Tag, I think there is a lot to learn for me 🙂

However, all my sequences do not really slow down my program, and I took care of race conditions whenever I used local variables.

The only thing that slows down my program is that array or better say my style of adding data to it.

Message Edited by johanneshoer on 09-11-2009 11:03 AM

[johanneshoer]

---

Darin.K wrote:

Locals, stacked sequences, and memory thrashing.  I anticipate some fun comments here. 

My advice is to do a search of the forums on Memory Thrashing, and among other things initialize a large array and replace elements instead of constantly appending. 

---

Darin,

thank you for the help. I tried initialising the array and then replacing elements instead of appending but it made it worse! 

I think I am on the wrong track.

[johanneshoer]

I found a solution:

I use a shift register (see first attachment). Wow! That is fast! But I cannot see what is written into the array.

If I put the array within the loop, it becomes miserable slow again....