LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

how do i create arrays more efficiently


Yamaeda wrote:
I did a small test program. 10 million random numbers build an array (*2), one through indexing, 1 through preallocation with Initialize array and then replace. Loop and rebuild 100 times and check the result. This is my result!
It might be affected by version and system, but automatic indexing is FASTER than Replacing a preallocated array. 🙂

/Y


I have been mislead for a long time.  Smiley Mad  I took an intermediate class once, and I'm sure it was after 6.1.  The instructor said to pre-allocate the array since it was faster than auto-indexing.  Now I know better.

 

- tbob

Inventor of the WORM Global
0 Kudos
Message 21 of 27
(1,597 Views)

tbob wrote:

Yamaeda wrote:
I did a small test program. 10 million random numbers build an array (*2), one through indexing, 1 through preallocation with Initialize array and then replace. Loop and rebuild 100 times and check the result. This is my result!
It might be affected by version and system, but automatic indexing is FASTER than Replacing a preallocated array. 🙂

/Y


I have been mislead for a long time.  Smiley Mad  I took an intermediate class once, and I'm sure it was after 6.1.  The instructor said to pre-allocate the array since it was faster than auto-indexing.  Now I know better.

 


 

Still valid for While loops since they can't anticipate how to size the array.

 

Ben

Retired Senior Automation Systems Architect with Data Science Automation LabVIEW Champion Knight of NI and Prepper LinkedIn Profile YouTube Channel
0 Kudos
Message 22 of 27
(1,593 Views)

Ben wrote:

I have been mislead for a long time.  Smiley Mad  I took an intermediate class once, and I'm sure it was after 6.1.  The instructor said to pre-allocate the array since it was faster than auto-indexing.  Now I know better.

 


 

Still valid for While loops since they can't anticipate how to size the array.

 

Ben


It is also important to note that it depends on how you are determining how many times to iterate the for loop. If you have an array that is created dynamically earlier in your program and you use an array size function wired to the N terminal to determine how many times the for loop will spin, there is no way for the memory to be fully allocated before hand(it doesn't know the array size). However, if you use a constant wired to the N terminal, or array size VI with a constant array known at compile time, then the memory will be preallocated (as far as I understand it). Correct me if I'm wrong. That said, I feel the most efficient way can often depend on the situation. Now because the question specifically says initialize array, this may be a bit out of scope but I feel it is an important point.

0 Kudos
Message 23 of 27
(1,551 Views)

for(imstuck) wrote:

Ben wrote:

I have been mislead for a long time.  Smiley Mad  I took an intermediate class once, and I'm sure it was after 6.1.  The instructor said to pre-allocate the array since it was faster than auto-indexing.  Now I know better.

 


 

Still valid for While loops since they can't anticipate how to size the array.

 

Ben


It is also important to note that it depends on how you are determining how many times to iterate the for loop. If you have an array that is created dynamically earlier in your program and you use an array size function wired to the N terminal to determine how many times the for loop will spin, there is no way for the memory to be fully allocated before hand(it doesn't know the array size). However, if you use a constant wired to the N terminal, or array size VI with a constant array known at compile time, then the memory will be preallocated (as far as I understand it). Correct me if I'm wrong. That said, I feel the most efficient way can often depend on the situation. Now because the question specifically says initialize array, this may be a bit out of scope but I feel it is an important point.


Maybe its your understanding of the term "before hand".

 

If the For Loop indexes off that array that was created earlier

AND

It has an auto-indexing output tunnel then LV will allocate an array for the output tunnel array as indicated by the in-coming array BEFORE the For Loop starts to execute so there is no need to size-up the buffer holding the output tunnel data.

 

So it is a quick one time operation prior the For loop starting to spin.

 

Now if you have the luxury to set the buffer size at development time, you do save move that time used to allocate the memory from execution time to load time and you also increase the size of your VI on disk.

 

Not trying to prove you wrong or similar. Just sharing my understanding too

 

1) Help you,

 

2) Get an early warning as to when I should give it up and retire.

 

Please feel free to Correct me if that seems wrong.

 

Keeping each other honest,

 

Ben

Retired Senior Automation Systems Architect with Data Science Automation LabVIEW Champion Knight of NI and Prepper LinkedIn Profile YouTube Channel
0 Kudos
Message 24 of 27
(1,543 Views)

Ben wrote:

Still valid for While loops since they can't anticipate how to size the array.

 

Ben



True, but maybe less so than you think. I've added While loops to my test, and Autoindexing (if the design allows it) in a while loop is half as fast as Replacing a preallocated array. It might sound like alot, but Build Array is actually 8 times slower than a autoindexed for-loop!

As many theorize it's probably due to memory management. With autoindex the compiler can try to allocate memory and let the loop fill it (which not surprisingly seems easier with a for-loop). With Initialize Array i suppose it actually fills the array costing time though the result is practically identical between for and while.
Build array increases memory usage is steps, causing several copies/resizes, nullifying the for-while difference. (I assume it works similar to Java, start at 50 bytes and then doubles each time it gets full)

/Y
G# - Award winning reference based OOP for LV, for free! - Qestit VIPM GitHub

Qestit Systems
Certified-LabVIEW-Developer
0 Kudos
Message 25 of 27
(1,523 Views)
I can't run your vi.  It gives me a memory full error around the third Flat Sequence.
- tbob

Inventor of the WORM Global
0 Kudos
Message 26 of 27
(1,508 Views)

Yamaeda wrote:

Ben wrote:

Still valid for While loops since they can't anticipate how to size the array.

 

Ben



True, but maybe less so than you think. I've added While loops to my test, and Autoindexing (if the design allows it) in a while loop is half as fast as Replacing a preallocated array. It might sound like alot, but Build Array is actually 8 times slower than a autoindexed for-loop!

As many theorize it's probably due to memory management. With autoindex the compiler can try to allocate memory and let the loop fill it (which not surprisingly seems easier with a for-loop). With Initialize Array i suppose it actually fills the array costing time though the result is practically identical between for and while.
Build array increases memory usage is steps, causing several copies/resizes, nullifying the for-while difference. (I assume it works similar to Java, start at 50 bytes and then doubles each time it gets full)

/Y

THe theory is backed-up by a simple Benchmark program that times how long it takes for each iteration of a loop using a Build Array. I ran such a benchmark back in LV 5.1 or LV 6i and what I found was "0 ms" average for the first 1K inserts and then a bump, followed by another 1K of "0ms" then a bump and then "0ms" until I got to 4K, 8K, 16K...

 

THe maginitude of each bump also increased to back-up the THEORY that the buffer had to be copied to an ever larger new allocation.

 

It has been a long time but I believe this THEORY was confirmed one of Greg McKaskle's posting years ago.

 

Ben

Retired Senior Automation Systems Architect with Data Science Automation LabVIEW Champion Knight of NI and Prepper LinkedIn Profile YouTube Channel
0 Kudos
Message 27 of 27
(1,483 Views)