2D array, Array Subset, a fast methode

[Blokk]

Hello,

I need to access data from a shift register where I store 40 rows in a 2D array of column size of 86400. I need to read out 12 rows arbitrarily picked from the 40. Also, a start and end index is varied to select subset of the rows, but the same start and end index applied for all 12 rows. The following snippet comes to my mind, I wonder if there is any way to do it better. If I configure iteration parallelism for the FOR loop, I guess it can help? Or the compiler anyway will optimize it for multicore?

thanks!

[mcduff]

Do not know if this is faster, but may or may not be more memory efficient. (I really did not say anything here.) You should try in your system. Everything should be in-place, however, the in-place structures are usually slower, but if your system is memory constrained then may be better as you minimize memory allocations.

VI is Version 2015.

mcduff

[crossrulz]

---

@mcduff wrote:

Do not know if this is faster, but may or may not be more memory efficient.

---

Blok's code already operates in place, mostly due to the FOR loop (auto-indexing tunnels are very memory efficient).  In this case, your In Place Element Structures are just adding sequence boundaries for no good reason.

[mcduff]

@Crossrulz

Yes and no.

I believe you when you say Blok's operates in-place, and I have had code similar to that in the past. However, when I have used the Trace Execution kit, I would get a memory resize of 0 after an iteration or two with code similar to that. So the buffer is in place, but there is a constant resize of 0. When I replaced it with something similar to what I posted, then the constant resize of 0 is gone, and everything operates slightly snappier. The resize of 0 does not seem do be a null operation, but I am not sure.

Cheers,

mcduff

[altenbach]

Yes, I would place the subtraction before the loop, but most likely the compiler will do that for you anyway as part of loop invariant code handling. 😄

Not sure where the problem is, but I I try to insert your snippet into LabVIEW 2015, it say it requires 2017, in contradiction to the snippet header. Is this just me?

I'll do some testing ...

[altenbach · Accepted solution]

---

@altenbach wrote:

I'll do some testing ...

---

Just doing a "two step" inside the loop (index array..subset of 1D array) speeds things up by about 2.5x. Try it 😄

In this particular case, start was 45, end 576 and 13 rows selected. But you get the same ratio for much longer or shorter ranges too.

[altenbach]

---

@Blokk wrote:

If I configure iteration parallelism for the FOR loop, I guess it can help?  

---

Some casual testing shows that it does not help (at least on my code version. It actually slows things down (parallelization overhead is large compared to the difficulty of the task: splitting the problem up and reassembling the partial results at the end) YMMV. 😄

[mcduff]

@Altenbach

Would you mind posting your bench marking code?

When I disabled debugging I could not see the difference between the earlier two methods. With debugging on, the in-place one I posted was faster, but I do not think that means anything.

Lastly, you get one memory allocation from the array subset function, which is missing from the in-place structure.

cheers,

mcduff

[altenbach]

---

@mcduff wrote:

@altenbach

Would you mind posting your bench marking code?

---

Here you go (downconverted to 2015).

(I initially did not even touched your code, because even if it would perform better, It looks unmaintainable :D).

I tried your code with my benchmark, but the results are different (Mine and Blokk's agree in the result for the same inputs, while yours gives different numeric results, but with the right array size. Maybe I hooked something up wrong?)

Note that for each algorithm, the second and first run differ slightly, so I run each twice and pick the fastest. I have not investigated further.

[altenbach]

Here's another version that is about the same speed as mine.

(Warning: Of course if you would do the loop first and the subset at the end, it'll cost you, even though the result is the same :D!)