Reshape 2D Array and Remove Zeros in 2D Array

[altenbach · Accepted solution]

So basically, each column can have a different lenght, but since 2D arrays cannot be ragged, teh shorter columns will be padded with zeroes.

See if this can give you some ideas.

[SDuffyLV]

This is exactly the understanding I needed to complete the problem.

I have extended the problem to fit everything that was discussed in the last replies. See VI (4) attached.

It would be interesting to know if this could be made cleaner (e.g. if we had 100 columns, then what?) but this isn't necessary. I'm marking this reply as the Accepted Solution.

Edit: VI (5) shows this solution!

Thank you for taking the time to understand the problem and come up with a great solution. You really are a hero of these forums! 😁

[altenbach · Accepted solution]

---

@SDuffyLV wrote:

It would be interesting to know if this could be made cleaner (e.g. if we had 100 columns, then what?) but this isn't necessary. I'm marking this reply as the Accepted Solution.

Edit: VI (5) shows this solution!

---

NO! Still way too convoluted. Why do you think you need to constantly go in and out of loops? All you need is one outer FOR loop to process one column!

Note that the "search for zeroes" is a snipe hunt, because there will never be any zeroes inside the loop here (given the current inputs and limits). These zeroes only get added in the indexing output tunnels to pad the shorter 1D arrays to the longest to create a 2D array. They don't exist inside the FOR loop.

One solution would be to create a ragged array: You can insert a bundle right before the output tunnel, creating a 1SD array of cluster where each cluster contains a 1D DBL array of arbitrary length. (not shown).

Also note that it is not necessary to create an array subset first (Array subset requires a new buffer allocation because the memory order of elements changes! Expensive!). You can limit the number or columns to be processed by wiring N of the outer FOR loop. You could even eliminate the first transpose by indexing out columns based on [i].

[altenbach]

Also note that if you are dealing with gigantic input arrays and speed matters, you can parallelize the outer loop to process several columns at the same time (most computers have several CPU cores!).

(Don't do it until you can prove that it is absolutely needed, though!)

Be aware that this requires careful testing and benchmarking to see if it is worth, especially if there are other CPU intensive processes already running.

Also, do you really need the outputs for "filtered array" and "median array" or are these just for debugging?

[SDuffyLV]

This is exactly what is needed. Perfect answer!

I can understand what you mean about the Snipe Hunt now.

It isn't necessary for the loops to be paralleled but it's a good technique that you've made me aware of that I'm sure I'll use in other tasks.

Thank you so much for the assistance.