Replace negative values from 2d array

[charmand79]

Hi all,

I have a large dataset that I want to post-process.. My data is in the form of a 2D array of typically 1e5+ rows by 100+ collumns! I want to replace all negative values in the array with zeros.

My initial though was to use two indexed for-loops with a case structure inside looking for the >0 condition. However, this way I am practically rebuilding the array from scratch and it takes forever (had to abort after about 5 mins of calcs while I was still less than 10% done). I have attached a snapshot of my sample vi for your consideration.

Is there a more efficient and quicker way of doing that?

Thanks a lot for your time,

Harry

[RavensFan]

I think it is taking a long time because you are rebuilding such a large array from scratch.

I would work on the original array.  Loop through row by row, then element by element.  Use replace array subset whenever your condition is met, otherwise pass the array through unchanged.

[JÞB]

This should be pretty fast

-n  * 0  /2 = 0

0   * 1 / 2 = 0

+n  *2 / 2 =+n

[Mads · Accepted solution]

There are lots of ways to do this, but in your case I would just wire the 2D array to a In Range and Coerce primitive (from the comparison palette) with the range set from 0 to +Infinity.

[Wart]

I've never used the in place structure before, but I would try something like this.

[JÞB]

Mads-  simple solution!  I'd be interested in benchmarking it against mine.

Wart- that would be the way to use the IPE.  Bet it would be faster if you you used the iteration terminals to replace array elements

Perhaps a combo. Try the  "Mad-Warts" approach

there should be no buffer allocation outside a scalar.  A good benchmark would give us a clue to the complier optomizer

[Darin.K]

While Mads was busy posting I was busy benchmarking. IR&C is about 5 times faster on my machine. Someday I'll learn to post first and benchmark later. 🙂

[charmand79]

Guys,

thanks a lot for all the excellent ideas.. I was about to start benchmarking and let you know but then I saw Darin's message... So I'll trust his findings and use the "in range and coerce" method.. There you go Darin. Please stick to benchmarking first and posting later !!! 

Harry

[charmand79]

Could not resist setting up a quick benchmark myself.. It appears that mads solution is indeed the faster but, according to my benchmark, only by 5-10% when compared to Jeff's..

Just for the record, Mads IR&C replaces all negatives with zeros in a 184000 x 150 array in 310-320 ms using a rather dated PC

Thanks again to all for your time..

[Mads]

The primitives have a tendency to be more efficient than anything you can do by writing your own logic (fair enough, but it does take a bit of fun out of things ).

Here is a benchmark of some of the options:

I have not checked the memory usage, but I'm sure the primitive is the winner there too.