Removing a list of indices from an array

[Ben]

" I must be doing an extra scalar copy or something."

I think you nailed it!

I think your approach is moving two values were Christian is only moving one.

Ben

[altenbach]

OK, I got an version with the "in place" structure, that is significantly faster. It can even do the 50M(!) I32 element array (200MB for a single copy)  in under one second. Even there, the time is still linear with the array size.

ArraySize        In Place Version

==================================

    5000        0ms|   0ms|   0ms

   50000        1ms|   1ms|   1ms

  500000        6ms|   6ms|   6ms

 5000000       67ms|  98ms|  66ms

50000000      678ms| 979ms| 649ms  (this is an additional data point!)

Maybe somebody understands why it is faster, or if it is even used correctly. 😉 Interestingly, I don't even read the element at the insert point, The index wire of the in place structure is just needed to define the insert point index at the right boundary. The other case is "-1" and just wired across.

(Btw, make sure to close the FP of all VIs under test. Having it open slows them down significantly.)

Message Edited by altenbach on 08-30-2007 11:58 PM

[altenbach]

---

@altenbach wrote:
OK, I got an version with the "in place" structure, that is significantly faster.

---

Sorry, I have to take that back.

If I replace the "in place" structure wuth a "replace array subset", I get about he same time, maybe even a bit faster. Originally, the "replace" version was slower than the "inplace" version only because I had debugging enabled by accident.

[Marc_A]

I see a problem with both of these solutions that use the in place element structure. You are both replacing elements in the array with -1 if they are to be deleted. However, what happens if the element is actually -1? I think you need to create another array, as Christian did in his original example, that contains all the indexes and replaces each of those elements with -1.

Message Edited by Marc A on 08-31-2007 08:52 AM

[Ben]

Christian,

I really appreciate the public service you are doing with these comparisons between old and new.

My gut feel is that LV was already damn good at minimizing buffer copies prior to the new constructs, PROVIDED you watched your buffer copies.

I suspect there was little room for improvement of the under-lying code but what ws improved was our ability to clearly designate what gets copied when. This by the way will be very helpful because the "old way" relied heavily on "gestures and inuendo (sp?)" to tell LV what to do with buffers.

Ben

[Ben]

Marc wrote "You are both replacing elements in the array with -1 if they are to be deleted. "

That thought crossed my mind as well.

If you read the fine print in an earlier post, they are exploring the special case were all values are positive so the "-1" flagging is valid for this special condition.

Ben

[Marc_A]

Oh, ok. It's just that the other solutions handled it with the index array or even the boolean array. If the input to this sub VI is an I32, I should be able to put a -1 in the array. Once Christian gets here, he can have a little extra work.

[altenbach]

---

@Ben wrote:
I suspect there was little room for improvement of the under-lying code but what ws improved was our ability to clearly designate what gets copied when. This by the way will be very helpful because the "old way" relied heavily on "gestures and inuendo (sp?)" to tell LV what to do with buffers.

---

Yes, the "in place" structure does not give any advantage here, because the entire code is already "in place" anyway, so there is nothing to improve.

It should probably only provide an advantage in cases where the compiler is too careful and thinks it needs another datacopy, while you know things can be done in place.

[Ray.R]

Euh..  Wouldn't this topic be a very nice nugget???

(or did I miss that somewhere!!)

🙂

[Ben]

Good observation Ray.

Why don't you link this thread into Darren's "New Feature Nugget" as an apendix?

Ben