Exclusion of values in mean calculation

[dan07]

Andre, I want that some values disappear from the mean calculation because I am performing a visual analysis of segments from a large (30 minutes) series of values. The segments that present artefacts are discarded, and this set of values should not contribute to the mean. I will use a boolean (green and red), and when I pushed the button it will turn in red and not allow that this set of numbers contribute to the mean calculation.

I used in the example only single values because is more easy to explain, but the solutions presented are very suitable to my problem. I solved my problem! Thanks all!


Another question, if I am working on a waveform and I want to perform mean calculation of only a segment of a whole waveform (plotted on a single graph), how do I build a VI that allow me to select (using mouse) only a segment of the whole waveform, and at the moment that I release the mouse key, the area selected remains selected and one operation is made in the values, for example the mean calculation.

Thanks

Daniel

[andre.buurman@carya]

If you're value are in an array. You can delete portions of the array with "delete from array" in a for loop. The delete array is just as easily used to delete one sample or many sample at once. Just iterate use a for loop through all the portions to be deleted.

You're waveform is just an array of Y values. You can apply the same methodology to it using the "array subset" function to get a subset of the array to calculate the mean from.

[dan07]

Andre.

I understood what you said, but how do I control the index and leght of the array subset using the active selection with the mouse?  The index should be the start of the selection and the leght should be the difference between the last point of the selection and the first point (both in X axis).

Thanks

Daniel

[altenbach]

---

andre.buurman@carya wrote: If you're value are in an array. You can delete portions of the array with "delete from array" in a for loop. The delete array is just as easily used to delete one sample or many sample at once. Just iterate use a for loop through all the portions to be deleted.

---

Be aware that using "delete from array" in a loop is a very expensive operation for large arrays, because with every iteration the array needs to be resized. My religion forbids me to use "insert into array" and "delete from array" inside a loop for this reason.

We have discussed this many times in the past, and the differences to more efficient versions that operate "in place" can be astronomical!

For example in this old VI we delete elements from two arrays when the value in array #1 is >1., I now get the following times (the table with the times on the FP is very old)

If you use delete from array and the array size is 100000:

1. delete from array: 8500ms
2. built new array: 15ms
3. replace I: ~0ms
4. replace II: ~0ms

The delete from array is so slow that we really need to wait. The "replace" versions are so fast that we need a bigger array to see the difference.

So, lets try with an array that is 10x larger (size 1000000)

1. delete from array: infinite (not measured) Probably about 1000seconds!!!!
2. built new array: 328ms
3. replace I: 50ms
4. replace II: 30ms

With an array that is 100x larger (size 10000000), the built array version starts also to lag behind:

1. delete from array: ~infinite (not measured)
2. built new array: 18000ms
3. replace I: 470ms
4. replace II: 300ms

In summary, "delete from array" is 30x slower on an array that is 100x smaller!!!! The difference actually gets worse with larger arrays. It is useless inside a loop.

A useful measure is the Big_O_notation. "Delete from array" is probably around O(n²), meaning a 10x increase in size will take 100x longer. The "replace" versions are ~O(n), meaning that the time is linear with input size, a 10x bigger array takes 10x longer).

The crucial piont is the fact that the "replace" versions operate on the array "in place", eliminating all memory allocations inside the loop.

HERE is the old VI, try for yourself. (Just select a size and an algorithm and run to meaure the time).

For a similar discussion, see also:

http://forums.ni.com/ni/board/message?board.id=170&view=by_date_ascending&message.id=268602#M268602

http://forums.ni.com/ni/board/message?board.id=170&view=by_date_ascending&message.id=137319#M137319

[lmtis]

Altenbach,
that is very odd, but valuable to know.  Thank you.

[altenbach]

---

lmtis wrote:
that is very odd...

---

No, it's not odd at all if you think about it. Arrays must always be contiguous in memory.

If you have a large array and delete an element, the compiler has e.g. the following options (and I don't know which one it uses).

1. It can delete that element, move all higher elements down one slot, and them define the lenght as one shorter.
2. It can allocate a new (shorter) array and copy all retained elements over.
3. How else would you do it?

Both options are a lof of work and for each element deletion, most of the other elements of the entire array need to be touched and manipulated. That's why the complexity goes with the square of the array size.

If you use the the "replace" version, all operations are "in place" in memory and each array element needs to be touched only once, no matter now many delete operations you perform. The complexity is liner with the array size.

"delete from array" is highly optimized code and works very well if you only need to delete a single element or subset. As soon as you need to do repetitive work in a loop it fails misearbly in terms of performance. The reason is not a flaw in the code, but a fundamental issue dictated by the task at hand. 🙂

[dan07]

Altenbach... I understood what you said....

And the problem might be quite serious because I work with large files.... The performance of computer will dropped down....

But one thing that I still don´t know hot to do, is the selection with mouse. I have a waveform in the screen and I want to select a region of the waveform with the mouse and when I release the button of the mouse, an operation runs on the region selected, for example a mean calculation. Is there any way to build this with Labview?

Thanks

Daniel

[altenbach]

---

daniel.penteado wrote:

 I have a waveform in the screen and I want to select a region of the waveform with the mouse and when I release the button of the mouse, an operation runs on the region selected, for example a mean calculation. Is there any way to build this with Labview?

---

Yes, this is easily possible using mouse down events on the graph and parsing the coordinates.

Since it has really nothing to do with the current topic, you probably should start a new thread with a new catchy title. 🙂