Filtering values before writing to an ARRAY

[Parny]

Hello All,

I have an array which will be written with "100 values at a time" continuously for a period of time, so it could end of few thousands of samples at the end. During this event, I would like to filter some of the values(lets say if any value is <100) before writing to that array. The other way of "storing the values into the array, and filtering using a for loop" is a slight time consuming for my task (also doing some database tasks)". Hence I am hunting for any possibility of killing the value before it enters the array. But it is not a single value that is written to the array, is rather a bunch of hundred values(those are samples being collected).

I have seen some popular posts regarding this situation, but they seem to be relying on indexes, the array is static(already having some values), and using "for loops".

How this can be handled somehow?

Thanks in advance.

[TCPlomp]

Hello Parny,

See this post, where I do something similar for a set of two arrays using bounds.
You won't be able to do this without loops.

Ton

[altenbach]

There is nothing wrong with loops, the main performance issue with this kind of problems is the fact thay the size of your array changes with every operation. This causes constant expensive memory reallocations if not done right..

Is the final array size constant?

The most efficient way would probably be to allocate an array sized to an upper limit, then conditionally fill it up with valid data, e.g. doing the comparison, keeping track of the insert point in a shift register, and conditionally replace elements with valid data. The key word is "in place".

---

Parny wrote:

... the array is static(already having some values), and using "for loops".

---

You lost me right there! Can you be a bit more specific? I have no idea what you are saying here.

Message Edited by altenbach on 02-25-2008 11:28 PM

[Parny]

Ton,

I have installed OpenG llbs and gone through the vi that you posted. As you said, it is been using loops and which is somewhat similar to my sub vi that I created. As Altenbach said, may be I must be wise choosing the upper limit of the array.

Altenbach,

The final array size is not same always, so its not constant.

Regarding the other quote, may be my wording is not correct . I was trying to say that the array is not processed in a real time, rather is done once the storage of the data into the arry is completed(in the example which I found in old posts). I said static, because at the end of storage the array is fixed.

-------------------------

 I have to write soon processed data to the database and prepare for the next cycle again(this may occur after few seconds), similar kind of (5)more vis running parallelly. This is a 24/7 operation. To also avoid overwriting of the samples(this may happen if my vi spends more time in database operation or processing larger arrays, resulting error-200279), I am trying to be minimum and conservative at processing time in any vi.

[TCPlomp]

Yes, but the OpenG VI isn't doing any resizing of arrays.

Ton

[altenbach]

---

Parny wrote:

The final array size is not same always, so its not constant.

---

Still, you probably have a reasonable upper limit so you can make an initial allocation and do things "in place" even if you waste a certain amount of memory. Even if you overestimate by a factor of 2, you'll be much better off this way, because a change in array size forces a new copy to be made in memory, a very expensive operation. Keep the array in a shift register for the duration of the program. If you have LabVIEW 8.5, use the "in place memory structure", it sometimes can save an extra buffer allocations. Use flat code for critical sections instead of subVIs.

Doing things right can be orders of magnitude more efficient. 🙂

Can you write a small simulator with sample data?

Just try a few things! Display buffer allocations, do some profiling and timings. Make sure to aviod local variables and value property nodes. Don't display huge amounts of data in graphs and other indicators, etc.

Message Edited by altenbach on 02-27-2008 08:31 AM