LabVIEW
remove empty elements of array taking soo much time
Solved!
Don't use Remove from array.
Simply use a conditional array at the loop border and let it build a new array with all non empty rows. That way you're only making one memory allocation (or a few) instead of a full array copy each iteration.
/Y
Two things that your VI is doing that is really bad.
1. Delete From Array causes a memory allocation each time it is called. So for your example, I see 129 memory allocations of large arrays. That is very expensive in both time and memory.
2. Each time you find an empty string, you start searching from the beginning again. This means you hit that first element at least 128 times. Again, a waste of time.
As Yamaeda pointed out, the more efficient option is to use Index Array tunnels and make the output tunnel be conditional. Only hitting each element once and only a total of 2 possible memory allocations (the output tunnel defaults to the full array in size and then is stripped down due to the conditional).
@mabaig wrote:*Note : I have created this 1D input array from Intel Hex file of 1776 lines and it is created in 64ms.
Depending how you are "Creating" the 1D output from the file (I don't even know what a "HEX file" is!), you could probably wrap the filtering together with the parsing so the array is created without empty elements from the beginning. No need to create a large array just to throw most of it away a nanosecond later. 😄 1776 lines is not much.
Can you show us the rest of the code and a typical file?
@mabaig wrote:
Awsome guys.... That was silly of me,lol.
Thanks alot. Help is truely appriciated.
sorry for more trouble, but I have one more question, Is it ok to use string subset frequently?? have used string subset like nearly 10 times in one VI (in a state machine where in each state I am seperating string at different offset address) ... Just need an advice!!
It's not easy to know that Remove from Array is such an expensive operation, but now you know. 🙂
I don't think String subset is that bad, you should be fine, but you can always create a performance test to compare a couple of solutions if you're wondering. 🙂
I remember a similar problem to yours, but i think i was using Match pattern or something, and the difference in speed was surprising!
/Y
@mabaig wrote:I have attached the hex file sample.
No, this is just an image of a hex file that is useless for us. Please attach the actual file (you can put in inside a zip file, together with the VI you are using to read it!).
We cannot debug images, for example we cannot see unprintable characters and delimiters. We can't see 60k characters and we also don't know how the fields are defined. Which part of the file shown would result in an empty array element?
@altenbach wrote:
@mabaig wrote:I have attached the hex file sample.
No, this is just an image of a hex file that is useless for us. Please attach the actual file (you can put in inside a zip file, together with the VI you are using to read it!).
We cannot debug images, for example we cannot see unprintable characters and delimiters. We can't see 60k characters and we also don't know how the fields are defined. Which part of the file shown would result in an empty array element?
Not to mention that I really don't feel like recreating all that text.
Going back to my earlier suggestion, you should be able to parse that file from the beginning so no empty strings are ever being generated. It just needs to be a tiny little bit smarter with a little more (or possibly much less!) code. That's why we also need your VI that we can assume works correctly but is just too slow.
