Memory/Speed of Split 1D array vs Delete from array

[DavidU]

Just wondering how Split 1D array works - does it create two new arrays in a new section of memory or does it just split the existing array in two, reusing the memory in place. (assuming that the original array is not needed elsewhere). If the latter is the case then presumably it is more efficient to use split array than delete from array if I want to remove element(s) from the beginning or end of the array. Is there a speed advantage as well?

If I use split array but don't then use one of the output arrays is that memory deallocated or does the array remain in memory?

Thanks
Dave

[muks]

Well I tried the profile Vis to check If there is any difference with respect the execution time of both.I didnt find any difference.

[tst]

I would suggest simply testing it with a large array.

Based on experience with the way these things work in LabVIEW, I wouldn't be surprised if both cases were optimized not to cause data copies. If memory serves (I don't feel like looking it up and I may very well be wrong), an array has the dimension sizes together and then a pointer to the actual data. If that's true, then there shouldn't be a problem splitting it (or deleting either from the start or the end) in-place. You should try looking up the documentation for how LV holds arrays in memory.

This of course might be different for arrays where the elements have variable sizes (e.g. strings, variants, pink clusters).

If you do this research, post back to tell us the results. It might be interesting.

P.S. I seem to remember something about a bug in an older version which caused LabVIEW to leak memory if the bottom output of the Delete primitive was unwired, although I don't remember the specifics. Presumably, this was fixed, and if you don't wire the array out its memory should be freed (within the general memory handling behavior of LabVIEW, of course. If you want to make sure, use the Request Deallocation primitive).

[altenbach]

It seems to me you want to obtain an array subset (since "split array" and "delete from array" can be used for what you want).

In this case you should use "array subset". It is the correct tool for the job. 🙂

(still I recommend to do some beckmarks with typical data. Is this a one-time operation or do you do it repeately in a loop, for example?)

Message Edited by altenbach on 08-11-2008 12:30 PM

[Wiebe@CARYA]

If you are interrested in the first part of the array, you can also use the
Resize Array function. I think split string is actually more efficient, but
haven't tested this for a long time. It might also differ between LV
versions.

Regards,

Wiebe.

[DavidU]

Hi,

Thanks for all the replies. At the moment my array is very small, and it probably doesn't make any difference which way I do it. I asked more out of interest than anything else. I'll follow up some of your reading suggestions.

I have done some benchmarking for the different methods available on a 1D Double array, results below. Each value is an average of 100 trials. Times in ms.

Interestingly,  it depends on whereabouts you divide the array, particularly for Subset and Split methods, less so for Reshape . In the table below the (split point)*(array size) gives you the index where the array was divided.

Also note that the choice of method will depend on what you want out at the end, Subset and Reshape only give you one array out, whereas Delete and Split give you two arrays.

Thanks
Dave



Message Edited by DavidU on 08-12-2008 12:24 PM

Message Edited by DavidU on 08-12-2008 12:25 PM

Message Edited by DavidU on 08-12-2008 12:27 PM

Message Edited by DavidU on 08-12-2008 12:27 PM

[tst]

It looks like your system is quite unreliable.

I would suggest getting rid of the smaller arrays. They don't give enough accuracy anyway.

Also, I would advise wiring each of the outputs into an index array and wiring an indicator out of that to have consistent memory allocation.

Additionally, either closing LabVIEW or allocating a large amount of memory for LabVIEW in advance might help.

If you post the VIs, other people can test this too.

[DavidU]

Ok please ignore the results I posted earlier, they are rubbish - not least because I got the column headings mixed up but also because my code was full of bugs

So, here is a revised table, and the code - which hopefully only contains a few bugs... I'm not clued into benchmarking yet so please feel free to rip the code apart.

I still get different results depending on where in the array I split it, most noticeably with subset and reshape. There is no effect with split. I'm guessing this is to do with the memory allocation. (I have not preallocated memory in my code, but I did wire the output arrays to Index Array)





Message Edited by DavidU on 08-12-2008 04:49 PM

[altenbach]

As expected, "array subset" is the correct way to do things and the time is somewhat proportional to the size of the resulting array, again as expected.

About your benchmark code, a few things could be improved:

1. You should disable debugging.
2. To compare identical outputs, you should only retain one relevant array from the case structure, otherwise you are comparing apples and oranges.
3. You should NOT place indicators inside a frame that you are timing, else you run the chance that sometimes you measure the updating of the indicators. Even then, asynchronous updated of the FP indicators could randomly interfere with the timings, especially on single processor systems. You should use a three-frame flat sequence for the tick-code-tick and it should not contain any controls or indicators or possible parallel code.
4. I would increase the priority and lower the loop count.
5. Taking the "mean" is mathematically incorrect. You should use "array min". All external influences add to the "real" execution time!

[tst]

I'm not sure that your testing is accurate, because you might have memory allocation issues (going over the same array in the loop might force LabVIEW to create a new copy of the data each time. Unlike Altenbach, I was expecting both split and and delete to reuse the memory if possible, so I wrote a different implementation (8.0).

I'm not actually sure if this implementation is any more accurate, but at least for me subset, reshape and split all take 0 ms. That makes sense since you should be dealing with pointers (if you look at the context help for split you will see it returns subarrays).

Delete, which I was expecting to be optimized the same way, does seem to create a copy. I'm not sure why that is, since it should be able to behave the same way that split does, at least for the case where the array is not used elsewhere.