topic Re: memory manager optimization in LabVIEW

memory manager optimization

QFang — Wed, 07 Oct 2015 17:57:29 GMT

I'm carrying a pre-allocated U8 buffer array on a shift register, dumping the 'active' portion of it to disk when it is getting 'close to full'.

My overall goal is to reduce calls to the memory manager as much as possible for this particular code. I insert data into the buffer array using in-place "array split / replace sub-arrays" which seems like a good idea(?), but when I go to get the 'data' portion of the buffer, am I better off using the in-place to get the sub-array (the size of which is close to the max size of the buffer array itself), or is it better to use 'Array Subset'?

My thinking is that the 'in-place' might be better since I'm not creating a copy of the main array, while if I wire the main array to 'Array subset' I have a branch/split on that wire, which creates a (new) copy of the whole array (except buffer allocation seems to indicate I don't on the input, but I do on the output.. sort of opposite of the in-place structure)?

On the other hand, the in-place shows two buffer allocations; the total size equaling the size of the buffer array, and another allocation on the write binary file VI anyway...

I always have a hard time trying to 'reason' or 'think' myself to the ideal solution in this type of a problem. 😕 Pointers and insight would be appreciated. Again, I'm trying to reduce dynamic memory alloc/dealloc as much as possible, not necessarily looking to maximize (cpu) performance.

Re: memory manager optimization

crossrulz — Wed, 07 Oct 2015 18:11:21 GMT

Unfortunately, I think somebody from NI will have to chime in here to get a definate answer.

But based on my limited understanding, the Array Subset will not make a copy of the entire array, but the subset you are getting. Supposedly, the compiler is smart enough to allow the read functions for the array happen before the write functions when on the same wire (branches count). If multiple write functions, then a copy will be make for that second write function.

I'm not completely sold on the IPES helping here based on what I just stated.

Re: memory manager optimization

mcduff — Wed, 07 Oct 2015 18:16:41 GMT

I always use a construct like your first diagram, the in-place structure.

Cheers,

mcduff

Re: memory manager optimization

Bob_Schor — Wed, 07 Oct 2015 18:16:51 GMT

Good question. I don't know the answer, and even if I said I did, why should you believe me (and who says that this would be true on your machine and with your version of LabVIEW)?

Besides, I'm a Scientist, so my answer would be "Do the Experiment". Write a little test routine, put timing code around it, and see how long it takes to do a reasonable number of writes similar to what you'd want to do with your code. Do it with the In Place method, then with the Subset method, and compare.

Several years ago, Darren Nattinger (who I believe still holds the title of The World's Fastest LabVIEW Coder) gave a very interesting talk at NI Week asking "Which way is faster?" The audience voted among three alternatives, and in most cases, the correct answer (which he demonstrated by rinning the code with timers surrounding it) was chosen by the smallest nujmber of people.

You have to "do the experiment" ...

Bob Schor

P.S. -- when you've done it, let us know the results!

Re: memory manager optimization

mcduff — Wed, 07 Oct 2015 18:24:45 GMT

I would lie to chime in if you are testing for memory.

Use the Windows Memory Task Manager to watch the memory when you do the two tests. Make array of 1 million or so of the data type you have, with 1 million points it should be easier to tell the memory usage. Check the task manager to see if any copies of the data are made. I did this in the past, do not remember which LabVIEW version, the IPE was better for memory, not sure about speed. I do not know if new versions have changed so case 2 uses the same amount of memory as case 1.

Cheers,

mcduff

Re: memory manager optimization

Bob_Schor — Wed, 07 Oct 2015 18:29:38 GMT

McDuff is clearly another Scientist, who says "Do the Experiment" ... Lay On, McDuff ...

Re: memory manager optimization

QFang — Wed, 07 Oct 2015 18:52:12 GMT

Oh I run 'experiments' all day long, trust me.. My issue here is that I am not aware of a good way to run a MEMORY test on an RT target. I already know that IPE often (but not always) take LONGER to run than not using it... But I am at a loss for how to test memory allocation/deallocation (on an RT system).

I'm actually oposite of mcduff when it comes to usage of IPE's. A year ago or so, I started making extensive use of IPE's, until I found that it really ate up a lot of CPU, at least in the cases and in the manner I tried to use them back then, so my general take is to never use them unless there is a very good reason/chance that it will actually be beneficial.

Even on a Windows platform, I'm not quite sure how I would get detailed information. Keep in mind, memory usage isn't the exact metric I'm looking for.. I'm looking for if the memory is re-used without calls to the memory handler (on an RT target) or not. blergh.. I feeling like I'm doing a poor job explaining what I mean. sorry guys..

Re: memory manager optimization

mcduff — Wed, 07 Oct 2015 18:53:08 GMT

According to the LabVIEW 2014 Profile Perfromance and Memory ...

This Case 1 snippet is the best

Strangely, without the DVR it is worse than the array subset, snippet below

I think you need to test in your application, ie, in a subVI with an array in and array out, if everything is inlined than not copies should be made.

Cheers,

mcduff

Re: memory manager optimization

QFang — Wed, 07 Oct 2015 18:58:14 GMT

By the way, the reason I'm taking such a keen interest here, is that I'm reworking a 'system log engine' that runs on my RT targets and logs messages along with time and system memory/cpu statistics to a file from time to time. The old version of the engine carried a string on a shift register that it kept bundling to until it hit a certain threshold and the string was written to file.. this would create a (noticeable) 'ramp' effect over time on memory usage on the RT targets. Now, I'm also chasing and testing for memory leaks on these targets as they go in the field and are expected to operate 24/7 for hundreds of days. The task of positively identifying the presence or absence of a memory leak is MUCH simpler if my code overall is 'quiet' and 'consistent' in its memory usage. As such, most my RT code make use of pre-allocated arrays and other techniques to reduce and prevent memory allocations and de-allocations.. This makes even smaller (4 byte references anyone) leaks stand out much sooner in my testing. This system log engine is one of the largest/last sources of 'noise' on my application memory usage. 🙂

Also, it is already better than the old version of this system logging loop.. now I'm just going full on OCD to make it as good as I can while I'm in there tinkering with it.. Once done, it is published on our VIPM repository so I want it as 'good as I can get it'. 🙂

Re: memory manager optimization

QFang — Wed, 07 Oct 2015 19:00:49 GMT

Mcduff, could you snag a screen shot of the Profile performance and memory result for Case 1 and Case 2? What metric do you use to say case 1 is 'best'? (sorry if that sounds like a retarded question, I'm just curious how you made that determination of 'best'?)

Re: memory manager optimization

mcduff — Wed, 07 Oct 2015 19:05:02 GMT

the LabVIEW 2014 Profile Perfromance and Memory says case 1 is using 3 MBy and the other case is using 4MBy

That was the metric. This is on Windows, not RT.

mcduff

See http://forums.ni.com/t5/LabVIEW/Queue-Memory-Allocation-Weirdness/m-p/1990809#M656598

for strange memory probelms regarding queues

Re: memory manager optimization

QFang — Wed, 07 Oct 2015 19:49:54 GMT

Understood. When I run the Profile Performance and Memory on the RT system, you're basically saying that I can run it one way, look at the "Max Bytes", then run it with the other code option and look at "Max Bytes" for this particular VI and compare?

My code with in-place shows Max Bytes = 72.34k, avg blocks = 25, min blocks = 25, max blocks = 26

My code with array sub-set shows Max = 88.67k, avg blocks = 27, min blocks = 27, max blocks = 27

Re-factoring to use DVR I getMax Bytes = 9.27k, avg blocks = 24, min blocks = 24, max blocks = 24

Presumably the rest of the buffer now resides in the top level vi where the DVR is created... but for some strange reason that VI shows 0 across all tabs in the profile... 0 ram, 0 time... but I know it was running..

Thanks for pushing me to check the DVR wrapped version.. looks fairly superior and as an added benefit I'm only passing a reference in and out of the 2-3 subVI's I have, instead of the array.

Re: memory manager optimization

QFang — Wed, 07 Oct 2015 19:52:40 GMT

As a side note on weirdness and Queues, obtain Q with fixed max depth on an RT does not pre-allocate memory (or at least not last I checked, maybe fixed in 2015). Instead you had to fill the Q to the max size.. Then, if you used 'flush' memory COULD in some instances be de-allocated... so to prevent that, the recomendation was/is to dequeue (and ignore) the queue back to 0... Queues on RT of course is potentially a really bad idea anyway, if you are on RT for the sake of deterministic execution times anyway. If you don't care about jitter etc., then they are still somewhat fickle, but have less CPU overhead than the RT FIFO's.

Re: memory manager optimization

mcduff — Wed, 07 Oct 2015 20:00:21 GMT

Yes.

I have no experience with RT, only the Windows side. That being said, the Profile Performance and Memory usually gives a good indication of the memory BUT it is not always right. When I looked at the Windows Task Manager for some situations I would see a difference, I do not know if there is anything similar to the Task Manager on the RT side. So, what I guess I am saying I would optimize first with the Profile Performance and Memory then with the Task Manager if those gave me different answers. Generally, I use DVRs for large arrays. It is easier to pass around a reference than an array.

Cheers,

mcduff

Re: memory manager optimization

nathand — Wed, 07 Oct 2015 20:22:01 GMT

There's a bunch of misleading information here.

First, there's no reason to use an IPE structure. LabVIEW will allocate a subarray for the array subset, which means allocating only a new starting point, array length, and stride (the gap between elements, if you were decimating or reversing the array), but pointing at the original data (NOT making a copy). For more information on subarrays, see https://lavag.org/topic/7307-another-reason-why-copy-dots-is-a-bad-name-for-buffer-allocations/

Mcduff's example snippets also may not be that helpful, because the array size is constant, so the Initialize Array can get folded into an array constant. This makes sense if the array is actually constant, but if you do something that modifies the array (which could include splitting and recombining it, depending on how smart the compiler is), LabVIEW then needs to make a copy of that entire constant array, because constants are read-only. I suspect that the DVR works around this in some way, which would explain why the non-DVR case is much worse - but I would still recommend avoiding the IPE entirely here, and if you want accurate measurements, replace the array sizes with controls. See https://lavag.org/topic/7063-why-does-the-replace-array-subset-double-the-used-memory/

Re: memory manager optimization

QFang — Wed, 07 Oct 2015 20:35:27 GMT

Pure gold those links nathand. THANKS!

Since my buffer array is 'carried around' a bit, and is indeed of a fixed size, in general I think it might still make sense to pass the DVR reference around instead of the array... for the example shown (where I split the array and write a portion to file), I see what you mean nathand, and will re-work that portion to not use IPE. (Where I put data into the fixed array buffer, I assume I would still be better off to use IPE's? After all, I'm specifically looking to replace big chunks of the array with new values, in-place.. seems like the text book example of when to use IPE?)

I will do some more benchmarks tomorrow to look into performance; both speed and ram use, and see if there are any big up or down sides on either of those metrics one way or the other.

Thanks to all of you who are posting and participating!

Re: memory manager optimization

mcduff — Wed, 07 Oct 2015 20:44:13 GMT

You are probably correct on all accounts as usual. I believe a buffer will be allocated for the File Write part regardless of whether you use an IPE or Array Subset, the only question is if the IPE or Array Subset makes a copy. I always use the IPE becasue at one point when I rewrote the Min-Max Decimation VI there was a memory copy due to the Array subset, in the IPE there was no extra buffer. Again, I do not know what version, I can place the Min Max decimation here if anybody wants it to examine.

Cheers,

mcduff

Re: memory manager optimization

nathand — Wed, 07 Oct 2015 20:50:00 GMT

@QFang wrote:

(Where I put data into the fixed array buffer, I assume I would still be better off to use IPE's? After all, I'm specifically looking to replace big chunks of the array with new values, in-place.. seems like the text book example of when to use IPE?)

I think you're attributing some magic to an IPE. The compiler already arranges code to do work in-place when it determines that's possible. There should be no advantage to doing Replace Array Subset on its own, versus splitting the array in an IPE and then substituting part of the array with the new data before recombining. In both cases, you're copying new data into a portion of the existing array.

The textbook use of an IPE is when you want to operate in-place on the elements of the array (as shown for example in the IPE help for Array Split/Replace Subarrays). If your input to one of the subarrays originates from somewhere other than the corresponding IPE subarray outputs, then you gain nothing from the IPE, because there's nothing that can be done in-place there. The two subarrays (the existing data and the new values you want to replace them with) are both in memory at the same time, in different memory locations, and you have to copy one over the other.

Re: memory manager optimization

mcduff — Wed, 07 Oct 2015 21:13:15 GMT

Nathand is correct

Case A is the Best as far memory as code simplicity.

Case B uses the same memory as Case A but mode code.

Case C uses the most memory.

(Note the delay is to monitor the memory usuage on the Windows Task Manager, Should be not more than 3 Mby which is true for A & B)

Case A

Case B

Case C

Re: memory manager optimization

nathand — Wed, 07 Oct 2015 21:40:14 GMT

What about the case with no DVR? I doubt the DVR makes a difference in memory use; it might slightly negatively impact performance because there's an extra level of de-referencing needed to get to the data.