LabVIEW Memory Management VI question, a special request to you

[cordm]

4, 5, 6, 7 are only 70MB buffers, the argument to DSNewPtr is number of bytes.

If Martin's assesment is correct that comes out to 12*140MB + 4*70MB = 1960 MB which is also what the profiler says.

There is a tool called Show Buffer Allocations directly below the Performance and Memory menu entry that gives you hard to see black boxes for each allocation site. I'm only counting 11 allocations, the DSNewPtr allocations and #12 are missing.

The third argument to MoveBlock is also in bytes. That is why the elements starting at 35M are 050A instead of 0. That brings up the question why the buffers allocated with DSNewPtr are zero-initialized. Is that the operating zeroing it out before handing it over? I would assume that on later uses there would be something in the buffer and you should use DSNewPClr.

[*3d0g]

What's a knot, as you used the term?

[GerdW]

Hi 3d0g,

"knot" is "Knoten" in German, which also is the translation of "node"…

So Martin was talking about nodes in your block diagram…

---

@*3d0g wrote:

What's a knot, as you used the term?

---

When you ask someone named "you" then you should also provide a citation so we know who is referenced by "you"…

[*3d0g]

As I understand the DVR, it's a pointer to a variable's contents in memory.  In this case the variable is a giant array.  I wanted to show that same giant array to each of the MemoryMoves.

As I understand it, the MemoryMove uses the (source) pointer presented to it to read contiguous memory contents starting at the source pointer, and then it stores a copy of the read contents starting at the destination pointer (here the DVR'd buffer.)  In this case it's overwriting the memory pointed to by the DVR with the contents seen in the block of memory of the source pointer, the pointer had from the memory allocation by the preceding DSNewPtr function.

I just keep rewriting the buffer of the DVR with whatever memory is seen by using the latest source pointer.  Had I not used a DVR I'd have to come up with three more giant arrays, a new one for each memory move.  (It isn't really moving anything.  It's copying what is read at one block into another block.)  In the end, thanks to dataflow, I finally dispose of the DVR, that pointer to the giant array I keep overwriting as I allocate more and more memory, stealing that memory from LabVIEW's use, until I release it all, in a series, block by block.  ...but does LabVIEW get that memory back?  Or is it gone until I close and reopen LabVIEW?

But like I said, I'm just trying to understand what's happening.   

[GerdW]

Hi 3d0g,

---

@*3d0g wrote:

As I understand it, the MemoryMove uses the (source) pointer presented to it to read contiguous memory contents starting at the source pointer, and then it stores a copy of the read contents starting at the destination pointer (here the DVR'd buffer.)  In this case it's overwriting the memory pointed to by the DVR with the contents seen in the block of memory of the source pointer, the pointer had from the memory allocation by the preceding DSNewPtr function.

But you don't use that DVR pointer/reference nowhere in your VI (apart from creating a DVR and killing it)!

The MemoryMove uses the pointer to the memory block behind the wire that hold the giant array.

Keep in mind: the wire is the variable in LabVIEW!

You can delete all the DVR stuff in your VI and it will work the same…

---

@*3d0g wrote:

I just keep rewriting the buffer of the DVR with whatever memory is seen by using the latest source pointer. 

---

You don't use the DVR, you use the array wire!

You keep rewriting the buffer behind the wire (which sounds dangerous to me)…

[rolfk]

There is a serious discrepancy already. Your DVR uses an I16 array with 70E6 elements, your array pointers use 70E6 bytes. A factor of two. Luckily you pass 70E6 as length to the MoveBlock functions, so you are not over-reading the pointer beyond its end, but you are only writing half of the array buffer of the DVR.

But you make a bad assumption. The DVR does absolutely nothing in your case. The DVR is only a pointer as far as LabVIEW is concerned inside the In Place Structure. Once you wire the contents of the DVR outside of the In Place Structure, LabVIEW data flow rules mandate that a real copy is made of the data, and the "pointer" inside your DVR is irrelevant at that point.

Each split of the resulting array that you pull out of the DVR, at some point creates a new array buffer of 70E6 I16 elements, passes it to the MoveBlock function to write data into the first half of the buffer and then displays it in a control. In total you have about 5 times 70E6 I16 arrays in the control buffers for your buffer and pDest0-3 controls, one more during runtime inside the DVR and at least one temporary copy for the array to the MoveBlock function (it could be more but the LabVIEW compiler tends to optimize execution of the nodes to avoid unnecessary copies. Since all the Call Library Nodes are set to run in the UI thread (dark yellow background) they can't possibly run in parallel so there is no need to create a separate array copy for each of them. However that optimization algorithm may not be able to recognize this specific case since you did not mark the pSource parameter in the MoveBlock Call Library Node as Constant. If you check the Constant checkbox for this parameter this tells the LabVIEW compiler that this parameter is never written to and that it is safe to schedule the other Call Library Nodes in parallel, so the compiler is much more likely to not create additional temporary copies of the array.

In addition you have during runtime 4 times 70E6 bytes buffers in the memory pointers.

And yes! Never ever, and I really mean never ever! mark a parameter in the Call Library Node as constant, if the DLL behind it is trying to write to even a single byte in that parameter!!!!!! The result of doing that could be anything from strange results, to crashes, usually much later and it can depend on the moon phase, your cats mood or anything else if nothing happens, the code performs illogical things, or actually crashes. In reality it of course depends on the LabVIEW compilers decision to schedule things in a certain order, but that order can be influenced by seemingly unrelated changes anywhere in your code base.

[Martin_Henz]

Thanks to everyone who corrected and supplemented my statements.

[rolfk]

---

@*3d0g wrote:

What's a knot, as you used the term?

---

I interpret knot as the little dot you get on every branching of a wire. It could be also a node as explained by others, but in terms of potential buffer allocations by LabVIEW, the dot on a wire branch is the more correct view.

[*3d0g]

I've been playing with this, and I was wrong, above.  The buffer contents are not overwritten.  Rather, the outputs of the memory moves/copies are newly created arrays, each with a new pointer.  The buffer remains intact.  Therefore, the DVR is a separate chunk of memory.   

[*3d0g]

Thank you Rolfk!

I have to keep reading what you said, trying to wrap my head around it.  It seems like you just stated what I discovered via experiment.  Still, I'm not sure, and I'll keep rereading what you've said.

Thank you for taking the time to type that for me.  I can't say I understand it yet, but I'll keep trying.