Memory considerations while using the Queue and Functional Global techniques

[DFGray]

My apologies.  Attached is the file in 8.5 (using inplace element) and 7.1 versions.  Unzip with folders so you don't confuse the two.

[Ben]

---

DFGray wrote:

... If it matters, it is usually worth the time to single step through your code and validate the buffer viewer.

---

Thank you Dr Gray for "entering the battle". 

What should I be looking for when doing the "validation"?

To all,

Sometimes can "loose sight of the forest because of the trees", with the "trees" being buffer dots and the forest being performance. The bootm line is performance so benchmarking various code constructs is the final determining factor.

Stepping back (and reinforcing a comment by nathand early in this thread)...

1) Scalars and their buffers are very difficult to even measure how little time they require, so don't worry about scalars.

2) Arrays are an issue for more than one reason;

  A) Copying of a large block of memory from one location to another takes CPU cycles for each move. The larger the array, the more important it is to avoid the copy.

  B) The memory allocated for a buffer must be contiguous and is only availble to LV by making request to the OS's memory manager (handled by LV automatically, don't you love it!). This process is extremely slow because it requires working in the OS kernal to allocate the physical memory and allocate the virtual memory (Page and Swap files) used by the app.

So benchmark often the code that is demanding the most CPU/Time and when you think you have found "The ultimate construct", post a copy of it here to see if anyone (Christian?) can beat your code. If they can't you win by default. If they can, you win by getting faster code and another tid-bit to file away.

Take care,

Ben

[yenknip]

On the topic of using queues, Is it more efficient to use the name, or wire in the queue reference? I remember being told the wire is always more efficient, so I typedef the queue ref and pass it around that way, with self initiating shift registers.

I've been using something like the below example for several months for my live databases, after finding this post / thread where Damien Gray suggested the single element queue

 

Message Edited by yenknip on 01-15-2009 03:15 PM

[DFGray]

Ben,

"Validation" means you have your operating system memory monitor active and you watch for changes in the LabVIEW process memory use as you single-step through your code.  In theory, every time you see a buffer allocation, you should see a change in the memory allocation of LabVIEW; in practice, this does not always happen due to a variety of issues (some of which are mentioned above).  However, changes due to copies of large buffers are easy to see.

In my previously posted example, if you set the buffer size to 10MBytes, you should see a 10MByte jump in the size of the LabVIEW process when you first run it and it hits the breakpoint (the buffer should be allocated by the FOR loop).  At that point, stepping through the example will show no real change until you hit the queue release or the VI finishes (depends on LabVIEW version), at which point the buffer is deallocated and the LabVIEW process will decrease in size.

Message Edited by DFGray on 01-16-2009 08:06 AM

[Ben]

---

DFGray wrote:

Ben,

"Validation" means you have your operating system memory monitor active and you watch for changes in the LabVIEW process memory use as you single-step through your code.  ...

---

That answer my Q!

Ben

[amaglio]

Is the Request Deallocation used often? Is there certain rules of thumb to go by when using this fucntion? I am tempted to use it in one of my Functional Globals (FG) but am not sure if there is any reason not too, or if mayber there is a better way. In my FG I have loads of data in a 1-Dimensional array and want to convert this 1D array to a 2D array. After the conversion I no longer need the data in the 1D array. If I convert the data using the reshape array fucntion and then send an empty array back into the shift register of the 1D array no additional memory is used as long as I have the Request Deallocation function on the block diagram with the input set to true. If I don't have the Request Deallocation function, even though I am sending an empty array back into the shift register of the 1D array, the memory doubles. Is it okay to use The Request Deallocation function for this type of application? Is there a better way?

Message Edited by amaglio on 01-22-2009 05:00 PM

[nathand]

I've never needed to use "Request Deallocation" but you might have run into one of the few instances where it's useful.  You might want to put it inside of your case structure, though, so that it only deallocates when that one specific function is called.  If you're calling that VI more than once to convert the 1-D data to 2-D, it should re-use the memory from the previous 1-D array on the next call, so you could be slowing your code by forcing deallocation and the reallocating that 1D array.

Why is your function selector not an enumeration type definition?

Also, I think a while loop with a constant wired to the stop terminal is the more conventional way to implement a functional global, although your for-loop approach produces identical results.

[amaglio]

I will be calling this VI more than once to convert 1-D data to 2-D but there will be a different set of 1-D data (more than likey this data will not be the same size as the 1-D data that was there on a previous call). If I don't use the Request Deallocation function, and on the second call of this FG  will reallocation have to occur anyway if the new 1-D array of data has a different number of elements as the 1-D of data that was there on the previous call?

The function selector that I am using is a ring selector. I guess I have just always been in the habit of using rings for these types of applications and have never used the enumerations type... Is there advantages to using this type of a control?

Thanks for your help nathand

[RavensFan]

---

amaglio wrote:

The function selector that I am using is a ring selector. I guess I have just always been in the habit of using rings for these types of applications and have never used the enumerations type... Is there advantages to using this type of a control?

---

Yes.  With an enum type, the case selector will show the meaningful name for the case as opposed to just a number.  At a glance, you don't know what case 2 is supposed to be, but you would know if it said Initialize, or Clear, or whatever.

[Ben]

Quick comments:

I agree with nathand that if you are only calling this VI once then the deallocate is useful.

If you are re-using this VI, then the deallocate will force more work (so slower) since it has to allocate what was there previously. If the second call is happy with the already allocated memory there will be no need to allocate more.

Again I agree with the enum for the Action Engine. If you would like to read more about Actin Engines then please take a look at my Action Engine Nugget found here.

Ben