Memory management / unmanaged instances????

[Risotto]

Hi,

I have some problems with memory management with MeasurementStudio.NET 8.1.2 on WinXP, .NET 2.0, VisualStudio 2005

My application:

We are running a C#.NET application with MeasurementStudio 8.1.2 to control a PCI-6229 on a test bench and acquire continiously data from the DAQ card. The measurement may run for several days. We show the current measured data in a windows form which use a WaveformGraph object. The data are saved on the drive and each 90 minutes, the measurement file is closed, the window forms is closed and then a new measurement file and a new window form are created.

We begin to be interested in memory management because after some times, our RAM were getting full which caused sometimes the application to crash. To help us, we used a memory profiler tool: .NET Memory Profiler (

http://memprofiler.com/Default.aspx).

We found and solve several memory leak which improve the memory management of the application but we finally reached a point where we cannot improve the memory increase because the memory leaks seems not to come from our code but from MeasurementStudio.

To show the kind of problem we have, I attached an example application. Please extract the attached zip-file. To run, you need a PCI-6229 card (simulated or not, it doesn't matters) and the device name must be Dev1.

The use of the example application is easy. It measures continiously 10 samples with a sample rate of 10Hz for 2 channels (Dev1/ai0 and Dev1/ai1). The data are put in a AnalogWaveform<double>[] array and then shown in the WaveformGraph. To start the measurement, click on start. To stop the measurement, click on stop. When you click on stop, the task will be disposed and the ClearDate-method of the graph is called. You may start and stop the measurement several time if you like.

Problem1: Instances of unmanged classes are garbage collected in the #1 or #2 generation.

Protocol:

Start application from .NET Memory Profiler
Select "RealTime" tabs.
Select "Types" (or may "Classes" depending on your system language)
Select the "Dispose Info" view and click on the "Undisposed instances" colum to sort them by descending size.

The column show how much instances of an unmanaged classes are currently in the heap. The number of theses instances should be 0. This is not our case. If you let run the measurement, you will see that a lot of unmanaged classes or created but not deleted by the garbage collector. Moreover the difference between the numbers of undisposed instances and the number of live instances for the same class show that the memory management is may be not the best.

For example:

Class                                                                      # Live instance               # Undisposed instances
NationalInstrument.Restriced.Native.TimeAbsolute      77                                  5603

==> There is 5526 instances of the TimeAbsolute class which are no more need but which are no deleted by the garbage collector. But if you wait long enough, some instance this class are garbage collected in the generation #1 and mostly are they garbage collected in the generation #2.

List of classes whith indisposed instances:

Class                                                                     # Live instance                 # Undisposed instances
System.ComponentModel.Design.ServiceContainer   5                                       7095 <== created by WaveformPlot constructor
NationalInstrument.Restriced.Native.TimeAbsolute    77                                      5603 <== created by WaveformPlot constructor
NationalInstrument.Restriced.Native.TimeValue        62                                      4183 <== created by WaveformPlot constructor
NationalInsrument.Restricted.CallbackManager.<GetCallbacks>d__e<Delegate> # Live instance: 0 # Undisposed instances: 3734 <== created by WaveformPlot constructor

This memory management causes in our application that the RAM is getting full before the garbage collector delete the unmanaged code instances in the generation #2.

==> Is it a problem of MeasurementStudio or is something wrong with my code?

Problem2: MeasurementStudio control resources are not realesed.

I have another problem with the controls of MeasurementStudio. For this I don't have an example - sorry. But the memory problem is not so important and is easy to explain.

My acquisition application is opening the window form in an extra thread - to allow user interaction and updating of the graph during measurement.

By closing the window form, some MeasurementStudio resource are not released. It doesn't matter if I call ClearData of the graph or the Dispose method for the WaveformGraph object or for other MeasurementStudio controls: the instance stay in memory and will only -may be - deleted when the acquisition application is closing.

The concerned classes are:

Class                                                                         Memory not freed
                                                                                 (Live instances size)
System.Windows.Forms.HScrollProperties                   40 bytes <== created during call of WaveformPlotCollection.AddRange(WaveformPlot
System.Windows.Forms.VScrollProperties                   40 bytes <== created during call of WaveformPlotCollection.AddRange(WaveformPlot[])
System.Windows.Forms.Application.ParkingWindow     220 bytes <== created during call of WaveformPlotCollection.AddRange(WaveformPlot[])
…. + some others classes….

==> As I told, the problem is not so important because it only some bytes or kBytes which stay in the memory each time I close and reopen my form (this appens may be 100 times for a measurement). I would be only interested to understand what appends here.

Thank in advance for all good advice!

Risotto

[JoBa]

This issue is being discussed with NI Support via e-mail.

[JonathanN]

Hi Risotta,

I read through your post and tried to reproduce the majority of memory issues you are seeing and so far haven't been able to. However, I first wanted to mention something I quickly saw in your example app which was the following:

In your DAQ example, you are spawning off a new thread and then trying to update the UI from that thread. This is breaking one of the fundamental rules of UI programming in the sense that you are only supposed to update a UI element from the thread that owns that UI element. If you are trying to update a UI element from another thread, you must marshal that call onto the UI thread.  Accessing Windows Forms controls from other threads results in undefined behavior. The Visual Studio 2005 debugger can detect this scenario and reports the following: Cross-thread operation not valid: Control '...' accessed from a thread other than the thread it was created on. If you want to use multiple threads with DAQ, you should use our built-in asynchronous reading/writing model discussed in the Asynchronously Reading and Writing with the NI-DAQmx .NET Class Library help topic. In this case, we spawn off a worker thread to do the data acquisition and then marshal the data back to the UI thread for you. So we handle all the underlying specifics.

Now as far as the memory leaks, go, in Measurement Studio 8.1.2 we did spend a great deal of time tracking down memory leaks in our API so I wanted to first make sure that you were using 8.1.2 by opening Visual Studio, selecting the Measurement Studio menu, and then clicking the About item in the menu drop-down.  This will have a Version section. See what version that says.

Now, in a couple cases (i.e. NationalInstruments.Restricted.CallbackManager), you mentioned not seeing live instances but rather indisposed instances.  In this case, I was able to reproduce similar behavior but this is not a memory leak as the memory was reclaimed by the garbage collector but the item was disposed "improperly". Now if the live instances field was not 0, then I would be concerned.  This is something we can improve on but again, this doesn't take up memory as the memory has been reclaimed.

Now as far as some of the other live instances that you found, I was not able to reproduce them. In some case, I found there was 1 live instance available (i.e. there should be 1 instance for NationalInstrument.Restriced.Native.TimeAbsolute type as this references the static constructor being called; There should always be only 1 but you are seeing 77 which is a problem; This instance will only go away when the app is closed but again, you should just be seeing 1)

Now in all the other cases (i.e. even the ones of opening and closing a form repeatedly) I wasn't able to reproduce anything remotely similar. I tried all sorts of combinations and didn't come up with anything useful. If you could post some more example projects that show these leaks, it would be helpful as we definitely want to get to the bottom of these problems.

Best Regards,

[Risotto]

Hi Jonathan,

I checked my version of MeasurementStudio: 8.1.2.472

About the rule of UI programming. Yeap, I am not an expert. So, what is "marshalling a call onto a UI thread"? What do you mean with this? May be can I improve my application here.

About the live instance of TimeAbsolute. I re-check my example and get exactly the same problem. I forgot to write it in my first contribution but... did you click on the "start" button?
I have re-run my example and do some screenshots to show you what happens here:
1. Start the application "DAQ_Cont_SWTriggered_Memory_Management.exe" from .NET Memory Profiler
2. Click on the "start" button, let run for about 5 minutes than click on the "quit" button
3. In .NET Memory Profiler, click on Real-Time / select standard view if needed / select "Total byte" and "Live bytes" to show them on the graph (pic1_standardview.gif)
    As you can see, the total memory is going down to the live memory for each generation #2.
4. Select the "Dispose Info" view / Select "Undisposed instances". (pic2_disposeview.gif)
   The number of undisposed instances is continiously increasing...
5. Select the register "Types" in the register "Real-time" and sort the column "Undisposed instanced" by decreasing number (pic2_disposeview.gif)
   These are the number of undisposed instances after 5 minutes.

About the opening/closing case: I will try to do another example, that I will post later.

Thanks,
Risotto

Message Edited by Risotto on 05-07-2008 07:15 AM

[JonathanN]

Hi Risotto,

Marshalling to the UI thread simply means that you are forcing the execution of your code (i.e. in this case, calling the Plot method) to run on the UI thread so it can properly update the UI component. There is a variety of .NET books and articles available out there describing this common problem. Check out the Basic Instincts: Updating the UI from a Secondary Thread tutorial on MSDN.

Now, unfortunately I wasn't able to run your exact application as the app crashed after running several 2nds and it has to do with the cross-thread rule you are breaking.  So, what I did was remove that secondary thread and just had one thread with your same code. Use this modified code, I wasn't able to see any leaks.  Just so we are both on the same basis, I am attaching the modified code so that you can run this on your end and tell me where you see leaks. It is important to remember that Generation 0 is collected more frequently than the other generations. Just because objects are promoted to another generation, and they are not immediately collected (which they won't be since the CLR doesn't collect from Gen#1 and #2 as often), doesn't indicate a leak. 

Best Regards,