Memory usage/ memory leak

[pbrowntwi]

Are there any know problems with using 8.0.1 and releasing memory ?

We currently have an application that creates lots of graphs, however it eats memory and never seems to release it. So I've tried to create a small test app that shows the problem and found that doing ClearData (which I assumed would remove data from memory), and then doing a manual GC collect left residual memory behind. This was a basic form with a scatter plot and 2 buttons that added 210,000 items to the plot. (We typically have 15 plots with ~210000 points each on them in 7 lines).

This means that when I first add the data to the plot the memory jumps by ~14Mb, removing the data drops the memory back by ~2mb and then seems to stabilise aroung that number. Is there something else that needs to be done to free up memory ?

Also is the a list of know issues with 8.0.1 somewhere ?

This is the guts of the code:

        private void button1_Click(object sender, EventArgs e)
        {
            Random rnd = new Random();
            List<double> x = new List<double>();
            List<double> y = new List<double>();
            for (int i = 0; i < 210000; i++)
            {
                x.Add( i );
                y.Add( rnd.NextDouble() );
            }
            scatterGraph1.PlotXY(x.ToArray(), y.ToArray());
        }

        private void button2_Click(object sender, EventArgs e)
        {
            scatterGraph1.ClearData();
            GC.Collect();
            GC.WaitForPendingFinalizers();
        }


Thanks

Paul

[Mike J]

I am having the same problem. I have tried ClearData(), and tried clearing plots and axis. I also tried to force the GC, but when I used the Ants memory profiler, it became obvious that some internal NI classes are holding data. It looks like NI obfuscated some of the internal classes and methods so it is hard to get a clue as to what is going on. It is possible that some collection classes are being held by events. When an object gets tied to an event, it is not garbage collected unless it is manually removed. It is a commong mistake, and there are several articles on the net about solutions to this problem.

Can someone at NI look at their code and see what is going on? My app can't ship until this is fixed.

Mike

[breeve]

When you plot data on a plot for the first time, we (Measurement Studio) cache screen points and other things so drawing will be faster. The plot object holds on to this data. When you call ClearData, the plot only clears the raw data and not the screen points etc associated with the data (we will look into this).

If you plot data over and over in a loop the memory remains constant. Each time new data is plotted the old data is released and when it comes time to draw the new screen points the old screen points are released. I verified this with the code above by plotting the data in a loop.

Because calling ClearData does not clear the screen points (and other cached values), we need to have the plot object get garbage collected. Here is some code that works. Let me know if this works out for you.

public partial class Form1 : Form
    {
        private ScatterPlot _plot;

        public Form1()
        {
            InitializeComponent();
        }

        private void button1_Click(object sender, EventArgs e)
        {
            Random rnd = new Random();
            List x = new List();
            List y = new List();
            for (int i = 0; i < 210000; i++)
            {
                x.Add(i);
                y.Add(rnd.NextDouble());
            }

            _plot = new ScatterPlot(xAxis1, yAxis1);
            scatterGraph1.Plots.Add(_plot);
            _plot.PlotXY(x.ToArray(), y.ToArray());
        }

        private void button2_Click(object sender, EventArgs e)
        {
            if (_plot != null)
            {
                scatterGraph1.Plots.Remove(_plot);
                _plot.Dispose();
                _plot = null;
            }

            GC.Collect();
            GC.WaitForPendingFinalizers();
        }
    }

[Mike J]

Calling Dispose() seems to work. I was already clearing the plots, but I was not calling Dispose().

Thanks,

Mike

[Sof]

breeve,

I had a question for you.  What kind of performance do you get plotting 210000 points on a plot?  We had such bad performance we had to go back to the ActiveX control.

Scott

[pbrowntwi]

Performance isn't too bad, we only have problems when adding annotations to the charts and then it goes like treacle.

[breeve]

ScatterPlot

Here is what I get from running the code below using the Stopwatch class. I set both axes to fixed; [0-1] for the YAxis and [0-210000] for the XAxis at design time. I timed both the ImmediateUpdate case (Graph.ImmediateUpdates=true to time the drawing as well) and non-immediate update case, which is the default. I timed the code 4 times for each and got the average, throwing out the first time.

ImmediateUpdates=false (default)

.0637 seconds

ImmediateUpdates = true (times the drawing as well)

1.2185 seconds

SpeedPlot

Now, I was curious if I could make that time faster by not copying the data. When you call ScatterPlot.PlotXY, all your data is copied to an internal buffer. We do that to maintain the integrity of the plotted data as well as to support charting. I created a SpeedPlot object in the code below. This keeps a reference to the data and maps and draws the points itself, keeping the same timing process as above the following times where obtained:

ImmediateUpdates=false (default)

.0000755 seconds

ImmediateUpdates = true (times the drawing as well)

.9413 seconds

Now before we get all excited about the speedup in the ImmedateUpdates=false case, we need to remember what we are actually timing here. All we are timing is a field set of an address. This does not take long to do. A more accurate estimate of the speedup would be the second case where ImmediateUpdates=true because this times the drawing operation as well. Also remember that with the SpeedPlot, I am doing all the drawing of the plot myself. This means certain features like PointStyles, LineStyles, etc will not be effective. This plot is purely for speed where speed is really needed. I don't recommend using it for anything else, because certain features of the graph will not work.

Give it a try.

 public partial class Form1 : Form
    {
        private SpeedPlot _plot;
        public Form1()
        {
            InitializeComponent();
            _plot = new SpeedPlot(new Range(0, 210000), new Range(0, 1), xAxis1, yAxis1);
            scatterGraph1.Plots.Add(_plot);
        }

        private void button1_Click(object sender, EventArgs e)
        {
            Random rnd = new Random();
            List x = new List();
            List y = new List();
            for (int i = 0; i < 210000; i++)
            {
                x.Add(i);
                y.Add(rnd.NextDouble());
            }
            double[] xData = x.ToArray();
            double[] yData = y.ToArray();
           
            Stopwatch watch = Stopwatch.StartNew();
            //switch line below to _plot from scatterGraph1 to test NoCopyPlot.
            scatterGraph1.PlotXY(xData, yData);
            //_plot.PlotData(xData, yData);
            watch.Stop();
            textBox1.Text = watch.Elapsed.TotalSeconds.ToString();
        }

        private class SpeedPlot : ScatterPlot
        {
            private double[] _xData;
            private double[] _yData;

            public SpeedPlot(Range xRange, Range yRange, XAxis xAxis, YAxis yAxis)
                : base(xAxis, yAxis)
            {
                XAxis.Mode = AxisMode.Fixed;
                YAxis.Mode = AxisMode.Fixed;
                XAxis.Range = xRange;
                YAxis.Range = yRange;
            }

            public void PlotData(double[] xData, double[] yData)
            {
                _xData = xData;
                _yData = yData;
                //Need to redraw new points.
                Invalidate();
            }

            protected override void OnBeforeDraw(BeforeDrawXYPlotEventArgs e)
            {
                base.OnBeforeDraw(e);
                if (_xData != null && _yData != null)
                {
                    PointF[] points = MapDataPoints(e.Bounds, _xData, _yData);
                    using (Pen pen = new Pen(LineColor))
                        e.Graphics.DrawLines(pen, points);
                    e.Cancel = true;
                }
            }
        }
    }

[bilalD]

I wanted to add that there is a large performance difference between using GDI (native Win32 drawing API) vs. GDI+ (.NET drawing API).  GDI+ lacks hardware support and with Vista coming, it doesn't look like video drivers will be adding support for it anytime soon. On the other hand, video card drivers do provided hardware acceleration for GDI.

GDI+ is what we use for all of our .NET UI controls since these are managed controls and GDI+ is the Microsoft recommended managed drawing API. We use GDI for the ActiveX controls which is why there is a difference in their performance.

[pbrowntwi]

I don't know if I'm missing something very obvious, however I've been trying to get the charts to dispose correctly and ended up comming to the conclusion that the only way to do this is to unsubscribe all events manually.  i.e. if I attach a handler to say the AfterMove event of a cursor, before disposing of the the form I end up having to do a -= on that cursor.

I've ended up finding this out by using Son of strike and getting gcroot on the undisposed form and finding the following

DOMAIN(0014C0C0):HANDLE(Pinned):ba13ec:Root:02564b68(System.Object[])->
0166b720(System.ComponentModel.WeakHashtable)->
01764618(System.Collections.Hashtable+bucket[])->
0166b9f4(System.ComponentModel.TypeDescriptor+TypeDescriptionNode)->
0166b9e0(System.ComponentModel.ReflectTypeDescriptionProvider)->
0166ba34(System.Collections.Hashtable)->
0195a36c(System.Collections.Hashtable+bucket[])->
019755c0(System.ComponentModel.ReflectTypeDescriptionProvider+ReflectedTypeData)->
0197a710(System.ComponentModel.PropertyDescriptorCollection)->
0197a68c(System.Object[])->
01979e9c(System.ComponentModel.ReflectPropertyDescriptor)->
019802f0(System.Collections.Hashtable)->
01c33f24(System.Collections.Hashtable+bucket[])->
01b5ad28(NationalInstruments.UI.XYCursor)->
01b5ad68(NationalInstruments.Restricted.CallbackManager)->
01b5ad7c(NationalInstruments.Restricted.MultiKeyDictionary`3[[System.Object, mscorlib],[System.Delegate, mscorlib],[NationalInstruments.Restricted.CallbackManager+a, NationalInstruments.Common]])->
01b5ad88(System.Collections.Generic.Dictionary`2[[System.Object, mscorlib],[System.Collections.Generic.Dictionary`2[[System.Delegate, mscorlib],[NationalInstruments.Restricted.CallbackManager+a, NationalInstruments.Common]], mscorlib]])->
01b5b0c8(System.Collections.Generic.Dictionary`2+Entry[[System.Object, mscorlib],[System.Collections.Generic.Dictionary`2[[System.Delegate, mscorlib],[NationalInstruments.Restricted.CallbackManager+a, NationalInstruments.Common]], mscorlib]][])->
01b5b82c(System.Collections.Generic.Dictionary`2[[System.Delegate, mscorlib],[NationalInstruments.Restricted.CallbackManager+a, NationalInstruments.Common]])->
01b5b878(System.Collections.Generic.Dictionary`2+Entry[[System.Delegate, mscorlib],[NationalInstruments.Restricted.CallbackManager+a, NationalInstruments.Common]][])->
01b5b7e8(NationalInstruments.UI.BeforeMoveXYCursorEventHandler)->
01becab4(Teletest.Views.FrmGeneralDispersionCurve)
DOMAIN(0014C0C0):HANDLE(WeakSh):ba290c:Root:01becd6c(System.Windows.Forms.Control+ControlNativeWindow)->
01becab4(Teletest.Views.FrmGeneralDispersionCurve)
DOMAIN(0014C0C0):HANDLE(WeakLn):ba312c:Root:01c4ce2c(System.Windows.Forms.NativeMethods+WndProc)->
01becd6c(System.Windows.Forms.Control+ControlNativeWindow)

I can only assume that this one is left as I add it manually on the form, I remove all of the +='s that are assigned in the designer before truing to dispose and have the following method to remove everything else

        internal static void Remover(ScatterGraph graph)
        {
            while (graph.Plots.Count > 0)
            {
                ScatterPlot plt = graph.Plots[graph.Plots.Count - 1];
                graph.Plots.RemoveAt(graph.Plots.Count - 1);
                plt.Dispose();
                plt = null;
            }

            while (graph.XAxes.Count > 0)
            {
                XAxis xa = graph.XAxes[graph.XAxes.Count - 1];
                graph.XAxes.Remove(xa);
                xa.Dispose();
                xa = null;
            }

            while (graph.YAxes.Count > 0)
            {
                YAxis ya = graph.YAxes[graph.YAxes.Count - 1];
                graph.YAxes.Remove(ya);
                ya.Dispose();
                ya = null;
            }

            while (graph.Annotations.Count > 0)
            {
                Annotation ann = graph.Annotations[graph.Annotations.Count - 1];
                graph.Annotations.RemoveAt(graph.Annotations.Count - 1);
                ann.Dispose();
                ann = null;
            }

            while (graph.Cursors.Count > 0)
            {
                XYCursor csr = graph.Cursors[graph.Cursors.Count - 1];
                graph.Cursors.RemoveAt(graph.Cursors.Count - 1);
                csr.Dispose();
                csr = null;
            }
        }


Is this really required, or is there a way to have the graph dispose correctly ?

Also is there a timescale for a patch to address current issues ?

Thanks

Paul

[breeve]

Can you post a simple project? Some details are unclear.

• Are you trying to Dispose the entire form and then recreate another form?
• What version of MStudio