topic Re: Memory usage in a while loop with an xy graph in LabVIEW

Memory usage in a while loop with an xy graph

stephencox — Fri, 04 Mar 2016 02:39:32 GMT

Hi all,

I am having memory usage issues with a vi that loops, records data, and plots it in an x-y graph as it runs. I wrote it using shift registers and "array subset" functions, which I assumed would reduce the overhead, but apparently this is incorrect. If it runs for several days (normal, since this is mostly a vi for tracking and viewing a backgroudn process that runs continuously), it takes several minutes just to stop the loop. I assume it would eventually just stop working entirely. I have searched for this issue, and I see suggestions to auto-index or pre-allocate. However, I also want to plot every point as the loop runs, so I need shift registers. Do I need to use some convoluted pre-allocation, array shift, and then insert into array approach, or is there a better way? Will that even work?

Thanks in advance

Re: Memory usage in a while loop with an xy graph

James.M — Fri, 04 Mar 2016 04:10:38 GMT

If your array gets infinitely large, it will use infinite amount of data, especially if you're displaying that data. Why do you need to display all of your data for days? Could you compress the data, like take an average down to a smaller sample rate just for display purposes? Same extra data to a file that can be loaded if the user wants to see farther back in to the past.

Re: Memory usage in a while loop with an xy graph

stephencox — Fri, 04 Mar 2016 04:38:28 GMT

Thanks for the response. I don't need to display all the data--I am subsampling the array continually to take only the last 1000 points (which is about 10 minutes). But the behavior of the vi, gobbling up memory and taking 15+ minutes to stop running after several days running, suggests to me that it's somehow allocating memory for everything anyway.

I found a mention of the array subset function online failing to release memory, so I thought maybe that is the problem. I'm not sure how to test this directly, as I definitely only have two 2x1000 arrays showing up in my block diagram at any one time.

Re: Memory usage in a while loop with an xy graph

stephencox — Fri, 04 Mar 2016 04:46:23 GMT

Just to be clear, I subsample the array and then pass that result to an xy graph and to the next iteration of the loop through a shift register. So as far as I can see, there is no infinitely large array. I'm just assuming that is the reason for the performance issues.

Re: Memory usage in a while loop with an xy graph

James.M — Tue, 06 May 2025 20:37:10 GMT

It sounds like you're going to need to share your code. If you array is size limited like you say, then there shouldn't be a memory issue there.

Please include a Snippet of your code, or attach your VIs.

Re: Memory usage in a while loop with an xy graph

JÞB — Fri, 04 Mar 2016 05:20:32 GMT

We'll need to see the rest of the code to fully understand what is going on. Untill then a general observation:

you are monitoring a contueous process that runs for days. Decouple the data display from the data acquisition- no user is staring at the display for hours! Suggestion: collect the data and write it to a TDMS file and display offline with a TDMS file viewer (Like the built in LabVIEW one, Scout, Excell or Diadem) Without doing anything else you change the scope of what the DAQ and File loops do and simplify the problem greatly.

Re: Memory usage in a while loop with an xy graph

crossrulz — Fri, 04 Mar 2016 10:47:39 GMT

Typically an application like this should be using a Chart, not a Graph. A Chart does the circular buffer for you. But it does not give you control over the x-axis like the x-y graph. So I would like some more clearification about what exactly this data is. Perhaps we can change it some so that it works with a chart and then life becomes very simple.

Re: Memory usage in a while loop with an xy graph

stephencox — Fri, 04 Mar 2016 18:23:03 GMT

I shoud have stopped messing with this or put the previous version into source control after I posted my previous message, but I didn't. I tried switching to a chart--maybe it will fix the problem if the graph was the cause of it. I was plotting time on the x-axis, but since the readings are usually roughly equally spaced, I can live without that. The script uses some old sub-vis to talk to hardware and some shared variables to talk to VIs on this and other computers, so I replaced those things with random number generators and local variables so that this can be run by other people.

Re: Memory usage in a while loop with an xy graph

James.M — Fri, 04 Mar 2016 18:36:07 GMT

You are correct that your array is not going to infinity. You're removing an element and inserting a single element, so the array is always size 1000. (By the way, this can be done with a single Replace Array Subset function). Your chart will accrue memory, but will max out at the Chart History Length setting on the chart.

But you are writing to file every single loop two data points... to infinity and beyond. You are opening and closing that file every single loop, but the real problem is that the file will get bigger and bigger and take longer and longer to open and close. It will eventually limit your application loop rate even.

You should adapt your code to write chunks of data at a time, not every single loop. You should also set it up so it creates new files once the current data file gets to a certain size.

Re: Memory usage in a while loop with an xy graph

stephencox — Fri, 04 Mar 2016 18:36:26 GMT

I confess I have been manually archiving the log file when it grows large. I should make it poll the file size and automatically archive when it hits a certain limit. I didn't think opening and closing the file would build overhead as the loop runs, though--is that possible? Could it be that the contents of the file are being written over and over into memory and not cleared out for some reason?

Re: Memory usage in a while loop with an xy graph

James.M — Fri, 04 Mar 2016 18:45:34 GMT

Opening and closing the file every loop won't build up memory, but as the file gets larger, that specific code will take longer. It's really just not a best practice. Up to you.

If you double-click the Write Delimited Spreadsheet VI, you can see that it is made up of LabVIEW primitives. If you use those primitives to create your own version of the File I/O, you will end up with a file refnum. You can use the Advanced File I/O palette to get the file size and then just close the file and re-open a new one. There's an awesome new function in LV2015 called Create File WIth Incremented Suffix.

Re: Memory usage in a while loop with an xy graph

stephencox — Fri, 04 Mar 2016 18:48:48 GMT

Thanks, that's a good suggestion. I will implement that and hope in the meantime that switching from xy graph to xy chart fixes my multi-day performance problem. If not, I can try to decouple logging and plotting, although I would rather understand the problem rather than just changing things until it disappears mysteriously.

Re: Memory usage in a while loop with an xy graph

James.M — Fri, 04 Mar 2016 18:54:27 GMT

If the XY Graph is the issue, then it's the memory associated with the array that you're writing to the graph. If you downsample that array (because you probably don't need 2Hz data across multiple days on a graph) by say taking every 10th point, your array will be a 10th the size and the graph won't change much.

If you really need all that data for the user to see when they zoom in, you can do some fancy stuff with old datafiles to load old data when they zoom in. In the end it's a compromise between how much time you want to spend to make the code smart enough for the user to have an awesome experience.

If you would, please look back through the thread and pick the answer the best matches your question and Mark as Solution. This helps other find what they need in the future.

Re: Memory usage in a while loop with an xy graph

stephencox — Fri, 04 Mar 2016 20:53:48 GMT

Well, my array is never more than 1000 points. So if it is the xy graph, it is caching data in a way that I don't understand.

Re: Memory usage in a while loop with an xy graph

James.M — Fri, 04 Mar 2016 21:06:12 GMT

Nope, you should be fine… You can go back to using a graph if you want if you're maxing out at 1000 points. The problem was most likely just the File I/O.

Re: Memory usage in a while loop with an xy graph

JÞB — Fri, 04 Mar 2016 21:51:40 GMT

@James.M wrote:
Opening and closing the file every loop won't build up memory, but as the file gets larger, that specific code will take longer. It's really just not a best practice. Up to you.

If you double-click the Write Delimited Spreadsheet VI, you can see that it is made up of LabVIEW primitives. If you use those primitives to create your own version of the File I/O, you will end up with a file refnum. You can use the Advanced File I/O palette to get the file size and then just close the file and re-open a new one. There's an awesome new function in LV2015 called Create File WIth Incremented Suffix.

An consider changing the file type to TDMS. They are much easier to work with for these types of systems since they allow you to display the existing data without stopping writes to the file at the same time. Hence you only have to open a ref to the file once rather than for every "Write operation" There are good shipping examples and white papers on the advantages.

Re: Memory usage in a while loop with an xy graph

altenbach — Fri, 04 Mar 2016 22:18:26 GMT

You are using a chart, not an xy graph, so simply set the chart history to 1000 and get rid of all the array operations. (If you would be using an xy graph, you should be using 1D complex array instead of cluster arrays. Make things simpler. You would also initialze a fixed size array and replace the oldest point with each iteration and use a plot style without interpolation so the order of the data in the array does not matter)

You have several race conditions due to the overuse of local variables. E.g. the "Cryo setpoint" local will get read before the local inside the event gets written. Why don't you use the actual terminal? It sits all outside, completely bored. You are also creating confusion between controls and indicators.

You have may to many different event conditions for the send button. Whenever you press and release it, you are triggering four events

(button is switch until released, so you get a value changed event for the on and one for the off transition. Then you also get events on mouse up and mouse down. That's way too much. Make the button "latch when released" and only keep the value change event condition. Now the event will only fire exactly once. However, since you are spinning the loop anyway, get rid of the event structure and simply poll the boolean.

Here's how the core code could look like. Keep it simple!

Re: Memory usage in a while loop with an xy graph

stephencox — Wed, 16 Mar 2016 23:29:56 GMT

Ok, I broke the big vi up into several smaller pieces, which is more optimal anyway. They run either at different frequencies or on demand to serve different needs. It quickly became clear that the cause of the problems was not what I thought, but was rather the calls to a low-level vi (by reference) on an older machine running Labview 8.6 and doing nothing but talk to some old hardware.

I noticed that sometimes after the vi ran for many days and developed the problem of moving slowly and taking forever to stop, it would also cause Labview 8.6 on the other machine to crash when I did stop it. I dug into the vi reference calls and found one that was missing a "close reference." I won't be sure for a few more days, but I bet that this is what was causing the problems.