Memory leak??? or more likely not knowing proper cleanup

[Yamaeda]

Do you need the 15 digit precision of doubles, or would 7 digit precision of Singles be enough? That'd half the memory in one go. 5GB says you have data duplication, and if you plot all data that's a culprit (thus the previous suggestions of decimation).

Watch out for Local variables as that can also mean data duplication. Altenbachs mention of shift registers is a good one, keep the data in such.

/Y

[Alexander_Sobolev]

>> Then I display them on one graph and change the line colors to black and the symbol to a hollow circle. 

Visible property of the plot would be simpler and better.

[mcduff]

Suggestions:

1. In your i=0 True case, instead of re-initializing an array, use the the resize array function and pass the wire through the case structure. You may be getting a copy there.
2. No need to use local variables for your display, extra data copies there. Just attach the indicator directly to the wire.
3. You are constantly resizing your array for Kinetics & Spectrum Graph, set it to size needed before the for loop and use replace array subset instead.
4. As said earlier, decimate data for display, and avoid local variables when possible.

mcduff

[gnzork]

Wow this generated a lot more feedback than I though it would.  Thank you all for the input and suggestions!  I’d like to address all the questions and comments:

The computer is 64 bits, LabVIEW is 64 bits

“Use a producer-consumer architecture”  - I just recently heard about this and want to implement it.

“Try storing the data as a binary file” – I will give this a try and see if it helps.

“Decimate your data” – I will give this a try

“Don’t use local variable” “Use shift registers” – I was told local variables are better than property nodes. My event case has 37 events for different function with more to come.  How should I reference the graph in multiple events?  Should I use shift registers for everything?

“Do not display the data” – Actually that’s exactly what I want to do.  I realize that I cannot see all of the data points plotted on top of each other.  The goal is to remove unwanted artifacts and bad data, so in fact the purpose of plotting all the data together is to see the outliers or oddballs, not see every data point. 

“Display the statistics of the data instead of all the data” – This is an interesting idea and I will have to give it some thought to see if it makes sense for me to do.

“Defer front panel update” – How do I do that?

“Why are you using a timed loop” – When the program is waiting for user input I do not need it free running and using lots of CPU and battery.  Once a button is clicked the program executes inside the timed loop and the timed loop does not limit it.  I think the timed loop is appropriate.

“Use single float instead of double” – This is a possibility and I will have to think about it.  My gut feeling is it’s better to keep using double just because the program will be more universal and I think others have said that 300 data sets should be about 1.44 GB, which in today’s world is not too much.

“Visible property of the plot would be simpler and better” – I’m unfamiliar with this, what is it?

“In your i=0 True case, instead of re-initializing an array, use the resize array function” – I didn’t realize it could create a copy.  I will replace and see if it helps.

“You are constantly resizing your array for Kinetics & Spectrum Graph, set it to size needed before the for loop and use replace array subset instead” – I will give this a try

Thank you everyone again!  This is really helpful!

[mcduff]

---

---

“Use a producer-consumer architecture”  - I just recently heard about this and want to implement it.

Note, this can also make copies. In your producer loop, you have a data copy which is then consumed in a consumer loop, making another copy. You can use FGVs or pass around a DVR reference to eliminate copies.

“Decimate your data” – I will give this a try

See this link.

There used to be example program, but I cannot find it now.

“Don’t use local variable” “Use shift registers” – I was told local variables are better than property nodes. My event case has 37 events for different function with more to come.  How should I reference the graph in multiple events?  Should I use shift registers for everything?

Make a giant clustersarous and bundle/unbundle your data in the loop. Down the JKI State Machine from VIPM and you can see how it works.

“Defer front panel update” – How do I do that?

Look in the application Control Palette.

“Visible property of the plot would be simpler and better” – I’m unfamiliar with this, what is it?

Look in the application Control Palette.

mcduff

[AYanez]

“Don’t use local variable” “Use shift registers” – I was told local variables are better than property nodes. My event case has 37 events for different function with more to come.  How should I reference the graph in multiple events?  Should I use shift registers for everything?

You should use this approach to store control values, of this way you can update the shift register when a value change happens in a specific control and reuse the value in other cases. You might review the kind of events and what are you processing in each one because you could reduce the number of events using Dynamic Registration of events to group logic into your application: https://forums.ni.com/t5/LabVIEW/Dynamic-registering-of-events-using-an-array-of-refnums/td-p/326568...

“Defer front panel update” – How do I do that? I left a link in my message: https://forums.ni.com/t5/LabVIEW/Example-for-Defer-Panel-Update/td-p/3688130,  read it, basically it is achieved using property nodes.

“Visible property of the plot would be simpler and better” – I’m unfamiliar with this, what is it? It refers to the visible property for each plot, you need to use Current Active Plot and after set the Visible attribute with the same property node.

I hope with the suggestions you can improve your software.

[gnzork]

Hi All

I figured it's polite to provide an update, so here it is.  I started by focusing on the part that holds all the data I load in, and changed it to a shift register instead of using local variable.  Doing that basically solved the memory issue (which is what I cared about the most).  Incidentally, that was also the first suggestion that was made before even looking at the program. 

I'm still trying some of the suggestions for speeding up the response, but now that this is working better, it's become a side project and I might not get to it for a few more weeks.  

[chembo]

I didn't look at the code, but noticed that it was suggested to use TDMS files.

During my recent troubleshooting of a memory leak and found something regarding TDMS in the LabVIEW known issues list: CAR 239631: Memory Growth with TDMS Write

I thought that TDMS is used a lot and I am wondering why this was not fixed. The CAR was open in April 2012.

[rolfk]

Because there is nothing to be fixed! The "fix" would be worse than the effect you see now as the TDMS write function would have to seek through the whole file everytime a new dataset needs to be written to disk and as the file grows, the seek time would increase dramatically. Your TDMS write would come to a big crawling halt eventually.

If you really want to avoid this issue the only solution is to use raw binary files by using a fixed 2D array of data and simply streaming that to disk. You won't be able to have different samplin reates for different channels in the data set and can't change the sampling time of some channels in the middle of the streaming, or add or remove a channel temporily midstream either but your writes will be consistently fast. TDMS allows for all these features but that requires bookkeeping of indices for every channel and as the file grows that bookkeeping data grows too.