Memory Leak suspected in TCP/MQTT (re-)connection

[CarcS]

Hi LabVIEW community,

Issue: I am having a memory "leak" in my application which I have high conviction is in my MQTT connection. Running my application over the weekend adds up to a memory consumption around 4 - 8 GB. Leaving it running longer I have reached 14 GB it just keeps going.

Application: the application reads measurement data from test-rigs and streams it to an MQTT broker. Reading test-rig time-series is done over TCP or API and stored in a waveform. This waveform is not extending, only the last value for all channels. The streaming to the (Solace) broker happens over secured TCP connection via MQTT application.

General details:
Running on Windows 10. Currently using LV24/Q3 64-bit but the issue was already there in LV22/Qx 32-bit so don't think this is related.
Additional libraries are the JKI State Machine, that takes care of all user interaction and the MQTT library from 'LabVIEW Open Source Project'  library to stream the test-rig data to the broker.

What did I try?
I have tried a ton of things to locate the memory consumption. Eventually disassembled my code into sub segments and ran them all overnight to see which part was responsible.
--> Result: memory consumption is increased by my streaming.vi subVI which only (re-) connects and sends the data to the broker.
To further allocate where in the streaming the issue is I changed from streaming over TCP-SSL to Solace broker over MQTT to streaming over TCP to a Mosquitto broker.
--> Result: running over the weekend memory consumption was still the same as at startup, 350 MB (instead of 4 - 8 GB).

The TCP-SSL MQTT connection to Solace (that has a leak) keeps disconnecting the client about 20% of the iterations. Though the regular TCP to Mosquitto connection remains stable and keeps the client connected.
Attachments: I created 2 snippets of the differences that have and didn't have the memory leak while keeping all of the other code identical.These snippets are in an over coupling case selector which only executes this connection case when no client is connected. I researched general TCP & MQTT connections but couldn’t quite figure what is wrong in my configuration.

1. Why does it disconnect in the first place, am I missing something in my secured TCP/MQTT connection to Solace (left) that causes it to disconnect the client?
2. Memory issue could be related to the re-connecting together with the way its setup. Am I doing something wrong in reconnecting the client while MQTT is still connected that causes memory consumption to increase when it re-connects?

So I think there is an issue in the way I implemented the TCP & MQTT connection for both cases but it only shows up in the TCP-SSL Solace connection due to the client constantly disconnecting. It might be that multiple 'ghost' connections are open but this is strange since connection to is only setup if no client is connected.

Sorry if I am missing any crucial details but would someone with more knowledge about MQTT and TCP-SSL be able to help me out here?
I know snippets are not ideal but I guess sufficient

Thanks in advance!
Grtz, CarcS

[santo_13]

I did not get to look at your VIs, in general, have you tried using Desktop Execution Trace Toolkit to track the memory leak?

https://www.ni.com/docs/en-US/bundle/teststand/page/tracing-with-the-dett.html

[CarcS · Accepted solution]

Hi santo_13,

To give some words on the issue. It did of course not solve it but it confirmed that the leak was in the MQTT connection in my streaming.vi, thanks.

Started digging a little deeper and noticed I was using the same client name for a dozen test-rigs. This of course caused clients being kicked out and continuously attempt to re-connect all of them again with the same client name. The client name is now dynamic including test-rig identifier and the connection with the broker is stable.
Since the application is not re-connecting the clients constantly anymore the memory consumption stopped increasing. Even over the last week, zero increase. Though it could still be that the design for re-connecting is not memory-leak-tight this is accepted for now.

Edit: looking at my initial post to clarify why streaming to Mosquitto did work fine and to Solace didn't. I always only streamed to Mosquitto for 1 test-rig simultaneously. This way the client name is no issue obviously. There is thus no difference in the 2 provided snippets, they both had the same issue. Difference was in the way I used them.

Cheers!

[Yamaeda]

Nice find!

Yes, reconnections often seem to leave a reference behind (even when doing it correctly sometimes), so try to keep them open and reuse them. (As you found out)