This is expected behavior. The CVI runtime manages its own memory. For performance reasons, it has its own logic for when it requests memory from and returns memory to the system. So you cannot use the task manager as a true indication of how much memory your program is using.
You can see a more detailed explanation from a previous thread here.
It depends on the set of controls that you are discarding versus the ones that you are creating. In theory, I wouldn't expect it to allocate more memory from the system if the sets are identical. And I would certainly not expect it to keep allocating memory if you enter into a cycle of creating and discarding controls. That would be a memory leak that would need to be investigated.
Is your program set up to do a quick test where you iterate through several cycles of creating and discarding controls, while monitoring the system memory usage?