If the LV VI is executing in a separate process, it will load the dll again in that process, which would explain what you are seeing.
To verify this, select Specify Module on your LabVIEW step and checkmark the Advanced Settings...>>Alway Run VI in LabVIEW Run-Time Engine option and see if that fixes the problem.
The runtime engine will run the VI in the TestStand process, so it should use the same dll instance. It is also a little faster to call the VI, but you can't debug the LV VI when this option is set.
If it still doesn't work, your CVI code might also be running in a separate process. To see if that is the problem, either call your CVI dll with the DLL adapter instead of the CVI adapter, or set the Configure>>Adapters>>LabWindows/CVI>>Configure>>Execute Steps In-Process option.