You should always call CA_DiscardObjHandle on all open handles to properly cleanup memory. There is no general rule about how ActiveX EXE servers behave as far as exiting when all open handles are closed. The general convention is that the server should exit if there is no interactive user using it - the server implementor might determine this based on visibility or actual user interaction (mouse-clicks, keyboard input, etc).
If you want to leave the application running after 'automating' it from your CVI program, then I recommend that you make it visible if you are not already doing so. There should be a method/property in the server to do this. Even if you make some servers visible, they may exit when all handles are closed - they may or may not expose some other method/property (UserControl, Activate, etc) that affects this. In the worst case, you can leave a handle open if that is what it takes to keep the server running - this will result in a small memory leak, but should not be a big deal.
If you run the client program repeatedly, then you may want to ensure that your client program is not launching too many instances of the server. You can try to do this, by first calling the 'Active_Application' function instead of the 'New_Application' function. The Active function will try to connect to an already running instance of the server. If there is no instance running, then it returns an error, and you can then call the New_Application function. But this may not be what you want to do - because when you do this, you will affect the usability of any interactive users currently using the server program.
On the other hand, if you want to ensure that the server exits after you disconnect from the client program, you should call the exit or quit method of the server. In general, this is a better solution. You launch a new instance of the server program, keep it invisible, automate it from the client program to do what you want, and then call its quit method to make sure it exits. But in your case, this may not be desired.