Gavin:
What kind of error does the setup VI give? If you can open the port, then it sounds like you might be trying to set other fields inconsistently.
You say you cannot open the port from MAX. Do you mean you right-clicked and chose Open VISA Session, and it gave you an error? If so, what was it?
Only 1 software component can own/use a COM port at any given time. There is no way to grab it away from another component, nor is there a way to reserve it for a given component (such as LabVIEW or VISA).
You may also wish to run NI Spy to capture the offending calls. The resulting .spy file is something you can attach to an email to NI tech support.
There is one small W2K/WXP thing that comes to my mind. If you set hardware flow control and then
try to manually set the state of the corresponding output line (such as RTS or DTR) then the OS will generate an error. None of the earlier Windows OS's did this. But if you do not use flow control, then this is not the problem you are seeing.
Dan Mondrik
Senior Software Engineer, NI-VISA
National Instruments