Kent:
I've got several random thoughts that might help:
1) The latest version of NI-VISA is 3.0.1 and we have fixed a number of things related to Serial. In particular, if I recall correctly, we fixed a case where there was a couple of error conditions that we had not been returning, and now do. So that might clue you in to the problem if you upgrade and at least get a VISA error code when this happens.
2) Call viSetBuf to set the I/O input (aka receive) buffer to a value higher than what you expect to receive. That means that as long as the Windows driver can keep up, it will hold the data until VISA gets it.
3) Try using hardware flow control (RTS/CTS) if possible. A lot of people forget that if you don't specify flow control, data loss is quite
likely. Using hardware flow control means that even if the Windows driver can't keep up, the UART can keep the device from sending more data.
I hope you find at least one of these suggestions helpful.
Dan Mondrik
National Instruments