I looked at the spec sheet for your driver, and did see the information about how the input line defaults to 3.4 volts when floating. It looks like when the 6514 is connected to the input pin of your driver and driven to LOW, it is not referencing the pin to ground but instead allowing it to float. I have attached an image of the typical connection scheme for the 6514. Make sure that you have the grounds connected so that when the card is set to output a LOW it references the input pin on your driver to ground rather than having it float.
Concerning having the 24V supply being off, the 4V that you see sounds like it is because the voltage is dividing down due to a lower impedance than when power is supplied to the driver.
Regards,
Andrew