The USB-6008 is rated to handle 8.5 mA on the digital outputs. Does your relay draw more than that? I do not know how the digital output would behave under a moderate overload, but it might explain what you are observing. If your relay does require more current, you will need to add an amplifier or a low coil current relay as a pilot relay.
Lynn