Unfortunately, the delay you are seeing of the instrument to get out of the 'repeat until scripttrigger0' loop is a an expected behavior of how this instruction works on the 564xR devices.
This is not the exact diagram, but this can give you an idea of what is happening on the FPGA:
http://zone.ni.com/reference/en-XX/help/373680F-01/rfvst/waveform_sequencer_overview/
There is a single FIFO in the FPGA that has the next N samples/waveforms that are to be generated, in order to make sure the generation does not underflow and due to the latency for getting data out of DRAM.
I thought the delay was not a function of the waveform size, but you may be right. If that's the case, you could try to break up your waveform into further smaller waveforms so the instrument can flush out data out faster and move on to the instruction outside the repeat until trigger loop.
