Hello. I am writing a VI for scheduled data acquisition system. The way this system works is that it reads data from a DAQmx device continuously for as many capture buffers are needed as it takes to record the set length of data, typically up to 1 minute, but could be several minutes. Data acquisition can occur over multiple channels, and usually at a fairly high sample rate. In order to keep memory usage low, data is streamed to disk with TDMS, and then when acquisition is done, it is read out of the TDMS file and saved as a waveform. Up until now, there have been no problems with the program.
A few days ago, it was decided to look into allowing continuous data acquisition. This changed the requirements of the program slightly - before, data would be transcoded from TDMS to waveform during the idle period of data acquisition. With continuous data acquisition, the data will be streamed to TDMS, and after the first recording is complete, the data will be transcoded from TDMS to wave while data acquisition occurs simultaneously.
The problem with this new approach is that CPU usage spikes very high during the transcode. When this happens, the VI appears to freeze, and when it resumes after a few seconds, the DAQmx read function returns an error because the device buffer overflows.
The solution seemed simple enough - I could throttle the transcode proceess manually by specifying the number of samples to transcode per iteration of a lood, and then insert a wait after each loop iteration. This works, to some extent, however the TDMS read function seems to be inefficient. At first, the transcode operates quickly, but as the loop works further into the TDMS file, the data rate slows down. This slows down the file save, which results in larger memory usage from excessive queue usage.
My question is, how can I ensure that the DAQmx read won't be interrupted by data saving tasks? There doesn't seem to be any ability to control thread priority other than the scheduling in the VI properties dialog, and it doesn't appear to do anything.
