Thank you for your interest in DSA hardware and analysis! My name is Bryan Fry -- I am writing from the DSA group in Engineering at NI. I am fairly certain that the issue you are experiencing relates to "spectral leakage" if the tone you are generating is not precisely on-bin for the FFT analysis. This possibility is something that is inherent in the THD method used in the CVI analysis library.
The THD functions in LabVIEW (version 6.0 and higher) use a more advanced peak detection algorithm that is insensitive to whether or not the tones are on-bin. I'd suggest that you send me an email at bryan.fry@ni.com and we can discuss the best workaround for this issue.