Something that I noticed. If you have a driver loaded like the sound board. My experience was that even though you are not using the sound board device if it's set to a high sample rate it will hog the processor. So I make sure if it's installed it's sample rate is turned way down.
JLW