What you are seeing is probably due to your UUT and the method you are using. Beginning at a different frequency looks like it is changing the response of your system.
What swept sine analysis are you doing (toolkit, environment, board, etc.)? If you try outputting sine waves at stepped static frequencies, you may be able to allow the system to come to equilibrium before moving on to the next analysis. You could use the test panels of your output board to generate a static sine wave then measure the response and wait for an equilibrium.