DSP filter problem in modifying the Oscilloscope

[CityuTerry]

The hardware and software I am using are:
LabVIEW: 5.1.1
DAQ device: PCI-MIO-16XE-50
VI programs: 2 Channel Oscilloscope(sample VI)

I want to modify the function of the sample VI of 2 Channel Oscilloscope to simulate the AC-GND-DC function of Real Oscilloscope.
I try it by using "Butterworth Filter" under
Function-->Signal Processing-->Filters to cut the frequency at 0Hz for "AC" function (cut all DC).
However, the output is not good as real Oscilloscope becasue the signal seen is not good and stable.
I even use order 10 filter to do it. But the result is also the same.
Any suggestions to my problem?

[LocalDSP]

The ac-coupling of an oscilloscope is not a filter a "0" Hz. It is normally just a simple capacitance that, with the 1MOhm input impedance of the oscilloscope, acts like a first order high-pass filter with a cut-off frequency of few hertz.

Try to implement a highpass Butterworth filter of order 1 and with a low cut-off frequency of 10 Hz.

You can then play with the cut-off frequency and see which value works best for you (remember to connect the right value to the sample frequency input of your filter). You can also try to increase the filter order to 2 (some measurement devices have input high-pass filters of order 2 or even 3).