When you perform a measurement, the source impedance (that is the output impedance of your "generator") is generally much smaller than the input impedance of your measurement device. (For high frequency measurements you normally try to make both impedances equal and matching your cable impedance ... but that's an other story).
In a low frequency situation like yours, the generator may have an output impedance impedance of few ohm or less, while the acquisition device has 100 Gohm input impedance. But in general you can still perform decent measurements in situations where the output impedance is few kOhms and the input impedance, for example, 1 MOhm.
In your case, with open input, you may experience two problems.
1 - Any signal that is co
nnected to your input through an impedance that is smaller than 100 GOhm will fulfill the above measurement conditions. This can happen if your connections are "leaking" (resistance or capacitance).
2 - Most likely in your case, when your multiplexer switches from channel 1 to channel 2, the small capacitance at the input of your amplifier will keep its charge, and normally that charge would be discharged in the output impedance of your generator ... but since nothing is connected, the input capacitance will stay charged and therefore you are "measuring" the same signal again.
Anyway, the measurement is useless since you don't have anything to measure connected 🙂
Place a resistance between the two input terminals (differential or grounded) and you'll then provide the needed discharging resistance.
