In this paper, we propose an adaptive lifting-based Motion-Compensated Temporal Filtering (MCTF) framework which can remove both long and short range temporal redundancies by choosing wavelet kernels with different support sets according to temporal correlations in the video sequence. The framework can also separate noise and sampling artifacts from the reconstructed frames by embedding a spatial predicted update step into each lifting step. To support the adaptive MCTF, we also use an improved hierarchical variable size block matching algorithm for motion trajectory estimation. Experimental results confirm that the proposed framework improves both the rate-distortion performance and the visual quality of the reconstructed video significantly.