In the digital image stabilization system, Kalman filter is the most commonly used filter for motion correction. When the wanted movements have large assumptions deviation with the movement model, the result of motion correction will cause divergence and even error. For this problem, a novel motion correction method with adaptive Karlman filter is proposed. The back and forth characteristic of the unwanted motion and the smoothness characteristic of the wanted motion is used to adjust the system noise and the observation error adaptively. Experiment results show that the proposed method can effectively distinguish the wanted and the unwanted movement. Compared with the method with fixed parameters, the proposed method takes into account the smoothness and delay of wanted motion at the same time and it is more adaptively.