This paper presented a high real time target tracking algorithm – mixed difference tracking algorithm MDT. In the proposed algorithm, frame difference and background difference algorithms are combined to get the location of the target. With background difference algorithm the shape of the target can be extracted. Due to the affection of the dynamic background, single background difference algorithm can not get the location of the moving target. To solve this issue the frame difference algorithm is used to estimate the location, and then combine the results of the background difference algorithm and the frame difference algorithm the location and the size of the target can be extracted. And then filtering algorithm is used to remove noise and isolated points. In the experiment it can be seen that the proposed algorithm can tracking object precisely in real time.