Kernel-based density estimation technique, especially Mean-shift based tracking technique, is a successful application to target tracking, which has the characteristics such as with few parameters, robustness, and fast convergence. However, classic Mean-shift based tracking algorithm uses fixed kernel-bandwidth, which limits the performance when the target’s orientation and scale change. An Improved adaptive kernel-based object tracking is proposed, which extend 2-dimentional mean shift to 3-dimentional, meanwhile combine multiple scale theory into tracking algorithm. Such improvements can enable the algorithm not only track zooming objects, but also track rotating objects. The experimental results validate that the new algorithm can adapt to the changes of orientation and scale of the target effectively.