With respect to the human tracking with multi-cameras in the video surveillance system, a human tracking method based on MSER (Maximally Stable Extremal Regions) was established. The approach transforms the human tracking into elliptic region matching. The method does elliptic region fitting to each MSER, and then selects the elliptic regions which meet some constraints. These selected elliptic regions are normalized to unity circular regions. The right matched elliptic regions are gotten by rotational invariant vectors calculation, histogram density estimation and weighted average distance calculation. Experimental results show that the approach can effectively realize the human tracking with multi-cameras.