In order to extract ground information more accurately, it is important to find an image segmentation method to make the segmented features match the ground objects. We proposed an image segmentation method based on mean shift and region merging. With this method, we first segmented the image by using mean shift method and small-scale parameters. According to the region merging homogeneity rule, image features were merged and large-scale image layers were generated. What’s more, Multi-level image object layers were created through scaling method. The test of segmenting remote sensing images showed that the method was effective and feasible, which laid a foundation for object-oriented information extraction.