An improved multi-scale segmentation algorithm is proposed in this paper. In order to get segmentation result more efficiently and accurately, watershed transformation is used as an initial segmentation algorithm, and then the objects of regions are merged based on the improved merge rule. The improved regulation for region merging is mainly based on the scale parameter of area-based while the heterogeneity parameter is considered as well. In this way, the failure of considering that some regions with large heterogeneity with their neighborhood are not suitable for merging will be prevented. Experimental results show that the quality and efficiency of remote sensing image segmentation can be greatly improved by the improved multi-scale segmentation algorithm.