The ultimate goal of remote sensing image processing is to analyze and interpret the image. The classification is the most basic question of remote sensing image information extraction. Object-oriented classification is proposed in recent years, whose image classification is based on image segmentation. This paper introduces the spatial pyramid matching kernel method (SPM) for feature extraction, the segmentation algorithm uses mean shift, and the classifier is support vector machines（SVM）. Taking a piece of land in southern California for example, we do two experiments, including our approach and a comparing test .Comparing the results, we can see that the object-oriented classification of remote sensing image which based on SPM feature extraction can greatly improve the accuracy.