To obtain better region extraction results of medical image, a new segmentation algorithm is proposed based on improved Adaboost algorithm. The seed pixel is selected with background subtraction. The neighborhood point is judged. The primary selected seed is calibrated with label, and then the range of seed is reduced through growing label and the maximal saliency. The optimized Adaboost algorithm is taken as growing criterion to optimally combine the scrappy region when the region growing is over. The experiment result shows that the accuracy and robustness of the algorithm both meet the actual application required.