In the paper, a new spatio-temporal segmentation algorithm is proposed to extract moving objects from video sequences, the sequences were taken by stationary cameras. First, the motion detection is used to achieve the mask representing moving regions with a estimation noise parameter. Which can effectively improve noise immunity. Due to the shortage of the moving video object textures, the eight-neighbor motion detection is present, which is used to smooth the mask boundary and fill the interior holes. Then a morphological filter is applied to refine the moving mask. Second, spatial segmentation is detected by the Canny operator. Then utilize the gradient histogram to select the high threshold to increase the adaptivity of Canny algorithm. Finally, merge the temporal and spatial mask by neighborhood matching algorithm to ensure further reliability and efficiency of our algorithm. Experiments on typical sequences have successfully demonstrated the validity of the proposed algorithm.