In order to overcome these deficiencies that computation of recognition algorithm based on template matching is very high and the recognition rate of recognition algorithms based on skin-color segmentation is low, and is vulnerable to the impact of background which is similar with skin-color, face recognition algrithom based on skin color segmentation and template matching is presented in this paper. According to the clustering properties that the skin-color of human faces have emerged in the YCbCr color space, the regions closing to facial skin color are separated from the image by using Gaussian mixture model in order to achieve the purpose of rapidly detecting the external face of human face. Adaptive template matching is used to overcome the affect of the backgrounds which are similar with skin color on face recognition. Computation in the matching process is reduced by using the second matching algorithm. Extraction of face images by using singular value features is used to identify faces and to reduce the dimensions of the eigenvalue matrix in the course of facial feature extraction. Experimental results show that proposed method can rapidly recongnise human faces, and improve the accuracy of face recognition.