In this paper, we proposed an efficient method to address the problem of color face image segmentation that is based on color information and saliency map. This method consists of three stages. At first, skin colored regions is detected using a Bayesian model of the human skin color. Then, we get a chroma chart that shows likelihoods of skin colors. This chroma chart is further segmented into skin region that satisfy the homogeneity property of the human skin. The third stage, visual attention model are employed to localize the face region according to the saliency map while the bottom-up approach utilizes both the intensity and color features maps from the test image. Experimental evaluation on test shows that the proposed method is capable of segmenting the face area quite effectively，at the same time, our methods shows good performance for subjects in both simple and complex backgrounds, as well as varying illumination conditions and skin color variances.