Automatic Eyeglasses Removal of Frontal Facial Images for Face Recognition

Abstract:

As the most common type of facial occlusion, eyeglasses may cause great accuracy degradation in face recognition. In this paper, we proposed an improved approach on automatically detecting and removing eyeglasses from grayscale facial images. First, we normalized a face image by the result of face detection and eyes localization. Then we used a fast judging method to detect eyeglasses’ presence. For images with eyeglasses, we used PCA reconstruction error and edge feature to determine the occluded area, and synthesized the area through image inpainting. Experimental results show that our approach can detect the presence of eyeglasses very accurately and obtain generally natural looking images without eyeglasses. In face recognition test, our approach greatly contributed to the accuracy of recognition, achieving higher improvement than other approaches such as simple PCA reconstruction, iterative error compensation, and weighted fusion.

• Periodical: Applied Mechanics and Materials (Volumes 571-572)
• Pages: 697-704
• DOI: https://doi.org/10.4028/www.scientific.net/AMM.571-572.697
• Online since: June 2014
• Authors: Han Ying*, Guang Da Su, Jian Sheng Chen
• Keywords: Eyeglasses Removal, Face Recognition, Image Inpainting, Occlusion Extraction

References

[1] Y. Saito, Y. Kenmochi, and K. Kotani. Estimation of eyeglassless facial images using principal component analysis. Proceedingsof the International Conference on Image Processing, 4: 172-201, (1999).

DOI: 10.1109/icip.1999.819578

[2] C. Du, and G. Su. Eyeglasses removal from facial images. Pattern Recognition Letters, 26: 2215-2220, (2005).

DOI: 10.1016/j.patrec.2005.04.002

[3] J. -S. Park, Y. H. Oh, S. C. Ahn, et al. Glasses removal from facial image using recursive error compensation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(5): 805-811, (2005).

DOI: 10.1109/tpami.2005.103

[4] Z. Wang, and J. Tao. Face occlusion detection and reconstruction. Journal of Computer Research and Development, 47(1): 16-22, 2010. (In Chinese).

[5] G. Hu, and Z. Lou. Eyeglasses removal method based on occlusion region compensation. Application Research of Computers, 30(12): 3863-3865. (In Chinese).

[6] Y. Freund, and R. E. Schapire. Experiments with a new boosting algorithm. International Conference on Machine Learning, 96: 148-156, (1996).

[7] S. Liao, X. Zhu, Zhen Lei, et al. Learning multi-scale block local binary patterns for face recognition. International Conference on Biometrics, 828-837, (2007).

DOI: 10.1007/978-3-540-74549-5_87

[8] A. Telea. An image inpainting technique based on the fast marching method. Journal of Graphics Tools, 9(1): 23-34, (2004).

DOI: 10.1080/10867651.2004.10487596

[9] A. Criminisi, P. Perez, and K. Toyama. Object removal by exemplar-based inpainting. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2: II-721, (2003).

DOI: 10.1109/cvpr.2003.1211538

[10] N. Otsu. A threshold selection method from gray-level histograms. Automatica, 11(285-296): 23-27, (1975).

[11] Y. Xiang, and G. Su. Multi-parts and multi-feature fusion in face verification. IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 1-6, (2008).

DOI: 10.1109/cvprw.2008.4563107

[12] P. Viola, and M. Jones. Rapid object detection using a boosted cascade of simple features. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 1: I-511-I-518. (2001).

DOI: 10.1109/cvpr.2001.990517

[13] T. F. Cootes, and C. J. Taylor. Statistical models of appearance for computer vision. Technical report, Imaging Science and Biomedical Engineering, University of Manchester, (2004).

[14] S. Milborrow, and F. Nicolls. Active shape models with SIFT descriptors and MARS. International Conference on Computer Vision Theory and Applications, 1(2): 5, (2014).

DOI: 10.5220/0004680003800387