Specularity Removal for Single Image Based on Inpainting and Blending with Parameter Estimation by Neural Networks over Multiple Feature Spaces

Abstract:

Specularity removal is useful for image related applications that need consistent object surface appearance. For a single image it can be more challenging problem due to presence of different shapes, sizes and colors of specular regions, which may have some parts with totally missing data. The problem can become more difficult if the specular regions having partial information grow bigger, because the exact boundaries are difficult to mark. Any region filling method can provide unusual results because the appropriate boundaries selection is important for these methods. In this work, we address this problem and propose a scheme which can handle specular regions by segmenting both types of sub-regions of specularity. Our segmentation algorithm is multistage which uses Luminance as well as principal components for the identification of specular regions. For specularity removal, we proposed a three step scheme which includes balancing illumination, inpainting and blending. Finally feed-forward neural network is proposed to estimate the tunning parameters, which not only automate the whole process but also simplifies the difficult task of choosing parameters like size of specular regions or preprocessing selection. The results demonstrates the effectiveness of the proposed method for a variety of images having natural specular reflection.

• Periodical: Applied Mechanics and Materials (Volume 555)
• Pages: 773-780
• DOI: https://doi.org/10.4028/www.scientific.net/AMM.555.773
• Online since: June 2014
• Authors: Saqib Yousaf*, Shi Yin Qin
• Keywords: Feed-Forward, GLARE, Highlights, Image Restoration, Inpainting, Specularity Removal

References

[1] S. P. Mallick et al., Specularity removal in images and videos: A PDE approach, Computer Vision–ECCV 2006, pp.550-563: Springer, (2006).

DOI: 10.1007/11744023_43

[2] Y. Weiss, Deriving intrinsic images from image sequences, Computer Vision, 2001. ICCV 2001. Proceedings. Eighth IEEE International Conference on, (2001).

DOI: 10.1109/iccv.2001.937606

[3] M. Dellepiane et al., Improved color acquisition and mapping on 3d models via flash-based photography, Journal on Computing and Cultural Heritage (JOCCH), (2010).

DOI: 10.1145/1709091.1709092

[4] S. Lin et al., Diffuse-specular separation and depth recovery from image sequences, Computer Vision—ECCV 2002, pp.210-224: Springer, (2002).

DOI: 10.1007/3-540-47977-5_14

[5] A. Artusi et al., A survey of specularity removal methods, Computer Graphics Forum, (2011).

[6] T. Chen et al., Mesostructure from specularity, Computer Vision and Pattern Recognition, 2006 IEEE Computer Society Conference on, (2006).

DOI: 10.1109/cvpr.2006.182

[7] B. Lamond et al., Fast image-based separation of diffuse and specular reflections, ACM SIGGRAPH 2007 sketches, (2007).

DOI: 10.1145/1278780.1278869

[8] W. -C. Ma et al., Rapid acquisition of specular and diffuse normal maps from polarized spherical gradient illumination, Proceedings of the 18th Eurographics conference on Rendering Techniques, (2007).

[9] A. Agrawal et al., Removing photography artifacts using gradient projection and flash-exposure sampling, ACM Transactions on Graphics (TOG), (2005).

DOI: 10.1145/1073204.1073269

[10] G. J. Klinker et al., The measurement of highlights in color images, International Journal of Computer Vision, (1988).

[11] K. Schluns et al., Global and local highlight analysis in color images, Proc. 1st Int. Conf. Color Graphics Image Processing, (2000).

[12] R. Bajcsy et al., Detection of diffuse and specular interface reflections and inter-reflections by color image segmentation, International Journal of Computer Vision, (1996).

DOI: 10.1007/bf00128233

[13] H. -L. Shen et al., Simple and efficient method for specularity removal in an image, Applied Optics, (2009).

[14] M. F. Tappen et al., Recovering intrinsic images from a single image, Pattern Analysis and Machine Intelligence, IEEE Transactions on, (2005).

DOI: 10.1109/tpami.2005.185

[15] P. Tan et al., Separation of highlight reflections on textured surfaces, Computer Vision and Pattern Recognition, 2006 IEEE Computer Society Conference on, (2006).

DOI: 10.1109/cvpr.2006.273

[16] E. Angelopoulou, Specular highlight detection based on the Fresnel reflection coefficient, Computer Vision, 2007. ICCV 2007. IEEE 11th International Conference on, (2007).

DOI: 10.1109/iccv.2007.4409097

[17] S. Lin et al., Highlight removal by illumination-constrained inpainting, Computer Vision, 2003. Proceedings. Ninth IEEE International Conference on, (2003).

DOI: 10.1109/iccv.2003.1238333

[18] P. Koirala et al., Highlight removal from single image, Advanced Concepts for Intelligent Vision Systems, (2009).

[19] W. Li et al., Automated image analysis of uterine cervical images, Medical Imaging, (2007).

[20] A. C. Bovik, Handbook of image and video processing: Academic Press, (2010).

[21] R. T. Tan et al., Separating reflection components of textured surfaces using a single image, Pattern Analysis and Machine Intelligence, IEEE Transactions on, (2005).

DOI: 10.1109/tpami.2005.36

[22] H. Lange, Automatic glare removal in reflectance imagery of the uterine cervix, Proc. SPIE, (2005).

DOI: 10.1117/12.596012

[23] J. B. Roerdink et al., The watershed transform: Definitions, algorithms and parallelization strategies, Fundamenta Informaticae, (2000).

DOI: 10.3233/fi-2000-411207

[24] G. D. Finlayson et al., Removing shadows from images, ECCV (4), (2002).

[25] Z. Xu et al., Image inpainting algorithm based on partial differential equation, Computing, Communication, Control, and Management, 2008. CCCM'08. ISECS International Colloquium on, (2008).

DOI: 10.1109/cccm.2008.89

[26] A. Criminisi et al., Region filling and object removal by exemplar-based image inpainting, IEEE transactions on image processing : a publication of the IEEE Signal Processing Society, (2004).

DOI: 10.1109/tip.2004.833105

[27] H. Grossauer, A combined PDE and texture synthesis approach to inpainting, Computer Vision-ECCV 2004, pp.214-224: Springer, (2004).

DOI: 10.1007/978-3-540-24671-8_17

[28] S. Grover et al., A unified approach for digital image inpainting using bounded search space, International Journal on Graphics, Vision and Image Processing, (2005).

[29] P. Pérez et al., Poisson image editing, ACM Transactions on Graphics (TOG), (2003).