2D Sensor Based Design of a Dynamic Hand Gesture Interpretation System

Abstract:

A complete 2D sensor based system for dynamic gesture interpretation is presented in this paper. A hand model is devised for this purpose, composed of the palm area and the fingertips. Multiple cues are integrated in a feature space. Segmentation is carried out in this space to output the hand model. The robust technique of mean shift mode estimation is used to estimate the parameters of the hand model, making it adaptive and robust. The model is validated in various experiments concerning difficult situations like occlusion, varying illumination, and camouflage. Real time requirements are also met. The gesture interpretation approach refers to dynamic hand gestures. A collection of fingertip locations is collected from the hand model. Tensor voting approach is used to smooth and reconstruct the trajectory. The final output is represented by an encoding sequence of local trajectory directions. These are obtained by mean shift mode detection on the trajectory representation on Radon space. This module was tested and proved highly accurate.

• Periodical: Advanced Engineering Forum (Volumes 8-9)
• Pages: 553-562
• DOI: https://doi.org/10.4028/www.scientific.net/AEF.8-9.553
• Online since: June 2013
• Authors: David Ciprian, Vasile Gui
• Keywords: Dynamic Hand Gestures, Human-Computer Interface, Trajectory Encoding

References

[1] W.T. Freeman, C.D. Weissman, Television control by hand gestures. Intl. Workshop on Automatic Face and Gesture Recognition, Zurich, Switzerland, June, (1995).

[2] N. Stefanov, A. Galata, R. Hubbold, Real-time hand tracker using variable-length markov models of behaviour. Computer Vision and Image Understanding, 108(1-2), 98–115, (2007).

DOI: 10.1016/j.cviu.2006.10.017

[3] C. Wang, K. Wang, Hand posture recognition using adaboost with sift for human robot interaction. Intl. Conf. on Advanced Robotics, Jeju Island, South Korea, 21-24 August, (2007).

[4] A. Kurakin, Z. Zhang, Z. Liu, Real Time System for Dynamic Hand Gesture Recognition with a Depth Sensor, 20th European Signal Processing Conf., Bucharest, Romania, August 27-31, (2012).

[5] M. Van den Berg, L. Van Gool, Combining RGB and ToF Cameras for Real-time 3D Hand Gesture Interaction, IEEE Workshop on Applications of Computer Vision, 5-7 January (2011).

DOI: 10.1109/wacv.2011.5711485

[6] Z. Li, R. Jarvis, Real Time Hand Gesture Recognition Using a Range Camera. Australasian Conf. on Robotics and Automation, Sydney, Australia, December 2-4, (2009).

[7] M. Kolsch, M. Turk, Fast 2D hand tracking with flocks of features and multi-cue integration. IEEE Workshop on Real-Time Vision for Human-Computer Interaction, p.158–165, (2004).

DOI: 10.1109/cvpr.2004.345

[8] I. Oikonomidis, N. Kyriazis, A. Argyros, Markerless and efficient 26-dof hand pose recovery. Asian Conf. on Computer Vision, Queenstown, New Zealand, 8-12 November (2010).

DOI: 10.1007/978-3-642-19318-7_58

[9] B. Stenger, A. Thayananthan, P.H.S. Torr, R. Cipolla, Model-based hand tracking using a hierarchical bayesian filter. IEEE Trans. Pattern Analysis and Machine Intell., 28(9), 1372–1384, (2006).

DOI: 10.1109/tpami.2006.189

[10] M. J. Black, and A. D. Jepson, Recognizing Temporal Trajectories using the Condensation Algorithm. Third IEEE Intl. Conf. on Automatic Face and Gesture Recognition, Nara, Japan, April (1998).

DOI: 10.1109/afgr.1998.670919

[11] G. Shaogang, M. Walter, A. Psarrou, Recognition of Temporal Structures: Learning Prior and Propagating Observation Augmented Densities via Hidden Markov States. Proc. Seventh Intl. Conf. on Computer Vision, (1999).

DOI: 10.1109/iccv.1999.791212

[12] S. Rajko, G. Qian, T. Ingalls, J. James, Real-time gesture recognition with minimal training requirements and on-line learning. Intl. Conf. on Computer Vision and Pattern Recognition, (2007).

DOI: 10.1109/cvpr.2007.383330

[13] S. Wang, A. Quattoni, L. P. Morency, D. Demirdjian, T. Darrell. Hidden conditional random fields for gesture recognition. Intl. Conf. on Computer Vision and Pattern Recognition, (2006).

DOI: 10.1109/cvpr.2006.132

[14] H. -I. Suk, B. -K. Sin, S. -W. Lee, Hand Gesture Recognition Based on Dynamic Bayesian Network Framework. Pattern Recognition, 2010, 43(9), 3059-3072.

DOI: 10.1016/j.patcog.2010.03.016

[15] X. Shen, G. Hua, L. Williams, Y. Wu, Motion Divergence Fields for Dynamic Hand Recognition. Proc. of Automatic Face and Gesture Recognition, 2011, 492-499.

DOI: 10.1109/fg.2011.5771447

[16] X. Shen, G. Hua, L. Williams, Y. Wu, Dynamic Hand Gesture Recognition: An Exemplar- Based Approach from Motion Divergence Fields. Journal of Image and Vision Computing, 2012, 30(3), 227-235.

DOI: 10.1016/j.imavis.2011.11.003

[17] Comaniciu, D.; Meer, P. Mean Shift: A Robust Approach Toward Feature Space Analysis, IEEE Trans. Pattern Recognition and Machine Intell., 24(5), 603-619, (2002).

DOI: 10.1109/34.1000236

[18] M. Nicolescu, and G. Medioni, Perceptual grouping from motion cues - a 4-D voting approach. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25(4), p.492–501, (2003).

DOI: 10.1109/tpami.2003.1190574

[19] K. Kim, T.H. Chalidabhongse, D. Harwood, L. Davis, Real-time Foreground–Background Segmentation Using Codebook Model. Real-time Imaging, 11(3), 167-256, (2005).

DOI: 10.1016/j.rti.2004.12.004

[20] A. Cheddad, J. Condell, K. Curran, P. A. Mc Kevitt, Skin Tone Detection Algorithm for an Adaptive Approach to Steganography. Signal Processing, 2009, 89, 2465–2478.

DOI: 10.1016/j.sigpro.2009.04.022