Robot Learning from Demonstration Using 3D Computer Vision

Abstract:

For robot application to proliferate in industry, and in unregulated environments, a simple means of programming is required. This paper describes methods for robot Learning from Demonstration (LfD). These methods used an RGB-D sensor for demonstration observation, and used finite state machines (FSMs) for policy derivation. Particularly, a method for object recognition was developed, which required only a single frame of data for training, and was able to perform real-time recognition. A planning method for object grasping was also developed. Experiments with a pick-and-place robot show that the developed methods resulted in object recognition accuracy greater than 99% in cluttered scenes, and manipulation accuracies of below 3mm in linear motion and 2° in rotation.

• Periodical: Advanced Materials Research (Volumes 875-877)
• Pages: 1994-1999
• DOI: https://doi.org/10.4028/www.scientific.net/AMR.875-877.1994
• Online since: February 2014
• Authors: James Aaron Debono, Gu Fang
• Keywords: 3D Computer Vision, Automated Manufacturing, Learning from Demonstration

References

[1] B. Argall, S. Chernova, M. Veloso, and B. Browning: A survey of robot learning from demonstration, Robotics and Autonomous Systems, vol. 57, no. 5, (2009). 469-83.

DOI: 10.1016/j.robot.2008.10.024

[2] N.A. Melchior, and R. Simmons: Dimensionality reduction for trajectory learning from demonstration, in 2010 IEEE International Conference on Robotics and Automation. 2953-8.

DOI: 10.1109/robot.2010.5509913

[3] A. Billard, S. Calinon, R. Dillmann, and S. Schaal: Robot Programming by Demonstration, in B Siciliano & O Khatib (eds), Springer Handbook of Robotics, Springer (2008), 1371-94.

DOI: 10.1007/978-3-540-30301-5_60

[4] B. Price, and C. Boutilier: Accelerating reinforcement learning through implicit imitation., J. Artif. Intell. Res. (JAIR) 19 (2003): 569-629.

DOI: 10.1613/jair.898

[5] S. Schaal: Learning From Demonstration, Advances in Neural Information Processing Systems, vol. 9, (1997) pp.1040-6.

[6] J. Vogel, C. Castellini, and P. van der Smagt: EMG-based teleoperation and manipulation with the DLR LWR-III, in 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), (2011). pp.672-8.

DOI: 10.1109/iros.2011.6094739

[7] Dillmann, R 2004, Teaching and learning of robot tasks via observation of human performance, Robotics and Autonomous Systems, vol. 47, no. 2–3, 109-16.

DOI: 10.1016/j.robot.2004.03.005

[8] D.A. Forsyth, and J. Ponce: Computer vision : a modern approach, 2nd edn, Pearson, (2012).

[9] D. Kragic, and H.I., Christensen: Advances in robot vision, Robotics and Autonomous Systems, (2005), vol. 52, no. 1, 1-3.

[10] R. Szeliski: Computer Vision: Algorithms and Applications, Springer, (2011).

[11] S. Arivazhagan, R. Shebiah, S. Nidhyanandhan, and L. Ganesan: Fruit Recognition using Color and Texture Features, Journal of Emerging Trends in Computing and Information Sciences, (2010) vol. 1, no. 2,. 80 - 9.

DOI: 10.1109/icccnt.2010.5591562

[12] L. Bo, K. Lai, R. Xiaofeng, and D. Fox: Object recognition with hierarchical kernel descriptors, in 2011 IEEE Conference on Computer Vision and Pattern Recognition. 1729-36.

DOI: 10.1109/cvpr.2011.5995719

[13] J. Shotton, A. Fitzgibbon, M. Cook, T. Sharp, M. Finocchio, R. Moore, A. Kipman, and A. Blake: Real-time human pose recognition in parts from single depth images, in 2011 IEEE Conference on Computer Vision and Pattern Recognition. 1297-304.

DOI: 10.1109/cvpr.2011.5995316

[14] D.H. Grollman and O.C. Jenkins: Incremental learning of subtasks from unsegmented demonstration, in 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems, 261-6.

DOI: 10.1109/iros.2010.5650500

[15] C. Bishop: Embracing uncertainty: applied machine learning comes of age., Machine Learning and Knowledge Discovery in Databases (2011): 4-4.

DOI: 10.1007/978-3-642-23780-5_3

[16] Y. Zhu and K. Fujimura: Constrained Optimization for Human Pose Estimation from Depth Sequences, in Y Yagi, S Kang, I Kweon & H Zha (eds), Computer Vision – ACCV 2007, Springer (2007) vol. 4843, 408-18.

DOI: 10.1007/978-3-540-76386-4_38

[17] R.B. Rusu and S. Cousins: 3D is here: Point Cloud Library (PCL), in 2011 IEEE International Conference on Robotics and Automation, 1-4.

DOI: 10.1109/icra.2011.5980567

[18] M.A. Fischler, and R.C. Bolles: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, (1981) 24(6), 381-395.

DOI: 10.1145/358669.358692

[19] N.M. Kwok, Q.P. Ha, and G. Fang: Effect of color space on color image segmentation. " In 2nd International Congress on Image and Signal Processing, 2009. CISP, 09., 1-5. IEEE, (2009).

DOI: 10.1109/cisp.2009.5304250