On the Development of a Voice and Gesture Based HMI for the Control of a Mobile Robot

Abstract:

This paper presents the development of a voice and gesture based interface (HMI – Human - Machine Interface) for the human control of a mobile robot. The application utilizes the Microsoft Kinect sensor and its Visual Studio C# libraries for voice and gestures recognition. Once, the specific voice commands or gestures are recognized, the controller (a PC or Laptop) sends specific commands to a microcontroller (ATmega2560 on an Arduino Mega board). The microcontroller is mounted on the mobile robot and controls the robot’s motors. Also there are presented some limitations of the Kinect sensor to be taken into consideration when using it.

• Periodical: Applied Mechanics and Materials (Volume 762)
• Pages: 201-204
• DOI: https://doi.org/10.4028/www.scientific.net/AMM.762.201
• Online since: May 2015
• Authors: Cristian Moldovan, Andrei Craciun, Valer Dolga, Erwin Christian Lovasz, Inocentiu Maniu, Carmen Sticlaru
• Keywords: Control, Gesture, Interface, KINECT, Robot, Sensor

References

[1] International Federation of Robotics, History of Industrial Robots, www. ifr. org/uploads/media/History_of_Industrial_Robots_online_brochure_by_IFR_2012. pdf.

[2] B. Dumas, D. Lalanne, S. Oviatt, Multimodal Interfaces: A Survey of Principles, Models and Frameworks, in Denis Lalanne, Jürg Kohlas (Eds), Human Machine Interaction, Springer-Verlag, Berlin/Heidelberg, 2009, pp.3-26.

DOI: 10.1007/978-3-642-00437-7_1

[3] Minimum requirements on PC with Kinect, http: /msdn. microsoft. com/en-us/library/ hh855359. aspx.

[4] J. Shotton et al., Real-Time Human Pose Recognition in Parts from Single Depth Images, Microsoft Research Cambridge & Xbox Incubation, http: /research. microsoft. com/pubs/145347/bodypartrecognition. pdf.

[5] Kinect for Windows – Human Interface Guidlines, http: /channel9. msdn. com/coding4fun/kinect/Kinect-for-Windows-Human-Interface-Guidelines-HIG.