Touch-Less Java Based Hand Tracking System to Control an Artificial Arm

Abstract:

Modern sensing technologies create new possibilities to control mobile robots without any dedicated manipulators. In this article authors present a novel method that enables driving of the Mindstorms NXT artificial arm with Microsoft Kinect, using gesture recognition and hand tracking. To imitate the movement of an artificial robotic arm, an algorithm of the human-computer interaction is employed using skeleton tracking and gesture control in 3D space. The solution is implemented using free, open source software, Java based.

• Periodical: Solid State Phenomena (Volume 214)
• Pages: 1-10
• DOI: https://doi.org/10.4028/www.scientific.net/SSP.214.1
• Online since: February 2014
• Authors: Grzegorz Baron, Piotr Czekalski, Daniel Malicki, Krzysztof Tokarz
• Keywords: Artificial Arm, Gesture Control, Java, KINECT, LeJOS, Mobile Robot, NXT, Remote Control

References

[1] K. Tokarz, S. Kiełtyka, Geometric approach to inverse kinematics for arm manipulator, Latest trends on systems, 14th WSEAS International Conference on Systems, Corfu, vol. II (2010) 682-687.

[2] K. Tokarz, C. Manger, Geometric and rough set approach to inverse kinematics for arm manipulator, International Journal of Mathematical Models and Methods in Applied Sciences, 5 (1), NAUN (2011) 1-8.

[3] G. Borenstein, Making Things See – 3D vision with Kinect, Processing, Arduino, and MakerBot, O'Reilly Media, 2012.

[4] Kinect Open Source programming secrets: Java programming tutorial for Kinect, http://fivedots.coe.psu.ac.th/~ad/kinect/

[5] OpenNI: OpenNI documentation, http://openni.org/Documentation/

[6] Microsoft: MSDN site with Kinect specification, http://msdn.microsoft.com/en-us/library/hh855348.aspx

[7] T. Grzejszczak, M. Mikulski, T. Szkodny, K. Jędrasiak, Gesture Based Robot Control, Proceedings of International conference Computer vision and graphics. ICCVG 2012, Warsaw, Poland, September 24-26, 2012., pp.407-413, Lecture Notes in Computer Science; vol. 7594 Springer, Berlin, 2012.

DOI: 10.1007/978-3-642-33564-8_49

[8] B. Bagnal, Maximum LEGO NXT: Building Robots with Java Brains, 3rd edition, (2012)

[9] NASA ISS, Filming of Space Robot "Jitter" Assembled out of Legos (Konstructor), http://www.nasa.gov/mission_pages/station/research/experiments/Konstructor.html#description, 2012.

[10] The LEGO Group: LEGO Mindstorms NXT Communication Protocol, Bluetooth Development Kit, 2006.

[11] The LEGO Group: 8527 – T-56 Building Manual, http://mindstorms.lego.com/en-us/support/buildinginstructions/8527-/T-56.aspx, 2010.

[12] P. Hurbain, NXT motor characteristics: http://www.philohome.com/nxtmotor/nxtmotor.htm, 2013.

[13] PrimeSense, http://www.primesense.com/, 2013.

[14] LEGO Mindstorms, http://mindstorms.lego.com, 2013.

[15] NXT Intelligent Brick, http://en.wikipedia.org/wiki/Lego_Mindstorms_NXT#NXT_Intelligent_Brick, 2013.

DOI: 10.3888/tmj.15-3

[16] Tracking Modes (Seated and Default), http://msdn.microsoft.com/en-us/library/hh973077.aspx, 2013.

[17] 3Gear gesture PC control, http://www.threegear.com/, 2013.

[18] Controlling robots using wrist band – MYO. https://developer.thalmic.com/apply/, 2013.

[19] Codasign, http://learning.codasign.com/index.php?title=Skeleton_Tracking_with_the_Kinect, 2013.

[20] PrimeSense Middleware, http://www.primesense.com/solutions/nite-middleware/, 2013.

[21] Microsoft Corporation, Kinect for Windows. Human Interface Guidelines, http://msdn.microsoft.com/en-us/library/jj663791.aspx, v1.8, 2014.