Information Feedback for Human Motor Skill Transferring System Using Augmented Reality


Article Preview

This paper presents the study and development of information feedback for human motor skill transferring system. The proposed system has an ability of capturing the demonstrator’s bimanual movement using Kinect which is the Microsoft’s sensing device. Then, the spatial and temporal data of the recorded movement will be encoded into the probabilistic model. When the learner wants to learn the demonstrator’s movement, the reproduced trajectory will be retrieved from the encoded model. After that, the augmented information guidance which is in the form of computer graphics using augmented reality technique will be presented to the learner during the practicing process. A virtual music performance is selected as the case study of transferring the demonstrator skill to the novice. The performance of proposed system guidance on the human motor skill transferring will be investigated. The experimental results indicate that augmented visual feedback leads to reducing the average position error of the novice’s movement with 67.2%. Moreover, 75% of the participants can improve their movement accuracies after being trained with augmented system guidance.



Edited by:

Prof. Jong Wan Hu




T. Tonggoed and S. Charoenseang, "Information Feedback for Human Motor Skill Transferring System Using Augmented Reality", Applied Mechanics and Materials, Vol. 851, pp. 603-610, 2016

Online since:

August 2016




* - Corresponding Author

[1] G. Westerfield, A. Mitrovic, M. Billinghurst, Intelligent Augmented Reality Training for Assembly Tasks, in Artificial Intelligence in Education: 16th International Conference, AIED 2013, Memphis, TN, USA, July 9-13, 2013. Proceedings, H. C. Lane, K. Yacef, J. Mostow, P. Pavlik, Eds., ed Berlin, Heidelberg: Springer Berlin Heidelberg, 2013, 542-551.


[2] N. Pathomaree, S. Charoenseang, Augmented reality for skill transfer in assembly task, in Robot and Human Interactive Communication, 2005, ROMAN 2005, IEEE International Workshop on, 2005, 500-504.


[3] A. C. Boud, D. J. Haniff, C. Baber, S. J. Steiner, Virtual reality and augmented reality as a training tool for assembly tasks, in Information Visualization, 1999, Proceedings, 1999 IEEE International Conference on, 1999, 32-36.


[4] U. Yang, G. J. Kim, Implementation and evaluation of just follow me,: an immersive, VR-based, motion-training system, Presence: Teleoper. Virtual Environ., 11 (2002) 304-323.


[5] Your Shape Fitness Evolved, (n. d. ). https: /en. wikipedia. org/wiki/Your_Shape: _Fitness_Evolved (accessed 6 March 2016).

[6] A. P. Hauge, C. S. Kragegaard, E. B. Kjæhr, M. Kraus, Transfer of Juggling Skills Acquired in a Virtual Environment, in 8th International Conference on Computer Graphics Theory and Applications (Grapp 2013), Spain, 2013, 385-388.

[7] F. Anderson, T. Grossman, J. Matejka, G. Fitzmaurice, YouMove: enhancing movement training with an augmented reality mirror, presented at the Proceedings of the 26th annual ACM symposium on User interface software and technology, St. Andrews, Scotland, United Kingdom, (2013).


[8] D. C. Halbert, Programming by example, University of California, Berkeley, (1984).

[9] Watch what I do: programming by demonstration, MIT Press, 1993, p. ^pp. Pages.

[10] A. Billard, S. Calinon, R. Dillmann, S. Schaal, Survey: Robot Programming by Demonstration, in Handbook of Robotics, chapter 59, MIT Press, (2008).


[11] S. Calinon, Robot Programming by Demonstration: A Probabilistic Approach: EPFL Press, (2009).

[12] S. Calinon, F. D'Halluin, E. L. Sauser, D. G. Caldwell, A. G. Billard, Learning and Reproduction of Gestures by Imitation, Robot. Autom. Mag. IEEE, 17(2) (2010) 44-54.


[13] M. F. Ghilardi, C. Moisello, G. Silvestri, C. Ghez, J. W. Krakauer, Learning of a Sequential Motor Skill Comprises Explicit and Implicit Components That Consolidate Differently, J. Neurophysiology, 101(5) (2009) 2218-2229.