Replicare: Real-Time Human Arms Movement Replication by a Humanoid Torso

Abstract:

Robotics is a field that has actively been working to reduce the involvement of humans in dangerous environments by automating tasks. In this paper, we propose a method to be able to remotely control a humanoid torso in a telekinetic method. The humanoid torso replicates the pose of the person controlling it remotely by detecting the pose of their arms through an input from an RGB camera in real-time using computer vision techniques based on machine learning algorithms. By detecting the pose, the humanoid’s joints (shoulders and elbows) are positioned to replicate the pose of the person controlling it. This task is achieved by mapping the positions of the joints of the person controlling the robot to a set of equations using vector algebra. Such a systemensures that the movements executed are not only oriented to the end-effector reaching the desired location, but it also ensures that the position of every part of the robot can be controlled to move in the required manner. This level of control eliminates the complexities of collision detection in teleoperated robotic systems and also increases the range of applications such a system can be used in efficiently.

• Periodical: Advances in Science and Technology (Volume 124)
• Pages: 28-36
• DOI: https://doi.org/10.4028/p-5s1927
• Online since: February 2023
• Authors: Vivek Jagannath*, Shushrut Kumar, P. Visalakshi
• Keywords: Humanoids, Pose Estimation, Robot Telekinesis

References

[1] Z. Cao, G. Hidalgo, T. S.-S. E. W. and Sheikh, Y. (2021). Openpose: Realtime multi-person 2d pose estimation using part affinity fields., IEEE Transactions on Pattern Analysis and Machine Intelligence.

DOI: 10.1109/tpami.2019.2929257

[2] Go, R. and Aoki, Y. (2016). Flexible top-view human pose estimation for detection system via cnn., IEEE 5th Global Conference on Consumer Electronics.

DOI: 10.1109/gcce.2016.7800406

[3] George Papandreou, Tyler Zhu, L.-C. C. S. G. J. T. K. M. (2018). Personlab: Person pose estimation and instance segmentation with a bottom-up, part-based, geometric embedding model." 15th European Conference on Computer Vision (ECCV), AITS.[4] et. al., T. L. M. "The progress of human pose estimation: A survey and taxonomy of models applied in 2d human pose estimation., IEEE Access, vol. 8.

DOI: 10.1007/978-3-030-01264-9_17

[5] Alexander Toshev, C. S. (2014). Deeppose: Human pose estimation via deep neural networks., IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

DOI: 10.1109/cvpr.2014.214

[6] Kaiming He, Xiangyu Zhang, S. R.-J. S. (2016). Deep residual learning for image recognition., IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

DOI: 10.1109/cvpr.2016.90

[7] Simonyan, K. and Zisserman, A. (2015). Very deep convolutional networks for large-scale image recognition., 3rd International Conference on Learning Representations (ICLR).

[8] S. Wei, V. Ramakrishna, T. K. and Sheikh, Y. (2016). Convolutional pose machines., IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

DOI: 10.1109/cvpr.2016.511

[9] Hao-Shu Fang, Shuqin Xie, Y.-W. T. C. L. (2017). Rmpe: Regional multi-person pose estimation., IEEE International Conference on Computer Vision (ICCV).

DOI: 10.1109/iccv.2017.256

[10] Bin Xiao, H. W. and Wei, Y. (2018). Simple baselines for human pose estimation and tracking., 15th European Conference on Computer Vision (ECCV).

DOI: 10.1007/978-3-030-01231-1_29

[11] Baoshi Cao, Kui Sun, M. J.-C. H. Y. Z. H. L. (2016). Design and development of a two-dof torso for humanoid robot., International Conference on Advanced Intelligent Mechatronics (AIM).

DOI: 10.1109/aim.2016.7576741

[12] Z. Huang, F. L. and Xu, L. (2020). Modeling and simulation of 6 dof robotic arm based on gazebo., 6th International Conference on Control, Automation and Robotics (ICCAR).

DOI: 10.1109/iccar49639.2020.9107989

[13] Dafni Antotsiou, Guillermo Garcia-Hernando, and Tae-Kyun Kim (2018). Task-oriented hand motion retargeting for dexterous manipulation imitation., European Conference on Computer Vision (ECCV).

DOI: 10.1007/978-3-030-11024-6_19

[14] Aude G Billard, Sylvain Calinon, and Florent Guenter (2006). Discriminative and adaptive imitation in uni-manual and bi-manual tasks., Robotics and Autonomous Systems, 54 (5):370-384.

DOI: 10.1016/j.robot.2006.01.007

[15] Ankur Handa, Karl Van Wyk, Wei Yang, Jacky Liang, Yu-Wei Chao, Qian Wan, Stan Birchfield, Nathan Ratliff, and Dieter Fox (2020). Dexpilot: Vision-based teleoperation of dexterous robotic hand-arm system., IEEE International Conference on Robotics and Automation (ICRA), pages 9164-917.

DOI: 10.1109/icra40945.2020.9197124

[16] Shuang Li, Xiaojian Ma, Hongzhuo Liang, Michael Gorner, Philipp Ruppel, Bin Fang, Fuchun Sun, and Jianwei Zhang (2019). Vision-based teleoperation of shadow dexterous hand using end-to-end deep neural network., International Conference on Robotics and Automation (ICRA), pages 416-422.

DOI: 10.1109/icra.2019.8794277