Paper Titles
Analytical Calculation, Numerical Structural Analysis and Design Optimization of the Diaphragm Spring of a Mechanical Clutch
p.151
Kinematic and Dynamic Determinations of the Movement of the Car when Moving between Cones
p.163
Study about the Application of Statistical Process Control for Process Quality Improvement in Automotive Industry
p.169
Automation of Installations for Solar Drying of Grain Seeds and Fluidized Bed Technical Plants
p.177
Real Time Prediction of Sclera Force with LSTM Neural Networks in Robot-Assisted Retinal Surgery
p.183
Optimal Design of Components for a Robotic Arm Based on a Matrix Approach of Performance Specifications
p.195
Dynamic and Modal Analysis of a Snake like Robot
p.203
Robotic System for Catheter Navigation during Medical Procedures
p.211
Studies on the Variation of the Process Parameters when Braking DC Motors with Shoe Brakes
p.218

Real Time Prediction of Sclera Force with LSTM Neural Networks in Robot-Assisted Retinal Surgery

Article Preview

Abstract:

Retinal microsurgery is one of the most technically demanding surgeries, during which the surgical tool needs to be inserted into the eyeball and is constantly constrained by the sclerotomy port. During the surgery, any unexpected manipulation could cause extreme tool-sclera contact force leading to sclera damage. Although, a robot assistant could reduce hand tremor and improve the tool positioning accuracy, it cannot prevent or alarm the surgeon about the upcoming danger caused by surgeon’s misoperations, i.e., applying excessive force on the sclera. In this paper, we present a new method based on a Long Short Term Memory recurrent neural network for predicting the user behavior, i.e., the contact force between the tool and sclera (sclera force) and the insertion depth of the tool from sclera contact point (insertion depth) in real time (40Hz). The predicted force information is provided to the user through auditory feedback to alarm any unexpected sclera force. The user behavior data is collected in a mock retinal surgical operation on a dry eye phantom with Steady Hand Eye Robot and a novel multi-function sensing tool. The Long Short Term Memory recurrent neural network is trained on the collected time series of sclera force and insertion depth. The network can predict the sclera force and insertion depth 100 milliseconds in the future with 95.29% and 96.57% accuracy, respectively, and can help reduce the fraction of unsafe sclera forces from 40.19% to 15.43%.

You might also be interested in these eBooks
Achievements and Solutions in Mechanical Engineering II View Preview

Info:

Periodical:

Applied Mechanics and Materials (Volume 896)

Pages:

183-194

DOI:

https://doi.org/10.4028/www.scientific.net/AMM.896.183

Citation:

Cite this paper

Online since:

February 2020

Authors:

Chang Yan He, Niravkumar Patel, Marin Kobilarov, Iulian Iordachita*

Keywords:

Neural Network, Robot-Assisted Retinal Surgery, Scleral Force

Export:

RIS, BibTeX

Price:

Permissions CCC:

Request Permissions

Permissions PLS:

Request Permissions

Сopyright:

© 2020 Trans Tech Publications Ltd. All Rights Reserved

Share:

Citation:

* - Corresponding Author

References

[1] S. Singhy and C. Riviere, Physiological tremor amplitude during reti- nal microsurgery,, in Bioengineering Conference, 2002. Proceedings of the IEEE 28th Annual Northeast. IEEE, 2002, p.171–172.

DOI: 10.1109/nebc.2002.999520

Google Scholar

[2] P. K. Gupta, P. S. Jensen, and E. de Juan, Surgical forces and tactile perception during retinal microsurgery,, in International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 1999, p.1218–1225.

DOI: 10.1007/10704282_132

Google Scholar

[3] F. Walsh, Robot operates inside eye in world first, 2016. [Online]. Available: http://www.bbc.com/news/health-37246995.

Google Scholar

[4] T. Donald, Auditory force feedback substitution improves surgical precision during simulated ophthalmic surgery,, Retina Today, vol. April, p.59–61, (2017).

Google Scholar

[5] C. He, L. Huang, Y. Yang, Q. Liang, and Y. Li, Research and realization of a master-slave robotic system for retinal vascular bypass surgery,, Chinese Journal of Mechanical Engineering, vol. 31, no. 1, p.78, (2018).

DOI: 10.1186/s10033-018-0278-6

Google Scholar

[6] A. Gijbels, N. Wouters, P. Stalmans, H. Van Brussel, D. Reynaerts, and E. Vander Poorten, Design and realisation of a novel robotic manipulator for retinal surgery,, in Intelligent Robots and Systems (IROS), 2013 IEEE/RSJ International Conference on. IEEE, 2013, p.3598–3603.

DOI: 10.1109/iros.2013.6696869

Google Scholar

[7] M. D. de Smet, T. C. Meenink, T. Janssens, V. Vanheukelom, G. J. Naus, M. J. Beelen, C. Meers, B. Jonckx, and J.-M. Stassen, Robotic assisted cannulation of occluded retinal veins,, PloS one, vol. 11, no. 9, p. e0162037, (2016).

DOI: 10.1371/journal.pone.0162037

Google Scholar

[8] R. A. MacLachlan, B. C. Becker, J. C. Tabar e´s, G. W. Podnar, L. A. Lobes Jr, and C. N. Riviere, Micron: an actively stabilized handheld tool for microsurgery,, IEEE Transactions on Robotics, vol. 28, no. 1, p.195–212, (2012).

DOI: 10.1109/tro.2011.2169634

Google Scholar

[9] M. P. Kummer, J. J. Abbott, B. E. Kratochvil, R. Borer, A. Sengul, and B. J. Nelson, Octomag: An electromagnetic system for 5-dof wireless micromanipulation,, IEEE Transactions on Robotics, vol. 26, no. 6, p.1006–1017, (2010).

DOI: 10.1109/tro.2010.2073030

Google Scholar

[10] J. Hubschman, J. Bourges, W. Choi, A. Mozayan, A. Tsirbas, C. Kim, and S. Schwartz, the microhand: a new concept of micro-forceps for ocular robotic surgery,, Eye, vol. 24, no. 2, p.364, (2010).

DOI: 10.1038/eye.2009.47

Google Scholar

[11] A. Uneri, M. A. Balicki, J. Handa, P. Gehlbach, R. H. Taylor, and I. Iordachita, New steady-hand eye robot with micro-force sensing for vitreoretinal surgery,, in Biomedical Robotics and Biomechatronics (BioRob), 2010 3rd IEEE RAS and EMBS International Conference on. IEEE, 2010, p.814–819.

DOI: 10.1109/biorob.2010.5625991

Google Scholar

[12] I. Iordachita, Z. Sun, M. Balicki, J. U. Kang, S. J. Phee, J. Handa, P. Gehlbach, and R. Taylor, A sub-millimetric, 0.25 mn resolution fully integrated fiber-optic force-sensing tool for retinal microsurgery,, International journal of computer assisted radiology and surgery, vol. 4, no. 4, p.383–390, (2009).

DOI: 10.1007/s11548-009-0301-6

Google Scholar

[13] X. He, M. Balicki, P. Gehlbach, J. Handa, R. Taylor, and I. Iordachita, A multi-function force sensing instrument for variable admittance robot control in retinal microsurgery,, in Robotics and Automation (ICRA), 2014 IEEE International Conference on. IEEE, 2014, p.1411–1418.

DOI: 10.1109/icra.2014.6907037

Google Scholar

[14] N. Cutler, M. Balicki, M. Finkelstein, J. Wang, P. Gehlbach, J. Mc-Gready, I. Iordachita, R. Taylor, and J. T. Handa, Auditory force feedback substitution improves surgical precision during simulated ophthalmic surgery,, Investigative ophthalmology & visual science, vol. 54, no. 2, p.1316–1324, (2013).

DOI: 10.1167/iovs.12-11136

Google Scholar

[15] S. Hochreiter and J. Schmidhuber, Long short-term memory,, Neural computation, vol. 9, no. 8, p.1735–1780, (1997).

DOI: 10.1162/neco.1997.9.8.1735

Google Scholar

[16] Z. C. Lipton, D. C. Kale, C. Elkan, and R. Wetzel, Learning to diagnose with lstm recurrent neural networks,, arXiv preprint arXiv:1511.03677, (2015).

Google Scholar

[17] M. F. Stollenga, W. Byeon, M. Liwicki, and J. Schmidhuber, Parallel multi-dimensional lstm, with application to fast biomedical volumetric image segmentation,, in Advances in neural information processing systems, 2015, p.2998–3006.

Google Scholar

[18] M. Sundermeyer, R. Schluter, and H. Ney, Lstm neural networks for language modeling,, in Thirteenth Annual Conference of the International Speech Communication Association, (2012).

Google Scholar

[19] A. Graves, A. Mohamed, and G. Hinton, Speech recognition with deep recurrent neural networks,, in Acoustics, speech and signal processing (ICASSP), 2013 IEEE international conference on. IEEE, 2013, p.6645–6649.

DOI: 10.1109/icassp.2013.6638947

Google Scholar

[20] K. Greff, R. K. Srivastava, J. Koutn ´ık, B. R. Steunebrink, and J. Schmidhuber, Lstm: A search space odyssey,, IEEE transactions on neural networks and learning systems, vol. 28, no. 10, p.2222– 2232, (2017).

DOI: 10.1109/tnnls.2016.2582924

Google Scholar

[21] D. P. Kingma and J. Ba, Adam: A method for stochastic optimization,, arXiv preprint arXiv:1412.6980, (2014).

Google Scholar

[22] F. Chollet et al., Keras,, https://github.com/keras-team/keras, (2015).

Google Scholar

[23] Y. Horise, X. He, P. Gehlbach, R. Taylor, and I. Iordachita, FBG-based sensorized light pipe for robotic intraocular illumination facilitates bi-manual retinal microsurgery,, in Engineering in Medicine and Biology Society (EMBC), 2015 37th Annual International Conference of the IEEE. IEEE, 2015, p.13–16.

DOI: 10.1109/embc.2015.7318249

Google Scholar

[24] C. He, N. Patel, I. Iordachita, and M. Kobilarov, Enabling technology for safe robot-assisted retinal surgery: Early warning for unsafe scleral force,, in Robotics and Automation (ICRA), 2019 IEEE International Conference on., (2019).

DOI: 10.1109/icra.2019.8794427

Google Scholar

[25] C. He, N. Patel, A. Ebrahimi, M. Kobilarov, and I. Iordachita, Preliminary study of an rnn-based active interventional robotic system (airs) in retinal microsurgery,, International journal of computer assisted radiology and surgery, p.1–10, (2019).

DOI: 10.1007/s11548-019-01947-9

Google Scholar