Improving Robot-Human Communication by Integrating Visual Attention and Auditory Localization Using a Biologically Inspired Model of Superior Colliculus

Abstract:

Effective agent communication is always been an important modern area of research. This paper focuses on achieving greater precision in common by improving agent-human communication with the help of visual attention and auditory localization based on a simple model of the superior colliculus in the human brain system. The model receives individual visual and auditory sensory stimuli and combines them to generate an integrated stimulus predicting the location of the sound source. This integrated stimulus is used to generate a motor saccade of the visual system to attend to the sound. The computational model is based on a neural network approach with learning and is explored in experiments reflecting varied conditions to determine whether it mimes the performance of superior colliculus in auditory and visual stimuli integration. Finally with a evaluation strategy carried between unimodal and multimodal data, the efficiency of the computational model of Superior Colliculus is determined. Performance of the neural network based computational model has proven effective in terms of learning, the better performance of the integrated response over unimodal response and providing a realistic communication experience.

• Periodical: Advanced Materials Research (Volumes 403-408)
• Pages: 4711-4717
• DOI: https://doi.org/10.4028/www.scientific.net/AMR.403-408.4711
• Online since: November 2011
• Authors: Kiran Kumar Ravulakollu, Harry Erwin, Kevin Burn
• Keywords: Autonomous Agents, Multimodal Integration, Saccade Generation, Stimuli Integration, Superior Colliculus

References

[1] Roel Vertegaal, Robert Slagter, Gerrit van der Veer and Anton Nijholt, (2001).

[2] Quaia, C., Lefevre, P., Optican, L.M.: Model of the Control of Saccades by Superior Colliculus and Cerebellum. Journal of Neurophysiology 82(2), 999–1018 (1999).

DOI: 10.1152/jn.1999.82.2.999

[3] Girard B and Berthoz A, (2005), From brainstem to cortex: Computational models of saccade generation circuitry,. Progress in Neurobiology, Vol. 77, p.215 – 251.

DOI: 10.1016/j.pneurobio.2005.11.001

[4] Stein, B.E., Meredith, M.A.: The Merging of the Senses. Cognitive Neuroscience Series. MIT Press, Cambridge (1993).

[5] Cucchiara, R.: Multimedia Surveillance Systems. In: 3rd International Workshop on Video Surveillance and Sensor Networks (VSSN 2005), Singapore, p.3–10 (2005) ISBN: 1-59593-242-9.

DOI: 10.1145/1099396.1099399

[6] King, A.J.: The Superior Colliculus. Current Biology 14(9), R335–R338 (2004).

DOI: 10.1016/j.cub.2004.04.018

[7] Casey, M.C., Pavlou, A.: A Behavioural Model of Sensory Alignment in the Superficial and Deep Layers of the Superior Colliculus. In: Proceeding of International Joint Conference on Neural Networks (IJCNN 2008), p.2750–2755 (2008).

DOI: 10.1109/ijcnn.2008.4634184

[8] Kiran Ravulakollu, Michael Knowles, Jindong Liu and Stefan Wermter, Towards Computational Modelling of Neural Multimodal Integration Based on the Superior Colliculus Concept, In: Innovations in Neural Information Paradigms & Applications, Springer-Verlag, p.269 – 291 (2009).

DOI: 10.1007/978-3-642-04003-0_11

[9] Rothwell, J.C., Schmidt, R.F.: Experimental Brain Research, vol. 221. Springer, Heidelberg, SSN: 0014-4819.

[10] Hakan Yavuz, An integrated approach to the conceptual design and development of an intelligent autonomous mobile robot, Robotics and Autonomous Systems, (2007), Vol. 55, p.498 – 512.

DOI: 10.1016/j.robot.2006.12.010