VisibleSound: Perceiving Environmental Sound with 4D Form

Abstract:

The progression of application software and high-speed Internet access has, in general, the ability to commonly engage synesthetic experience. However, current synesthesia systems do not meet their full potential, as they are restricted by the simple display of non-intelligent two-dimensional form. This paper is going to present a system for enhancing visual and auditory attributes connection by augmenting the traditional sound visualization system with natural perceptual properties beyond two-dimensional space. A method of expanding the analysis capabilities of the system over changes in the frequencies and sound pressure levels of the live environmental sound stream was explored. With this method, a semantically rich four-dimensional form of sound containing information regarding audio properties and location could be generated. Some specific scenarios are discussed in order to explore visual form of natural perception of environmental sounds, multimedia voice and musical performance.

• Periodical: Advanced Materials Research (Volumes 718-720)
• Pages: 1777-1781
• DOI: https://doi.org/10.4028/www.scientific.net/AMR.718-720.1777
• Online since: July 2013
• Authors: Fang Tian Ying, Ye Tao, Guan Yun Wang, Qi Wang
• Keywords: Four-Dimensional Form, Perception, Synesthesia, Visual Design

References

[1] Haverkamp M, Hanser Verlag, Synaesthetic Design, Munich / Vienna, 2009.

[2] Cytowic Richard E, Eagleman David M, Wednesday is Indigo Blue: Discovering the Brain of Synesthesia (with an afterword by Dmitri Nabokov), Cambridge: MIT Press. p.309. ISBN 0-262-01279-9 (2009).

DOI: 10.1016/s0262-4079(09)61454-8

[3] Nicolas Plouznikoff, Alexandre Plouznikoff, Jean-Marc Robert, Artificial Grapheme-Color Synesthesia for Wearable Task Support, in Proceedings of the Ninth IEEE International Symposium on Wearable Computers(ISWC'2005).

DOI: 10.1109/iswc.2005.11

[4] Matthias Waldeck, Synesthetic Visual Graphics Triggered by Hearing Piano Sounds, In: Color-Light-Music, Jewanski J, Sidler N (eds), Zu¨richer Musikstudien Band V. Peter Lang Verlag, Bern (2006) 90–91.

[5] Natalia Sidler, Synesthetic Sketches Triggered by Different Sounds from the Color Light Organ, In: Color-Light-Music, Jewanski J, Sidler N (eds), Zu¨richer Musikstudien Band V. Peter Lang Verlag, Bern (2006) 440–441.

[6] Filip Bruyneel, Lieve Lanoye, Optical Simulations for Ambilight TV Systems, in Proc. of SPIE Vol. 8429 84290B-1 (2012).

DOI: 10.1117/12.921295

[7] Hidehiro Nakahara, Shinichi Furuya, Tsutomu Masuko, Peter R. Francis d, Hiroshi Kinoshita, Performing Music Can Induce Greater Modulation of Emotion-related Psychophysiological Responses than Listening to Music, International Journal of Psychophysiology 81 (2011) 152–158.

DOI: 10.1016/j.ijpsycho.2011.06.003