Congruence and Simultaneity Judgment with a Single Multisensory Target: Preliminary Results

Abstract:

The presence of congruent features may suggest to the brain that stimuli in different modalities are due to a common cause. Previous work has presented conflicting results regarding the impact of congruence on simultaneity judgments, widely thought to be related to judgments of common cause. Here, a novel experimental paradigm using a single changing multi-sensory object is described, with the goal of contrasting results obtained using a single multi-sensory target with those obtained from previous work using arrays of moving dots and staggered asynchronous pairs of targets. Some preliminary results from a pilot validating the experimental procedure are discussed.

• Periodical: Advanced Materials Research (Volumes 546-547)
• Pages: 937-942
• DOI: https://doi.org/10.4028/www.scientific.net/AMR.546-547.937
• Online since: July 2012
• Authors: Qi Li, Steven Langsford, Jing Jing Yang
• Keywords: Inference, Multi-Modal Congruence, Multisensory Integration, Simultaneity Judgments

References

[1] J. V. Stone, N. M. Hunkin, J. Porrill, R. Wood, V. Keeler, M. Beanland, M. Port and N. R. Porter (2001) When is now? Perception of simultaneity, Proc. R. Soc. Lond. B 268 31-38.

DOI: 10.1098/rspb.2000.1326

[2] C. Spence, and S. Squire (2003) Multisensory integration: Maintaining the Perception of Synchrony, Current Biology 13: 13 R519-R521.

DOI: 10.1016/s0960-9822(03)00445-7

[3] L.R. Harris, V. Harrar, P.M. Jaekl, and A. Kopinska (2009) Mechanisms of simultaneity constancy. In: Nijhawan R (ed) Issues of space and time in perception and action. Cambridge University Press.

DOI: 10.1017/cbo9780511750540.015

[4] D. Alais and S. Carlile. (2005). Synchronizing to real events: subjective audiovisual alignment scales with perceived auditory depth and speed of sound., Proc Natl Acad Sci U S A 102: 2244–2247.

DOI: 10.1073/pnas.0407034102

[5] W. Fujisaki, S. Shimojo, M. Kashino, and S. Nishida (2004) Recalibration of audio–visual simultaneity., Nat. Neurosci. 7: 773–778.

DOI: 10.1038/nn1268

[6] K. P Körding, U. Beierholm, W.J. Ma, S. Quartz, J.B. Tenenbaum et al. (2007) Causal Inference in Multisensory Perception., PLoS ONE 2(9): e943.

DOI: 10.1371/journal.pone.0000943

[7] G. H Recanzone (2003) Auditory influences on visual temporal rate perception, J. Neurophysiol 89 1078–1093.

DOI: 10.1152/jn.00706.2002

[8] J. Vroomen, and B. de Gelder (2004) Temporal Ventriloquism: Sound Modulates the Flash-Lag Effect., Journal of Experimental Psychology: Human Perception and Performance, 30: 3 513-518.

DOI: 10.1037/0096-1523.30.3.513

[9] Y. Wada, N. Kitagawa, and K. Noguchi (2003) Audio-visual integration in temporal perception, International Journal of Psychophysiology 50: 1-2 117-124.

DOI: 10.1016/s0167-8760(03)00128-4

[10] W. J Ma, and A. Pouget (2008) Linking neurons to behaviour in multisensory perception: a computational review, Brain Research 10: 1016.

[11] S. Deneve, and A. Pouget (2004) Bayesian multisensory integration and cross-modal spatial links, Journal of Physiology –Paris 98 249-258.

DOI: 10.1016/j.jphysparis.2004.03.011

[12] U. Beierholm, K. Kording, L. Shams, and W.J. Ma, (2007). Comparing Bayesian models for multisensory cue combination without mandatory integration., In: Advances in Neural Information Processing Systems.

[13] M. Keetels , and J. Vroomen (2009) Perception of synchrony between the senses. In M. M. Murray & M. T. Wallace (Eds. ), Frontiers in the neural basis of multisensory processes.

DOI: 10.1201/b11092-12

[14] A. Vatakis and C. Spence (2007) Evaluating the influence of the 'unity assumption' on the temporal perception of realistic audiovisual stimuli, B. Acta Psychologica 127: 1 12-23.

DOI: 10.1016/j.actpsy.2006.12.002

[15] Baumann and Greenlee (2007) Neural correlates of coherent audiovisual motion perception, Cerebral Cortex 17: 1433-1443.

DOI: 10.1093/cercor/bhl055

[16] G.F. Meyer and S. M Wuerger (2001) Cross-modal integration of auditory and visual motion signals, Neuroreport 12: 118 2557-2560.

DOI: 10.1097/00001756-200108080-00053

[17] C. Parise, and C. Spence. (2008). Synesthetic congruency modulates the temporal ventriloquism effect., Neuroscience Letters, 442, 257–261.

DOI: 10.1016/j.neulet.2008.07.010

[18] C. Parise, and C. Spence. (2009). When birds of a feather flock together: Synesthetic correspondences modulate audiovisual integration in non-synesthetes., PLoS ONE, 4, e5664.

DOI: 10.1371/journal.pone.0005664

[19] J. Vroomen and M. Keetels (2011) No effect of synesthetic congruency on temporal ventriloquism., Atten. Percept. Psychophys. 73 209-218.

DOI: 10.3758/s13414-010-0019-0

[20] N. Naber, T Carlson, F. Verstraten, W. Einhauser (2011) Perceptual benefits of objecthood, Journal of Vision 11(4): 8, 1-9.

DOI: 10.1167/11.4.8

[21] L. Busse, K. Roberts, R. Crist, D. Weissman, M. Woldorff (2005) The spread of attention across modalities and space in a multisensory object, Proceedings of the Nat. ional Academy of Sciences 102: 51 18751-18756.

DOI: 10.1073/pnas.0507704102

[22] J McDonald, W. Teder-Salejarvi, F. Di Russo, and S. Hillyard (2003) Neural substrates of perceptual enchancement by cross-modal spatial attention, Journal of Cognitive Neuroscience 15: 1 10-19.

DOI: 10.1162/089892903321107783

[23] L. Marks, R Hammeal, and M. Bornstein (1987) Children's comprehension of cross-modal similarity: perception and metaphor., Monographs in social research and child development 42: 11-91.

[24] Marks (2004) Cross-modal interactions in speeded classification In The handbook of multisensory processes, Calvert, Spence, and Stein (Eds) MIT Press (2004).

DOI: 10.7551/mitpress/3422.003.0009

[25] R. Malera (1989) Dimensional interaction between color and pitch, Journal of Experimantal Psychology: Human Perception and Performance 15: 1 69-79.

[26] H. Levitt (1971) Transformed up‐down methods in psychoacoustics., J Acoust Soc Am 1971; 49 Suppl 2: 467–76.

[27] R. Alcala-Quintana, and M. Garcia-Perez (2007) A comparison of fixed-step-size and Bayesian staircases for sensory threshold estimation, Spatial Vision, 20: 3 197-218.

DOI: 10.1163/156856807780421174

[28] D. Slutsky, and G. Recanzone (2001) Temporal and spatial dependency of the ventriloquism effect, Auditory and Vestibular Systems 12: 1 7-10.

DOI: 10.1097/00001756-200101220-00009

[29] G. Recanzone (2002) Auditory influences on visual temporal rate perception, Journal of Neurophysiology 89: 2 1078-1093.

DOI: 10.1152/jn.00706.2002