Your browser does not fully support modern features. Please upgrade for a smoother experience.
The Senses of Music: Towards a Theoretical Model of Multisensory Musical Experience: History
View Latest Version
Please note this is an old version of this entry, which may differ significantly from the current revision.
Contributor: Cristiane Nogueira , Ana Isabel Pereira , Helena Rodrigues

A growing number of studies have highlighted the various sensory interactions involved in the musical experience, as relationships between music and dimensions of taste, olfaction, sound, and visual qualities, such as associations between pitch and the size of images or objects, spatial location and frequency, and instrumental timbres and visual shapes. These studies share the premise that the way we relate to the musical phenomenon, whether in the processes of production, perception, or understanding, emerges from an integrated and intrinsically multisensory perceptual event. Nevertheless, because music is present daily in everyday life and because this experience is inherently subjective, such interactions tend to occur so naturally and seem so obvious that they have been relegated to common sense. On the other hand, evidence indicates that sensory interactions constitute a fundamental ancestral mechanism for cognitive and neuronal development governed by non-arbitrary tendencies, multiple variables, and patterns of predictability. The novel contribution of this review is to advance a dynamic theoretical model of multisensory musical experience that takes crossmodal correspondences as its central organising axis, articulated through three structuring principles (universality, congruence effect, hierarchical tendency) and their interaction with musical organisation, cognitive structure, and the sensory systems mobilised by music. A future research agenda is also proposed to broaden and deepen investigations in the field of music psychology and human development.

  • music
  • multisensory
  • music cognition
  • musical senses
  • crossmodal correspondence
  • synesthesia
As an experience involving different sensory mechanisms, music has been recognised for its intrinsically multisensory nature [1]. In fact, from the manipulation of sound through musical instruments to the reading of signs and notation to physical involvement, whether creating, playing, or listening, the musical phenomenon activates and interacts with different perceptual senses, such as auditory, tactile, visual, and kinesthetic, as well as emotional and cognitive mechanisms.
Although seemingly obvious, a growing body of research shows that sensory interactions involving the musical phenomenon are governed by non-arbitrary tendencies, as in the case of the relationships between pitch and the size of images and objects [2][3][4][5][6][7], spatial location and frequency [8], brightness and visual forms [9], in addition to instrumental timbres and visual forms [10]. Other studies have reported connections between music and colours [11][12][13], icons and figures [14][15][16], and paintings [17][18][19]. In addition, relationships between music and taste [20][21][22][23][24][25] and between music and the sense of smell [26] have been reported. These studies share the premise that the way we relate to the musical phenomenon, whether in its production, perception, or understanding, emerges from an integrated perceptual event. There is evidence that combining information across sensory systems constitutes a fundamental ancestral mechanism for cognitive and neural development [27], manifesting itself in different ways among individuals. Consequently, interactions between sensory domains are believed to be central features of the human mind [28]. In seeking to understand the foundations that underpin this integration, this review draws on philosophical assumptions and empirical evidence to propose a theoretical model that helps to understand the multisensory nature of music, while also offering an agenda for future research in this area.

This entry is adapted from the peer-reviewed paper 10.3390/encyclopedia6050094

References

  1. Tan, S.; Pfordresher, P.; Harré, R. Psychology of Music: From Sound to Significance; Psychology Press: New York, NY, USA, 2010.
  2. Deroy, O.; Spence, C. Why we are not all synesthetes (not even weakly so). Psychon. Bull. Rev. 2013, 20, 643–664.
  3. Gallace, A.; Spence, C. Multisensory synesthetic interactions in the speeded classification of visual size. Percept. Psychophys. 2006, 68, 1191–1203.
  4. Parise, C.; Spence, C. Synesthetic congruency modulates the temporal ventriloquism effect. Neurosci. Lett. 2008, 442, 257–261.
  5. Parise, C.; Spence, C. “When birds of a feather flock together”: Synesthetic correspondences modulate audiovisual integration in non-synesthetes. PLoS ONE 2009, 4, e5664.
  6. Parise, C.; Spence, C. Audiovisual crossmodal correspondences and sound symbolism: A study using the implicit association test. Exp. Brain Res. 2012, 220, 319–333.
  7. Walker, P.; Smith, S. Stroop interference based on the multimodal correlates of haptic size and auditory pitch. Perception 1985, 14, 729–736.
  8. Evans, K.; Treisman, A. Natural cross-modal mappings between visual and auditory features. J. Vis. 2010, 10, 6.
  9. Marks, L. On Cross-Modal Similarity: Auditory-Visual Interactions in Speeded Discrimination. J. Exp. Psychol. 1987, 16, 384–394.
  10. Adeli, M.; Rouat, J.; Molotchnikoff, S. Audiovisual correspondence between musical timbre and visual shapes. Front. Hum. Neurosci. 2014, 8, 352.
  11. Palmer, S.; Schloss, K.; Xu, Z.; Prado-León, L. Music–color associations are mediated by emotion. Proc. Natl. Acad. Sci. USA 2013, 110, 8836–8841.
  12. Lindborg, P.; Friberg, A. Colour Association with Music Is Mediated by Emotion: Evidence from an Experiment Using a CIE Lab Interface and Interviews. PLoS ONE 2015, 10, e0144013.
  13. Whiteford, K.L.; Schloss, K.B.; Helwig, N.E.; Palmer, S.E. Color, music, and emotion: Bach to the blues. i-Perception 2018, 9, 2041669518808535.
  14. Murari, M.; Schubert, E.; Rodà, A.; Da Pos, O.; De Poli, G. How >:(is Bizet? Icon ratings of music. Psychol. Music 2017, 46, 749–760.
  15. Marin, M.; Gingras, B.; Bhattacharya, J. Crossmodal transfer of arousal, but not pleasantness, from the musical to the visual domain. Emotion 2012, 12, 618–631.
  16. Simurra, I.; Vanzella, P.; Sato, J. Timbre and Visual Forms: A crossmodal study relating acoustic features and the Bouba- Kiki Effect. In Proceedings of the 2nd International Conference on Timbre (Timbre 2020), Thessaloniki, Greece (online), 3–4 September 2020; Available online: http://timbre2020.mus.auth.gr/assets/papers/Timbre2020_proceedings.pdf (accessed on 3 April 2026).
  17. Albertazzi, L.; Canal, L.; Micciolo, R. Cross-modal associations between materic painting and classical Spanish music. Front. Psychol. 2015, 6, 424.
  18. Albertazzi, L.; Canal, L.; Micciolo, R.; Hachen, I. Cross-Modal Perceptual Organization in Works of Art. i-Perception 2020, 11, 2041669520950750.
  19. Braun Janzen, T.; de Oliveira, B.; Ventorim Ferreira, G.; Sato, J.R.; Feitosa-Santana, C.; Vanzella, P. The effect of background music on the aesthetic experience of a visual artwork in a naturalistic environment. Psychol. Music 2022, 51, 16–32.
  20. Guetta, R.; Loui, P. When music is salty: The crossmodal associations between sound and taste. PLoS ONE 2017, 12, e0173366.
  21. Reinoso-Carvalho, F.; Wang, Q.; van Ee, R.; Persoone, D.; Spence, C. “Smooth operator”: Music modulates the perceived creaminess, sweetness, and bitterness of chocolate. Appetite 2017, 108, 383–390.
  22. Kantono, K.; Hamid, N.; Shepherd, D.; Yoo, M.J.Y.; Carr, B.T.; Grazioli, G. The effect of background music on food pleasantness ratings. Psychol. Music 2016, 44, 1111–1125.
  23. Lin, Y.; Hamid, N.; Shepherd, D.; Kantono, K.; Spence, C. Musical and Non-Musical Sounds Influence the Flavour Perception of Chocolate Ice Cream and Emotional Responses. Foods 2022, 11, 1784.
  24. Galmarini, M.; Silva Paz, R.; Enciso Shoquehuanca, D.; Zamora, D.; Mesz, B. Impact of music on the dynamic perception of coffee and evoked emotions evaluated by temporal dominance of sensations (TDS) and emotions (TDE). Food Res. Int. 2021, 150, 110795.
  25. Wang, Q.; Spence, C. Assessing the Effect of Musical Congruency on Wine Tasting in a Live Performance Setting. i-Perception 2015, 6, 2041669515593027.
  26. Velasco, C.; Balboa, D.; Marmolejo-Ramos, F.; Spence, C. Crossmodal effect of music and odor pleasantness on olfactory quality perception. Front. Psychol. 2014, 5, 1352.
  27. Sathian, K.; Ramachandran, V. Multisensory Perception: From Laboratory to Clinic; Academic Press: Cambridge, MA, USA, 2020.
  28. Ramachandran, V.; Marcus, Z.; Chunharas, C. Bouba-Kiki: Cross-domain resonance and the origins of synesthesia, metaphor, and words in the human mind. In Multisensory Perception: From Laboratory to Clinic; Sathian, K., Ramachandran, V., Eds.; Academic Press: Cambridge, MA, USA, 2020; pp. 3–40.
More
© Text is available under the terms and conditions of the Creative Commons Attribution (CC BY) license; additional terms may apply. By using this site, you agree to the Terms and Conditions and Privacy Policy.
This entry is offline, you can click here to edit this entry!
Academic Video Service
Feedback

Quick Survey

Encyclopedia MDPI is conducting a targeted survey to identify the specific barriers hindering efficient research. We invite you to spend 3 minutes defining the priorities for our next generation of structured knowledge tools.
Take Survey