Augmenting art crossmodally: possibilities and pitfalls

In this narrative historical review, we take a closer look at the question of whether it is possible to augment works of art through crossmodal (specifically audiovisual) means. We start by highlighting an important distinction between three classes of audiovisual crossmodal correspondence: Namely t...

Full description

Saved in:
Bibliographic Details
Main Authors: Charles Spence, Nicola Di Stefano
Format: Article
Language:English
Published: Frontiers Media S.A. 2025-07-01
Series:Frontiers in Psychology
Subjects:
Online Access:https://www.frontiersin.org/articles/10.3389/fpsyg.2025.1605110/full
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In this narrative historical review, we take a closer look at the question of whether it is possible to augment works of art through crossmodal (specifically audiovisual) means. We start by highlighting an important distinction between three classes of audiovisual crossmodal correspondence: Namely those operating on individual sensory stimuli (so-called basic correspondences), those operating on dynamically-changing stimuli, or else on combinations of unisensory stimuli (so-called mid-level correspondences), and those operating on complex and often aesthetically-meaningful stimuli, such as music and paintings. We also highlight another important distinction between the literature on crossmodal matching and that dedicated to demonstrating crossmodal effects. The latter distinction aligns, in some sense, onto the distinction between crossmodal mapping and crossmodal effects. Although it may not be possible, in any meaningful sense, to translate works of art from one modality into another, that does not deny the possibility of augmenting a work of art by the deliberate addition of stimulation presented to another sensory modality. The aims and objectives of those who have attempted to augment works of art by introducing additional sensory stimulation are discussed. We also draw attention to a number of challenges and/or pitfalls (such as the distraction offered by recourse to the phenomenon of synaesthesia) for those interested in augmenting auditory/visual art crossmodally.
ISSN:1664-1078