A comparative evaluation of auditory-visual mappings for sound visualisation

Giannakis, Kostas


Organised Sound 11(3): 297-307.

Language: English

The significant role of visual communication in modern computer applications is indisputable. In the case of music, various attempts have been made from time to time to translate non-visual ideas into visual codes. In computer music research, most current sound design tools allow the direct manipulation of visual representations of sound such as time-domain and frequency-domain representations, with the most notable examples being the UPIC system, Phonogramme, Lemur, and MetaSynth, among others. Associations between auditory and visual dimensions have also been extensively studied in other scientific domains such as visual perception and cognitive psychology, as well as inspired new forms of artistic expression. Although there has been such a cross-disciplinary interest in the investigation and application of visual metaphors for musical or related purposes, the quite distinct research methodologies incorporated in the above disciplines have not facilitated interdisciplinary attempts and coordination of research efforts. The proposed auditory-visual associations are primarily based on subjective judgements rather than on empirical evidence. Furthermore, the evaluation of auditory visual mappings is an issue that has generally been disregarded. This situation makes it rather difficult to find well-established methods to evaluate a particular mapping or to compare different mappings against the same evaluation criteria. This article presents efforts to set the necessary groundwork for future investigations in this area.