The auditory and the visual percept evoked by the same audiovisual stimuli
2007 (English)In: Proceedings of the international conference AVSP 2007: International Conference on Auditory-Visual Speech Processing, 2007Conference paper (Refereed)
In analyses and models of audiovisual speech perception, it has been common to consider three percepts: (1) the auditory percept evoked by acoustic stimuli, (2) the visual percept evoked by optic stimuli and (3) a common percept evoked by synchronous optic and acoustic stimuli. Here, it is shown that a vocal percept that is heard and influenced by vision has to be distinguished from a gestural percept that is seen and influenced by audition. In the two experiments reported, syllables distinguished solely by their vowels [i], [y] or [e] were presented to phonetically sophisticated subjects auditorily, visually and in incongruently cross-dubbed audiovisual form. In the first, the subjects rated roundedness, lip spreading, openness and backness of the vowels they heard - in the second of those they saw. The results confirmed that roundedness is mainly heard by eye while openness is heard by ear. Heard backness (retraction) varied with the acoustic and optic presence of roundedness. Seen openness was substantially influenced by acoustic cues, while there was no such influence on seen roundedness. The results are discussed in the context of theories and models.
Place, publisher, year, edition, pages
auditory-visual integration, vowel perception, visual speech perception, audio-visual speech perception.
General Language Studies and Linguistics
IdentifiersURN: urn:nbn:se:su:diva-22185OAI: oai:DiVA.org:su-22185DiVA: diva2:188712
AVSP 2007 International Conference on Auditory-Visual Speech Processing