Search

International audience; The McGurk effect has been shown to be modulated by attention. However, it remains unclear whether attentional effects are due to changes in unisensory processing or in the fusion mechanism. In this paper, we used published experimental data showing that distraction of visual attention weakens the McGurk effect, to fit either the Fuzzy Logical Model of Perception (FLMP) in which the fusion mechanism is fixed, or a variant of it in which the fusion mechanism could be varied depending on attention. The latter model was associated with a larger likelihood when assessed with a Bayesian Model Selection criterion. Our findings suggest that distraction of visual attention affects fusion by decreasing the weight of the visual input.

Speech perception is audiovisual, as demonstrated by the McGurk effect in which discrepant visual speech alters the auditory speech percept. We studied the role of visual attention in audiovisual speech perception by measuring the McGurk effect in two conditions. In the baseline condition, attention was focused on the talking face. In the distracted attention condition, subjects ignored the face and attended to a visual distractor, which was a leaf moving across the face. The McGurk effect was weaker in the latter condition, indicating that visual attention modulated audiovisual speech perception. This modulation may occur at an early, unisensory processing stage, or it may be due to changes at the stage where auditory and visual information is integrated. We investigated this issue by conventional statistical testing, and by fitting the Fuzzy Logical Model of Perception (Massaro, 1998) to the results. The two methods suggested different interpretations, revealing a paradox in the current methods of analysis.

McGurk discovered that when a videotape of a talker is dubbed with similar but mismatched utterances, the optic information exerts a strong influence on the speech that is perceived, often without the observer's awareness. In several studies, we explored further the effects of conflicting acoustic and optic information on consonant perception. The following questions were asked: (1) Do subjects report a consonant even when no consonantal cues are contained in the acoustic signal (an isolated vowel)? (2) How is acoustic‐optic place of articulation conflict resolved when acoustically specified manner (e.g., nasal, stop) restricts possible places of articulation? (3) How far does observers' awareness of acoustic‐optic discrepancy reduce cross‐modal integration? [Work supported by NICHD.]

In the McGurk effect, observers typically report the illusory syllable /da/ when they hear the auditory syllable /ba/ presented in synchrony with a video display of a talker saying /ga/. While the effect itself has been well established, there is still little research on the conditions under which the effect occurs. In the experiment reported here, the number of illusory /d/ responses to the auditory /b/‐visual /g/ combination is examined in three vowel environments: /a/, /i/, and /u/. The results of this study indicate that the magnitude of the illusion is not the same across different vowel environments. It appears to be strongest for the /i/ vowel, moderate for /a/, and almost nonexistent for /u/. The results thus show that vowel environment is an important factor in determining the magnitude of the McGurk effect, which needs to be considered in accounts of auditory‐visual integration during speech perception. [Work supported by NIH.]

The McGurk effect is a phenomenon that demonstrates a perceptual fusion between auditory and visual (lip‐read) information in speech perception under visual‐auditory discrepancy condition (using dubbed video tapes). This paper examined the relation between the “McGurk effects” and the intelligibility of auditory stimuli. A female narrator's speech was video taped for ten Japanese syllables (/bad,/pad,/mad, /wad,/da/,/tad,/na/,/ra/,/ga,/ka/). The video and audio signals for these ten syllables were combined, resulting in 100 audio‐visual stimuli. These stimuli were presented to ten subjects who were required to identify the stimuli as heard speech in both noisy and noise‐free conditions. For both conditions, the intelligibility of the auditory stimuli was measured, presenting the auditory stimuli alone. In the noise‐free condition, the McGurk effect was small and found only in conditions in which the intelligibility of the auditory stimuli was not 100%. In the noisy condition, the McGurk effect was very strong and widespread. These results suggest that incomplete intelligibility of auditory stimuli is necessary for the McGurk effect.

The McGurk effect recently has been variously conjectured to be based upon a precategorical interaction or integration of information from the auditory and visual representation of the linguistic event. The conjecture of precategorical interactions, which rejects the original notion of post‐categorical interactions, may be in the form of a specialized module for speech or language, or may contribute to a logic‐decision process, either of which leads to a relatively discrete perception of speech. While not disputing such precategorical interactions for language, the research to be described demonstrates post‐categorical interaction of auditory and visual representations of phonemes. The stimuli are nonword CVC syllables. The set of auditory stimuli is edited from natural speech, while the set of visual stimuli includes the orthographic representations of the auditory stimuli syllables. Evidence for interactions in identification of initial phonemes is based upon both error rates and reaction times. [Research supported in part by an NSF grant to the first author.]