Search

International audience; In natural listening situations, understanding spoken sentences requires interactions between several multisensory to linguistic levels of information. In two electroencephalographical studies, we examined the neuronal oscillations of linguistic prediction produced by unimodal and bimodal sentence listening to observe how these brain correlates were affected by the sensory streams delivering linguistic information. Sentence contexts which were strongly predictive of a particular word were ended by a possessive adjective matching or not the gender of the predicted word. Alpha, beta and gamma oscillations were investigated as they were considered to play a crucial role in the predictive process. During the audiovisual or auditory-only listening to sentences, no evidence of word prediction was observed. In contrast, in a more challenging listening situation during which bimodal audiovisual streams switched to unimodal auditory stream, gamma power was sensitive to word prediction based on prior sentence context. Results suggest that prediction spreading from higher sentence levels to lower word levels is optional during unimodal and bimodal sentence listening and is observed when the listening situation is more challenging. Alpha and beta oscillations were found to decrease when semantically constraining sentences were delivered in the audiovisual modality in comparison with the auditory-only modality. Altogether, our findings bear major implications for our understanding of the neural mechanisms that support predictive processing in multimodal language comprehension.

In everyday communication, natural spoken sentences are expressed in a multisensory way through auditory signals and speakers’ visible articulatory gestures. An important issue is to know whether audiovisual speech plays a main role in the linguistic encoding of an utterance until access to meaning. To this end, we conducted an event-related potential experiment during which participants listened passively to spoken sentences and a lexical recognition task. The results revealed that N200 and N400 waves had a greater amplitude after semantically incongruous words than after expected words. This effect of semantic congruency was increased over N200 in the audiovisual trials. Words presented audiovisually also elicited a reduced amplitude of the N400 wave and a facilitated recovery in memory. Our findings shed light on the influence of audiovisual speech on the understanding of natural spoken sentences by acting on the early stages of word recognition in order to access a lexical-semantic network.

International audience; Speakers use prosodic emphasis to express the content of their message in order to help listeners to infer meaning. By measuring event-related potentials (ERPs) to semantically congruent and incongruent final words embedded in a sentential context that was emphasised or de-emphasised, we investigated whether prosodic emphasis conveyed by a sentential context leads listeners to a finer semantic analysis. The negative shift (N400) triggered by the difficulty to combine the incongruent word with the sentence representation was increased by prosodic emphasis at an early stage. Over the later stages, the amplitude of the N400 wave was increased by prosodic emphasis of the sentential context, whatever the semantic congruency of final words. As shown by the N400 wave, emphasising a sentential context affected the lexical-semantic processing of the following word. This study provides clear evidence that prosodic emphasis plays a role in the semantic analysis of sentences by inducing a deeper analysis. © 2018, © 2018 Informa UK Limited, trading as Taylor & Francis Group.

Cette etude de cas s'inscrit dans un travail de recherche plus etendu sur les liens entre prosodie et geste realise a partir d'une banque de donnees constituee de courts extraits filmes de recits d'evenements personnels par de jeunes britanniques. Un logiciel, mis au point a l'Universite de Lille 3, permet de synchroniser la lecture des deviations de tete du locuteur, vers la droite ou vers la gauche, vers le haut ou vers le bas, avec celle des variations de hauteur de la voix obtenues dans Praat. L'objectif est de montrer en quoi les caracteristiques vocales et gestuelles mises en evidence dans l'emploi par le locuteur des prepositions "in", "on" et "off" structurent son recit et permettent d'elaborer graduellement une representation mentale de l'espace/temps dans lequel s'inscrit l'evenement narre. On montrera, par le biais de l'analyse pluri-parametrique, le role joue par les prepositions, role a la fois localisant, introduisant des espaces concrets (dont un terrain de football est le centre) mais aussi abstraits, discursifs. Toute l'architecture du recit tient a la mise en forme orale et gestuelle de l'evenement vecu dans un espace recree par la parole.