Search

17 maart 2023, Item does not contain fulltext, During listening, brain activity tracks the rhythmic structures of speech signals. Here, we directly dissociated the contribution of neural envelope tracking in the processing of speech acoustic cues from that related to linguistic processing. We examined the neural changes associated with the comprehension of Noise-Vocoded (NV) speech using magnetoencephalography (MEG). Participants listened to NV sentences in a 3-phase training paradigm: (1) pre-training, where NV stimuli were barely comprehended, (2) training with exposure of the original clear version of speech stimulus, and (3) post-training, where the same stimuli gained intelligibility from the training phase. Using this paradigm, we tested if the neural responses of a speech signal was modulated by its intelligibility without any change in its acoustic structure. To test the influence of spectral degradation on neural envelope tracking independently of training, participants listened to two types of NV sentences (4-band and 2-band NV speech), but were only trained to understand 4-band NV speech. Significant changes in neural tracking were observed in the delta range in relation to the acoustic degradation of speech. However, we failed to find a direct effect of intelligibility on the neural tracking of speech envelope in both theta and delta ranges, in both auditory regions-of-interest and whole-brain sensor-space analyses. This suggests that acoustics greatly influence the neural tracking response to speech envelope, and that caution needs to be taken when choosing the control signals for speech-brain tracking analyses, considering that a slight change in acoustic parameters can have strong effects on the neural tracking response.

Radboud University, 29 maart 2023, Promotores : Hagoort, P., Hoop, H. de Co-promotor : Martin, A.E., Contains fulltext : 290384.pdf (Publisher’s version ) (Closed access)

Item does not contain fulltext, According to the language marker hypothesis language has provided homo sapiens with a rich symbolic system that plays a central role in interpreting signals delivered by our sensory apparatus, in shaping action goals, and in creating a powerful tool for reasoning and inferencing. This view provides an important correction on embodied accounts of language that reduce language to action, perception, emotion and mental simulation. The presence of a language system has, however, also important consequences for perception, action, emotion, and memory. Language stamps signals from perception, action, and emotional systems with rich cognitive markers that transform the role of these signals in the overall cognitive architecture of the human mind. This view does not deny that language is implemented by means of universal principles of neural organization. However, language creates the possibility to generate rich internal models of the world that are shaped and made accessible by the characteristics of a language system. This makes us less dependent on direct action-perception couplings and might even sometimes go at the expense of the veridicality of perception. In cognitive (neuro)science the pendulum has swung from language as the key to understand the organization of the human mind to the perspective that it is a byproduct of perception and action. It is time that it partly swings back again.

Radboud University, 01 juni 2023, Promotor : Hagoort, P. Co-promotor : Ostarek, M., Contains fulltext : 292962.pdf (Publisher’s version ) (Closed access), The neural correlates of language processing have been mostly studied from a perceptive point of view (i.e. while listening or reading). Studies with speaking are instead rarer, because fMRI is very sensitive to movement in the scanner, such as the movement of the jaw elicited by speaking. In addition, it is harder to run experiments where participants speak, because it is almost impossible to have experimental control over what participants will say, without telling them exactly what to say (for example by reading). Therefore, the aim of this thesis was to focus on brain activity during the productive side of language, and to understand to what extent what we learnt about language comprehension can be extended to language production. This thesis presents four studies that focus on different aspects of sentence production and comprehension with different tasks. Three chapters focus on brain activity and the connectivity between brain regions during syntactic processing, using both a task design and a naturalistic paradigm. One chapter focuses on the neural responses to compositional processing during sentence production. The results suggest that there are some differences in the brain responses to production and comprehension, both spatially and in the timing of responses, but overall it appears that the same brain regions are used for linguistic processing during speaking and listening.

18 april 2023, Item does not contain fulltext, Recent artificial neural networks that process natural language achieve unprecedented performance in tasks requiring sentence-level understanding. As such, they could be interesting models of the integration of linguistic information in the human brain. We review works that compare these artificial language models with human brain activity and we assess the extent to which this approach has improved our understanding of the neural processes involved in natural language comprehension. Two main results emerge. First, the neural representation of word meaning aligns with the context-dependent, dense word vectors used by the artificial neural networks. Second, the processing hierarchy that emerges within artificial neural networks broadly matches the brain, but is surprisingly inconsistent across studies. We discuss current challenges in establishing artificial neural networks as process models of natural language comprehension. We suggest exploiting the highly structured representational geometry of artificial neural networks when mapping representations to brain data.

22 juni 2023, Item does not contain fulltext, When perceiving the world around us, we are constantly integrating pieces of information. The integrated experience consists of more than just the sum of its parts. For example, visual scenes are defined by a collection of objects as well as the spatial relations amongst them and sentence meaning is computed based on individual word semantic but also syntactic configuration. Having quantitative models of such integrated representations can help evaluate cognitive models of both language and scene perception. Here, we focus on language, and use a behavioral measure of perceived similarity as an approximation of integrated meaning representations. We collected similarity judgments of 200 subjects rating nouns or transitive sentences through an online multiple arrangement task. We find that perceived similarity between sentences is most strongly modulated by the semantic action category of the main verb. In addition, we show how non-negative matrix factorization of similarity judgment data can reveal multiple underlying dimensions reflecting both semantic as well as relational role information. Finally, we provide an example of how similarity judgments on sentence stimuli can serve as a point of comparison for artificial neural networks models (ANNs) by comparing our behavioral data against sentence similarity extracted from three state-of-the-art ANNs. Overall, our method combining the multiple arrangement task on sentence stimuli with matrix factorization can capture relational information emerging from integration of multiple words in a sentence even in the presence of strong focus on the verb.

INSAR 2023: Annual Meeting International Society for Autism Research (INSAR) (Stockholm, Sweden, 306 May 2023), Item does not contain fulltext