Search

Syntax, the grammatical structure of sentences, is a fundamental aspect of language. It remains debated whether reduced syntactic complexity is unique to schizophrenia spectrum disorder (SSD) or whether it is also present in major depressive disorder (MDD). Furthermore, the association of syntax (including syntactic complexity and diversity) with language-related neuropsychology and psychopathological symptoms across disorders remains unclear. Thirty-four SSD patients and thirty-eight MDD patients diagnosed according to DSM-IV-TR as well as forty healthy controls (HC) were included and tasked with describing four pictures from the Thematic Apperception Test. We analyzed the produced speech regarding its syntax delineating measures for syntactic complexity (the total number of main clauses embedding subordinate clauses) and diversity (number of different types of complex sentences). We performed cluster analysis to identify clusters based on syntax and investigated associations of syntactic, to language-related neuropsychological (verbal fluency and verbal episodic memory), and psychopathological measures (positive and negative formal thought disorder) using network analyses. Syntax in SSD was significantly reduced in comparison to MDD and HC, whereas the comparison of HC and MDD revealed no significant differences. No associations were present between speech measures and current medication, duration and severity of illness, age or sex; the single association accounted for was education. A cluster analysis resulted in four clusters with different degrees of syntax across diagnoses. Subjects with less syntax exhibited pronounced positive and negative symptoms and displayed poorer performance in executive functioning, global functioning, and verbal episodic memory. All cluster-based networks indicated varying degrees of domain-specific and cross-domain connections. Measures of syntactic complexity were closely related while syntactic diversity appeared to be a separate node outside of the syntactic network. Cross-domain associations were more salient in more complex syntactic production.

The neurocognitive mechanism underlying negation processing remains controversial. While negation is suggested to modulate the access of word meaning, no such evidence has been observed in the event-related potential (ERP) literature on sentence processing. In the current study, we applied multivariate pattern analysis (MVPA) methods to examine the processing of sentence negation. We investigated two types of negative congruent/incongruent sentence pairs with truth-value evaluation (e.g., ‘A robin is a/not a bird’) and without (e.g., ‘The woman reads a/no book’). At the target word, ERP results showed consistent N400 effects for semantic congruency, whose amplitudes remain unmodulated across both sentence types and polarity conditions. With MVPA, however, for negative vs. positive sentences, decoding across time analysis showed decreased decoding accuracy for semantic congruency within the N400 window. In addition, temporal generalization revealed serial bi-phasic N400-P600 patterns for semantic congruency across sentence types and polarity. Further, with the cross-decoding approach, we showed that the cognitive processes underlying the N400 window for both affirmative and negative sentences are comparable; whereas in the P600 window, negative sentences showed a distinct pattern from their affirmative counterparts. Our results from MVPA—but not the ERPs—thus speak for a more interactive, but nevertheless serial and bi-phasic processing account of negation processing. We also discuss the advantage of applying MVPA in addition to the classical univariate methods for a better understanding of the neurobiology of negation processing and language comprehension alike.

Reduzierte Leistungen der Wortflussigkeit (WF) sind bei vielen psychiatrischen Populationen, wie Personen mit Schizophrenie (Berberian et al., 2016), Depression (Henry & Crawford, 2005) oder auch bipolarer Erkrankung (Raucher-Chene et al., 2017), belegt. Weiterhin werden diese Defizite in Verbindung mit storungsubergreifenden Symptomen, wie der positiven (z. B. assoziative Auflockerung) und der negativen formalen Denkstorung (z. B. Spracharmut), gebracht (Docherty et al., 2011; Rosenkranz et al., 2019). Viele Studien berichten ausschlieslich die semantische WF und lediglich die Gesamtzahl der korrekten Worter. Ruckschlusse auf eine Ursache der Defizite sind dann nur eingeschrankt moglich. Erganzende Analysen, wie die Auswertung von Fehlern, der generierten Unterkategorien (Cluster) und deren Wechsel konnten Hinweise auf spezifische lexikalisch-semantische Defizite liefern. Die vorliegende Studie untersucht die WF bei Personen mit psychiatrischen Erkrankungen und setzt sie in Relation zu storungsubergreifenden Symptomen sowie weiteren kognitiven Leistungen.

Body orientation of gesture entails social‐communicative intention, and may thus influence how gestures are perceived and comprehended together with auditory speech during face‐to‐face communication. To date, despite the emergence of neuroscientific literature on the role of body orientation on hand action perception, limited studies have directly investigated the role of body orientation in the interaction between gesture and language. To address this research question, we carried out an electroencephalography (EEG) experiment presenting to participants (n = 21) videos of frontal and lateral communicative hand gestures of 5 s (e.g., raising a hand), followed by visually presented sentences that are either congruent or incongruent with the gesture (e.g., “the mountain is high/low…”). Participants underwent a semantic probe task, judging whether a target word is related or unrelated to the gesture‐sentence event. EEG results suggest that, during the perception phase of handgestures, while both frontal and lateral gestures elicited a power decrease in both the alpha (8–12 Hz) and the beta (16–24 Hz) bands, lateral versus frontal gestures elicited reduced power decrease in the beta band, source‐located to the medial prefrontal cortex. For sentence comprehension, at the critical word whose meaning is congruent/incongruent with the gesture prime, frontal gestures elicited an N400 effect for gesture‐sentence incongruency. More importantly, this incongruency effect was significantly reduced for lateral gestures. These findings suggest that body orientation plays an important role in gesture perception, and that its inferred social‐communicative intention may influence gesture‐language interaction at semantic level., We collected EEG when participants process visual sentences primed by frontal and lateral gestures. Lateral gestures elicited reduced beta band power decrease during perception. Additionally, in the visual sentences, the N400 effect, as derived from semantic incongruency between the critical word and the gesture prime, was reduced for lateral gestures.

The feeling of being addressed is the first step in a complex processing stream enabling successful social communication. Social impairments are a relevant characteristic of patients with major depressive disorder (MDD). Here, we investigated a mechanism which—if impaired—might contribute to withdrawal or isolation in MDD, namely, the neural processing of social cues such as body orientation and gesture. During funtional magnetic resonance imaging (fMRI) data acquisition, 33 patients with MDD and 43 healthy control subjects watched video clips of a speaking actor: one version with a gesture accompanying the speech and one without gesture. Videos were filmed simultaneously from two different viewpoints: one with the actor facing the viewer head‐on (frontal) and one side‐view (lateral). After every clip, the participants were instructed to evaluate whether they felt addressed or not. Despite overall comparable addressment ratings and a large overlap in activation patterns in MDD and healthy subjects for gesture processing, the anterior cingulate cortex, bilateral superior/middle frontal cortex, and right angular gyrus were more strongly activated in patients than in healthy subjects for the frontal conditions. Our analyses revealed that patients showed specifically higher activation than healthy subjects for the frontal condition without gesture in regions including the posterior cingulate cortex, left prefrontal cortex, and the left hippocampus. We conclude that MDD patients can recognize and interpret social cues such as gesture or body orientation; however, they seem to require more neural resources. This additional effort might affect successful communication and contribute to social isolation in MDD.

Background There is an ongoing discussion about which neurobiological correlates or symptoms separate the major psychoses (i.e. Major Depressive Disorder MDD, Bipolar Disorder BD, and Schizophrenia SZ). Psychopathological factor analyses within one of these disorders have resulted in models including one to five factors. Factor analyses across the major psychoses using a comprehensive set of psychopathological scales in the same patients are lacking. It is further unclear, whether hierarchical or unitarian models better summarize phenomena. Method Patients (n = 1182) who met DSM-IV criteria for MDD, BD, SZ or schizoaffective disorder were assessed with the SANS, SAPS, HAMA, HAM-D, and YMRS. The sample was split into two and analyzed using explorative and confirmatory factor analyses to extract psychopathological factors independent of diagnosis. Results In the exploratory analysis of sample 1 (n = 593) we found 5 factors. The confirmatory analysis using sample 2 (n = 589) confirmed the 5-factor model (χ2 = 1287.842, df = 571, p Conclusion This is the first study delineating psychopathological factors in a large group of patients across the spectrum of affective and psychotic disorders. In future neurobiological studies, we should consider transdiagnostic syndromes besides the traditional diagnoses.

Two experiments were conducted to investigate the effects of visual segmentation, complexity, and context on the cognitive processing of compounds in Easy Language. By presenting compounds in different boundary conditions, we determined whether a segmentation cue facilitates the processing of compounds presented with and without contextual information. The study was conducted with unimpaired adults and with hearing-impaired pupils, representing one of the Easy Language target groups. The results indicate that visual segmentation facilitates processing of compounds for pupils with low literacy skills. However, they only benefit from segmentation when morpheme boundaries are marked in a subtle way, i. e., without strikingly deviating from the standard version. Pupils with higher literacy skills and unimpaired adults do not profit from segmentation. Even though hyphenation slows down compound processing for unimpaired readers, initial processing advantages of hyphenated over concatenated compounds emerged, which is explained by the fact that hyphenation forces a morpheme-based access and enables fast recognition of the compound’s first constituent. However, it hinders readers from accessing the compound via the direct route and thus slows down the processing of the compound as a whole.

There is a lack of knowledge regarding the relationship between dimensional psychopathological syndromes and neurocognitive functions, particularly across the major psychiatric disorders (i.e., Major Depressive Disorder (MDD), Bipolar Disorder (BD), and Schizophrenia (SZ)).SANS, SAPS, HAMA, HAM-D, and YMRS were assessed in 1064 patients meeting DSM-IV criteria for MDD, BD, SZ or schizoaffective disorder (SZA). In addition, a comprehensive neuropsychological test battery was administered. Psychopathological syndromes derived from factor analysis and present state of illness were used to explore psychopathology-cognition relationships. Correlational analyses were corrected for age, sex, verbal IQ, years of education, and DSM-IV diagnosis. Age of onset and total duration of hospitalizations as proxies for illness severity were tested as moderators on the cognition - psychopathology relationship.The negative syndrome, positive formal thought disorder as well as the paranoid-hallucinatory syndrome exhibited associations with neuro-cognition in an illness state-dependent manner, while the psychopathological factors depression and increased appetite only showed weak associations. Illness severity showed moderating effects on the neurocognitive-psychopathology relationship only for the negative syndrome and positive formal thought disorder.No healthy control subjects were entered into the analyses because of lack of variance in psychopathological symptoms, which prevents from drawing conclusions regarding the relative level of potential cognitive impairments.This study suggests the relationship of neuro-cognition and psychopathology to be highly state of illness-dependent across affective and psychotic disorders. Results hint at the moderating effects of illness severity on psychopathological factors that might be more treatment resistant.

Dysfunctional social communication is one of the most stable characteristics in patients with schizophrenia spectrum disorder (SSD) that severely affects quality of life. Interpreting abstract speech and integrating nonverbal information is particularly affected.Considering the difficulty to treat communication dysfunctions with usual intervention, we investigated the possibility to apply a multimodal speech-gesture (MSG) training.In the MSG training, we offered 8 sessions (60 min each) including perceptive and expressive tasks as well as meta-learning elements and transfer exercises to 29 patients with SSD. In a within-group crossover design, patients were randomized to a TAU-first (treatment as usual first, then MSG training) group (N = 20) or a MSG-first (MSG training first, then TAU only) group (N = 9), and were compared to healthy controls (N = 17). Outcomes were quality of life and related changes in the neural processing of abstract speech-gesture information, which were measured pre-post training through standardized psychological questionnaires and functional Magnetic Resonance Imaging, respectively.Pre-training, patients showed reduced quality of life as compared to controls but improved significantly during the training. Strikingly, this improvement was correlated with neural activation changes in the middle temporal gyrus for the processing of abstract multimodal content. Improvement during training, self-report measures and ratings of relatives confirmed the MSG-related changes.Together, we provide first promising results of a novel multimodal speech-gesture training for patients with schizophrenia. We could link training induced changes in speech-gesture processing to changes in quality of life, demonstrating the relevance of intact communication skills and gesture processing for well-being.

Two eye-tracking experiments were conducted to investigate the effects of visual segmenta-tion, complexity, and context on the cognitive processing of compounds in German Easy Language. By presenting compounds in different boundary conditions, we determined whether a segmentation cue facilitates the processing of compounds presented with and without contextual information. The study was conducted with unimpaired adults and with hearing-impaired pupils, representing one of the target groups of Easy Language. The results indicate that visual segmentation facilitates processing of compounds for pupils with low literacy skills. However, they only benefit from segmentation when morpheme boundaries are marked in a subtle way, i.e., without strikingly deviating from the standard version. Pupils with higher literacy skills and unimpaired adults do not profit from segmentation. Even though hyphenation slows down compound processing for unimpaired readers, initial processing advantages of hyphenated over concatenated compounds emerged, which is explained by the fact that hyphenation forces a morpheme-based access and enables fast recognition of the compound’s first constituent. However, it hinders readers from accessing the compound via the direct route and thus slows down the processing of the compound as a whole. Furthermore, unimpaired readers and hearing-impaired pupils process compounds faster when presented with context.

Introduction: Dysfunctional social communication is one of the most stable characteristics in patients with schizophrenia that severely affects quality of life. Interpreting abstract speech and integrating nonverbal information is particularly affected which has been associated with neural dysfunctions in temporal lobe regions. Objectives: Considering the difficulty to treat communication dysfunctions with usual intervention, we investigated the possibility to improve quality of life and co-verbal gesture processing in patients with schizophrenia by applying a multimodal speech-gesture training (MSG training).Methods: In the MSG training, we offered eight sessions (60 min each) of training including perceptive and expressive tasks as well as meta-learning elements and transfer exercises to 29 patients with schizophrenia spectrum disorder (SSD). Patients were randomized to a waiting-first group (N=20) or a training-first group (N=9), and were compared to healthy controls (N=17). Outcomes were quality of life and related changes in the neural processing of abstract speech-gesture information, which were measured pre-post training through standardized psychological questionnaires and functional Magnetic Resonance Imaging, respectively.Results: Pre-training, patients showed reduced quality of life as compared to controls but improved significantly during the training. Strikingly, this improvement was correlated with neural activation changes in the middle temporal gyrus for the processing of abstract multimodal content. Conclusion: With this study, we provide first promising results of a novel multimodal speech-gesture training for patients with schizophrenia. We could link training induced changes in speech-gesture processing to changes in quality of life, demonstrating the relevance of intact communication skills and gesture processing for well-being.

The ability to generate humor gives rise to positive emotions and thus facilitate the successful resolution of adversity. Although there is consensus that inhibitory processes might be related to broaden the way of thinking, the neural underpinnings of these mechanisms are largely unknown. Here, we use functional Magnetic Resonance Imaging, a humorous alternative uses task and a stroop task, to investigate the brain mechanisms underlying the emergence of humorous ideas in 24 subjects. Neuroimaging results indicate that greater cognitive control abilities are associated with increased activation in the amygdala, the hippocampus and the superior and medial frontal gyrus during the generation of humorous ideas. Examining the neural mechanisms more closely shows that the hypoactivation of frontal brain regions is associated with an hyperactivation in the amygdala and vice versa. This antagonistic connectivity is concurrently linked with an increased number of humorous ideas and enhanced amygdala responses during the task. Our data therefore suggests that a neural antagonism previously related to the emergence and regulation of negative affective responses, is linked with the generation of emotionally positive ideas and may represent an important neural pathway supporting mental health.

Gestures are an integral part of in-person conversations and complement the meaning of the speech they accompany. The neural processing of co-speech gestures is supported by a mostly left-lateralized network of fronto-temporal regions. However, in contrast to iconic gestures, metaphoric as well as unrelated gestures have been found to more strongly engage the left and right inferior frontal gyrus (IFG), respectively. With this study, we conducted the first systematic comparison of all three types of gestures and resulting potential laterality effects. During collection of functional imaging data, 74 subjects were presented with 5 s videos of abstract speech with related metaphoric gestures, concrete speech with related iconic gestures and concrete speech with unrelated gestures. They were asked to judge whether the content of the speech and gesture matched or not. Differential contrasts revealed that both abstract related and concrete unrelated compared to concrete related stimuli elicited stronger activation of the bilateral IFG. Analyses of lateralization indices for IFG activation further showed a left hemispheric dominance for metaphoric gestures and a right hemispheric dominance for unrelated gestures. Our results give support to the hypothesis that the bilateral IFG is activated specifically when processing load for speech-gesture combinations is high. In addition, laterality effects indicate a stronger involvement of the right IFG in mismatch detection and conflict processing, whereas the left IFG performs the actual integration of information from speech and gesture.

Internal states, e.g., emotions, cognitive states, or desires, are often verbalized by figurative means, in particular by embodied metaphors involving human senses, such as touch, taste, and smell. The present paper presents a database for German metaphorical expressions conveying internal states with human senses as their source domains. 168 metaphorical expressions from the source domains of vision, hearing, smell, taste, touch, and temperature combined with literal equivalents were collected and rated by 643 adults. The agreement between the metaphor and an equivalent literal expression, as well as emotional valence, arousal, and familiarity values were assessed using a 7-point Likert scale. Between the metaphorical expressions and their equivalents, familiarity, but not valence or arousal differed significantly while agreement ratings indicated high similarity in meaning. The novel database offers carefully controlled stimuli that can be used in both empirical metaphor research and research on internal state language. Using part of the stimuli in a sentence completion experiment revealed a significant preference for literal over metaphorical expressions that cannot be attributed to higher familiarity levels.

Humor is a ubiquitous human ability with important implications for both social and emotional functioning. Patients with neuropsychiatric disorders, such as schizophrenia, experience difficulties in the comprehension and appreciation of humor. However, the specific neural mechanisms underlying these deficits are unknown. In the current study, we sought to elucidate the neural correlates of humor processing in patients with schizophrenia spectrum disorders (SSD). Functional Magnetic Resonance Imaging (fMRI) was used in thirty-one patients with SSD and a control group, performing a humor processing paradigm. Both regional brain activation and parametric modulation of brain responses via subjective funniness ratings were collected during the scanning procedure. On a neural level, large overlaps in fronto-temporal brain activation was found in both groups. However, patients compared to healthy control subjects showed attenuated responses in frontal brain regions, including the anterior cingulate cortex, medial prefrontal cortex (mPFC), superior frontal gyrus and middle frontal gyrus, as well as in the striatum (i.e. caudate nucleus, putamen). Furthermore, parametric modulation of subjective funniness ratings resulted in attenuated responses in the mPFC, bilateral insula, and left hippocampus. Analysis of functional connectivity revealed alterations in mPFC-caudate nucleus coupling in patients with SSD, which might reflect impairments in reward-related processing. Notably, alterations in mPFC-caudate nucleus coupling in patients were significantly associated to subjective funniness ratings. Our results extend previous findings demonstrating the relevance of frontal hypoactivation for humor processing impairments in patients with SSD and first point towards alterations in humor appreciation processes.

More than half of the world’s population is multilingual, yet it is not known how the human brain encodes the perception of native vs. nonnative speech. To find out, we asked German native speakers to detect the onset of native and nonnative (English and Turkish) vowels in a roving standard stimulation. Using EEG, we show that nonnativeness is robustly registered by an increase in phase coherence in the alpha band (8-12 Hz), beginning as early as ∼100 ms after stimulus onset and lasting more than 200 ms. The alpha band effect is speech-specific, successfully predicts the response speed advantage of nonnative speech, and grants ∼90% decoding accuracy in distinguishing native vs. nonnative speech irrespective of language familiarity. We propose alpha phase coherence as a candidate neural channel for the online resolution of the native-nonnative contrast in the adult brain.Significance StatementWe show that the human brain takes only ∼ 100 ms to distinguish native vs. nonnative speech. This difference is neurally encoded in the alpha band in a reliable and specific way, supporting a processing model in which the mixture of top-down and bottom-up information represented in the speech signal is concurrently partitioned into different frequency channels.

Previous fMRI-studies investigating the production of nouns and verbs in healthy participants reported predominantly activation in the left inferior frontal gyrus (IFG) for both classes of words with increased neural responses for verbs. To date, comparable imaging data for spontaneous speech in patients with schizophrenia is missing. These results are novel and may contribute to understand the neural basis of noun and verb production in a "natural" environment. Fifteen patients with schizophrenia and fifteen healthy control participants described pictures for one minute each while BOLD signal changes were measured with fMRI. In an event-related design, activations related to noun and verb production were extracted in the imaging analysis. Imaging results revealed increased activation for nouns and decreased activation for verbs in the left IFG in the patients. A post-hoc analysis revealed that patients produced significantly more transitive verbs which were negatively associated with activation in the left IFG. We conclude that a subtle linguistic processing deficit in schizophrenia may lead to an increased use of transitive as compared to intransitive verbs in connected speech and to a deviant pattern of brain activation related to the processing of verbs.

How linguistic negation is processed online has been a long-standing issue in psycho- and neurolinguistics. In this study, we investigated negation processing in two distinct types of sentences with truth value evaluation (e.g., ‘A robin is a/not a bird’), and without (e.g., ‘The woman reads a/no book’), focusing on electroencephalogram (EEG) indices in terms of the N400 component and oscillatory power and inter-trial phase coherence (ITPC). Across both sentence constructions, we observed enhanced N400 and increased ITPC in the theta band for semantically unrelated target words, irrespective of negation. However for the theta power, we observed distinct modulation effects of negation on semantic-relatedness for two types of sentence constructions. In addition, we showed that direct comparison between affirmative and negative target words led to effects in the N400 and theta power in a nuanced manner. Our findings provide novel evidence on a more interactive role of negation during online sentence processing.

Highlights • Social/non-social information processing in three modalities was investigated in SZ. • SZ showed reduced activation for social information only in gesture modality. • Reduced activation in SZ was observed for non-social information only in speech. • Neural Neural processing in bimodal condition is not different between patients and controls., Schizophrenia is characterized by marked communication dysfunctions encompassing potential impairments in the processing of social-abstract and non-social-concrete information, especially in everyday situations where multiple modalities are present in the form of speech and gesture. To date, the neurobiological basis of these deficits remains elusive. In a functional magnetic resonance imaging (fMRI) study, 17 patients with schizophrenia or schizoaffective disorder, and 18 matched controls watched videos of an actor speaking, gesturing (unimodal), and both speaking and gesturing (bimodal) about social or non-social events in a naturalistic way. Participants were asked to judge whether each video contains person-related (social) or object-related (non-social) information. When processing social-abstract content, patients showed reduced activation in the medial prefrontal cortex (mPFC) only in the gesture but not in the speech condition. For non-social-concrete content, remarkably, reduced neural activation for patients in the left postcentral gyrus and the right insula was observed only in the speech condition. Moreover, in the bimodal conditions, patients displayed improved task performance and comparable activation to controls in both social and non-social content. To conclude, patients with schizophrenia displayed modality-specific aberrant neural processing of social and non-social information, which is not present for the bimodal conditions. This finding provides novel insights into dysfunctional multimodal communication in schizophrenia, and may have potential therapeutic implications.

Formal thought disorders (FTD) are a hallmark diagnostic feature of schizophrenia (SZ) and (bipolar) mania (MA). FTD can be separated into positive (pFTD) and negative dimensions. It is unclear whether there are differences in pFTD on a single symptom level between acutely ill patients with SZ and MA, which cannot be attributed to cognitive impairment. We compared single pFTD symptoms in two groups of acutely ill patients with ICD-10 bipolar mania and schizophrenia, closely matched for age, sex, pFTD TALD score, verbal IQ and neuropsychological test performance (executive function, verbal fluency, attention, and working memory). SZ patients had higher severity of the TALD symptoms “perseverations” and “poverty of content of speech” than those with MA (Mann–Whitney U, significant, Bonferroni corrected). Speech in acute SZ patients differs from MA in that it conveys little information and adheres to previously mentioned ideas and topics. Matching for confounding variables, such as IQ and cognition, is important when comparing patients with different diagnoses.

In vowel discrimination, commonly found discrimination patterns are directional asymmetries where discrimination is faster (or easier) if differing vowels are presented in a certain sequence compared to the reversed sequence. Different models of speech sound processing try to account for these asymmetries based on either phonetic or phonological properties. In this study, we tested and compared two of those often-discussed models, namely the Featurally Underspecified Lexicon (FUL) model (Lahiri and Reetz, 2002) and the Natural Referent Vowel (NRV) framework (Polka and Bohn, 2011). While most studies presented isolated vowels, we investigated a large stimulus set of German vowels in a more naturalistic setting within minimal pairs. We conducted an mismatch negativity (MMN) study in a passive and a reaction time study in an active oddball paradigm. In both data sets, we found directional asymmetries that can be explained by either phonological or phonetic theories. While behaviorally, the vowel discrimination was based on phonological properties, both tested models failed to explain the found neural patterns comprehensively. Therefore, we additionally examined the influence of a variety of articulatory, acoustical, and lexical factors (e.g., formant structure, intensity, duration, and frequency of occurrence) but also the influence of factors beyond the well-known (perceived loudness of vowels, degree of openness) in depth via multiple regression analyses. The analyses revealed that the perceptual factor of perceived loudness has a greater impact than considered in the literature and should be taken stronger into consideration when analyzing preattentive natural vowel processing.

Impaired social functioning is a hallmark of schizophrenia and altered functional integration between distant brain regions are expected to account for signs and symptoms of the disorder. The functional neuroarchitecture of a network relevant for social functioning, the mentalizing network, is however poorly understood. In this study we examined dysfunctions of the mentalizing network in patients with schizophrenia compared to healthy controls via dynamic causal modelling and an interactive social decision-making game. Network characteristics were analyzed on a single subject basis whereas graph theoretic metrics such as in-degree, out-degree and edge-connectivity per network node were compared between the groups. The results point to a sparser network structure in patients with schizophrenia and highlight the dorsomedial prefrontal cortex as a disconnected network hub receiving significantly less input from other brain regions in the network. Further analyses suggest that integrating pathways from the right and the left temporo-parietal junction into the dorsomedial prefrontal cortex were less frequently found in patients with schizophrenia. Brain and behavior analyses further suggest that the connectivity-intactness within the entire network is associated with functional interpersonal behavior during the task. Thus, the neurobiological alterations within the mentalizing network in patients with schizophrenia point to a specific integration deficit between core brain regions underlying the generation of higher-order representations and thereby provide a potential treatment target.

Body orientation of gesture entails social-communicative intention, and may thus influence how gestures are perceived and comprehended together with auditory speech during face-to-face communication. To date, despite the emergence of neuroscientific literature on the role of body orientation on hand action perception, limited studies have directly investigated the role of body orientation in the interaction between gesture and language. To address this research question, we carried out an EEG experiment presenting to participants (n=21) videos of frontal and lateral hand gestures of five-seconds (e.g., raising a hand), followed by visually presented sentences that are either congruent or incongruent with the hand gesture (e.g., ‘the mountain ishigh/low…’). All participants underwent a semantic-probe task, judging whether a target word is related or unrelated to the gesture-speech event. EEG results suggest that, during the perception phase of hand-gestures, while both frontal and lateral gestures elicited power decrease in both the alpha (8-12Hz) and the beta (16-24Hz) bands, lateral gestures elicited reduced power decrease in the beta band when compared with frontal gestures. For sentence comprehension, at the critical word whose meaning is congruent/incongruent with the gesture, frontal gestures elicited an N400 effect for gesture-sentence incongruency. More importantly, this incongruency effect was significantly reduced for lateral gestures. The findings suggest that body orientation plays a crucial role in gesture perception, and that its inferred social-communicative intention influences gesture-sentence semantic integration in an interactive manner.

Dysfunctional social communication is one of the most stable characteristics in patients with schizophrenia that also affects quality of life. Interpreting abstract speech and integrating nonverbal modalities is particularly affected. Considering the impact of communication on social life but failure to treat communication dysfunctions with usual treatment, we will investigate the possibility to improve verbal and non-verbal communication in schizophrenia by applying a multimodal speech-gesture training (MSG training). Here we describe the newly developed MSG training program and the study design for the first clinical investigation. The intervention contains perceptive rating (match/mismatch of sentence and gesture) and memory tasks (n-back tasks), imitation and productive tasks (e.g., SG fluency—similar to verbal fluency where words are accompanied by gesture). In addition, we offer information about gesture as meta-learning element as well as homework for reasons of transfer to everyday life as part of every session. In the MSG training intervention, we offer eight sessions (60 min each) of training. The first pilot study is currently conducted as a single-center, randomized controlled trial of speech-gesture intervention versus wait-list control with a follow-up. Outcomes are measured through pre-post-fMRI and standardized psychological questionnaires comparing two subject groups (30 patients with schizophrenia and 30 healthy controls). Patients and healthy controls are randomized in two intervention groups (with 20 being in the wait-training group and 10 in the training-follow-up group). With our study design we will be able to demonstrate the beneficial effect of the MSG training intervention on behavioral and neural levels. Clinical Trial Registration DRKS.de, identifier DRKS00015118.

Schizophrenia is associated with impaired and exaggerated Theory of Mind processes, pointing on alterations in generating a representation of another person's mind. Despite recent work on healthy subjects suggesting that a coupling between the right temporoparietal junction (rTPJ) and the hippocampus is relevant for building representations of others' intentions, the neural basis of related dysfunctions in patients with schizophrenia remains unclear. Therefore, we used structural and functional magnetic resonance imaging together with a modified prisoner's dilemma game to test the hypotheses, that patients show dysfunctional social updating on behavioral level accompanied by altered rTPJ-hippocampus coupling on a functional and a structural level. During the task, 31 patients with schizophrenia and 20 healthy controls interacted with 3 playing partners, who behaved according to stable strategies competitively, cooperatively, or randomly. Our data show that patients adapted their social behavior less flexibly to the playing partners than healthy controls, indicating differences in forming mental representations of the counterparts' intentions. Patients showed lower functional connectivity between the rTPJ and temporal lobe regions such as the hippocampus, the fusiform gyrus, and the middle temporal gyrus, indicating that in patients the rTPJ fails to integrate memory-informed processing streams during mental state inferences. Remarkably, the rTPJ-hippocampus coupling accounted for the participants' adaptive social behavior in the task, suggesting that a neural pathway relevant for updating social knowledge and forming forward predictions in social interactions is altered in schizophrenia.

Formal thought disorder (FTD) is a core feature of schizophrenia, a marker of illness severity and a predictor of outcome. The underlying neural mechanisms are still a matter of debate. This study aimed at 1) reviewing the literature on the neural correlates of FTD in schizophrenia, and 2) testing the hypothesis that FTD correlates with structural and functional aberrations in the language network. Medline, PsychInfo, and Embase were searched for neuroimaging studies, which applied a clinical measure to assess FTD in adults with schizophrenia and were published in English or German in peer-reviewed journals until December 2016. Of 412 articles identified, 61 studies were included in the review. Volumetric studies reported bilateral grey matter deficits (L > R) to be associated with FTD in the inferior frontal gyrus, the superior temporal gyrus and the inferior parietal lobe. The same regions showed hyperactivity in resting state functional magnetic resonance imaging (fMRI) studies and both hyper- and hypoactivity in fMRI studies that employed semantic processing or free speech production tasks. Diffusion tensor imaging studies demonstrated white matter aberrations in fibre tracts that connect the frontal and temporo-parietal regions. FTD in schizophrenia was found to be associated with structural and functional aberrations in the language network. However, there are studies that did not find an association between FTD and neural aberrations of the language network and regions not included in the language network have been associated with FTD. Thus, future research is needed to clarify the specificity of the language network for FTD in schizophrenia.

The semantic integration between gesture and speech (GSI) is mediated by the left posterior temporal sulcus/middle temporal gyrus (pSTS/MTG) and the left inferior frontal gyrus (IFG). Evidence from electroencephalography (EEG) suggests that oscillations in the alpha and beta bands may support processes at different stages of GSI. In the present study, we investigated the relationship between electrophysiological oscillations and blood-oxygen-level-dependent (BOLD) activity during GSI. In a simultaneous EEG-fMRI study, German participants (n = 19) were presented with videos of an actor either performing meaningful gestures in the context of a comprehensible German (GG) or incomprehensible Russian sentence (GR), or just speaking a German sentence (SG). EEG results revealed reduced alpha and beta power for the GG vs. SG conditions, while fMRI analyses showed BOLD increase in the left pSTS/MTG for GG > GR ∩ GG > SG. In time-window-based EEG-informed fMRI analyses, we further found a positive correlation between single-trial alpha power and BOLD signal in the left pSTS/MTG, the left IFG, and several sub-cortical regions. Moreover, the alpha-pSTS/MTG correlation was observed in an earlier time window in comparison to the alpha-IFG correlation, thus supporting a two-stage processing model of GSI. Our study shows that EEG-informed fMRI implies multiple roles of alpha oscillations during GSI, and that the method is a best candidate for multidimensional investigations on complex cognitive functions such as GSI.

Identifying someone else's noncooperative intentions can prevent exploitation in social interactions. Hence, the inference of another person's mental state might be most pronounced in order to improve social decision‐making. Here, we tested the hypothesis that brain regions associated with Theory of Mind (ToM), particularly the right temporo–parietal junction (rTPJ), show higher neural responses when interacting with a selfish person and that the rTPJ‐activity as well as cooperative tendencies will change over time. We used functional magnetic resonance imaging (fMRI) and a modified prisoner's dilemma game in which 20 participants interacted with three fictive playing partners who behaved according to stable strategies either competitively, cooperatively or randomly during seven interaction blocks. The rTPJ and the posterior–medial prefrontal cortex showed higher activity during the interaction with a competitive compared with a cooperative playing partner. Only the rTPJ showed a high response during an early interaction phase, which preceded participants increase in defective decisions. Enhanced functional connectivity between the rTPJ and the left hippocampus suggests that social cognition and learning processes co‐occur when behavioral adaptation seems beneficial.

Previous research and theory implicate that personality traits, such as extraversion and neuroticism, influence the processing of humor, as indicated by alterations in the activation of fronto-temporal and mesocorticolimbic brain regions during humor processing. In the current study, we sought to complement these findings by testing whether inter-individual differences in functional connectivity of humor-related brain regions are modulated by stable personality characteristics during humor processing. Using fMRI techniques, we studied 19 healthy subjects during the processing of standardized humorous and neutral cartoons. In order to isolate the specific effects of humor appreciation, subjective funniness ratings, collected during the scanning procedure, were implemented in the analysis as parametric modulation. Two distinct clusters in the right amygdala and the left insula were identified. Seed-to-voxel connectivity analysis investigating the effects of personality on inter-individual differences in functional connectivity revealed that amygdala and insula connectivity with brain areas previously related to humor comprehension (e.g. middle temporal gyrus) and appreciation (e.g. caudate nucleus) were significantly modulated by personality dimensions. These results underscore the sensitivity of humor processing to moderating influences, such as personality, and call attention to the importance of brain connectivity measures for the investigation of inter-individual differences in the processing of humor.

Structural and functional changes in the lateral temporal language areas have been related to formal thought disorder (FTD) in schizophrenia. Continuous, natural speech production activates the right lateral temporal lobe in schizophrenia, as opposed to the left in healthy subjects. Positive and negative FTD can be elicited in healthy subjects by glutamatergic NMDA blockade with ketamine. It is unclear whether the glutamate system is related to the reversed hemispheric lateralization during speaking in patients. In a double-blind, crossover, placebo-controlled study, 15 healthy, male, right-handed volunteers overtly described 7 pictures for 3 min each while BOLD signal changes were acquired with fMRI. As a measure of linguistic demand, the number of words within 20 s epochs was correlated with BOLD responses. Participants developed S-ketamine-induced psychotic symptoms, particularly positive FTD. Ketamine vs placebo was associated with enhanced neural responses in the right middle and inferior temporal gyri. Similar to a previous fMRI study in schizophrenia patients vs healthy controls applying the same design, S-ketamine reversed functional lateralization during speech production in healthy subjects. Results demonstrate an association between glutamatergic imbalance, dysactivations in lateral temporal brain areas, and FTD symptom formation.

While listening to continuous speech, humans process beat information to correctly identify word boundaries. The beats of language are stress patterns that are created by combining lexical (word-specific) stress patterns and the rhythm of a specific language. Sometimes, the lexical stress pattern needs to be altered to obey the rhythm of the language. This study investigated the interplay of lexical stress patterns and rhythmical well-formedness in natural speech with fMRI. Previous electrophysiological studies on cases in which a regular lexical stress pattern may be altered to obtain rhythmical well-formedness showed that even subtle rhythmic deviations are detected by the brain if attention is directed toward prosody. Here, we present a new approach to this phenomenon by having participants listen to contextually rich stories in the absence of a task targeting the manipulation. For the interaction of lexical stress and rhythmical well-formedness, we found one suprathreshold cluster localized between the cerebellum and the brain stem. For the main effect of lexical stress, we found higher BOLD responses to the retained lexical stress pattern in the bilateral SMA, bilateral postcentral gyrus, bilateral middle fontal gyrus, bilateral inferior and right superior parietal lobule, and right precuneus. These results support the view that lexical stress is processed as part of a sensorimotor network of speech comprehension. Moreover, our results connect beat processing in language to domain-independent timing perception.

© 2019 Elsevier Inc. Gestures are elemental components of social communication and aid comprehension of verbal messages; however, little is known about the potential role of gestures in facilitating processing of semantic complexity in an ecologically valid setting. The goal of this study was to investigate whether cognitive load, as indexed by semantic complexity, is modulated by the presentation of gestures accompanying speech. Twenty healthy participants watched 16 video clips of a short narrative while instructed to carefully listen to and watch the narrator while functional magnetic resonance imaging (fMRI) data were acquired. The videos contained passages with and without various co-speech gestures, as well as passages where the semantic complexity was either low or high, as measured by the metric of idea density. Increasing semantic complexity led to reduced activation within the default mode network (DMN); whereas, presents of gestures decreased activation in language-related regions (left middle temporal gyrus and left inferior frontal gyrus) and increased activation in high-level visual and multimodal regions of occipitotemporal cortex. Most interestingly, an interaction between semantic complexity and gestures was observed in a language-related area in left anterior temporal cortex; specifically, increasing gestures led to a greater drop in activation with high vs. Low semantic complexity. These results provide evidence that the facilitation of gestures on semantic processing, particularly for complex narratives, is reflected in the neural substrates of language processing.