Search

Understanding how and where in the brain sentence-level meaning is constructed from words presents a major scientific challenge. Recent advances have begun to explain brain activation elicited by sentences using vector models of word meaning derived from patterns of word co-occurrence in text corpora. These studies have helped map out semantic representation across a distributed brain network spanning temporal, parietal, and frontal cortex. However, it remains unclear whether activation patterns within regions reflect unified representations of sentence-level meaning, as opposed to superpositions of context-independent component words. This is because models have typically represented sentences as "bags-of-words" that neglect sentence-level structure. To address this issue, we interrogated fMRI activation elicited as 240 sentences were read by 14 participants (9 female, 5 male), using sentences encoded by a recurrent deep artificial neural-network trained on a sentence inference task (InferSent). Recurrent connections and nonlinear filters enable InferSent to transform sequences of word vectors into unified "propositional" sentence representations suitable for evaluating intersentence entailment relations. Using voxelwise encoding modeling, we demonstrate that InferSent predicts elements of fMRI activation that cannot be predicted by bag-of-words models and sentence models using grammatical rules to assemble word vectors. This effect occurs throughout a distributed network, which suggests that propositional sentence-level meaning is represented within and across multiple cortical regions rather than at any single site. In follow-up analyses, we place results in the context of other deep network approaches (ELMo and BERT) and estimate the degree of unpredicted neural signal using an "experiential" semantic model and cross-participant encoding. [ABSTRACT FROM AUTHOR]

Objective: To define left temporal lobe regions where surgical resection produces a persistent postoperative decline in naming visual objects. Methods: Pre‐ and postoperative brain magnetic resonance imaging data and picture naming (Boston Naming Test) scores were obtained prospectively from 59 people with drug‐resistant left temporal lobe epilepsy. All patients had left hemisphere language dominance at baseline and underwent surgical resection or ablation in the left temporal lobe. Postoperative naming assessment occurred approximately 7 months after surgery. Surgical lesions were mapped to a standard template, and the relationship between presence or absence of a lesion and the degree of naming decline was tested at each template voxel while controlling for effects of overall lesion size. Results: Patients declined by an average of 15% in their naming score, with wide variation across individuals. Decline was significantly related to damage in a cluster of voxels in the ventral temporal lobe, located mainly in the fusiform gyrus approximately 4‐6 cm posterior to the temporal tip. Extent of damage to this region explained roughly 50% of the variance in outcome. Picture naming decline was not related to hippocampal or temporal pole damage. Significance: The results provide the first statistical map relating lesion location in left temporal lobe epilepsy surgery to picture naming decline, and they support previous observations of transient naming deficits from electrical stimulation in the basal temporal cortex. The critical lesion is relatively posterior and could be avoided in many patients undergoing left temporal lobe surgery for intractable epilepsy. [ABSTRACT FROM AUTHOR]

The brain is thought to combine linguistic knowledge of words and nonlinguistic knowledge of their referents to encode sentence meaning. However, functional neuroimaging studies aiming at decoding language meaning from neural activity have mostly relied on distributional models of word semantics, which are based on patterns of word co-occurrence in text corpora. Here, we present initial evidence that modeling nonlinguistic "experiential" knowledge contributes to decoding neural representations of sentence meaning. We model attributes of peoples' sensory, motor, social, emotional, and cognitive experiences with words using behavioral ratings. We demonstrate that fMRI activation elicited in sentence reading is more accurately decoded when this experiential attribute model is integrated with a text-based model than when either model is applied in isolation (participants were 5 males and 9 females). Our decoding approach exploits a representation-similarity-based framework, which benefits from being parameter free, while performing at accuracy levels comparable with those from parameter fitting approaches, such as ridge regression. We find that the text-based model contributes particularly to the decoding of sentences containing linguistically oriented "abstract" words and reveal tentative evidence that the experiential model improves decoding of more concrete sentences. Finally, we introduce a cross-participant decoding method to estimate an upper bound on model-based decoding accuracy. We demonstrate that a substantial fraction of neural signal remains unexplained, and leverage this gap to pinpoint characteristics of weakly decoded sentences and hence identify model weaknesses to guide future model development. [ABSTRACT FROM AUTHOR]

The capacity to process information in conceptual form is a fundamental aspect of human cognition, yet little is known about how this type of information is encoded in the brain. Although the role of sensory and motor cortical areas has been a focus of recent debate, neuroimaging studies of concept representation consistently implicate a network of heteromodal areas that seem to support concept retrieval in general rather than knowledge related to any particular sensory-motor content. We used predictive machine learning on fMRI data to investigate the hypothesis that cortical areas in this "general semantic network" (GSN) encode multimodal information derived from basic sensory-motor processes, possibly functioning as convergence-divergence zones for distributed concept representation. An encoding model based on five conceptual attributes directly related to sensory-motor experience (sound, color, shape, manipulability, and visual motion) was used to predict brain activation patterns associated with individual lexical concepts in a semantic decision task. When the analysis was restricted to voxels in the GSN, the model was able to identify the activation patterns corresponding to individual concrete concepts significantly above chance. In contrast, a model based on five perceptual attributes of the word form performed at chance level. This pattern was reversed when the analysis was restricted to areas involved in the perceptual analysis of written word forms. These results indicate that heteromodal areas involved in semantic processing encode information about the relative importance of different sensory-motor attributes of concepts, possibly by storing particular combinations of sensory and motor features. [ABSTRACT FROM AUTHOR]

Componential theories of lexical semantics assume that concepts can be represented by sets of features or attributes that are in some sense primitive or basic components of meaning. The binary features used in classical category and prototype theories are problematic in that these features are themselves complex concepts, leaving open the question of what constitutes a primitive feature. The present availability of brain imaging tools has enhanced interest in how concepts are represented in brains, and accumulating evidence supports the claim that these representations are at least partly “embodied” in the perception, action, and other modal neural systems through which concepts are experienced. In this study we explore the possibility of devising a componential model of semantic representation based entirely on such functional divisions in the human brain. We propose a basic set of approximately 65 experiential attributes based on neurobiological considerations, comprising sensory, motor, spatial, temporal, affective, social, and cognitive experiences. We provide normative data on the salience of each attribute for a large set of English nouns, verbs, and adjectives, and show how these attribute vectors distinguish a priori conceptual categories and capture semantic similarity. Robust quantitative differences between concrete object categories were observed across a large number of attribute dimensions. A within- versus between-category similarity metric showed much greater separation between categories than representations derived from distributional (latent semantic) analysis of text. Cluster analyses were used to explore the similarity structure in the data independent of a priori labels, revealing several novel category distinctions. We discuss how such a representation might deal with various longstanding problems in semantic theory, such as feature selection and weighting, representation of abstract concepts, effects of context on semantic retrieval, and conceptual combination. In contrast to componential models based on verbal features, the proposed representation systematically relates semantic content to large-scale brain networks and biologically plausible accounts of concept acquisition. [ABSTRACT FROM AUTHOR]

Patterns of resting state connectivity change dynamically and may represent modes of cognitive information processing. The diversity of connectivity patterns (global brain states) reflects the information capacity of the brain and determines the state of consciousness. In this work, computer simulation was used to explore the repertoire of global brain states as a function of cortical activation level. We implemented a modified spin glass model to describe UP/DOWN state transitions of neuronal populations at a mesoscopic scale based on resting state BOLD fMRI data. Resting state fMRI was recorded in 20 participants and mapped to 10,000 cortical regions (sites) defined on a group-aligned cortical surface map. Each site represented the population activity of a ∼20mm2 area of the cortex. Cross-correlation matrices of the mapped BOLD time courses of the set of sites were calculated and averaged across subjects. In the model, each cortical site was allowed to interact with the 16 other sites that had the highest pair-wise correlation values. All sites stochastically transitioned between UP and DOWN states under the net influence of their 16 pairs. The probability of local state transitions was controlled by a single parameter T corresponding to the level of global cortical activation. To estimate the number of distinct global states, first we ran 10,000 simulations at T = 0. Simulations were started from random configurations that converged to one of several distinct patterns. Using hierarchical clustering, at 99% similarity, close to 300 distinct states were found. At intermediate T, metastable state configurations were formed suggesting critical behavior with a sharp increase in the number of metastable states at an optimal T. Both reduced activation (anesthesia, sleep) and increased activation (hyper-activation) moved the system away from equilibrium, presumably incompatible with conscious mentation. During equilibrium, the diversity of large-scale brain states was maximum, compatible with maximum information capacity-a presumed condition of consciousness. [ABSTRACT FROM AUTHOR]

The National Institutes of Health-sponsored Epilepsy Connectome Project aims to characterize connectivity changes in temporal lobe epilepsy (TLE) patients. The magnetic resonance imaging protocol follows that used in the Human Connectome Project, and includes 20 min of resting-state functional magnetic resonance imaging acquired at 3T using 8-band multiband imaging. Glasser parcellation atlas was combined with the FreeSurfer subcortical regions to generate resting-state functional connectivity (RSFC), amplitude of low-frequency fluctuations (ALFFs), and fractional ALFF measures. Seven different frequency ranges such as Slow-5 (0.01-0.027 Hz) and Slow-4 (0.027-0.073 Hz) were selected to compute these measures. The goal was to train machine learning classification models to discriminate TLE patients from healthy controls, and to determine which combination of the resting state measure and frequency range produced the best classification model. The samples included age- and gender-matched groups of 60 TLE patients and 59 healthy controls. Three traditional machine learning models were trained: support vector machine, linear discriminant analysis, and naive Bayes classifier. The highest classification accuracy was obtained using RSFC measures in the Slow-4 + 5 band (0.01-0.073 Hz) as features. Leave-one-out cross-validation accuracies were ∼83%, with receiver operating characteristic area-under-the-curve reaching close to 90%. Increased connectivity from right area posterior 9-46v in TLE patients contributed to the high accuracies. With increased sample sizes in the near future, better machine learning models will be trained not only to aid the diagnosis of TLE, but also as a tool to understand this brain disorder.