Search

This work proposes a novel generative multimodal approach to jointly analyze multimodal data while linking the multimodal information to colors. We apply our proposed framework, which disentangles multimodal data into private and shared sets of features from pairs of structural (sMRI), functional (sFNC and ICA), and diffusion MRI data (FA maps). With our approach, we find that heterogeneity in schizophrenia is potentially a function of modality pairs. Results show (1) schizophrenia is highly multimodal and includes changes in specific networks, (2) non-linear relationships with schizophrenia are observed when interpolating among shared latent dimensions, and (3) we observe a decrease in the modularity of functional connectivity and decreased visual-sensorimotor connectivity for schizophrenia patients for the FA-sFNC and sMRI-sFNC modality pairs, respectively. Additionally, our results generally indicate decreased fractional corpus callosum anisotropy, and decreased spatial ICA map and voxel-based morphometry strength in the superior frontal lobe as found in the FA-sFNC, sMRI-FA, and sMRI-ICA modality pair clusters. In sum, we introduce a powerful new multimodal neuroimaging framework designed to provide a rich and intuitive understanding of the data which we hope challenges the reader to think differently about how modalities interact.

Resting-state functional network connectivity (rsFNC) has shown utility for identifying characteristic functional brain patterns in individuals with psychiatric and mood disorders, providing a promising avenue for biomarker development. However, several factors have precluded widespread clinical adoption of rsFNC diagnostics, namely a lack of standardized approaches for capturing comparable and reproducible imaging markers across individuals, as well as the disagreement on the amount of data required to robustly detect intrinsic connectivity networks (ICNs) and diagnostically relevant patterns of rsFNC at the individual subject level. Recently, spatially constrained independent component analysis (scICA) has been proposed as an automated method for extracting ICNs standardized to a chosen network template while still preserving individual variation. Leveraging the scICA methodology, which solves the former challenge of standardized neuroimaging markers, we investigate the latter challenge of identifying a minimally sufficient data length for clinical applications of resting-state fMRI (rsfMRI). Using a dataset containing rsfMRI scans of individuals with schizophrenia and controls (M = 310) as well as simulated rsfMRI, we evaluated the robustness of ICN and rsFNC estimates at both the subject- and group-level, as well as the performance of diagnostic classification, with respect to the length of the rsfMRI time course. We found individual estimates of ICNs and rsFNC from the full-length (5 min) reference time course were sufficiently approximated with just 3-3.5 min of data (r = 0.85, 0.88, respectively), and significant differences in group-average rsFNC could be sufficiently approximated with even less data, just 2 min (r = 0.86). These results from the shorter clinical data were largely consistent with the results from validation experiments using longer time series from both simulated (30 min) and real-world (14 min) datasets, in which estimates of subject-level FNC were reliably estimated with 3-5 min of data. Moreover, in the real-world data we found rsFNC and ICN estimates generated across the full range of data lengths (0.5-14 min) more reliably matched those generated from the first 5 min of scan time than those generated from the last 5 min, suggesting increased influence of "late scan" noise factors such as fatigue or drowsiness may limit the reliability of FNC from data collected after 10+ min of scan time, further supporting the notion of shorter scans. Lastly, a diagnostic classification model trained on just 2 min of data retained 97%-98% classification accuracy relative to that of the full-length reference model. Our results suggest that, when decomposed with scICA, rsfMRI scans of just 2-5 min show good clinical utility without significant loss of individual FNC information of longer scan lengths.

Characterizing neuropsychiatric disorders is challenging due to heterogeneity in the population. We propose combining structural and functional neuroimaging and genomic data in a multimodal classification framework to leverage their complementary information. Our objectives are two-fold (i) to improve the classification of disorders and (ii) to introspect the concepts learned to explore underlying neural and biological mechanisms linked to mental disorders. Previous multimodal studies have focused on naïve neural networks, mostly perceptron, to learn modality-wise features and often assume equal contribution from each modality. Our focus is on the development of neural networks for feature learning and implementing an adaptive control unit for the fusion phase. Our mid fusion with attention model includes a multilayer feed-forward network, an autoencoder, a bi-directional long short-term memory unit with attention as the features extractor, and a linear attention module for controlling modality-specific influence. The proposed model acquired 92% (p

In this work, we focus on explicitly nonlinear relationships in functional networks. We introduce a technique using normalized mutual information (NMI) that calculates the nonlinear relationship between different brain regions. We demonstrate our proposed approach using simulated data and then apply it to a dataset previously studied by Damaraju et al. This resting-state fMRI data included 151 schizophrenia patients and 163 age- and gender-matched healthy controls. We first decomposed these data using group independent component analysis (ICA) and yielded 47 functionally relevant intrinsic connectivity networks. Our analysis showed a modularized nonlinear relationship among brain functional networks that was particularly noticeable in the sensory and visual cortex. Interestingly, the modularity appears both meaningful and distinct from that revealed by the linear approach. Group analysis identified significant differences in explicitly nonlinear functional network connectivity (FNC) between schizophrenia patients and healthy controls, particularly in the visual cortex, with controls showing more nonlinearity (i.e., higher normalized mutual information between time courses with linear relationships removed) in most cases. Certain domains, including subcortical and auditory, showed relatively less nonlinear FNC (i.e., lower normalized mutual information), whereas links between the visual and other domains showed evidence of substantial nonlinear and modular properties. Overall, these results suggest that quantifying nonlinear dependencies of functional connectivity may provide a complementary and potentially important tool for studying brain function by exposing relevant variation that is typically ignored. Beyond this, we propose a method that captures both linear and nonlinear effects in a "boosted" approach. This method increases the sensitivity to group differences compared to the standard linear approach, at the cost of being unable to separate linear and nonlinear effects.

Schizophrenia (SZ) and autism spectrum disorder (ASD) sharing overlapping symptoms have a long history of diagnostic confusion. It is unclear what their differences at a brain level are. Here, we propose a multimodality fusion classification approach to investigate their divergence in brain function and structure. Using brain functional network connectivity (FNC) calculated from resting-state fMRI data and gray matter volume (GMV) estimated from sMRI data, we classify the two disorders using the main data (335 SZ and 380 ASD patients) via an unbiased 10-fold cross-validation pipeline, and also validate the classification generalization ability on an independent cohort (120 SZ and 349 ASD patients). The classification accuracy reached up to 83.08% for the testing data and 72.10% for the independent data, significantly better than the results from using the single-modality features. The discriminative FNCs that were automatically selected primarily involved the sub-cortical, default mode, and visual domains. Interestingly, all discriminative FNCs relating to the default mode network showed an intermediate strength in healthy controls (HCs) between SZ and ASD patients. Their GMV differences were mainly driven by the frontal gyrus, temporal gyrus, and insula. Regarding these regions, the mean GMV of HC fell intermediate between that of SZ and ASD, and ASD showed the highest GMV. The middle frontal gyrus was associated with both functional and structural differences. In summary, our work reveals the unique neuroimaging characteristics of SZ and ASD that can achieve high and generalizable classification accuracy, supporting their potential as disorder-specific neural substrates of the two entwined disorders.

The Enhancing NeuroImaging Genetics through Meta-Analysis copy number variant (ENIGMA-CNV) and 22q11.2 Deletion Syndrome Working Groups (22q-ENIGMA WGs) were created to gain insight into the involvement of genetic factors in human brain development and related cognitive, psychiatric and behavioral manifestations. To that end, the ENIGMA-CNV WG has collated CNV and magnetic resonance imaging (MRI) data from ~49,000 individuals across 38 global research sites, yielding one of the largest studies to date on the effects of CNVs on brain structures in the general population. The 22q-ENIGMA WG includes 12 international research centers that assessed over 533 individuals with a confirmed 22q11.2 deletion syndrome, 40 with 22q11.2 duplications, and 333 typically developing controls, creating the largest-ever 22q11.2 CNV neuroimaging data set. In this review, we outline the ENIGMA infrastructure and procedures for multi-site analysis of CNVs and MRI data. So far, ENIGMA has identified effects of the 22q11.2, 16p11.2 distal, 15q11.2, and 1q21.1 distal CNVs on subcortical and cortical brain structures. Each CNV is associated with differences in cognitive, neurodevelopmental and neuropsychiatric traits, with characteristic patterns of brain structural abnormalities. Evidence of gene-dosage effects on distinct brain regions also emerged, providing further insight into genotype-phenotype relationships. Taken together, these results offer a more comprehensive picture of molecular mechanisms involved in typical and atypical brain development. This "genotype-first" approach also contributes to our understanding of the etiopathogenesis of brain disorders. Finally, we outline future directions to better understand effects of CNVs on brain structure and behavior.

Schizophrenia is associated with widespread alterations in subcortical brain structure. While analytic methods have enabled more detailed morphometric characterization, findings are often equivocal. In this meta-analysis, we employed the harmonized ENIGMA shape analysis protocols to collaboratively investigate subcortical brain structure shape differences between individuals with schizophrenia and healthy control participants. The study analyzed data from 2,833 individuals with schizophrenia and 3,929 healthy control participants contributed by 21 worldwide research groups participating in the ENIGMA Schizophrenia Working Group. Harmonized shape analysis protocols were applied to each site's data independently for bilateral hippocampus, amygdala, caudate, accumbens, putamen, pallidum, and thalamus obtained from T1-weighted structural MRI scans. Mass univariate meta-analyses revealed more-concave-than-convex shape differences in the hippocampus, amygdala, accumbens, and thalamus in individuals with schizophrenia compared with control participants, more-convex-than-concave shape differences in the putamen and pallidum, and both concave and convex shape differences in the caudate. Patterns of exaggerated asymmetry were observed across the hippocampus, amygdala, and thalamus in individuals with schizophrenia compared to control participants, while diminished asymmetry encompassed ventral striatum and ventral and dorsal thalamus. Our analyses also revealed that higher chlorpromazine dose equivalents and increased positive symptom levels were associated with patterns of contiguous convex shape differences across multiple subcortical structures. Findings from our shape meta-analysis suggest that common neurobiological mechanisms may contribute to gray matter reduction across multiple subcortical regions, thus enhancing our understanding of the nature of network disorganization in schizophrenia.

The problem of poor reproducibility of scientific findings has received much attention over recent years, in a variety of fields including psychology and neuroscience. The problem has been partly attributed to publication bias and unwanted practices such as p-hacking. Low statistical power in individual studies is also understood to be an important factor. In a recent multisite collaborative study, we mapped brain anatomical left-right asymmetries for regional measures of surface area and cortical thickness, in 99 MRI datasets from around the world, for a total of over 17,000 participants. In the present study, we revisited these hemispheric effects from the perspective of reproducibility. Within each dataset, we considered that an effect had been reproduced when it matched the meta-analytic effect from the 98 other datasets, in terms of effect direction and significance threshold. In this sense, the results within each dataset were viewed as coming from separate studies in an "ideal publishing environment," that is, free from selective reporting and p hacking. We found an average reproducibility rate of 63.2% (SD = 22.9%, min = 22.2%, max = 97.0%). As expected, reproducibility was higher for larger effects and in larger datasets. Reproducibility was not obviously related to the age of participants, scanner field strength, FreeSurfer software version, cortical regional measurement reliability, or regional size. These findings constitute an empirical illustration of reproducibility in the absence of publication bias or p hacking, when assessing realistic biological effects in heterogeneous neuroscience data, and given typically-used sample sizes.

Deep learning methods hold strong promise for identifying biomarkers for clinical application. However, current approaches for psychiatric classification or prediction do not allow direct interpretation of original features. In the present study, we introduce a sparse deep neural network (DNN) approach to identify sparse and interpretable features for schizophrenia (SZ) case-control classification. An L0 -norm regularization is implemented on the input layer of the network for sparse feature selection, which can later be interpreted based on importance weights. We applied the proposed approach on a large multi-study cohort with gray matter volume (GMV) and single nucleotide polymorphism (SNP) data for SZ classification. A total of 634 individuals served as training samples, and the classification model was evaluated for generalizability on three independent datasets of different scanning protocols (N = 394, 255, and 160, respectively). We examined the classification power of pure GMV features, as well as combined GMV and SNP features. Empirical experiments demonstrated that sparse DNN slightly outperformed independent component analysis + support vector machine (ICA + SVM) framework, and more effectively fused GMV and SNP features for SZ discrimination, with an average error rate of 28.98% on external data. The importance weights suggested that the DNN model prioritized to select frontal and superior temporal gyrus for SZ classification with high sparsity, with parietal regions further included with lower sparsity, echoing previous literature. The results validate the application of the proposed approach to SZ classification, and promise extended utility on other data modalities and traits which ultimately may result in clinically useful tools.

Electroconvulsive therapy is regarded as the most effective antidepressant treatment for severe and treatment-resistant depressive episodes. Despite the efficacy of electroconvulsive therapy, the neurobiological underpinnings and mechanisms underlying electroconvulsive therapy induced antidepressant effects remain unclear. The objective of this investigation was to identify electroconvulsive therapy treatment responsive multimodal biomarkers with the 17-item Hamilton Depression Rating Scale guided brain structure-function fusion in 118 patients with depressive episodes and 60 healthy controls. Results show that reduced fractional amplitude of low frequency fluctuations in the prefrontal cortex, insula and hippocampus, linked with increased gray matter volume in anterior cingulate, medial temporal cortex, insula, thalamus, caudate and hippocampus represent electroconvulsive therapy responsive covarying functional and structural brain networks. In addition, relative to nonresponders, responder-specific electroconvulsive therapy related brain networks occur in frontal-limbic network and are associated with successful therapeutic outcomes. Finally, electroconvulsive therapy responsive brain networks were unrelated to verbal declarative memory. Using a data-driven, supervised-learning method, we demonstrated that electroconvulsive therapy produces a remodeling of brain functional and structural covariance that was unique to antidepressant symptom response, but not linked to memory impairment.

There is growing evidence that rather than using a single brain imaging modality to study its association with physiological or symptomatic features, the field is paying more attention to fusion of multimodal information. However, most current multimodal fusion approaches that incorporate functional magnetic resonance imaging (fMRI) are restricted to second-level 3D features, rather than the original 4D fMRI data. This trade-off is that the valuable temporal information is not utilized during the fusion step. Here we are motivated to propose a novel approach called "parallel group ICA+ICA" that incorporates temporal fMRI information from group independent component analysis (GICA) into a parallel independent component analysis (ICA) framework, aiming to enable direct fusion of first-level fMRI features with other modalities (e.g., structural MRI), which thus can detect linked functional network variability and structural covariations. Simulation results show that the proposed method yields accurate intermodality linkage detection regardless of whether it is strong or weak. When applied to real data, we identified one pair of significantly associated fMRI-sMRI components that show group difference between schizophrenia and controls in both modalities, and this linkage can be replicated in an independent cohort. Finally, multiple cognitive domain scores can be predicted by the features identified in the linked component pair by our proposed method. We also show these multimodal brain features can predict multiple cognitive scores in an independent cohort. Overall, results demonstrate the ability of parallel GICA+ICA to estimate joint information from 4D and 3D data without discarding much of the available information up front, and the potential for using this approach to identify imaging biomarkers to study brain disorders.

The brain is highly dynamic, reorganizing its activity at different interacting spatial and temporal scales, including variation within and between brain networks. The chronnectome is a model of the brain in which nodal activity and connectivity patterns change in fundamental and recurring ways over time. Most literature assumes fixed spatial nodes/networks, ignoring the possibility that spatial nodes/networks may vary in time. Here, we introduce an approach to calculate a spatially fluid chronnectome (called the spatial chronnectome for clarity), which focuses on the variations of networks coupling at the voxel level, and identify a novel set of spatially dynamic features. Results reveal transient spatially fluid interactions between intra- and internetwork relationships in which brain networks transiently merge and separate, emphasizing dynamic segregation and integration. Brain networks also exhibit distinct spatial patterns with unique temporal characteristics, potentially explaining a broad spectrum of inconsistencies in previous studies that assumed static networks. Moreover, we show anticorrelative connections to brain networks are transient as opposed to constant across the entire scan. Preliminary assessments using a multi-site dataset reveal the ability of the approach to obtain new information and nuanced alterations that remain undetected during static analysis. Patients with schizophrenia (SZ) display transient decreases in voxel-wise network coupling within visual and auditory networks, and higher intradomain coupling variability. In summary, the spatial chronnectome represents a new direction of research enabling the study of functional networks which are transient at the voxel level, and the identification of mechanisms for within- and between-subject spatial variability.

The analysis of time-varying activity and connectivity patterns (i.e., the chronnectome) using resting-state magnetic resonance imaging has become an important part of ongoing neuroscience discussions. The majority of previous work has focused on variations of temporal coupling among fixed spatial nodes or transition of the dominant activity/connectivity pattern over time. Here, we introduce an approach to capture spatial dynamics within functional domains (FDs), as well as temporal dynamics within and between FDs. The approach models the brain as a hierarchical functional architecture with different levels of granularity, where lower levels have higher functional homogeneity and less dynamic behavior and higher levels have less homogeneity and more dynamic behavior. First, a high-order spatial independent component analysis is used to approximate functional units. A functional unit is a pattern of regions with very similar functional activity over time. Next, functional units are used to construct FDs. Finally, functional modules (FMs) are calculated from FDs, providing an overall view of brain dynamics. Results highlight the spatial fluidity within FDs, including a broad spectrum of changes in regional associations, from strong coupling to complete decoupling. Moreover, FMs capture the dynamic interplay between FDs. Patients with schizophrenia show transient reductions in functional activity and state connectivity across several FDs, particularly the subcortical domain. Activity and connectivity differences convey unique information in many cases (e.g., the default mode) highlighting their complementarity information. The proposed hierarchical model to capture FD spatiotemporal variations provides new insight into the macroscale chronnectome and identifies changes hidden from existing approaches.

Despite increasing interest in the dynamics of functional brain networks, most studies focus on the changing relationships over time between spatially static networks or regions. Here we propose an approach to study dynamic spatial brain networks in human resting state functional magnetic resonance imaging (rsfMRI) data and evaluate the temporal changes in the volumes of these 4D networks. Our results show significant volumetric coupling (i.e., synchronized shrinkage and growth) between networks during the scan, that we refer to as dynamic spatial network connectivity (dSNC). We find that several features of such dynamic spatial brain networks are associated with cognition, with higher dynamic variability in these networks and higher volumetric coupling between network pairs positively associated with cognitive performance. We show that these networks are modulated differently in individuals with schizophrenia versus typical controls, resulting in network growth or shrinkage, as well as altered focus of activity within a network. Schizophrenia also shows lower spatial dynamical variability in several networks, and lower volumetric coupling between pairs of networks, thus upholding the role of dynamic spatial brain networks in cognitive impairment seen in schizophrenia. Our data show evidence for the importance of studying the typically overlooked voxel‐wise changes within and between brain networks. [ABSTRACT FROM AUTHOR]

Testosterone levels sharply rise during the transition from childhood to adolescence and these changes are known to be associated with changes in human brain structure. During this same developmental window, there are also robust changes in the neural oscillatory dynamics serving verbal working memory processing. Surprisingly, whereas many studies have investigated the effects of chronological age on the neural oscillations supporting verbal working memory, none have probed the impact of endogenous testosterone levels during this developmental period. Using a sample of 89 youth aged 6–14 years‐old, we collected salivary testosterone samples and recorded magnetoencephalography during a modified Sternberg verbal working memory task. Significant oscillatory responses were identified and imaged using a beamforming approach and the resulting maps were subjected to whole‐brain ANCOVAs examining the effects of testosterone and sex, controlling for age, during verbal working memory encoding and maintenance. Our primary results indicated robust testosterone‐related effects in theta (4–7 Hz) and alpha (8–14 Hz) oscillatory activity, controlling for age. During encoding, females exhibited weaker theta oscillations than males in right cerebellar cortices and stronger alpha oscillations in left temporal cortices. During maintenance, youth with greater testosterone exhibited weaker alpha oscillations in right parahippocampal and cerebellar cortices, as well as regions across the left‐lateralized language network. These results extend the existing literature on the development of verbal working memory processing by showing region and sex‐specific effects of testosterone, and are the first results to link endogenous testosterone levels to the neural oscillatory activity serving verbal working memory, above and beyond the effects of chronological age. [ABSTRACT FROM AUTHOR]

Brain networks extracted by independent component analysis (ICA) from magnitude‐only fMRI data are usually denoised using various amplitude‐based thresholds. By contrast, spatial source phase (SSP) or the phase information of ICA brain networks extracted from complex‐valued fMRI data, has provided a simple yet effective way to perform the denoising using a fixed phase change. In this work, we extend the approach to magnitude‐only fMRI data to avoid testing various amplitude thresholds for denoising magnitude maps extracted by ICA, as most studies do not save the complex‐valued data. The main idea is to generate a mathematical SSP map for a magnitude map using a mapping framework, and the mapping framework is built using complex‐valued fMRI data with a known SSP map. Here we leverage the fact that the phase map derived from phase fMRI data has similar phase information to the SSP map. After verifying the use of the magnitude data of complex‐valued fMRI, this framework is generalized to work with magnitude‐only data, allowing use of our approach even without the availability of the corresponding phase fMRI datasets. We test the proposed method using both simulated and experimental fMRI data including complex‐valued data from University of New Mexico and magnitude‐only data from Human Connectome Project. The results provide evidence that the mathematical SSP denoising with a fixed phase change is effective for denoising spatial maps from magnitude‐only fMRI data in terms of retaining more BOLD‐related activity and fewer unwanted voxels, compared with amplitude‐based thresholding. The proposed method provides a unified and efficient SSP approach to denoise ICA brain networks in fMRI data. [ABSTRACT FROM AUTHOR]

The study of human brain connectivity, including structural connectivity (SC) and functional connectivity (FC), provides insights into the neurophysiological mechanism of brain function and its relationship to human behavior and cognition. Both types of connectivity measurements provide crucial yet complementary information. However, integrating these two modalities into a single framework remains a challenge, because of the differences in their quantitative interdependencies as well as their anatomical representations due to distinctive imaging mechanisms. In this study, we introduced a new method, joint connectivity matrix independent component analysis (cmICA), which provides a data‐driven parcellation and automated‐linking of SC and FC information simultaneously using a joint analysis of functional magnetic resonance imaging (MRI) and diffusion‐weighted MRI data. We showed that these two connectivity modalities produce common cortical segregation, though with various degrees of (dis)similarity. Moreover, we show conjoint FC networks and structural white matter tracts that directly link these cortical parcellations/sources, within one analysis. Overall, data‐driven joint cmICA provides a new approach for integrating or fusing structural connectivity and FC systematically and conveniently, and provides an effective tool for connectivity‐based multimodal data fusion in brain. [ABSTRACT FROM AUTHOR]