Search

Brain connectivity can be estimated in many ways, depending on modality and processing strategy. Here we present the Krakencoder, a joint connectome mapping tool that simultaneously, bidirectionally translates between structural (SC) and functional connectivity (FC), and across different atlases and processing choices via a common latent representation. These mappings demonstrate unprecedented accuracy and individual-level identifiability; the mapping between SC and FC has identifiability 42-54% higher than existing models. The Krakencoder combines all connectome flavors via a shared low-dimensional latent space. This "fusion" representation i) better reflects familial relatedness, ii) preserves age- and sex-relevant information and iii) enhances cognition-relevant information. The Krakencoder can be applied without retraining to new, out-of-age-distribution data while still preserving inter-individual differences in the connectome predictions and familial relationships in the latent representations. The Krakencoder is a significant leap forward in capturing the relationship between multi-modal brain connectomes in an individualized, behaviorally- and demographically-relevant way., Competing Interests: Competing interests The authors declare no competing interests.

Despite the advantage of neuroimaging-based machine learning (ML) models as pivotal tools for investigating brain-behavior relationships in neuropsychiatric studies, these data-driven predictive approaches have yet to yield substantial, clinically actionable insights for mental health care. A notable impediment lies in the inadequate accommodation of most ML research to the natural heterogeneity within large samples. Although commonly thought of as individual-level analyses, many ML algorithms are unimodal and homogeneous and thus incapable of capturing the potentially heterogeneous relationships between biology and psychopathology. We review the current landscape of computational research targeting population heterogeneity and argue that there is a need to expand from brain subtyping and behavioral phenotyping to analyses that focus on heterogeneity at the relational level. To this end, we review and suggest several existing ML models with the capacity to discern how external environmental and sociodemographic factors moderate the brain-behavior mapping function in a data-driven fashion. These heterogeneous ML models hold promise for enhancing the discovery of individualized brain-behavior associations and advancing precision psychiatry., (© 2024 The Authors.)

Heart failure (HF) affects millions of individuals and causes hundreds of thousands of deaths each year in the United States. Despite the public health burden, medical and device therapies for HF significantly improve clinical outcomes and, in a subset of patients, can cause reversal of abnormalities in cardiac structure and function, termed "myocardial recovery." By identifying novel patterns in high-dimensional data, artificial intelligence (AI) and machine learning (ML) algorithms can enhance the identification of key predictors and molecular drivers of myocardial recovery. Emerging research in the area has begun to demonstrate exciting results that could advance the standard of care. Although major obstacles remain to translate this technology to clinical practice, AI and ML hold the potential to usher in a new era of purposeful myocardial recovery programs based on precision medicine. In this review, we discuss applications of ML to the prediction of myocardial recovery, potential roles of ML in elucidating the mechanistic basis underlying recovery, barriers to the implementation of ML in clinical practice, and areas for future research., Competing Interests: FML reports no conflicts related to this work. Outside of this work, FML reports equity options in Roivant Sciences and consulting fees from Roivant Sciences and Kinevant Sciences. The other authors have no conflicts to disclose., (Copyright: © 2024 The Author(s).)

Rationale and Objectives: Following autosomal dominant polycystic kidney disease (ADPKD) progression by measuring organ volumes requires low measurement variability. The objective of this study is to reduce organ volume measurement variability on MRI of ADPKD patients by utilizing all pulse sequences to obtain multiple measurements which allows outlier analysis to find errors and averaging to reduce variability., Materials and Methods: In order to make measurements on multiple pulse sequences practical, a 3D multi-modality multi-class segmentation model based on nnU-net was trained/validated using T1, T2, SSFP, DWI and CT from 413 subjects. Reproducibility was assessed with test-re-test methodology on ADPKD subjects (n = 19) scanned twice within a 3-week interval correcting outliers and averaging the measurements across all sequences. Absolute percent differences in organ volumes were compared to paired students t-test., Results: Dice similarlity coefficient > 97%, Jaccard Index > 0.94, mean surface distance < 1 mm and mean Hausdorff Distance < 2 cm for all three organs and all five sequences were found on internal (n = 25), external (n = 37) and test-re-test reproducibility assessment (38 scans in 19 subjects). When averaging volumes measured from five MRI sequences, the model automatically segmented kidneys with test-re-test reproducibility (percent absolute difference between exam 1 and exam 2) of 1.3% which was better than all five expert observers. It reliably stratified ADPKD into Mayo Imaging Classification (area under the curve=100%) compared to radiologist., Conclusion: 3D deep learning measures organ volumes on five MRI sequences leveraging the power of outlier analysis and averaging to achieve 1.3% total kidney test-re-test reproducibility., Competing Interests: Declaration of Competing Interest Study “Polycystic Kidney Disease Data Repository” is registered on ClinicalTrials.gov (NCT00792155). George Shih discloses co-chairman of the Society of Abdominal radiology Artificial Intelligence committee and the Society of Imaging Informatics machine learning committee. Daniil Shimonov discloses consulting for Accordant/CVS on chronic kidney disease education. None of the other authors have anything to disclose., (Copyright © 2024 The Association of University Radiologists. Published by Elsevier Inc. All rights reserved.)

In healthcare, machine learning (ML) shows significant potential to augment patient care, improve population health, and streamline healthcare workflows. Realizing its full potential is, however, often hampered by concerns about data privacy, diversity in data sources, and suboptimal utilization of different data modalities. This review studies the utility of cross-cohort cross-category (C 4 ) integration in such contexts: the process of combining information from diverse datasets distributed across distinct, secure sites. We argue that C 4 approaches could pave the way for ML models that are both holistic and widely applicable. This paper provides a comprehensive overview of C 4 in health care, including its present stage, potential opportunities, and associated challenges., Competing Interests: The authors declare no competing interests., (© 2023 The Author(s).)

Interpretability for machine learning models in medical imaging (MLMI) is an important direction of research. However, there is a general sense of murkiness in what interpretability means. Why does the need for interpretability in MLMI arise? What goals does one actually seek to address when interpretability is needed? To answer these questions, we identify a need to formalize the goals and elements of interpretability in MLMI. By reasoning about real-world tasks and goals common in both medical image analysis and its intersection with machine learning, we identify five core elements of interpretability: localization, visual recognizability, physical attribution, model transparency, and actionability. From this, we arrive at a framework for interpretability in MLMI, which serves as a step-by-step guide to approaching interpretability in this context. Overall, this paper formalizes interpretability needs in the context of medical imaging, and our applied perspective clarifies concrete MLMI-specific goals and considerations in order to guide method design and improve real-world usage. Our goal is to provide practical and didactic information for model designers and practitioners, inspire developers of models in the medical imaging field to reason more deeply about what interpretability is achieving, and suggest future directions of interpretability research.

Background: Histopathologic studies of Alzheimer's disease (AD) suggest that extracellular amyloid-β (Aβ) plaques promote the spread of neurofibrillary tau tangles. However, these two proteinopathies initiate in spatially distinct brain regions, so how they interact during AD progression is unclear., Objective: In this study, we utilized Aβ and tau positron emission tomography (PET) scans from 572 older subjects (476 healthy controls (HC), 14 with mild cognitive impairment (MCI), 82 with mild AD), at varying stages of the disease, to investigate to what degree tau is associated with cortical Aβ deposition., Methods: Using multiple linear regression models and a pseudo-longitudinal ordering technique, we investigated remote tau-Aβ associations in four pathologic phases of AD progression based on tau spread: 1) no-tau, 2) pre-acceleration, 3) acceleration, and 4) post-acceleration., Results: No significant tau-Aβ association was detected in the no-tau phase. In the pre-acceleration phase, the earliest stage of tau deposition, associations emerged between regional tau in medial temporal lobe (MTL) (i.e., entorhinal cortex, parahippocampal gyrus) and cortical Aβ in lateral temporal lobe regions. The strongest tau-Aβ associations were found in the acceleration phase, in which tau in MTL regions was strongly associated with cortical Aβ (i.e., temporal and frontal lobes regions). Strikingly, in the post-acceleration phase, including 96% of symptomatic subjects, tau-Aβ associations were no longer significant., Conclusions: The results indicate that associations between tau and Aβ are stage-dependent, which could have important implications for understanding the interplay between these two proteinopathies during the progressive stages of AD.

We present KeyMorph, a deep learning-based image registration framework that relies on automatically detecting corresponding keypoints. State-of-the-art deep learning methods for registration often are not robust to large misalignments, are not interpretable, and do not incorporate the symmetries of the problem. In addition, most models produce only a single prediction at test-time. Our core insight which addresses these shortcomings is that corresponding keypoints between images can be used to obtain the optimal transformation via a differentiable closed-form expression. We use this observation to drive the end-to-end learning of keypoints tailored for the registration task, and without knowledge of ground-truth keypoints. This framework not only leads to substantially more robust registration but also yields better interpretability, since the keypoints reveal which parts of the image are driving the final alignment. Moreover, KeyMorph can be designed to be equivariant under image translations and/or symmetric with respect to the input image ordering. Finally, we show how multiple deformation fields can be computed efficiently and in closed-form at test time corresponding to different transformation variants. We demonstrate the proposed framework in solving 3D affine and spline-based registration of multi-modal brain MRI scans. In particular, we show registration accuracy that surpasses current state-of-the-art methods, especially in the context of large displacements. Our code is available at https://github.com/alanqrwang/keymorph., Competing Interests: Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2023 Elsevier B.V. All rights reserved.)

Intracranial hypertension is a feared complication of acute brain injury that can cause ischemic stroke, herniation, and death. Identifying those at risk is difficult, and the physical examination is often confounded. Given the widespread availability and use of computed tomography (CT) in patients with acute brain injury, prior work has attempted to use optic nerve diameter measurements to identify those at risk of intracranial hypertension. We aimed to validate the use of optic nerve diameter measurements on CT as a screening tool for intracranial hypertension in a large cohort of brain-injured patients. We performed a retrospective observational cohort study in a single tertiary referral Neuroscience Intensive Care Unit. We identified patients with documented intracranial pressure (ICP) measures as part of their routine clinical care who had non-contrast CT head scans collected within 24 h, and then measured the optic nerve diameters and explored the relationship and test characteristics of these measures to identify those at risk of intracranial hypertension. In a cohort of 314 patients, optic nerve diameter on CT was linearly but weakly associated with ICP. When used to identify those with intracranial hypertension (> 20 mm Hg), the area under the receiver operator curve (AUROC) was 0.68. Using a previously proposed threshold of 0.6 cm, the sensitivity was 81%, specificity 43%, positive likelihood ratio 1.4, and negative likelihood ratio 0.45. CT-derived optic nerve diameter using a threshold of 0.6 cm is sensitive but not specific for intracranial hypertension, and the overall correlation is weak.

Understanding how human brains interpret and process information is important. Here, we investigated the selectivity and inter-individual differences in human brain responses to images via functional MRI. In our first experiment, we found that images predicted to achieve maximal activations using a group level encoding model evoke higher responses than images predicted to achieve average activations, and the activation gain is positively associated with the encoding model accuracy. Furthermore, anterior temporal lobe face area (aTLfaces) and fusiform body area 1 had higher activation in response to maximal synthetic images compared to maximal natural images. In our second experiment, we found that synthetic images derived using a personalized encoding model elicited higher responses compared to synthetic images from group-level or other subjects' encoding models. The finding of aTLfaces favoring synthetic images than natural images was also replicated. Our results indicate the possibility of using data-driven and generative approaches to modulate macro-scale brain region responses and probe inter-individual differences in and functional specialization of the human visual system., (© 2023. Springer Nature Limited.)

For hemodialysis patients, arteriovenous fistula (AVF) patency determines whether adequate hemofiltration can be achieved, and directly influences clinical outcomes. Here, we report the development and performance of a deep learning model for automated AVF stenosis screening based on the sound of AVF blood flow using supervised learning with data validated by ultrasound. We demonstrate the importance of contextualizing the sound with location metadata as the characteristics of the blood flow sound varies significantly along the AVF. We found the best model to be a vision transformer trained on spectrogram images. Our model can screen for stenosis at a performance level comparable to that of a nephrologist performing a physical exam, but with the advantage of being automated and scalable. In a high-volume, resource-limited clinical setting, automated AVF stenosis screening can help ensure patient safety via early detection of at-risk vascular access, streamline the dialysis workflow, and serve as a patient-facing tool to allow for at-home, self-screening., (© 2023. Springer Nature Limited.)

The convolutional neural network (CNN) is one of the most commonly used architectures for computer vision tasks. The key building block of a CNN is the convolutional kernel that aggregates information from the pixel neighborhood and shares weights across all pixels. A standard CNN's capacity, and thus its performance, is directly related to the number of learnable kernel weights, which is determined by the number of channels and the kernel size (support). In this paper, we present the hyper-convolution, a novel building block that implicitly encodes the convolutional kernel using spatial coordinates. Unlike a regular convolutional kernel, whose weights are independently learned, hyper-convolution kernel weights are correlated through an encoder that maps spatial coordinates to their corresponding values. Hyper-convolutions decouple kernel size from the total number of learnable parameters, enabling a more flexible architecture design. We demonstrate in our experiments that replacing regular convolutions with hyper-convolutions can improve performance with less parameters, and increase robustness against noise. We provide our code here: https://github.com/tym002/Hyper-Convolution., Competing Interests: Declaration of Competing Interest The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Mert Sabuncu reports financial support was provided by Cornell University. Mert Sabuncu reports a relationship with Cornell University that includes: employment and funding grants., (Copyright © 2023 Elsevier B.V. All rights reserved.)

Rationale and Objectives: Multiparametric magnetic resonance imaging (mpMRI) is increasingly used for risk stratification and localization of prostate cancer (PCa). Thanks to the great success of deep learning models in computer vision, the potential application for early detection of PCa using mpMRI is imminent., Materials and Methods: Deep learning analysis of the PROSTATEx dataset., Results: In this study, we show a simple convolutional neural network (CNN) with mpMRI can achieve high performance for detection of clinically significant PCa (csPCa), depending on the pulse sequences used. The mpMRI model with T2-ADC-DWI achieved 0.90 AUC score in the held-out test set, not significantly better than the model using K trans instead of DWI (AUC 0.89). Interestingly, the model incorporating T2-ADC- K trans better estimates grade. We also describe a saliency "heat" map. Our results show that csPCa detection models with mpMRI may be leveraged to guide clinical management strategies., Conclusion: Convolutional neural networks incorporating multiple pulse sequences show high performance for detection of clinically-significant prostate cancer, and the model including dynamic contrast-enhanced information correlates best with grade., (Copyright © 2022 The Association of University Radiologists. Published by Elsevier Inc. All rights reserved.)

Quantitative susceptibility mapping (QSM) involves acquisition and reconstruction of a series of images at multi-echo time points to estimate tissue field, which prolongs scan time and requires specific reconstruction technique. In this paper, we present our new framework, called Learned Acquisition and Reconstruction Optimization (LARO), which aims to accelerate the multi-echo gradient echo (mGRE) pulse sequence for QSM. Our approach involves optimizing a Cartesian multi-echo k-space sampling pattern with a deep reconstruction network. Next, this optimized sampling pattern was implemented in an mGRE sequence using Cartesian fan-beam k-space segmenting and ordering for prospective scans. Furthermore, we propose to insert a recurrent temporal feature fusion module into the reconstruction network to capture signal redundancies along echo time. Our ablation studies show that both the optimized sampling pattern and proposed reconstruction strategy help improve the quality of the multi-echo image reconstructions. Generalization experiments show that LARO is robust on the test data with new pathologies and different sequence parameters. Our code is available at https://github.com/Jinwei1209/LARO-QSM.git., Competing Interests: Declaration of Competing Interest YW and PS are inventors of QSM-related patents issued to Cornell University and hold equity in Medimagemetric LLC. PS is a paid consultant for Medimagemetric LLC., (Copyright © 2023. Published by Elsevier Inc.)

While Machine Learning (ML) models have been increasingly applied to a range of histopathology tasks, there has been little emphasis on characterizing these models and contrasting them with human experts. We present a detailed empirical analysis comparing expert neuropathologists and ML models at predicting IDH mutation status in H&E-stained histology slides of infiltrating gliomas, both independently and synergistically. We find that errors made by neuropathologists and ML models trained using the TCGA dataset are distinct, representing modest agreement between predictions (human-vs.-human κ = 0.656; human-vs.-ML model κ = 0.598). While no ML model surpassed human performance on an independent institutional test dataset (human AUC = 0.901, max ML AUC = 0.881), a hybrid model aggregating human and ML predictions demonstrates predictive performance comparable to the consensus of two expert neuropathologists (hybrid classifier AUC = 0.921 vs. two-neuropathologist consensus AUC = 0.920). We also show that models trained at different levels of magnification exhibit different types of errors, supporting the value of aggregation across spatial scales in the ML approach. Finally, we present a detailed interpretation of our multi-scale ML ensemble model which reveals that predictions are driven by human-identifiable features at the patch-level., (© 2022. The Author(s).)

Alzheimer's disease (AD) is a neurodegenerative condition that progresses over decades. Early detection of individuals at high risk of future progression toward AD is likely to be of critical significance for the successful treatment and/or prevention of this devastating disease. In this paper, we present an empirical study to characterize how predictable an individual subjects' future AD trajectory is, several years in advance, based on rich multi-modal data, and using modern deep learning methods. Crucially, the machine learning strategy we propose can handle different future time horizons and can be trained with heterogeneous data that exhibit missingness and non-uniform follow-up visit times. Our experiments demonstrate that our strategy yields predictions that are more accurate than a model trained on a single time horizon (e.g. 3 years), which is common practice in prior literature. We also provide a comparison between linear and nonlinear models, verifying the well-established insight that the latter can offer a boost in performance. Our results also confirm that predicting future decline for cognitively normal (CN) individuals is more challenging than for individuals with mild cognitive impairment (MCI). Intriguingly, however, we discover that prediction accuracy decreases with increasing time horizon for CN subjects, but the trend is in the opposite direction for MCI subjects. Additionally, we quantify the contribution of different data types in prediction, which yields novel insights into the utility of different biomarkers. We find that molecular biomarkers are not as helpful for CN individuals as they are for MCI individuals, whereas magnetic resonance imaging biomarkers (hippocampus volume, specifically) offer a significant boost in prediction accuracy for CN individuals. Finally, we show how our model's prediction reveals the evolution of individual-level progression risk over a five-year time horizon. Our code is available at https://github.com/batuhankmkaraman/mlbasedad., Competing Interests: The authors have declared that no competing interests exist., (Copyright: © 2022 Karaman et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.)

Neuroimaging studies are often limited by the number of subjects and cognitive processes that can be feasibly interrogated. However, a rapidly growing number of neuroscientific studies have collectively accumulated an extensive wealth of results. Digesting this growing literature and obtaining novel insights remains to be a major challenge, since existing meta-analytic tools are constrained to keyword queries. In this paper, we present Text2Brain, an easy to use tool for synthesizing brain activation maps from open-ended text queries. Text2Brain was built on a transformer-based neural network language model and a coordinate-based meta-analysis of neuroimaging studies. Text2Brain combines a transformer-based text encoder and a 3D image generator, and was trained on variable-length text snippets and their corresponding activation maps sampled from 13,000 published studies. In our experiments, we demonstrate that Text2Brain can synthesize meaningful neural activation patterns from various free-form textual descriptions. Text2Brain is available at https://braininterpreter.com as a web-based tool for efficiently searching through the vast neuroimaging literature and generating new hypotheses., Competing Interests: Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2022 Elsevier B.V. All rights reserved.)

Task-based and resting-state represent the two most common experimental paradigms of functional neuroimaging. While resting-state offers a flexible and scalable approach for characterizing brain function, task-based techniques provide superior localization. In this paper, we build on recent deep learning methods to create a model that predicts task-based contrast maps from resting-state fMRI scans. Specifically, we propose BrainSurfCNN, a surface-based fully-convolutional neural network model that works with a representation of the brain's cortical sheet. BrainSurfCNN achieves exceptional predictive accuracy on independent test data from the Human Connectome Project, which is on par with the repeat reliability of the measured subject-level contrast maps. Conversely, our analyses reveal that a previously published benchmark is no better than group-average contrast maps. Finally, we demonstrate that BrainSurfCNN can generalize remarkably well to novel domains with limited training data., (Copyright © 2021. Published by Elsevier Inc.)

Functional MRI (fMRI) is a powerful technique that has allowed us to characterize visual cortex responses to stimuli, yet such experiments are by nature constructed based on a priori hypotheses, limited to the set of images presented to the individual while they are in the scanner, are subject to noise in the observed brain responses, and may vary widely across individuals. In this work, we propose a novel computational strategy, which we call NeuroGen, to overcome these limitations and develop a powerful tool for human vision neuroscience discovery. NeuroGen combines an fMRI-trained neural encoding model of human vision with a deep generative network to synthesize images predicted to achieve a target pattern of macro-scale brain activation. We demonstrate that the reduction of noise that the encoding model provides, coupled with the generative network's ability to produce images of high fidelity, results in a robust discovery architecture for visual neuroscience. By using only a small number of synthetic images created by NeuroGen, we demonstrate that we can detect and amplify differences in regional and individual human brain response patterns to visual stimuli. We then verify that these discoveries are reflected in the several thousand observed image responses measured with fMRI. We further demonstrate that NeuroGen can create synthetic images predicted to achieve regional response patterns not achievable by the best-matching natural images. The NeuroGen framework extends the utility of brain encoding models and opens up a new avenue for exploring, and possibly precisely controlling, the human visual system., Competing Interests: Declaration of Competing Interest The authors declare no competing interests., (Copyright © 2021. Published by Elsevier Inc.)

Background: While Prostate Imaging Reporting and Data System (PI-RADS) 4 and 5 lesions typically warrant prostate biopsy and PI-RADS 1 and 2 lesions may be safely observed, PI-RADS 3 lesions are equivocal., Purpose: To construct and cross-validate a machine learning model based on radiomics features from T 2 -weighted imaging (T 2 WI) of PI-RADS 3 lesions to identify clinically significant prostate cancer (csPCa), that is, pathological Grade Group ≥ 2., Study Type: Single-center retrospective study., Population: A total of 240 patients were included (training cohort, n = 188, age range 43-82 years; test cohort, n = 52, age range 41-79 years). Eligibility criteria were 1) magnetic resonance imaging (MRI)-targeted biopsy between 2015 and 2020; 2) PI-RADS 3 index lesion identified on multiparametric MRI; (3) biopsy performed within 1 year of MRI. The percentages of csPCa lesions were 10.6% and 15.4% in the training and test cohorts, respectively., Field Strength/sequence: A 3 T; T 2 WI turbo-spin echo, diffusion-weighted spin-echo echo planar imaging, dynamic contrast-enhanced MRI with time-resolved T1-weighted imaging., Assessment: Multislice volumes-of-interest (VOIs) were drawn in the PI-RADS 3 index lesions on T 2 WI. A total of 107 radiomics features (first-order histogram and second-order texture) were extracted from the segmented lesions., Statistical Tests: A random forest classifier using the radiomics features as input was trained and validated for prediction of csPCa. The performance of the machine learning classifier, prostate specific antigen (PSA) density, and prostate volume for csPCa prediction was evaluated using receiver operating characteristic (ROC) analysis., Results: The trained random forest classifier constructed from the T 2 WI radiomics features good and statistically significant area-under-the-curves (AUCs) of 0.76 (P = 0.022) for prediction of csPCa in the test set. Prostate volume and PSA density showed moderate and nonsignificant performance (AUC 0.62, P = 0.275 and 0.61, P = 0.348, respectively) for csPCa prediction in the test set., Conclusion: The machine learning classifier based on T 2 WI radiomic features demonstrated good performance for prediction of csPCa in PI-RADS 3 lesions., Evidence Level: 4 TECHNICAL EFFICACY: 2., (© 2021 International Society for Magnetic Resonance in Medicine.)

White matter structural connections are likely to support flow of functional activation or functional connectivity. While the relationship between structural and functional connectivity profiles, here called SC-FC coupling, has been studied on a whole-brain, global level, few studies have investigated this relationship at a regional scale. Here we quantify regional SC-FC coupling in healthy young adults using diffusion-weighted MRI and resting-state functional MRI data from the Human Connectome Project and study how SC-FC coupling may be heritable and varies between individuals. We show that regional SC-FC coupling strength varies widely across brain regions, but was strongest in highly structurally connected visual and subcortical areas. We also show interindividual regional differences based on age, sex and composite cognitive scores, and that SC-FC coupling was highly heritable within certain networks. These results suggest regional structure-function coupling is an idiosyncratic feature of brain organisation that may be influenced by genetic factors., (© 2021. The Author(s).)

Naturalistic stimuli, such as movies, activate a substantial portion of the human brain, invoking a response shared across individuals. Encoding models that predict neural responses to arbitrary stimuli can be very useful for studying brain function. However, existing models focus on limited aspects of naturalistic stimuli, ignoring the dynamic interactions of modalities in this inherently context-rich paradigm. Using movie-watching data from the Human Connectome Project, we build group-level models of neural activity that incorporate several inductive biases about neural information processing, including hierarchical processing, temporal assimilation, and auditory-visual interactions. We demonstrate how incorporating these biases leads to remarkable prediction performance across large areas of the cortex, beyond the sensory-specific cortices into multisensory sites and frontal cortex. Furthermore, we illustrate that encoding models learn high-level concepts that generalize to task-bound paradigms. Together, our findings underscore the potential of encoding models as powerful tools for studying brain function in ecologically valid conditions., (Copyright © 2021 The Authors, some rights reserved; exclusive licensee American Association for the Advancement of Science. No claim to original U.S. Government Works. Distributed under a Creative Commons Attribution NonCommercial License 4.0 (CC BY-NC).)

Human cortex is patterned by a complex and interdigitated web of large-scale functional networks. Recent methodological breakthroughs reveal variation in the size, shape, and spatial topography of cortical networks across individuals. While spatial network organization emerges across development, is stable over time, and is predictive of behavior, it is not yet clear to what extent genetic factors underlie interindividual differences in network topography. Here, leveraging a nonlinear multidimensional estimation of heritability, we provide evidence that individual variability in the size and topographic organization of cortical networks are under genetic control. Using twin and family data from the Human Connectome Project ( n = 1,023), we find increased variability and reduced heritability in the size of heteromodal association networks ( h 2 : M = 0.34, SD = 0.070), relative to unimodal sensory/motor cortex ( h 2 : M = 0.40, SD = 0.097). We then demonstrate that the spatial layout of cortical networks is influenced by genetics, using our multidimensional estimation of heritability ( h 2 - multi; M = 0.14, SD = 0.015). However, topographic heritability did not differ between heteromodal and unimodal networks. Genetic factors had a regionally variable influence on brain organization, such that the heritability of network topography was greatest in prefrontal, precuneus, and posterior parietal cortex. Taken together, these data are consistent with relaxed genetic control of association cortices relative to primary sensory/motor regions and have implications for understanding population-level variability in brain functioning, guiding both individualized prediction and the interpretation of analyses that integrate genetics and neuroimaging., Competing Interests: The authors declare no competing interest., (Copyright © 2021 the Author(s). Published by PNAS.)

Segmentation of anatomical regions of interest such as vessels or small lesions in medical images is still a difficult problem that is often tackled with manual input by an expert. One of the major challenges for this task is that the appearance of foreground (positive) regions can be similar to background (negative) regions. As a result, many automatic segmentation algorithms tend to exhibit asymmetric errors, typically producing more false positives than false negatives. In this paper, we aim to leverage this asymmetry and train a diverse ensemble of models with very high recall, while sacrificing their precision. Our core idea is straightforward: A diverse ensemble of low precision and high recall models are likely to make different false positive errors (classifying background as foreground in different parts of the image), but the true positives will tend to be consistent. Thus, in aggregate the false positive errors will cancel out, yielding high performance for the ensemble. Our strategy is general and can be applied with any segmentation model. In three different applications (carotid artery segmentation in a neck CT angiography, myocardium segmentation in a cardiovascular MRI and multiple sclerosis lesion segmentation in a brain MRI), we show how the proposed approach can significantly boost the performance of a baseline segmentation method.

A thorough understanding of sex differences that exist in the brains of healthy individuals is crucial for the study of neurological illnesses that exhibit phenotypic differences between males and females. Here we evaluate sex differences in regional temporal dependence of resting-state brain activity in 195 adult male-female pairs strictly matched for total grey matter volume from the Human Connectome Project. We find that males have more persistent temporal dependence in regions within temporal, parietal, and occipital cortices. Machine learning algorithms trained on regional temporal dependence measures achieve sex classification accuracies up to 81%. Regions with the strongest feature importance in the sex classification task included cerebellum, amygdala, and frontal and occipital cortices. Secondarily, we show that even after strict matching of total gray matter volume, significant volumetric sex differences persist; males have larger absolute cerebella, hippocampi, parahippocampi, thalami, caudates, and amygdalae while females have larger absolute cingulates, precunei, and frontal and parietal cortices. Sex classification based on regional volume achieves accuracies up to 85%, highlighting the importance of strict volume-matching when studying brain-based sex differences. Differential patterns in regional temporal dependence between the sexes identifies a potential neurobiological substrate or environmental effect underlying sex differences in functional brain activation patterns., (© 2020 The Authors. Human Brain Mapping published by Wiley Periodicals, Inc.)

Background and Purpose: One-fifth of ischemic strokes are embolic strokes of undetermined source (ESUS). Their theoretical causes can be classified as cardioembolic versus noncardioembolic. This distinction has important implications, but the categories' proportions are unknown., Methods: Using data from the Cornell Acute Stroke Academic Registry, we trained a machine-learning algorithm to distinguish cardioembolic versus non-cardioembolic strokes, then applied the algorithm to ESUS cases to determine the predicted proportion with an occult cardioembolic source. A panel of neurologists adjudicated stroke etiologies using standard criteria. We trained a machine learning classifier using data on demographics, comorbidities, vitals, laboratory results, and echocardiograms. An ensemble predictive method including L1 regularization, gradient-boosted decision tree ensemble (XGBoost), random forests, and multivariate adaptive splines was used. Random search and cross-validation were used to tune hyperparameters. Model performance was assessed using cross-validation among cases of known etiology. We applied the final algorithm to an independent set of ESUS cases to determine the predicted mechanism (cardioembolic or not). To assess our classifier's validity, we correlated the predicted probability of a cardioembolic source with the eventual post-ESUS diagnosis of atrial fibrillation., Results: Among 1083 strokes with known etiologies, our classifier distinguished cardioembolic versus noncardioembolic cases with excellent accuracy (area under the curve, 0.85). Applied to 580 ESUS cases, the classifier predicted that 44% (95% credibility interval, 39%-49%) resulted from cardiac embolism. Individual ESUS patients' predicted likelihood of cardiac embolism was associated with eventual atrial fibrillation detection (OR per 10% increase, 1.27 [95% CI, 1.03-1.57]; c-statistic, 0.68 [95% CI, 0.58-0.78]). ESUS patients with high predicted probability of cardiac embolism were older and had more coronary and peripheral vascular disease, lower ejection fractions, larger left atria, lower blood pressures, and higher creatinine levels., Conclusions: A machine learning estimator that distinguished known cardioembolic versus noncardioembolic strokes indirectly estimated that 44% of ESUS cases were cardioembolic.

Background . Accurate prediction of clinical impairment in upper-extremity motor function following therapy in chronic stroke patients is a difficult task for clinicians but is key in prescribing appropriate therapeutic strategies. Machine learning is a highly promising avenue with which to improve prediction accuracy in clinical practice. Objectives . The objective was to evaluate the performance of 5 machine learning methods in predicting postintervention upper-extremity motor impairment in chronic stroke patients using demographic, clinical, neurophysiological, and imaging input variables. Methods . A total of 102 patients (female: 31%, age 61 ± 11 years) were included. The upper-extremity Fugl-Meyer Assessment (UE-FMA) was used to assess motor impairment of the upper limb before and after intervention. Elastic net (EN), support vector machines, artificial neural networks, classification and regression trees, and random forest were used to predict postintervention UE-FMA. The performances of methods were compared using cross-validated R 2 . Results . EN performed significantly better than other methods in predicting postintervention UE-FMA using demographic and baseline clinical data (median R EN 2 = 0 . 91 , R RF 2 = 0 . 88 , R ANN 2 = 0 . 83 , R SVM 2 = 0 . 79 , R CART 2 = 0 . 70 ; P < .05). Preintervention UE-FMA and the difference in motor threshold (MT) between the affected and unaffected hemispheres were the strongest predictors. The difference in MT had greater importance than the absence or presence of a motor-evoked potential (MEP) in the affected hemisphere. Conclusion . Machine learning methods may enable clinicians to accurately predict a chronic stroke patient's postintervention UE-FMA. Interhemispheric difference in the MT is an important predictor of chronic stroke patients' response to therapy and, therefore, could be included in prospective studies.

Deep learning (DL) is increasingly used to solve ill-posed inverse problems in medical imaging, such as reconstruction from noisy and/or incomplete data, as DL offers advantages over conventional methods that rely on explicit image features and hand engineered priors. However, supervised DL-based methods may achieve poor performance when the test data deviates from the training data, for example, when it has pathologies not encountered in the training data. Furthermore, DL-based image reconstructions do not always incorporate the underlying forward physical model, which may improve performance. Therefore, in this work we introduce a novel approach, called fidelity imposed network edit (FINE), which modifies the weights of a pre-trained reconstruction network for each case in the testing dataset. This is achieved by minimizing an unsupervised fidelity loss function that is based on the forward physical model. FINE is applied to two important inverse problems in neuroimaging: quantitative susceptibility mapping (QSM) and under-sampled image reconstruction in MRI. Our experiments demonstrate that FINE can improve reconstruction accuracy., (Copyright © 2020 The Authors. Published by Elsevier Inc. All rights reserved.)

Deep neural networks yield promising results in a wide range of computer vision applications, including landmark detection. A major challenge for accurate anatomical landmark detection in volumetric images such as clinical CT scans is that large-scale data often constrain the capacity of the employed neural network architecture due to GPU memory limitations, which in turn can limit the precision of the output. We propose a multi-scale, end-to-end deep learning method that achieves fast and memory-efficient landmark detection in 3D images. Our architecture consists of blocks of shift-equivariant networks, each of which performs landmark detection at a different spatial scale. These blocks are connected from coarse to fine-scale, with differentiable resampling layers, so that all levels can be trained together. We also present a noise injection strategy that increases the robustness of the model and allows us to quantify uncertainty at test time. We evaluate our method for carotid artery bifurcations detection on 263 CT volumes and achieve a better than state-of-the-art accuracy with mean Euclidean distance error of 2.81mm.

There is significant interest in the development and application of deep neural networks (DNNs) to neuroimaging data. A growing literature suggests that DNNs outperform their classical counterparts in a variety of neuroimaging applications, yet there are few direct comparisons of relative utility. Here, we compared the performance of three DNN architectures and a classical machine learning algorithm (kernel regression) in predicting individual phenotypes from whole-brain resting-state functional connectivity (RSFC) patterns. One of the DNNs was a generic fully-connected feedforward neural network, while the other two DNNs were recently published approaches specifically designed to exploit the structure of connectome data. By using a combined sample of almost 10,000 participants from the Human Connectome Project (HCP) and UK Biobank, we showed that the three DNNs and kernel regression achieved similar performance across a wide range of behavioral and demographic measures. Furthermore, the generic feedforward neural network exhibited similar performance to the two state-of-the-art connectome-specific DNNs. When predicting fluid intelligence in the UK Biobank, performance of all algorithms dramatically improved when sample size increased from 100 to 1000 subjects. Improvement was smaller, but still significant, when sample size increased from 1000 to 5000 subjects. Importantly, kernel regression was competitive across all sample sizes. Overall, our study suggests that kernel regression is as effective as DNNs for RSFC-based behavioral prediction, while incurring significantly lower computational costs. Therefore, kernel regression might serve as a useful baseline algorithm for future studies., (Copyright © 2019 Elsevier Inc. All rights reserved.)

Bulliform cells comprise specialized cell types that develop on the adaxial (upper) surface of grass leaves, and are patterned to form linear rows along the proximodistal axis of the adult leaf blade. Bulliform cell patterning affects leaf angle and is presumed to function during leaf rolling, thereby reducing water loss during temperature extremes and drought. In this study, epidermal leaf impressions were collected from a genetically and anatomically diverse population of maize inbred lines. Subsequently, convolutional neural networks were employed to measure microscopic, bulliform cell-patterning phenotypes in high-throughput. A genome-wide association study, combined with RNAseq analyses of the bulliform cell ontogenic zone, identified candidate regulatory genes affecting bulliform cell column number and cell width. This study is the first to combine machine learning approaches, transcriptomics, and genomics to study bulliform cell patterning, and the first to utilize natural variation to investigate the genetic architecture of this microscopic trait. In addition, this study provides insight toward the improvement of macroscopic traits such as drought resistance and plant architecture in an agronomically important crop plant., (Copyright © 2019 Qiao et al.)

Machine learning techniques have gained prominence for the analysis of resting-state functional Magnetic Resonance Imaging (rs-fMRI) data. Here, we present an overview of various unsupervised and supervised machine learning applications to rs-fMRI. We offer a methodical taxonomy of machine learning methods in resting-state fMRI. We identify three major divisions of unsupervised learning methods with regard to their applications to rs-fMRI, based on whether they discover principal modes of variation across space, time or population. Next, we survey the algorithms and rs-fMRI feature representations that have driven the success of supervised subject-level predictions. The goal is to provide a high-level overview of the burgeoning field of rs-fMRI from the perspective of machine learning applications., (Copyright © 2019 Elsevier Inc. All rights reserved.)

Individuals with Alzheimer's disease (AD) dementia exhibit significant heterogeneity across clinical symptoms, atrophy patterns, and spatial distribution of Tau deposition. Most previous studies of AD heterogeneity have focused on atypical clinical subtypes, defined subtypes with a single modality, or restricted their analyses to a priori brain regions and cognitive tests. Here, we considered a data-driven hierarchical Bayesian model to identify latent factors from atrophy patterns and cognitive deficits simultaneously, thus exploiting the rich dimensionality within each modality. Unlike most previous studies, our model allows each factor to be expressed to varying degrees within an individual, in order to reflect potential multiple co-existing pathologies. By applying our model to ADNI-GO/2 AD dementia participants, we found three atrophy-cognitive factors. The first factor was associated with medial temporal lobe atrophy, episodic memory deficits and disorientation to time/place ("MTL-Memory"). The second factor was associated with lateral temporal atrophy and language deficits ("Lateral Temporal-Language"). The third factor was associated with atrophy in posterior bilateral cortex, and visuospatial executive function deficits ("Posterior Cortical-Executive"). While the MTL-Memory and Posterior Cortical-Executive factors were discussed in previous literature, the Lateral Temporal-Language factor is novel and emerged only by considering atrophy and cognition jointly. Several analyses were performed to ensure generalizability, replicability and stability of the estimated factors. First, the factors generalized to new participants within a 10-fold cross-validation of ADNI-GO/2 AD dementia participants. Second, the factors were replicated in an independent ADNI-1 AD dementia cohort. Third, factor loadings of ADNI-GO/2 AD dementia participants were longitudinally stable, suggesting that these factors capture heterogeneity across patients, rather than longitudinal disease progression. Fourth, the model outperformed canonical correlation analysis at capturing associations between atrophy patterns and cognitive deficits. To explore the influence of the factors early in the disease process, factor loadings were estimated in ADNI-GO/2 mild cognitively impaired (MCI) participants. Although the associations between the atrophy patterns and cognitive profiles were weak in MCI compared to AD, we found that factor loadings were associated with inter-individual regional variation in Tau uptake. Taken together, these results suggest that distinct atrophy-cognitive patterns exist in typical Alzheimer's disease, and are associated with distinct patterns of Tau depositions before clinical dementia emerges., (Copyright © 2019 Elsevier Inc. All rights reserved.)

Probabilistic atlas priors have been commonly used to derive adaptive and robust brain MRI segmentation algorithms. Widely-used neuroimage analysis pipelines rely heavily on these techniques, which are often computationally expensive. In contrast, there has been a recent surge of approaches that leverage deep learning to implement segmentation tools that are computationally efficient at test time. However, most of these strategies rely on learning from manually annotated images. These supervised deep learning methods are therefore sensitive to the intensity profiles in the training dataset. To develop a deep learning-based segmentation model for a new image dataset (e.g., of different contrast), one usually needs to create a new labeled training dataset, which can be prohibitively expensive, or rely on suboptimal ad hoc adaptation or augmentation approaches. In this paper, we propose an alternative strategy that combines a conventional probabilistic atlas-based segmentation with deep learning, enabling one to train a segmentation model for new MRI scans without the need for any manually segmented images. Our experiments include thousands of brain MRI scans and demonstrate that the proposed method achieves good accuracy for a brain MRI segmentation task for different MRI contrasts, requiring only approximately 15 seconds at test time on a GPU.

The specificity and sensitivity of resting state functional MRI (rs-fMRI) measurements depend on preprocessing choices, such as the parcellation scheme used to define regions of interest (ROIs). In this study, we critically evaluate the effect of brain parcellations on machine learning models applied to rs-fMRI data. Our experiments reveal an intriguing trend: On average, models with stochastic parcellations consistently perform as well as models with widely used atlases at the same spatial scale. We thus propose an ensemble learning strategy to combine the predictions from models trained on connectivity data extracted using different (e.g., stochastic) parcellations. We further present an implementation of our ensemble learning strategy with a novel 3D Convolutional Neural Network (CNN) approach. The proposed CNN approach takes advantage of the full-resolution 3D spatial structure of rs-fMRI data and fits non-linear predictive models. Our ensemble CNN framework overcomes the limitations of traditional machine learning models for connectomes that often rely on region-based summary statistics and/or linear models. We showcase our approach on a classification (autism patients versus healthy controls) and a regression problem (prediction of subject's age), and report promising results., (Copyright © 2019 Elsevier Inc. All rights reserved.)

Classical deformable registration techniques achieve impressive results and offer a rigorous theoretical treatment, but are computationally intensive since they solve an optimization problem for each image pair. Recently, learning-based methods have facilitated fast registration by learning spatial deformation functions. However, these approaches use restricted deformation models, require supervised labels, or do not guarantee a diffeomorphic (topology-preserving) registration. Furthermore, learning-based registration tools have not been derived from a probabilistic framework that can offer uncertainty estimates. In this paper, we build a connection between classical and learning-based methods. We present a probabilistic generative model and derive an unsupervised learning-based inference algorithm that uses insights from classical registration methods and makes use of recent developments in convolutional neural networks (CNNs). We demonstrate our method on a 3D brain registration task for both images and anatomical surfaces, and provide extensive empirical analyses of the algorithm. Our principled approach results in state of the art accuracy and very fast runtimes, while providing diffeomorphic guarantees. Our implementation is available online at http://voxelmorph.csail.mit.edu., (Copyright © 2019 Elsevier B.V. All rights reserved.)

Global signal regression (GSR) is one of the most debated preprocessing strategies for resting-state functional MRI. GSR effectively removes global artifacts driven by motion and respiration, but also discards globally distributed neural information and introduces negative correlations between certain brain regions. The vast majority of previous studies have focused on the effectiveness of GSR in removing imaging artifacts, as well as its potential biases. Given the growing interest in functional connectivity fingerprinting, here we considered the utilitarian question of whether GSR strengthens or weakens associations between resting-state functional connectivity (RSFC) and multiple behavioral measures across cognition, personality and emotion. By applying the variance component model to the Brain Genomics Superstruct Project (GSP), we found that behavioral variance explained by whole-brain RSFC increased by an average of 47% across 23 behavioral measures after GSR. In the Human Connectome Project (HCP), we found that behavioral variance explained by whole-brain RSFC increased by an average of 40% across 58 behavioral measures, when GSR was applied after ICA-FIX de-noising. To ensure generalizability, we repeated our analyses using kernel regression. GSR improved behavioral prediction accuracies by an average of 64% and 12% in the GSP and HCP datasets respectively. Importantly, the results were consistent across methods. A behavioral measure with greater RSFC-explained variance (using the variance component model) also exhibited greater prediction accuracy (using kernel regression). A behavioral measure with greater improvement in behavioral variance explained after GSR (using the variance component model) also enjoyed greater improvement in prediction accuracy after GSR (using kernel regression). Furthermore, GSR appeared to benefit task performance measures more than self-reported measures. Since GSR was more effective at removing motion-related and respiratory-related artifacts, GSR-related increases in variance explained and prediction accuracies were unlikely the result of motion-related or respiratory-related artifacts. However, it is worth emphasizing that the current study focused on whole-brain RSFC, so it remains unclear whether GSR improves RSFC-behavioral associations for specific connections or networks. Overall, our results suggest that at least in the case for young healthy adults, GSR strengthens the associations between RSFC and most (although not all) behavioral measures. Code for the variance component model and ridge regression can be found here: https://github.com/ThomasYeoLab/CBIG/tree/master/stable&#95;projects/preprocessing/Li2019&#95;GSR., (Copyright © 2019 Elsevier Inc. All rights reserved.)

Individual differences in educational attainment are linked to differences in intelligence, and predict important social, economic, and health outcomes. Previous studies have found common genetic factors that influence educational achievement, cognitive performance and total brain volume (i.e., brain size). Here, in a large sample of participants from the UK Biobank, we investigate the shared genetic basis between educational attainment and fine-grained cerebral cortical morphological features, and associate this genetic variation with a related aspect of cognitive ability. Importantly, we execute novel statistical methods that enable high-dimensional genetic correlation analysis, and compute high-resolution surface maps for the genetic correlations between educational attainment and vertex-wise morphological measurements. We conduct secondary analyses, using the UK Biobank verbal-numerical reasoning score, to confirm that variation in educational attainment that is genetically correlated with cortical morphology is related to differences in cognitive performance. Our analyses relate the genetic overlap between cognitive ability and cortical thickness measurements to bilateral primary motor cortex as well as predominantly left superior temporal cortex and proximal regions. These findings extend our understanding of the neurobiology that connects genetic variation to individual differences in educational attainment and cognitive performance., (© The Author(s) 2018. Published by Oxford University Press. All rights reserved. For Permissions, please e-mail: journals.permissions@oup.com.)

Resting-state functional magnetic resonance imaging (rs-fMRI) offers the opportunity to delineate individual-specific brain networks. A major question is whether individual-specific network topography (i.e., location and spatial arrangement) is behaviorally relevant. Here, we propose a multi-session hierarchical Bayesian model (MS-HBM) for estimating individual-specific cortical networks and investigate whether individual-specific network topography can predict human behavior. The multiple layers of the MS-HBM explicitly differentiate intra-subject (within-subject) from inter-subject (between-subject) network variability. By ignoring intra-subject variability, previous network mappings might confuse intra-subject variability for inter-subject differences. Compared with other approaches, MS-HBM parcellations generalized better to new rs-fMRI and task-fMRI data from the same subjects. More specifically, MS-HBM parcellations estimated from a single rs-fMRI session (10 min) showed comparable generalizability as parcellations estimated by 2 state-of-the-art methods using 5 sessions (50 min). We also showed that behavioral phenotypes across cognition, personality, and emotion could be predicted by individual-specific network topography with modest accuracy, comparable to previous reports predicting phenotypes based on connectivity strength. Network topography estimated by MS-HBM was more effective for behavioral prediction than network size, as well as network topography estimated by other parcellation approaches. Thus, similar to connectivity strength, individual-specific network topography might also serve as a fingerprint of human behavior., (© The Author(s) 2018. Published by Oxford University Press. All rights reserved. For Permissions, please e-mail: journals.permissions@oup.com.)

Linking human behavior to resting-state brain function is a central question in systems neuroscience. In particular, the functional timescales at which different types of behavioral factors are encoded remain largely unexplored. The behavioral counterparts of static functional connectivity (FC), at the resolution of several minutes, have been studied but behavioral correlates of dynamic measures of FC at the resolution of a few seconds remain unclear. Here, using resting-state fMRI and 58 phenotypic measures from the Human Connectome Project, we find that dynamic FC captures task-based phenotypes (e.g., processing speed or fluid intelligence scores), whereas self-reported measures (e.g., loneliness or life satisfaction) are equally well explained by static and dynamic FC. Furthermore, behaviorally relevant dynamic FC emerges from the interconnections across all resting-state networks, rather than within or between pairs of networks. Our findings shed new light on the timescales of cognitive processes involved in distinct facets of behavior.

We propose a novel machine learning strategy for studying neuroanatomical shape variation. Our model works with volumetric binary segmentation images, and requires no preprocessing such as the extraction of surface points or a mesh. The learned shape descriptor is invariant to affine transformations, including shifts, rotations and scaling. Thanks to the adopted autoencoder framework, inter-subject differences are automatically enhanced in the learned representation, while intra-subject variances are minimized. Our experimental results on a shape retrieval task showed that the proposed representation outperforms a state-of-the-art benchmark for brain structures extracted from MRI scans.

The health and function of tissue rely on its vasculature network to provide reliable blood perfusion. Volumetric imaging approaches, such as multiphoton microscopy, are able to generate detailed 3D images of blood vessels that could contribute to our understanding of the role of vascular structure in normal physiology and in disease mechanisms. The segmentation of vessels, a core image analysis problem, is a bottleneck that has prevented the systematic comparison of 3D vascular architecture across experimental populations. We explored the use of convolutional neural networks to segment 3D vessels within volumetric in vivo images acquired by multiphoton microscopy. We evaluated different network architectures and machine learning techniques in the context of this segmentation problem. We show that our optimized convolutional neural network architecture with a customized loss function, which we call DeepVess, yielded a segmentation accuracy that was better than state-of-the-art methods, while also being orders of magnitude faster than the manual annotation. To explore the effects of aging and Alzheimer's disease on capillaries, we applied DeepVess to 3D images of cortical blood vessels in young and old mouse models of Alzheimer's disease and wild type littermates. We found little difference in the distribution of capillary diameter or tortuosity between these groups, but did note a decrease in the number of longer capillary segments (>75μm) in aged animals as compared to young, in both wild type and Alzheimer's disease mouse models., Competing Interests: The authors have declared that no competing interests exist.

We present VoxelMorph, a fast learning-based framework for deformable, pairwise medical image registration. Traditional registration methods optimize an objective function for each pair of images, which can be time-consuming for large datasets or rich deformation models. In contrast to this approach, and building on recent learning-based methods, we formulate registration as a function that maps an input image pair to a deformation field that aligns these images. We parameterize the function via a convolutional neural network (CNN), and optimize the parameters of the neural network on a set of images. Given a new pair of scans, VoxelMorph rapidly computes a deformation field by directly evaluating the function. In this work, we explore two different training strategies. In the first (unsupervised) setting, we train the model to maximize standard image matching objective functions that are based on the image intensities. In the second setting, we leverage auxiliary segmentations available in the training data. We demonstrate that the unsupervised model's accuracy is comparable to state-of-the-art methods, while operating orders of magnitude faster. We also show that VoxelMorph trained with auxiliary data improves registration accuracy at test time, and evaluate the effect of training set size on registration. Our method promises to speed up medical image analysis and processing pipelines, while facilitating novel directions in learning-based registration and its applications. Our code is freely available at https://github.com/voxelmorph/voxelmorph.

The human default mode network (DMN) is implicated in several unique mental capacities. In this study, we tested whether brain-wide interregional communication in the DMN can be derived from population variability in intrinsic activity fluctuations, gray-matter morphology, and fiber tract anatomy. In a sample of 10,000 UK Biobank participants, pattern-learning algorithms revealed functional coupling states in the DMN that are linked to connectivity profiles between other macroscopical brain networks. In addition, DMN gray matter volume was covaried with white matter microstructure of the fornix. Collectively, functional and structural patterns unmasked a possible division of labor within major DMN nodes: Subregions most critical for cortical network interplay were adjacent to subregions most predictive of fornix fibers from the hippocampus that processes memories and places., Competing Interests: The authors declare no conflict of interest., (Copyright © 2018 the Author(s). Published by PNAS.)

In this work, we consider the problem of predicting the course of a progressive disease, such as cancer or Alzheimer's. Progressive diseases often start with mild symptoms that might precede a diagnosis, and each patient follows their own trajectory. Patient trajectories exhibit wild variability, which can be associated with many factors such as geno-type, age, or sex. An additional layer of complexity is that, in real life, the amount and type of data available for each patient can differ significantly. For example, for one patient we might have no prior history, whereas for another patient we might have detailed clinical assessments obtained at multiple prior time-points. This paper presents a probabilistic model that can handle multiple modalities (including images and clinical assessments) and variable patient histories with irregular timings and missing entries, to predict clinical scores at future time-points. We use a sigmoidal function to model latent disease progression, which gives rise to clinical observations in our generative model. We implemented an approximate Bayesian inference strategy on the proposed model to estimate the parameters on data from a large population of subjects. Furthermore, the Bayesian framework enables the model to automatically fine-tune its predictions based on historical observations that might be available on the test subject. We applied our method to a longitudinal Alzheimer's disease dataset with more than 3,000 subjects [1] with comparisons against several benchmarks.

We present an algorithm for creating high resolution anatomically plausible images consistent with acquired clinical brain MRI scans with large inter-slice spacing. Although large data sets of clinical images contain a wealth of information, time constraints during acquisition result in sparse scans that fail to capture much of the anatomy. These characteristics often render computational analysis impractical as many image analysis algorithms tend to fail when applied to such images. Highly specialized algorithms that explicitly handle sparse slice spacing do not generalize well across problem domains. In contrast, we aim to enable application of existing algorithms that were originally developed for high resolution research scans to significantly undersampled scans. We introduce a generative model that captures fine-scale anatomical structure across subjects in clinical image collections and derive an algorithm for filling in the missing data in scans with large inter-slice spacing. Our experimental results demonstrate that the resulting method outperforms state-of-the-art upsampling super-resolution techniques, and promises to facilitate subsequent analysis not previously possible with scans of this quality. Our implementation is freely available at https://github.com/adalca/papago.