Search

Deep learning models usually suffer from domain shift issues, where models trained on one source domain do not generalize well to other unseen domains. In this work, we investigate the single-source domain generalization problem: training a deep network that is robust to unseen domains, under the condition that training data is only available from one source domain, which is common in medical imaging applications. We tackle this problem in the context of cross-domain medical image segmentation. Under this scenario, domain shifts are mainly caused by different acquisition processes. We propose a simple causality-inspired data augmentation approach to expose a segmentation model to synthesized domain-shifted training examples. Specifically, 1) to make the deep model robust to discrepancies in image intensities and textures, we employ a family of randomly-weighted shallow networks. They augment training images using diverse appearance transformations. 2) Further we show that spurious correlations among objects in an image are detrimental to domain robustness. These correlations might be taken by the network as domain-specific clues for making predictions, and they may break on unseen domains. We remove these spurious correlations via causal intervention. This is achieved by resampling the appearances of potentially correlated objects independently. The proposed approach is validated on three cross-domain segmentation tasks: cross-modality (CT-MRI) abdominal image segmentation, cross-sequence (bSSFP-LGE) cardiac MRI segmentation, and cross-center prostate MRI segmentation. The proposed approach yields consistent performance gains compared with competitive methods when tested on unseen domains., This is an early, non-peer-reviewed version. For the final peer-reviewed full version that has been substantially revised, please find: https://ieeexplore.ieee.org/document/9961940. Please find the code at https://github.com/cheng-01037/Causality-Medical-Image-Domain-Generalization

OBJECTIVES: The UK Biobank (UKBB) and German National Cohort (NAKO) are among the largest cohort studies, capturing a wide range of health-related data from the general population, including comprehensive magnetic resonance imaging (MRI) examinations. The purpose of this study was to demonstrate how MRI data from these large-scale studies can be jointly analyzed and to derive comprehensive quantitative image-based phenotypes across the general adult population. MATERIALS AND METHODS: Image-derived features of abdominal organs (volumes of liver, spleen, kidneys, and pancreas; volumes of kidney hilum adipose tissue; and fat fractions of liver and pancreas) were extracted from T1-weighted Dixon MRI data of 17,996 participants of UKBB and NAKO based on quality-controlled deep learning generated organ segmentations. To enable valid cross-study analysis, we first analyzed the data generating process using methods of causal discovery. We subsequently harmonized data from UKBB and NAKO using the ComBat approach for batch effect correction. We finally performed quantile regression on harmonized data across studies providing quantitative models for the variation of image-derived features stratified for sex and dependent on age, height, and weight. RESULTS: Data from 8791 UKBB participants (49.9% female; age, 63 ± 7.5 years) and 9205 NAKO participants (49.1% female, age: 51.8 ± 11.4 years) were analyzed. Analysis of the data generating process revealed direct effects of age, sex, height, weight, and the data source (UKBB vs NAKO) on image-derived features. Correction of data source-related effects resulted in markedly improved alignment of image-derived features between UKBB and NAKO. Cross-study analysis on harmonized data revealed comprehensive quantitative models for the phenotypic variation of abdominal organs across the general adult population. CONCLUSIONS: Cross-study analysis of MRI data from UKBB and NAKO as proposed in this work can be helpful for future joint data analyses across cohorts linking genetic, environmental, and behavioral risk factors to MRI-derived phenotypes and provide reference values for clinical diagnostics.

Large epidemiological studies such as the UK Biobank (UKBB) or German National Cohort (NAKO) provide unprecedented health-related data of the general population aiming to better understand determinants of health and disease. As part of these studies, Magnetic Resonance Imaging (MRI) is performed in a subset of participants allowing for phenotypical and functional characterization of different organ systems. Due to the large amount of imaging data, automated image analysis is required, which can be performed using deep learning methods, e. g. for automated organ segmentation. In this paper we describe a computational pipeline for automated segmentation of abdominal organs on MRI data from 20,000 participants of UKBB and NAKO and provide results of the quality control process. We found that approx. 90% of data sets showed no relevant segmentation errors while relevant errors occurred in a varying proportion of data sets depending on the organ of interest. Image-derived features based on automated organ segmentations showed relevant deviations of varying degree in the presence of segmentation errors. These results show that large-scale, deep learning-based abdominal organ segmentation on MRI data is feasible with overall high accuracy, but visual quality control remains important ensuring the validity of down-stream analyses in large epidemiological imaging studies.

Myocardial motion and deformation are rich descriptors that characterize cardiac function. Image registration, as the most commonly used technique for myocardial motion tracking, is an ill-posed inverse problem which often requires prior assumptions on the solution space. In contrast to most existing approaches which impose explicit generic regularization such as smoothness, in this work we propose a novel method that can implicitly learn an application-specific biomechanics-informed prior and embed it into a neural network-parameterized transformation model. Particularly, the proposed method leverages a variational autoencoder-based generative model to learn a manifold for biomechanically plausible deformations. The motion tracking then can be performed via traversing the learnt manifold to search for the optimal transformations while considering the sequence information. The proposed method is validated on three public cardiac cine MRI datasets with comprehensive evaluations. The results demonstrate that the proposed method can outperform other approaches, yielding higher motion tracking accuracy with reasonable volume preservation and better generalizability to varying data distributions. It also enables better estimates of myocardial strains, which indicates the potential of the method in characterizing spatiotemporal signatures for understanding cardiovascular diseases., Under review

Optical microrobots have a wide range of applications in biomedical research for both in vitro and in vivo studies. In most microrobotic systems, the video captured by a monocular camera is the only way for visualizing the movements of microrobots, and only planar motion, in general, can be captured by a monocular camera system. Accurate depth estimation is essential for 3D reconstruction or autofocusing of microplatforms, while the pose and depth estimation are necessary to enhance the 3D perception of the microrobotic systems to enable dexterous micromanipulation and other tasks. In this paper, we propose a data-driven method for pose and depth estimation in an optically manipulated microrobotic system. Focus measurement is used to obtain features for Gaussian Process Regression (GPR), which enables precise depth estimation. For mobile microrobots with varying poses, a novel method is developed based on a deep residual neural network with the incorporation of prior domain knowledge about the optical microrobots encoded via GPR. The method can simultaneously track microrobots with complex shapes and estimate the pose and depth values of the optical microrobots. Cross-validation has been conducted to demonstrate the submicron accuracy of the proposed method and precise pose and depth perception for microrobots. We further demonstrate the generalizability of the method by adapting it to microrobots of different shapes using transfer learning with few-shot calibration. Intuitive visualization is provided to facilitate effective human-robot interaction during micromanipulation based on pose and depth estimation results.

Differences in cardiac and aortic structure and function are associated with cardiovascular diseases and a wide range of other types of disease. Here we analyzed cardiovascular magnetic resonance images from a population-based study, the UK Biobank, using an automated machine-learning-based analysis pipeline. We report a comprehensive range of structural and functional phenotypes for the heart and aorta across 26,893 participants, and explore how these phenotypes vary according to sex, age and major cardiovascular risk factors. We extended this analysis with a phenome-wide association study, in which we tested for correlations of a wide range of non-imaging phenotypes of the participants with imaging phenotypes. We further explored the associations of imaging phenotypes with early-life factors, mental health and cognitive function using both observational analysis and Mendelian randomization. Our study illustrates how population-based cardiac and aortic imaging phenotypes can be used to better define cardiovascular disease risks as well as heart-brain health interactions, highlighting new opportunities for studying disease mechanisms and developing image-based biomarkers.

Quantification of anatomical shape changes currently relies on scalar global indexes which are largely insensitive to regional or asymmetric modifications. Accurate assessment of pathology-driven anatomical remodeling is a crucial step for the diagnosis and treatment of many conditions. Deep learning approaches have recently achieved wide success in the analysis of medical images, but they lack interpretability in the feature extraction and decision processes. In this work, we propose a new interpretable deep learning model for shape analysis. In particular, we exploit deep generative networks to model a population of anatomical segmentations through a hierarchy of conditional latent variables. At the highest level of this hierarchy, a two-dimensional latent space is simultaneously optimised to discriminate distinct clinical conditions, enabling the direct visualisation of the classification space. Moreover, the anatomical variability encoded by this discriminative latent space can be visualised in the segmentation space thanks to the generative properties of the model, making the classification task transparent. This approach yielded high accuracy in the categorisation of healthy and remodelled left ventricles when tested on unseen segmentations from our own multi-centre dataset as well as in an external validation set, and on hippocampi from healthy controls and patients with Alzheimer's disease when tested on ADNI data. More importantly, it enabled the visualisation in three-dimensions of both global and regional anatomical features which better discriminate between the conditions under exam. The proposed approach scales effectively to large populations, facilitating high-throughput analysis of normal anatomy and pathology in large-scale studies of volumetric imaging., Comment: Accepted for publication in IEEE Transactions on Medical Imaging (TMI)

Diastole is the sequence of physiological events that occur in the heart during ventricular filling and principally depends on myocardial relaxation and chamber stiffness. Abnormal diastolic function is related to many cardiovascular disease processes and is predictive of health outcomes, but its genetic architecture is largely unknown. Here, we use machine-learning cardiac motion analysis to measure diastolic functional traits in 39,559 participants of the UK Biobank and perform a genome-wide association study. We identified nine significant, independent loci near genes that are associated with maintaining sarcomeric function under biomechanical stress and genes implicated in the development of cardiomyopathy. Age, sex and diabetes were independent predictors of diastolic function and we found a causal relationship between genetically determined ventricular stiffness and incident heart failure. Our results provide insights into the genetic and environmental factors influencing diastolic function that are relevant for identifying causal relationships and potential tractable targets.

Recovering the 3D motion of the heart from cine cardiac magnetic resonance (CMR) imaging enables the assessment of regional myocardial function and is important for understanding and analyzing cardiovascular disease. However, 3D cardiac motion estimation is challenging because the acquired cine CMR images are usually 2D slices which limit the accurate estimation of through-plane motion. To address this problem, we propose a novel multi-view motion estimation network (MulViMotion), which integrates 2D cine CMR images acquired in short-axis and long-axis planes to learn a consistent 3D motion field of the heart. In the proposed method, a hybrid 2D/3D network is built to generate dense 3D motion fields by learning fused representations from multi-view images. To ensure that the motion estimation is consistent in 3D, a shape regularization module is introduced during training, where shape information from multi-view images is exploited to provide weak supervision to 3D motion estimation. We extensively evaluate the proposed method on 2D cine CMR images from 580 subjects of the UK Biobank study for 3D motion tracking of the left ventricular myocardium. Experimental results show that the proposed method quantitatively and qualitatively outperforms competing methods.

Case studies conducted after recovery from acute infection with SARS-CoV-2 have frequently identified abnormalities on CMR imaging, suggesting the possibility that SARS-CoV-2 infection commonly leads to cardiac pathology. However, these observations have not been able to distinguish between associations that reflect pre-existing cardiac abnormalities (that might confer a greater likelihood of more severe infection) from those that arise as consequences of infection. To address this question, UK Biobank volunteers (n=1285; 54.5% women; mean age at baseline, 59.8 years old; 96.3% white) who attended an imaging assessment including cardiac magnetic resonance (CMR) before the start of the COVID-19 pandemic were invited to attend a second imaging assessment in 2021. Cases with evidence of previous SARS-CoV-2 infection were identified through linkage to PCR-testing or other medical records, or a positive antibody lateral flow test; n=640 in data available on 22 Sep 2021) and were matched to controls with no evidence of previous infection (n=645). The majority of these infections were milder and did not involve hospitalisation. Measures of cardiac and aortic structure and function were derived from the CMR images obtained on the cases before and after SARS-CoV-2 infection from images for the controls obtained over the same time interval using a previously validated, automated algorithm. Cases and controls had similar cardiac and aortic imaging phenotypes at their first imaging assessment. Changes between CMR imaging measures in cases before and after infection were not significantly different from those in the matched control group. Additional adjustment for comorbidities made no material difference to the results. While these results are preliminary and limited to imaging metrics derived from automated analyses, they do not suggest clinically significant persistent cardiac pathology in the UK Biobank population after generally milder (non-hospitalised) SARS-CoV-2 infection.

Background Measurement of cardiac structure and function from images (e.g. volumes, mass and derived parameters such as left ventricular (LV) ejection fraction [LVEF]) guides care for millions. This is best assessed using cardiovascular magnetic resonance (CMR), but image analysis is currently performed by individual clinicians, which introduces error. We sought to develop a machine learning algorithm for volumetric analysis of CMR images with demonstrably better precision than human analysis. Methods A fully automated machine learning algorithm was trained on 1923 scans (10 scanner models, 13 institutions, 9 clinical conditions, 60,000 contours) and used to segment the LV blood volume and myocardium. Performance was quantified by measuring precision on an independent multi-site validation dataset with multiple pathologies with n = 109 patients, scanned twice. This dataset was augmented with a further 1277 patients scanned as part of routine clinical care to allow qualitative assessment of generalization ability by identifying mis-segmentations. Machine learning algorithm (‘machine’) performance was compared to three clinicians (‘human’) and a commercial tool (cvi42, Circle Cardiovascular Imaging). Findings Machine analysis was quicker (20 s per patient) than human (13 min). Overall machine mis-segmentation rate was 1 in 479 images for the combined dataset, occurring mostly in rare pathologies not encountered in training. Without correcting these mis-segmentations, machine analysis had superior precision to three clinicians (e.g. scan-rescan coefficients of variation of human vs machine: LVEF 6.0% vs 4.2%, LV mass 4.8% vs. 3.6%; both P Conclusion We present a fully automated algorithm for measuring LV structure and global systolic function that betters human performance for speed and precision.

Background Hypertrophic cardiomyopathy (HCM) is caused by rare variants in sarcomere-encoding genes, but little is known about the clinical significance of these variants in the general population. Purpose To determine the population prevalence of HCM-associated sarcomeric variants, characterise their phenotypic manifestations, estimate penetrance, and identify associations between sarcomeric variants and clinical outcomes, we performed an observational study of 218,813 adults in the UK Biobank (UKBB), of whom 200,584 have whole exome sequencing (WES). Methods We carried out an integrated analysis of WES and cardiac magnetic resonance (CMR) imaging in UK Biobank participants stratified by sarcomere-encoding variant status. Computer vision techniques were used to automatically segment the four chambers of the heart (Figure 1). Cardiac motion analysis was used to derive strain and strain rates. Regional analysis of left ventricular wall thickness was performed using three-dimensional modelling of these segmentations. Results Median age at recruitment was 58 (IQR 50–63 years), and participants were followed up for a median of 10.8 years (IQR 9.9–11.6 years) with a total of 19,507 primary clinical events reported. The prevalence of rare variants (allele frequency SARC-P/LP variants were associated with increased risk of death or major adverse cardiac events (MACE) compared to controls (HR 1.68, 95% CI 1.37–2.06, p While males had a higher overall risk of adverse outcomes, the incremental genetic risk from SARC-P/LP mutations was greater in females (HR for females: 2.18 CI 1.65–2.89, p In 21,322 participants with CMR, SARC-P/LP were associated with asymmetric increase in left ventricular maximum wall thickness (10.9±2.7 vs 9.4±1.6 mm, p Conclusions In the general population, SARC-P/LP variants have low aggregate penetrance for overt HCM but are associated with increased risk of adverse cardiovascular outcomes and a sub-clinical cardiomyopathic phenotype. Although absolute event rates are low, identification of these variants may enhance risk stratification beyond familial disease. Funding Acknowledgement Type of funding sources: Public grant(s) – National budget only. Main funding source(s): The study was supported by the Medical Research Council, UK (MC-A651-53301); National Institute for Health Research (NIHR) Imperial College Biomedical Research Centre; NIHR Royal Brompton Cardiovascular Biomedical Research Unit; British Heart Foundation (NH/17/1/32725, RG/19/6/34387, RE/18/4/34215).

Background: Hypertrophic cardiomyopathy (HCM) is caused by rare variants in sarcomere-encoding genes, but little is known about the clinical significance of these variants in the general population. Objectives: The goal of this study was to compare lifetime outcomes and cardiovascular phenotypes according to the presence of rare variants in sarcomere-encoding genes among middle-aged adults. Methods: This study analyzed whole exome sequencing and cardiac magnetic resonance imaging in UK Biobank participants stratified according to sarcomere-encoding variant status. Results: The prevalence of rare variants (allele frequency

Aortic dimensions and distensibility are key risk factors for aortic aneurysms and dissections, as well as for other cardiovascular and cerebrovascular diseases. We tested genome-wide associations of ascending and descending aortic distensibility and area derived from cardiac magnetic resonance imaging (MRI) data of up to 32,590 Caucasian individuals in UK Biobank. We identified 102 loci (including 31 novel associations) tagging genes related to cardiovascular development, extracellular matrix production, smooth muscle cell contraction and heritable aortic diseases. Functional analyses highlighted four signalling pathways associated with aortic distensibility (TGF-β, IGF, VEGF and PDGF). We identified distinct sex-specific associations with aortic traits. We developed co-expression networks associated with aortic traits and applied phenome-wide Mendelian randomization (MR-PheWAS), generating evidence for a causal role for aortic distensibility in development of aortic aneurysms. Multivariable MR suggested a causal relationship between aortic distensibility and cerebral white matter hyperintensities, mechanistically linking aortic traits and brain small vessel disease.

We propose a dynamic spatio-temporal graph convolutional network (DST-GCN) approach to learn the left ventricular (LV) motion patterns from cardiac MR cine images. We represent the myocardial geometry using a graph that is constructed from sample nodes on endo- and epicardial contours. The DST-GCN follows an encoder-decoder framework. The encoder accepts a given cardiac motion represented by a sequence of ST-GCN. The decoder employs a graph-based gated recurrent unit (G-GRU) to predict future cardiac motion. We show that the DST-GCN can automatically quantify the spatio-temporal patterns in cardiac MR that characterise cardiac motion. Experiments are performed on the UK Biobank dataset. We compare four methods from two architecture variances. Experiments show that the proposed method inputting node velocities with residual connection in the decoder outperform others, and achieves a mean squared error of 0.135 pixel between the ground truth node locations and our prediction.

One of the challenges in developing deep learning algorithms for medical image segmentation is the scarcity of annotated training data. To overcome this limitation, data augmentation and semi-supervised learning (SSL) methods have been developed. However, these methods have limited effectiveness as they either exploit the existing data set only (data augmentation) or risk negative impact by adding poor training examples (SSL). Segmentations are rarely the final product of medical image analysis - they are typically used in downstream tasks to infer higher-order patterns to evaluate diseases. Clinicians take into account a wealth of prior knowledge on biophysics and physiology when evaluating image analysis results. We have used these clinical assessments in previous works to create robust quality-control (QC) classifiers for automated cardiac magnetic resonance (CMR) analysis. In this paper, we propose a novel scheme that uses QC of the downstream task to identify high quality outputs of CMR segmentation networks, that are subsequently utilised for further network training. In essence, this provides quality-aware augmentation of training data in a variant of SSL for segmentation networks (semiQCSeg). We evaluate our approach in two CMR segmentation tasks (aortic and short axis cardiac volume segmentation) using UK Biobank data and two commonly used network architectures (U-net and a Fully Convolutional Network) and compare against supervised and SSL strategies. We show that semiQCSeg improves training of the segmentation networks. It decreases the need for labelled data, while outperforming the other methods in terms of Dice and clinical metrics. SemiQCSeg can be an efficient approach for training segmentation networks for medical image data when labelled datasets are scarce., MICCAI STACOM 2020

Diastole is the sequence of physiological events that occur in the heart during ventricular filling and principally depends on myocardial relaxation and chamber stiffness. Abnormal diastolic function is related to many cardiovascular disease processes and is predictive of health outcomes, but its genetic architecture is largely unknown. Here, we use machine learning cardiac motion analysis to measure diastolic functional traits in 39,559 participants of UK Biobank and perform a genome-wide association study. We identified 9 significant, independent loci near genes that are associated with maintaining sarcomeric function under biomechanical stress and genes implicated in the development of cardiomyopathy. Age, sex and diabetes were independent predictors of diastolic function and we found a causal relationship between ventricular stiffness and heart failure. Our results provide novel insights into the genetic and environmental factors influencing diastolic function that are relevant for identifying causal relationships and tractable targets in heart failure.

Background:Excessive alcohol consumption is associated with damage to various organs, but its multi-organ effects have not been characterised across the usual range of alcohol drinking in a large general population sample.Methods:We assessed global effect sizes of alcohol consumption on quantitative magnetic resonance imaging phenotypic measures of the brain, heart, aorta, and liver of UK Biobank participants who reported drinking alcohol.Results:We found a monotonic association of higher alcohol consumption with lower normalised brain volume across the range of alcohol intakes (–1.7 × 10−3 ± 0.76 × 10−3 per doubling of alcohol consumption, p=3.0 × 10−14). Alcohol consumption was also associated directly with measures of left ventricular mass index and left ventricular and atrial volume indices. Liver fat increased by a mean of 0.15% per doubling of alcohol consumption.Conclusions:Our results imply that there is not a ‘safe threshold’ below which there are no toxic effects of alcohol. Current public health guidelines concerning alcohol consumption may need to be revisited.Funding:See acknowledgements.

The heart muscle diseases hypertrophic (HCM) and dilated (DCM) cardiomyopathies are leading causes of sudden death and heart failure in young, otherwise healthy, individuals. We conducted genome-wide association studies and multi-trait analyses in HCM (1,733 cases), DCM (5,521 cases) and nine left ventricular (LV) traits (19,260 UK Biobank participants with structurally normal hearts). We identified 16 loci associated with HCM, 13 with DCM and 23 with LV traits. We show strong genetic correlations between LV traits and cardiomyopathies, with opposing effects in HCM and DCM. Two-sample Mendelian randomization supports a causal association linking increased LV contractility with HCM risk. A polygenic risk score explains a significant portion of phenotypic variability in carriers of HCM-causing rare variants. Our findings thus provide evidence that polygenic risk score may account for variability in Mendelian diseases. More broadly, we provide insights into how genetic pathways may lead to distinct disorders through opposing genetic effects.

BackgroundHypertrophic cardiomyopathy (HCM) is caused by rare variants in sarcomere-encoding genes, but little is known about the clinical significance of these variants in the general population.MethodsWe compared outcomes and cardiovascular phenotypes in UK Biobank participants with whole exome sequencing stratified by sarcomere-encoding variant status.ResultsThe prevalence of rare variants (allele frequency ConclusionsIn the general population, SARC-P/LP variants have low aggregate penetrance for overt HCM but are associated with an increased risk of adverse cardiovascular outcomes and a sub-clinical cardiomyopathic phenotype. In contrast, rare sarcomeric variants that do not meet criteria to be classified as P/LP appear to have minimal clinical impact.

Excessive alcohol consumption is associated with damage to various organs, but its multi-organ effects have not been characterised across the usual range of alcohol drinking in a large general population sample. We assessed global effects of alcohol consumption on quantitative magnetic resonance imaging phenotypic measures of the brain, heart, aorta and liver of UK-Biobank participants who reported drinking alcohol. We found a monotonic association of higher alcohol consumption with lower normalised brain volume across the range of alcohol intakes (–1.7×10−3±0.76×10−3 per doubling of alcohol consumption, P=3.0×10−14). Alcohol consumption also was associated directly with measures of left ventricular mass index and left ventricular and atrial volume indices. Liver fat increased by a mean of 0.15% per doubling of alcohol consumption. Our results imply that there is not a “safe threshold” below which there are no toxic effects of alcohol. Current public health guidelines concerning alcohol consumption may need to be revisited.

The success of neural networks on medical image segmentation tasks typically relies on large labeled datasets for model training. However, acquiring and manually labeling a large medical image set is resource-intensive, expensive, and sometimes impractical due to data sharing and privacy issues. To address this challenge, we propose AdvChain, a generic adversarial data augmentation framework, aiming at improving both the diversity and effectiveness of training data for medical image segmentation tasks. AdvChain augments data with dynamic data augmentation, generating randomly chained photo-metric and geometric transformations to resemble realistic yet challenging imaging variations to expand training data. By jointly optimizing the data augmentation model and a segmentation network during training, challenging examples are generated to enhance network generalizability for the downstream task. The proposed adversarial data augmentation does not rely on generative networks and can be used as a plug-in module in general segmentation networks. It is computationally efficient and applicable for both low-shot supervised and semi-supervised learning. We analyze and evaluate the method on two MR image segmentation tasks: cardiac segmentation and prostate segmentation with limited labeled data. Results show that the proposed approach can alleviate the need for labeled data while improving model generalization ability, indicating its practical value in medical imaging applications., Comment: Under review

Deep learning-based segmentation methods are vulnerable to unforeseen data distribution shifts during deployment, e.g. change of image appearances or contrasts caused by different scanners, unexpected imaging artifacts etc. In this paper, we present a cooperative framework for training image segmentation models and a latent space augmentation method for generating hard examples. Both contributions improve model generalization and robustness with limited data. The cooperative training framework consists of a fast-thinking network (FTN) and a slow-thinking network (STN). The FTN learns decoupled image features and shape features for image reconstruction and segmentation tasks. The STN learns shape priors for segmentation correction and refinement. The two networks are trained in a cooperative manner. The latent space augmentation generates challenging examples for training by masking the decoupled latent space in both channel-wise and spatial-wise manners. We performed extensive experiments on public cardiac imaging datasets. Using only 10 subjects from a single site for training, we demonstrated improved cross-site segmentation performance, and increased robustness against various unforeseen imaging artifacts compared to strong baseline methods. Particularly, cooperative training with latent space data augmentation yields 15% improvement in terms of average Dice score when compared to a standard training method.

In recent years, the research landscape of machine learning in medical imaging has changed drastically from supervised to semi-, weakly- or unsupervised methods. This is mainly due to the fact that ground-truth labels are time-consuming and expensive to obtain manually. Generating labels from patient metadata might be feasible but it suffers from user-originated errors which introduce biases. In this work, we propose an unsupervised approach for automatically clustering and categorizing large-scale medical image datasets, with a focus on cardiac MR images, and without using any labels. We investigated the end-to-end training using both class-balanced and imbalanced large-scale datasets. Our method was able to create clusters with high purity and achieved over 0.99 cluster purity on these datasets. The results demonstrate the potential of the proposed method for categorizing unstructured large medical databases, such as organizing clinical PACS systems in hospitals.

We present a novel spatio-temporal graph convolutional networks (ST-GCN) approach to learn spatio-temporal patterns of left ventricular (LV) motion in cardiac MR cine images for improving the characterization of heart conditions. Specifically, a novel GCN architecture is used, where the sample nodes of endocardial and epicardial contours are connected as a graph to represent the myocardial geometry. We show that the ST-GCN can automatically quantify the spatio-temporal patterns in cine MR that characterise cardiac motion. Experiments are performed on healthy volunteers from the UK Biobank dataset. We compare different strategies for constructing cardiac structure graphs. Experiments show that the proposed methods perform well in estimating endocardial radii and characterising cardiac motion features for regional LV analysis.

This paper shows that deep modelling of subtle changes of cardiac motion can help in automated diagnosis of early onset of cardiac disease. In this paper, we model left ventricular (LV) cardiac motion in MRI sequences, based on a hybrid spatio-temporal network. Temporal data over long time periods is used as inputs to the model and delivers a dense displacement field (DDF) for regional analysis of LV function. A segmentation mask of the end-diastole (ED) frame is deformed by the predicted DDF from which regional analysis of LV function endocardial radius, thickness, circumferential strain (Ecc) and radial strain (Err) are estimated. Cardiac motion is estimated over MR cine loops. We compare the proposed technique to two other deep learning-based approaches and show that the proposed approach achieves promising predicted DDFs. Predicted DDFs are estimated on imaging data from healthy volunteers and patients with primary pulmonary hypertension from the UK Biobank. Experiments demonstrate that the proposed methods perform well in obtaining estimates of endocardial radii as cardiac motion-characteristic features for regional LV analysis.

ObjectivesThis study sought to investigate whether shape-based late gadolinium enhancement (LGE) metrics and simulations of re-entrant electrical activity are associated with arrhythmic events in patients with nonischemic dilated cardiomyopathy (NIDCM).BackgroundThe presence of LGE predicts life-threatening ventricular arrhythmias in NIDCM; however, risk stratification remains imprecise. LGE shape and simulations of electrical activity may be able to provide additional prognostic information.MethodsCardiac magnetic resonance (CMR)-LGE shape metrics were computed for a cohort of 156 patients with NIDCM and visible LGE and tested retrospectively for an association with an arrhythmic composite endpoint of sudden cardiac death and ventricular tachycardia. Computational models were created from images and used in conjunction with simulated stimulation protocols to assess the potential for re-entry induction in each patient’s scar morphology. A mechanistic analysis of the simulations was carried out to explain the associations.ResultsDuring a median follow-up of 1,611 (interquartile range: 881 to 2,341) days, 16 patients (10.3%) met the primary endpoint. In an inverse probability weighted Cox regression, the LGE–myocardial interface area (hazard ratio [HR]: 1.75; 95% confidence interval [CI]: 1.24 to 2.47; p = 0.001), number of simulated re-entries (HR: 1.40; 95% CI: 1.23 to 1.59; p ConclusionsThe area of interface between scar and surviving myocardium, as well as simulated re-entrant activity, are associated with an elevated risk of major arrhythmic events in patients with NIDCM and LGE and represent novel risk predictors.

Neural network-based approaches can achieve high accuracy in various medical image segmentation tasks. However, they generally require large labelled datasets for supervised learning. Acquiring and manually labelling a large medical dataset is expensive and sometimes impractical due to data sharing and privacy issues. In this work, we propose an adversarial data augmentation method for training neural networks for medical image segmentation. Instead of generating pixel-wise adversarial attacks, our model generates plausible and realistic signal corruptions, which models the intensity inhomogeneities caused by a common type of artefacts in MR imaging: bias field. The proposed method does not rely on generative networks, and can be used as a plug-in module for general segmentation networks in both supervised and semi-supervised learning. Using cardiac MR imaging we show that such an approach can improve the generalization ability and robustness of models as well as provide significant improvements in low-data scenarios., 13 pages. This paper is accepted to MICCAI 2020

In recent years, convolutional neural networks have demonstrated promising performance in a variety of medical image segmentation tasks. However, when a trained segmentation model is deployed into the real clinical world, the model may not perform optimally. A major challenge is the potential poor-quality segmentations generated due to degraded image quality or domain shift issues. There is a timely need to develop an automated quality control method that can detect poor segmentations and feedback to clinicians. Here we propose a novel deep generative model-based framework for quality control of cardiac MRI segmentation. It first learns a manifold of good-quality image-segmentation pairs using a generative model. The quality of a given test segmentation is then assessed by evaluating the difference from its projection onto the good-quality manifold. In particular, the projection is refined through iterative search in the latent space. The proposed method achieves high prediction accuracy on two publicly available cardiac MRI datasets. Moreover, it shows better generalisation ability than traditional regression-based methods. Our approach provides a real-time and model-agnostic quality control for cardiac MRI segmentation, which has the potential to be integrated into clinical image analysis workflows., Comment: The paper is accepted to MICCAI 2020

The inner surfaces of the human heart are covered by a complex network of muscular strands that is thought to be a remnant of embryonic development1,2. The function of these trabeculae in adults and their genetic architecture are unknown. Here we performed a genome-wide association study to investigate image-derived phenotypes of trabeculae using the fractal analysis of trabecular morphology in 18,096 participants of the UK Biobank. We identified 16 significant loci that contain genes associated with haemodynamic phenotypes and regulation of cytoskeletal arborization3,4. Using biomechanical simulations and observational data from human participants, we demonstrate that trabecular morphology is an important determinant of cardiac performance. Through genetic association studies with cardiac disease phenotypes and Mendelian randomization, we find a causal relationship between trabecular morphology and risk of cardiovascular disease. These findings suggest a previously unknown role for myocardial trabeculae in the function of the adult heart, identify conserved pathways that regulate structural complexity and reveal the influence of the myocardial trabeculae on susceptibility to cardiovascular disease. A genome-wide association study shows that myocardial trabeculae are an important determinant of cardiac performance in the adult heart, identifies conserved pathways that regulate structural complexity and reveals the influence of trabeculae on the susceptibility to cardiovascular disease.