Search

Atrial fibrillation (AF) is the most common sustained cardiac arrhythmia, characterised by a rapid and irregular electrical activation of the atria. Treatments for AF are often ineffective and few atrial biomarkers exist to automatically characterise atrial function and aid in treatment selection for AF. Clinical metrics of left atrial (LA) function, such as ejection fraction (EF) and active atrial contraction ejection fraction (aEF), are promising, but have until now typically relied on volume estimations extrapolated from single-slice images.

The subject of ‘fairness’ in artificial intelligence (AI) refers to assessing AI algorithms for potential bias based on demographic characteristics such as race and gender, and the development of algorithms to address this bias. Most applications to date have been in computer vision, although some work in healthcare has started to emerge. The use of deep learning (DL) in cardiac MR segmentation has led to impressive results in recent years, and such techniques are starting to be translated into clinical practice. However, no work has yet investigated the fairness of such models. In this work, we perform such an analysis for racial/gender groups, focusing on the problem of training data imbalance, using a nnU-Net model trained and evaluated on cine short axis cardiac MR data from the UK Biobank dataset, consisting of 5,903 subjects from 6 different racial groups. We find statistically significant differences in Dice performance between different racial groups. To reduce the racial bias, we investigated three strategies: (1) stratified batch sampling, in which batch sampling is stratified to ensure balance between racial groups; (2) fair meta-learning for segmentation, in which a DL classifier is trained to classify race and jointly optimized with the segmentation model; and (3) protected group models, in which a different segmentation model is trained for each racial group. We also compared the results to the scenario where we have a perfectly balanced database. To assess fairness we used the standard deviation (SD) and skewed error ratio (SER) of the average Dice values. Our results demonstrate that the racial bias results from the use of imbalanced training data, and that all proposed bias mitigation strategies improved fairness, with the best SD and SER resulting from the use of protected group models.

Cardiovascular magnetic resonance (CMR) is an established, non-invasive technique to comprehensively assess cardiovascular structure and function in a variety of acquired and inherited cardiac conditions. In addition to the heart, a typical CMR examination will also image adjacent thoracic and abdominal structures. Consequently, findings incidental to the cardiac examination may be encountered, some of which may be clinically relevant. We compare two deep learning architectures to automatically detect extra cardiac findings (ECFs) in the HASTE sequence of a CMR acquisition. The first one consists of a binary classification network that detects the presence of ECFs and the second one is a multi-label classification network that detects and classifies the type of ECF. We validated the two models on a cohort of 236 subjects, corresponding to 5610 slices, where 746 ECFs were found. Results show that the proposed methods have promising balanced accuracy and sensitivity and high specificity.

Sickle Cell Disease (SCD) is one of the most common genetic diseases in the world. Splenomegaly (abnormal enlargement of the spleen) is frequent among children with SCD. If left untreated, splenomegaly can be life-threatening. The current workflow to measure spleen size includes palpation, possibly followed by manual length measurement in 2D ultrasound imaging. However, this manual measurement is dependent on operator expertise and is subject to intra- and inter-observer variability. We investigate the use of deep learning to perform automatic estimation of spleen length from ultrasound images. We investigate two types of approach, one segmentation-based and one based on direct length estimation, and compare the results against measurements made by human experts. Our best model (segmentation-based) achieved a percentage length error of 7.42%, which is approaching the level of inter-observer variability (5.47%–6.34%). To the best of our knowledge, this is the first attempt to measure spleen size in a fully automated way from ultrasound images.

Surface reconstruction from magnetic resonance (MR) imaging data is indispensable in medical image analysis and clinical research. A reliable and effective reconstruction tool should: be fast in prediction of accurate well localised and high resolution models, evaluate prediction uncertainty, work with as little input data as possible. Current deep learning state of the art (SOTA) 3D reconstruction methods, however, often only produce shapes of limited variability positioned in a canonical position or lack uncertainty evaluation. In this paper, we present a novel probabilistic deep learning approach for concurrent 3D surface reconstruction from sparse 2D MR image data and aleatoric uncertainty prediction. Our method is capable of reconstructing large surface meshes from three quasi-orthogonal MR imaging slices from limited training sets whilst modelling the location of each mesh vertex through a Gaussian distribution. Prior shape information is encoded using a built-in linear principal component analysis (PCA) model. Extensive experiments on cardiac MR data show that our probabilistic approach successfully assesses prediction uncertainty while at the same time qualitatively and quantitatively outperforms SOTA methods in shape prediction. Compared to SOTA, we are capable of properly localising and orientating the prediction via the use of a spatially aware neural network.

Maintaining good cardiac function for as long as possible is a major concern for healthcare systems worldwide and there is much interest in learning more about the impact of different risk factors on cardiac health. The aim of this study is to analyze the impact of systolic blood pressure (SBP) on cardiac function while preserving the interpretability of the model using known clinical biomarkers in a large cohort of the UK Biobank population. We propose a novel framework that combines deep learning based estimation of interpretable clinical biomarkers from cardiac cine MR data with a variational autoencoder (VAE). The VAE architecture integrates a regression loss in the latent space, which enables the progression of cardiac health with SBP to be learnt. Results on 3,600 subjects from the UK Biobank show that the proposed model allows us to gain important insight into the deterioration of cardiac function with increasing SBP, identify key interpretable factors involved in this process, and lastly exploit the model to understand patterns of positive and adverse adaptation of cardiac function.

Deep learning methods for classifying medical images have demonstrated impressive accuracy in a wide range of tasks but often these models are hard to interpret, limiting their applicability in clinical practice. In this work we introduce a convolutional neural network model for identifying disease in temporal sequences of cardiac MR segmentations which is interpretable in terms of clinically familiar measurements. The model is based around a variational autoencoder, reducing the input into a low-dimensional latent space in which classification occurs. We then use the recently developed ‘concept activation vector’ technique to associate concepts which are diagnostically meaningful (eg. clinical biomarkers such as ‘low left-ventricular ejection fraction’) to certain vectors in the latent space. These concepts are then qualitatively inspected by observing the change in the image domain resulting from interpolations in the latent space in the direction of these vectors. As a result, when the model classifies images it is also capable of providing naturally interpretable concepts relevant to that classification and demonstrating the meaning of those concepts in the image domain. Our approach is demonstrated on the UK Biobank cardiac MRI dataset where we detect the presence of coronary artery disease.

Estimating dimensional measurements of the left ventricle provides diagnostic values which can be used to assess cardiac health and identify certain pathologies. In this paper we describe our methodology of calculating measurements from left ventricle segmentations automatically generated using deep learning. We use a U-net convolutional neural network architecture built from residual units to segment the left ventricle and then process these segmentations to estimate the area of the cavity and myocardium, the dimensions of the cavity, and the thickness of the myocardium. Determining if an image is part of the diastolic or systolic portion of the cardiac cycle is done by analysing the cavity volume. The quality of our results are dependent on our training regime where we have generated a large derivative dataset by augmenting the original images with free-form deformations. Our expanded training set, in conjunction with simple affine image transforms, creates a sufficiently large training population to prevent over-fitting of the network while still creating an accurate and robust segmentation network. Assessing our method on the STACOM18 LVQuan challenge dataset we find that it significantly outperforms the previously published state-of-the-art on a 5-fold validation all tasks considered.

Patient-specific 3D printing of congenital heart anatomy demands an accurate segmentation of the thin tissue interfaces which characterise these diagnoses. Even when a label set has a high spatial overlap with the ground truth, inaccurate delineation of these interfaces can result in topological errors. These compromise the clinical utility of such models due to the anomalous appearance of defects. CNNs have achieved state-of-the-art performance in segmentation tasks. Whilst data augmentation has often played an important role, we show that conventional image resampling schemes used therein can introduce topological changes in the ground truth labelling of augmented samples. We present a novel pipeline to correct for these changes, using a fast-marching algorithm to enforce the topology of the ground truth labels within their augmented representations. In so doing, we invoke the idea of cardiac contiguous topology to describe an arbitrary combination of congenital heart defects and develop an associated, clinically meaningful metric to measure the topological correctness of segmentations. In a series of five-fold cross-validations, we demonstrate the performance gain produced by this pipeline and the relevance of topological considerations to the segmentation of congenital heart defects. We speculate as to the applicability of this approach to any segmentation task involving morphologically complex targets.

We present a novel method to explicitly incorporate topological prior knowledge into deep learning based segmentation, which is, to our knowledge, the first work to do so. Our method uses the concept of persistent homology, a tool from topological data analysis, to capture high-level topological characteristics of segmentation results in a way which is differentiable with respect to the pixelwise probability of being assigned to a given class. The topological prior knowledge consists of the sequence of desired Betti numbers of the segmentation. As a proof-of-concept we demonstrate our approach by applying it to the problem of left-ventricle segmentation of cardiac MR images of subjects from the UK Biobank dataset, where we show that it improves segmentation performance in terms of topological correctness without sacrificing pixelwise accuracy.

Surface reconstruction is a vital tool in a wide range of areas of medical image analysis and clinical research. Despite the fact that many methods have proposed solutions to the reconstruction problem, most, due to their deterministic nature, do not directly address the issue of quantifying uncertainty associated with their predictions. We remedy this by proposing a novel probabilistic deep learning approach capable of simultaneous surface reconstruction and associated uncertainty prediction. The method incorporates prior shape information in the form of a principal component analysis (PCA) model. Experiments using the UK Biobank data show that our probabilistic approach outperforms an analogous deterministic PCA-based method in the task of 2D organ delineation and quantifies uncertainty by formulating distributions over predicted surface vertex positions.

Incorrect ECG gating of cardiac magnetic resonance (CMR) acquisitions can lead to artefacts, which hampers the accuracy of diagnostic imaging. Therefore, there is a need for robust reconstruction methods to ensure high image quality. In this paper, we propose a method to automatically correct motion-related artefacts in CMR acquisitions during reconstruction from k-space data. Our method is based on the Automap reconstruction method, which directly reconstructs high quality MR images from k-space using deep learning. Our main methodological contribution is the addition of an adversarial element to this architecture, in which the quality of image reconstruction (the generator) is increased by using a discriminator. We train the reconstruction network to automatically correct for motion-related artefacts using synthetically corrupted CMR k-space data and uncorrupted reconstructed images. Using 25000 images from the UK Biobank dataset we achieve good image quality in the presence of synthetic motion artefacts, but some structural information was lost. We quantitatively compare our method to a standard inverse Fourier reconstruction. In addition, we qualitatively evaluate the proposed technique using k-space data containing real motion artefacts.

Quality assessment of medical images is essential for complete automation of image processing pipelines. For large population studies such as the UK Biobank, artefacts such as those caused by heart motion are problematic and manual identification is tedious and time-consuming. Therefore, there is an urgent need for automatic image quality assessment techniques. In this paper, we propose a method to automatically detect the presence of motion-related artefacts in cardiac magnetic resonance (CMR) images. As this is a highly imbalanced classification problem (due to the high number of good quality images compared to the low number of images with motion artefacts), we propose a novel k-space based training data augmentation approach in order to address this problem. Our method is based on 3D spatio-temporal Convolutional Neural Networks, and is able to detect 2D+time short axis images with motion artefacts in less than 1 ms. We test our algorithm on a subset of the UK Biobank dataset consisting of 3465 CMR images and achieve not only high accuracy in detection of motion artefacts, but also high precision and recall. We compare our approach to a range of state-of-the-art quality assessment methods.

A cardiac motion atlas provides a space of reference in which the cardiac motion fields of a cohort of subjects can be directly compared. From such atlases, descriptors can be learned for subsequent diagnosis and characterization of disease. Traditionally, such atlases have been formed from imaging data acquired using a single modality. In this work we propose a framework for building a multimodal cardiac motion atlas from MR and ultrasound data and incorporate a multiview classifier to exploit the complementary information provided by the two modalities. We demonstrate that our novel framework is able to detect non ischemic dilated cardiomyopathy patients from ultrasound data alone, whilst still exploiting the MR based information from the multimodal atlas. We evaluate two different approaches based on multiview learning to implement the classifier and achieve an improvement in classification performance from 77.5% to 83.50% compared to the use of US data without the multimodal atlas.

We present a methodology for reconstructing full-cycle respiratory and cardiac gated short-axis cine MR sequences from real-time MR data. For patients who are too ill or otherwise incapable of consistent breath holds, real-time MR sequences are the preferred means of acquiring cardiac images, but suffer from inferior image quality compared to standard short-axis sequences and lack cardiac ECG gating. To construct a sequence from real-time images which, as close as possible, replicates the characteristics of short-axis series, the phase of the cardiac cycle must be estimated for each image and the left ventricle identified, to be used as a landmark for slice re-alignment. Our method employs CNN-based deep learning to segment the left ventricle in the real-time sequence, which is then used to estimate the pool volume and thus the position of each image in the cardiac cycle. We then use manifold learning to account for the respiratory cycle so as to select images of the best quality at expiration. From these images a selection is made to automatically reconstruct a single cardiac cycle, and the images and segmentations are then aligned. The aligned pool segmentations can then be used to calculate volume over time and thus volume-based biomarkers.

Cardiac magnetic resonance (CMR) can be used for quantitative analysis of heart function. However, CMR imaging typically involves acquiring 2D image planes during separate breath-holds, often resulting in misalignment of the heart between image planes in 3D. Accurate quantitative analysis requires a robust 3D reconstruction of the heart from CMR images, which is adversely affected by such motion artifacts. Therefore, we propose a fully automated method for motion correction of CMR planes using segmentations produced by fully convolutional neural networks (FCNs). FCNs are trained on 100 UK Biobank subjects to produce short-axis and long-axis segmentations, which are subsequently used in an iterative registration algorithm for correcting breath-hold induced motion artifacts. We demonstrate significant improvements in motion-correction over image-based registration, with strong correspondence to results obtained using manual segmentations. We also deploy our automatic method on 9,353 subjects in the UK Biobank database, demonstrating significant improvements in 3D plane alignment.

Training a fully convolutional network for pixel-wise (or voxel-wise) image segmentation normally requires a large number of training images with corresponding ground truth label maps. However, it is a challenge to obtain such a large training set in the medical imaging domain, where expert annotations are time-consuming and difficult to obtain. In this paper, we propose a semi-supervised learning approach, in which a segmentation network is trained from both labelled and unlabelled data. The network parameters and the segmentations for the unlabelled data are alternately updated. We evaluate the method for short-axis cardiac MR image segmentation and it has demonstrated a high performance, outperforming a baseline supervised method. The mean Dice overlap metric is 0.92 for the left ventricular cavity, 0.85 for the myocardium and 0.89 for the right ventricular cavity. It also outperforms a state-of-the-art multi-atlas segmentation method by a large margin and the speed is substantially faster.

Cardiac phase determination of 3D ultrasound (US) imaging has numerous applications including intra- and inter-modality registration of US volumes, and gating of live images. We have developed a novel and potentially clinically useful real-time three-dimensional (3D) cardiac motion gating technique that facilitates and supports 3D US-guided procedures. Our proposed real-time 3D-Masked-PCA technique uses the Principal Component Analysis (PCA) statistical method in combination with other image processing operations. Unlike many previously proposed gating techniques that are either retrospective and hence cannot be applied on live data, or can only gate respiratory motion, the technique is able to extract the phase of live 3D cardiac US data. It is also robust to varying image-content; thus it does not require specific structures to be visible in the US image. We demonstrate the application of the technique for the purposes of real-time 3D cardiac gating of trans-oesophageal US used in electrophysiology (EP) and trans-catheter aortic valve implantation (TAVI) procedures. The algorithm was validated using 2 EP and 8 TAVI clinical sequences (623 frames in total), from patients who underwent left atrial ablation and aortic valve replacement, respectively. The technique successfully located all of the 69 end-systolic and end-diastolic gating points in these sequences.

Imaging of the heart during exercise can improve detection and treatment of heart diseases but is challenging using current clinically applied cardiac MRI (cMRI) techniques. Real-time (RT) imaging strategies have recently been proposed for exercise cMRI, but respiratory motion and unreliable cardiac gating introduce significant errors in quantification of cardiac function. Self-navigated cMRI sequences are currently not routinely available in a clinical environment. We aim to establish a method for cardiac and respiratory gated cine exercise cMRI that can be applied in a clinical cMRI setting. We developed a retrospective, image-based cardiac and respiratory gating and reconstruction framework based on widely available highly accelerated dynamic imaging. From the acquired dynamic images, respiratory motion was estimated using manifold learning. Cardiac periodicity was obtained by identifying local maxima in the temporal frequency spectrum of the spatial means of the images. We then binned the dynamic images in respiratory and cardiac phases and subsequently registered and averaged them to reconstruct a respiratory and cardiac gated cine stack. We evaluated our method in healthy volunteers and patients with heart diseases and demonstrate good agreement with existing RT acquisitions (R = .82). We show that our reconstruction pipeline yields better image quality and has lower inter- and intra-observer variability compared to RT imaging. Subsequently, we demonstrate that our method is able to detect a pathological response to exercise in patients with heart diseases, illustrating its potential benefit in cardiac diagnostic and prognostic assessment.

Cardiac motion is inherently tied to the disease state of the heart, and as such can be used to identify the presence and extent of different cardiac pathologies. Abnormal cardiac motion can manifest at different spatial scales of the myocardium depending on the disease present. The importance of spatial scale in the analysis of cardiac motion has not previously been explicitly investigated. In this paper, a novel approach is presented for analysing myocardial strains at different spatial scales using a cardiac motion atlas to find the optimal scales for (1) predicting response to cardiac resynchronisation therapy and (2) identifying the presence of strict left bundle-branch block in a patient cohort of 34. Optimal spatial scales for the two applications were found to be \(4\%\) and \(16\%\) of left ventricular volume with accuracies of \(84.8 \pm 8.4\%\) and \(81.3 \pm 12.6\%\), respectively, using a repeated, stratified cross-validation.

We propose a novel technique for the localisation of Left Ventricular LV scar based on local motion descriptors. Cardiac MR imaging is employed to construct a spatio-temporal motion atlas where the LV motion of different subjects can be directly compared. Local motion descriptors are derived from the motion atlas and dictionary learning is used for scar classification. Preliminary results on a cohort of 20 patients show a sensitivity and specificity of $$80\,\%$$ and $$87\,\%$$ in a binary classification setting.

A major challenge for cardiac motion analysis is the high-dimensionality of the motion data. Conventionally, the AHA model is used for dimensionality reduction, which divides the left ventricle into 17 segments using criteria based on anatomical structures. In this paper, a novel method is proposed to divide the left ventricle into homogeneous parcels in terms of motion trajectories. We demonstrate that the motion-driven parcellation has good reproducibility and use it for data reduction and motion description on a dataset of 1093 subjects. The resulting motion descriptor achieves high performance on two exemplar applications, namely gender and age predictions. The proposed method has the potential to be applied to groupwise motion analysis.

In this work we apply manifold learning to biophysical modeling of cardiac contraction with the aim of estimating material parameters characterizing myocardial stiffness and contractility. The set of cardiac cycle simulations spanning the parameter space of myocardial stiffness and contractility is used to create a manifold structure based on the motion pattern of the left ventricle endocardial surfaces. First, we assess the proposed method by using synthetic data generated by the model specifically to test our approach with the known ground truth parameter values. Then, we apply the method on cardiac magnetic resonance imaging (MRI) data of two healthy volunteers. The post-processed cine MRI for each volunteer were embedded into the manifold together with the simulated samples and the global parameters of contractility and stiffness for the whole myocardium were estimated. Then, we used these parameters as an initialization into an estimator of regional contractilities based on a reduced order unscented Kalman filter. The global values of stiffness and contractility obtained by manifold learning corrected the model in comparison to a standard model calibration by generic parameters, and a significantly more accurate estimation of regional contractilities was reached when using the initialization given by manifold learning.

Manifold alignment can be used to reduce the dimensionality of multiple medical image datasets into a single globally consistent low-dimensional space. This may be desirable in a wide variety of problems, from fusion of different imaging modalities for Alzheimer’s disease classification to 4DMR reconstruction from 2D MR slices. Unfortunately, most existing manifold alignment techniques require either a set of prior correspondences or comparability between the datasets in high-dimensional space, which is often not possible. We propose a novel technique for the ‘self-alignment’ of manifolds (SAM) from multiple dissimilar imaging datasets without prior correspondences or inter-dataset image comparisons. We quantitatively evaluate the method on 4DMR reconstruction from realistic, synthetic sagittal 2D MR slices from 6 volunteers and real data from 4 volunteers. Additionally, we demonstrate the technique for the compounding of two free breathing 3D ultrasound views from one volunteer. The proposed method performs significantly better for 4DMR reconstruction than state-of-the-art image-based techniques.

Cardiorespiratory phase determination has numerous applications during cardiac imaging. We propose a novel view-angle independent prospective cardiorespiratory motion gating technique for X-ray fluoroscopy images that are used to guide cardiac electrophysiology procedures. The method is based on learning coronary sinus catheter motion using principal component analysis and then applying the derived motion model to unseen images taken at arbitrary projections. We validated our technique on 7 sequential biplane sequences in normal and very low dose scenarios and on 5 rotational sequences in normal dose. For the normal dose images we established average systole, end-inspiration and end-expiration gating success rates of 100 %, 97.4 % and 95.2 %, respectively. For very low dose applications, the method was tested on images with added noise. Average gating success rates were 93.4 %, 90 % and 93.4 % even at the low SNR value of \(\sqrt{5}\), representing a dose reduction of more than 10 times. This technique can extract clinically useful motion information whilst minimising exposure to ionising radiation.

Motion, together with shape, reflect important aspects of cardiac function. In this work, a new method is proposed for learning of a cardiac motion descriptor from a data-driven perspective. The resulting descriptor can characterise the global motion pattern of the left ventricle with a much lower dimension than the original motion data. It has demonstrated its predictive power on two exemplar classification tasks on a large cohort of 1093 normal subjects.