Search

Longitudinal image registration is challenging and has not yet benefited from major performance improvements thanks to deep-learning. Inspired by Deep Image Prior, this paper introduces a different use of deep architectures as regularizers to tackle the image registration question. We propose a subject-specific deformable registration method called MIRRBA, relying on a deep pyramidal architecture to be the prior parametric model constraining the deformation field. Diverging from the supervised learning paradigm, MIRRBA does not require a learning database, but only the pair of images to be registered to optimize the network's parameters and provide a deformation field. We demonstrate the regularizing power of deep architectures and present new elements to understand the role of the architecture in deep learning methods for registration. Hence, to study the impact of the network parameters, we ran our method with different architectural configurations on a private dataset of 110 metastatic breast cancer full-body PET images with manual segmentations of the brain, bladder and metastatic lesions. We compared it against conventional iterative registration approaches and supervised deep learning-based models. Global and local registration accuracies were evaluated using the detection rate and the Dice score respectively, while registration realism was evaluated using the Jacobian's determinant. Moreover, we computed the ability of the different methods to shrink vanishing lesions with the disappearing rate. MIRRBA significantly improves the organ and lesion Dice scores of supervised models. Regarding the disappearing rate, MIRRBA more than doubles the best performing conventional approach SyNCC score. Our work therefore proposes an alternative way to bridge the performance gap between conventional and deep learning-based methods and demonstrates the regularizing power of deep architectures., Comment: 11 pages 3 figures in the main article 2 tables in the main article 2 figures in supplementary material

This paper addresses the domain shift problem for segmentation. As a solution, we propose OLVA, a novel and lightweight unsupervised domain adaptation method based on a Variational Auto-Encoder (VAE) and Optimal Transport (OT) theory. Thanks to the VAE, our model learns a shared cross-domain latent space that follows a normal distribution, which reduces the domain shift. To guarantee valid segmentations, our shared latent space is designed to model the shape rather than the intensity variations. We further rely on an OT loss to match and align the remaining discrepancy between the two domains in the latent space. We demonstrate OLVA's effectiveness for the segmentation of multiple cardiac structures on the public Multi-Modality Whole Heart Segmentation (MM-WHS) dataset, where the source domain consists of annotated 3D MR images and the unlabelled target domain of 3D CTs. Our results show remarkable improvements with an additional margin of 12.5\% dice score over concurrent generative training approaches., Comment: 10 pages, 3 figures, conference MICCAI

We study the fully convolutional neural networks in the context of malignancy detection for breast cancer screening. We work on a supervised segmentation task looking for an acceptable compromise between the precision of the network and the computational complexity., Comment: in Proceedings of iTWIST'20, Paper-ID: 30, Nantes, France, December, 2-4, 2020

In a low-statistics PET imaging context, the positive bias in regions of low activity is a burning issue. To overcome this problem, algorithms without the built-in non-negativity constraint may be used. They allow negative voxels in the image to reduce, or even to cancel the bias. However, such algorithms increase the variance and are difficult to interpret, since negative radioactive concentrations have no physical meaning. Here, we propose a post-processing strategy to remove negative intensities while preserving the local mean activities. Our idea is to transfer the negative intensities to neighboring voxels, so that the mean of the image is preserved. The proposed post-processing algorithm solves a linear programming problem with a specific symmetric structure, and the solution can be computed in a very efficient way. Acquired data from an yttrium-90 phantom show that on images produced by a non-constrained algorithm, a much lower variance in the cold area is obtained after the post-processing step., 3 pages, 4 figures, in Proceedings of iTWIST'20, Paper-ID: 27, Nantes, France, December, 2-4, 2020

Catheter segmentation in 3D ultrasound is important for computer-assisted cardiac intervention. However, a large amount of labeled images are required to train a successful deep convolutional neural network (CNN) to segment the catheter, which is expensive and time-consuming. In this paper, we propose a novel catheter segmentation approach, which requests fewer annotations than the supervised learning method, but nevertheless achieves better performance. Our scheme considers a deep Q learning as the pre-localization step, which avoids voxel-level annotation and which can efficiently localize the target catheter. With the detected catheter, patch-based Dual-UNet is applied to segment the catheter in 3D volumetric data. To train the Dual-UNet with limited labeled images and leverage information of unlabeled images, we propose a novel semi-supervised scheme, which exploits unlabeled images based on hybrid constraints from predictions. Experiments show the proposed scheme achieves a higher performance than state-of-the-art semi-supervised methods, while it demonstrates that our method is able to learn from large-scale unlabeled images.

Clinical neuroimaging has recently witnessed explosive growth in data availability which brings studying heterogeneity in clinical cohorts to the spotlight. Normative modeling is an emerging statistical tool for achieving this objective. However, its application remains technically challenging due to difficulties in properly dealing with nuisance variation, for example due to variability in image acquisition devices. Here, in a fully probabilistic framework, we propose an application of hierarchical Bayesian regression (HBR) for multi-site normative modeling. Our experimental results confirm the superiority of HBR in deriving more accurate normative ranges on large multi-site neuroimaging data compared to widely used methods. This provides the possibility i) to learn the normative range of structural and functional brain measures on large multi-site data; ii) to recalibrate and reuse the learned model on local small data; therefore, HBR closes the technical loop for applying normative modeling as a medical tool for the diagnosis and prognosis of mental disorders.

This work focuses on the automatic quantification of the breast density from digital mammography imaging. Using only categorical image-wise labels we train a model capable of predicting continuous density percentage as well as providing a pixel wise support frit for the dense region. In particular we propose a weakly supervised loss linking the density percentage to the mask size., Comment: MIDL 2019 [arXiv:1907.08612]

In this paper, we target the problem of fracture classification from clinical X-Ray images towards an automated Computer Aided Diagnosis (CAD) system. Although primarily dealing with an image classification problem, we argue that localizing the fracture in the image is crucial to make good class predictions. Therefore, we propose and thoroughly analyze several schemes for simultaneous fracture localization and classification. We show that using an auxiliary localization task, in general, improves the classification performance. Moreover, it is possible to avoid the need for additional localization annotations thanks to recent advancements in weakly-supervised deep learning approaches. Among such approaches, we investigate and adapt Spatial Transformers (ST), Self-Transfer Learning (STL), and localization from global pooling layers. We provide a detailed quantitative and qualitative validation on a dataset of 1347 femur fractures images and report high accuracy with regard to inter-expert correlation values reported in the literature. Our investigations show that i) lesion localization improves the classification outcome, ii) weakly-supervised methods improve baseline classification without any additional cost, iii) STL guides feature activations and boost performance. We plan to make both the dataset and code available., Comment: 7 pages, 4 figures

Activity recognition has primarily addressed the identification of either actions or well-defined interactions among objects in a scene. In this work, we extend the scope to the study of workflow monitoring. In a workflow, ordered groups of activities (phases) with different durations take place in a constrained environment and create temporal patterns across the workflow instances. We address the problem of recognizing phases, based on exemplary recordings. We propose to use Workflow-HMMs, a form of HMMs augmented with phase probability variables that model the complete workflow process. This model takes into account the full temporal context which improves on-line recognition of the phases, especially in case of partial labeling. Targeted applications are workflow monitoring in hospitals and factories, where common action recognition approaches are difficult to apply. To avoid interfering with the normal workflow, we capture the activity of a room with a multiple-camera system. Additionally, we propose to rely on real-time low-level features (3D motion flow) to maintain a generic approach. We demonstrate our methods on sequences from medical procedures performed in a mock-up operating room. The sequences follow a complex workflow, containing various alternatives.

This thesis investigates two key aspects of learning deep-based image representations for medical diagnosis. The two are confronted with common challenges of medical image databases, namely, the limited number of samples, the presence of unreliable annotations and class-imbalance; as well as, domain shift and data privacy constraints for collaborative learning across institutions. The first part of this thesis concerns the architectural design of deep learning approaches. We explore the importance of localizing the region of interest in the image prior to the classification and the implicit capsule networks’ approach to model spatial information. We verify the importance of localization as a preliminary step to the classification, provide a sensitivity analysis of the size of the region of interest, and discuss image retrieval as a clinical use case. We also validate that capsules create equivariance, thus requiring to see fewer viewpoints of the object of interest. The second part of the thesis focuses on easing the optimization of the deep network parameters by gradually increasing the difficulty of the training samples. This gradual increase is based on the concept of curriculum learning and achieved with a data scheduler that controls the order and pace of the samples. We validate the beneficial effect of the curriculum data schedulers in two scenarios. First, we leveraged prior knowledge and uncertainty for the fine-grained classification of proximal femur fractures. In this case, we demonstrated the benefits of our proposed curriculum method under controlled scenarios: with limited amounts of data, under class-imbalance, and in the presence of label noise. Second, we verified the positive effect of the curriculum data scheduler for multi-site breast cancer classification in a federated learning setup. Esta tesis investiga dos aspectos fundamentales del aprendizaje de representaciones profundas de imágenes para el diagnóstico médico. Ambos se enfrentan a los retos comunes de las bases de datos de imágenes médicas, a saber, el número limitado de muestras, la presencia de anotaciones poco fiables y el desequilibrio de clases; así como, la adaptación al dominio (“domain adaptation”) y las restricciones de privacidad de datos para el aprendizaje colaborativo entre instituciones. La primera parte de esta tesis se centra en el diseño de arquitecturas para métodos de aprendizaje profundo (“deep learning”). Exploramos la importancia de localizar la región de interés en la imagen antes de la clasificación y el enfoque implícito de redes capsulares (“capsule networks”) para modelar la información espacial. Verificamos la importancia de la localización como paso previo a la clasificación, proporcionamos un análisis de sensibilidad del tamaño de la región de interés y discutimos la recuperación de imágenes como caso de uso clínico. También validamos que las cápsulas crean equidistancia, por lo que requieren ver menos puntos de vista del objeto de interés. La segunda parte de la tesis se enfoca en facilitar la optimización de los parámetros de la red aumentando gradualmente la dificultad de las muestras de entrenamiento. Este aumento gradual se basa en el concepto de aprendizaje curricular (“curriculum learning”) y se consigue con un programador de datos (“data scheduler”) que controla el orden y el ritmo de las muestras. Validamos el efecto beneficioso de los programadores de datos curriculares en dos escenarios. En primer lugar, aprovechamos el conocimiento previo y la incertidumbre para la clasificación granular de las fracturas de fémur proximal. En este caso, demostramos los beneficios de nuestro método basado en aprendizaje curricular bajo escenarios controlados: con cantidades limitadas de datos, desequilibrio de clases y en presencia de anotaciones imprecisas. En segundo lugar, verificamos el efecto positivo del planificador de datos para la clasificación del cáncer de mama en una configuración de aprendizaje federado (“federated learning”).

International audience; 3D ultrasound imaging has high potential for various clinical applications, but often suffers from high operator-dependency and the directionality of the acquired data. State-of-the-art systems mostly perform compounding of the image data prior to further processing and visualization, resulting in 3D volumes of scalar intensities. This work presents computational sonography as a novel concept to represent 3D ultrasound as tensor instead of scalar fields, mapping a full and arbitrary 3D acquisition to the reconstructed data. The proposed representation compactly preserves significantly more information about the anatomy-specific and direction-depend acquisition, facilitating both targeted data processing and improved visualization. We show the potential of this paradigm on ultrasound phantom data as well as on clinically acquired data for acquisitions of the femoral, brachial and antebrachial bone.

International audience; Automatic localization of multiple anatomical structures in medical images provides important semantic information with potential benefits to diverse clinical applications. Aiming at organ-specific attenuation correction in PET/MR imaging, we propose an efficient approach for estimating location and size of multiple anatomical structures in MR scans. Our contribution is three-fold: (1) we apply supervised regression techniques to the problem of anatomy detection and localization in whole-body MR, (2) we adapt random ferns to produce multidimensional regression output and compare them with random regression forests, and (3) introduce the use of 3D LBP descriptors in multi-channel MR Dixon sequences. The localization accuracy achieved with both fern- and forest-based approaches is evaluated by direct comparison with state of the art atlas-based registration, on ground-truth data from 33 patients. Our results demonstrate improved anatomy localization accuracy with higher efficiency and robustness.

Diffusion magnetic resonance imaging (dMRI) is a non-invasive method that allows connectivity mapping of the brain. However, despite major advances in this field, accurate inference of these patterns and its applicability within a clinical context is still in its early stages. This thesis describes a conceptually novel method for reconstructing neuronal pathways inside the brain from diffusion-weighted imaging (DWI) measurements with high angular resolution and short data acquisition time. The proposed method combines recent theoretical advances on spherical sampling and noise reduction techniques from the field of compressed sensing. Numerical simulations were performed to study the best sampling strategy under a novel sampling theorem on the sphere in order to reduce the acquisition time during dMRI scans. Furthermore, these results were combined with the recently proposed spherical deconvolution technique to reconstruct the distribution of neuronal tracts (or fibers) within one voxel with high angular resolution between multiple crossing fibers. The spherical deconvolution problem was hereby formulated as an inverse problem and solved using techniques adopted from the field of compressed sensing. Since the result of the spherical deconvolution step is sparse in nature, the basis pursuit denoising formulation of the inverse problem is optimal within this context. Finally, the resulting fiber orientation reconstruction was compared with diffusion spectrum imaging (DSI) – a classic model-free acquisition method. Simulations revealed that the proposed approach is superior to DSI in terms of both, acquisition time and angular resolution of crossing fibers (>=40° with at least 90% sensitivity). Our investigations showed that the application of spherical deconvolution stated as a basis pursuit denoising problem holds great promise for high angular resolution dMRI.

International audience; Postprocedural analysis of gastrointestinal (GI) endoscopic videos is a difficult task because the videos often suffer from a large number of poor-quality frames due to the motion or out-of-focus blur, specular highlights and artefacts caused by turbid fluid inside the GI tract. Clinically, each frame of the video is examined individually by the endoscopic expert due to the lack of a suitable visualisation technique. In this work, we introduce a low dimensional representation of endoscopic videos based on a manifold learning approach. The introduced endoscopic video manifolds (EVMs) enable the clustering of poor-quality frames and grouping of different segments of the GI endoscopic video in an unsupervised manner to facilitate subsequent visual assessment. In this paper, we present two novel inter-frame similarity measures for manifold learning to create structured manifolds from complex endoscopic videos. Our experiments demonstrate that the proposed method yields high precision and recall values for uninformative frame detection (90.91% and 82.90%) and results in well-structured manifolds for scene clustering.