Search

International audience; Frontotemporal dementia and amyotrophic lateral sclerosis are rare neurodegenerative diseases with no effective treatment. The development of biomarkers allowing an accurate assessment of disease progression is crucial for evaluating new therapies. Concretely, neuroimaging and transcriptomic (microRNA) data have been shown useful in tracking their progression. However, no single biomarker can accurately measure progression in these complex diseases. Additionally, large samples are not available for such rare disorders. It is thus essential to develop methods that can model disease progression by combining multiple biomarkers from small samples. In this paper, we propose a new framework for computing a disease progression score (DPS) from cross-sectional multimodal data. Specifically, we introduce a supervised multimodal variational autoencoder that can infer a meaningful latent space, where latent representations are placed along a disease trajectory. A score is computed by orthogonal projections onto this path. We evaluate our framework with multiple synthetic datasets and with a real dataset containing 14 patients, 40 presymptomatic genetic mutation carriers and 37 controls from the PREV-DEMALS study. There is no ground truth for the DPS in real-world scenarios, therefore we use the area under the ROC curve (AUC) as a proxy metric. Results with the synthetic datasets support this choice, since the higher the AUC, the more accurate the predicted simulated DPS. Experiments with the real dataset demonstrate better performance in comparison with stateof-the-art approaches. The proposed framework thus leverages cross-sectional multimodal datasets with small sample sizes to objectively measure disease progression, with potential application in clinical trials.

Convolutional neural networks (CNN) have demonstrated their ability to segment 2D cardiac ultrasound images. However, despite recent successes according to which the intra-observer variability on end-diastole and end-systole images has been reached, CNNs still struggle to leverage temporal information to provide accurate and temporally consistent segmentation maps across the whole cycle. Such consistency is required to accurately describe the cardiac function, a necessary step in diagnosing many cardiovascular diseases. In this paper, we propose a framework to learn the 2D+time apical long-axis cardiac shape such that the segmented sequences can benefit from temporal and anatomical consistency constraints. Our method is a post-processing that takes as input segmented echocardiographic sequences produced by any state-of-the-art method and processes it in two steps to (i) identify spatio-temporal inconsistencies according to the overall dynamics of the cardiac sequence and (ii) correct the inconsistencies. The identification and correction of cardiac inconsistencies relies on a constrained autoencoder trained to learn a physiologically interpretable embedding of cardiac shapes, where we can both detect and fix anomalies. We tested our framework on 98 full-cycle sequences from the CAMUS dataset, which are available alongside this paper. Our temporal regularization method not only improves the accuracy of the segmentation across the whole sequences, but also enforces temporal and anatomical consistency., 12 pages, accepted for publication in IEEE TMI

We propose a novel method to generate a single image product from a multi-image strip acquired by a push-frame satellite imaging system. The images of the push-frame strips are combined into a large scale mosaic simulating a perfect sensor geometry. The local camera models of the input images are leveraged to produce a new localization model that covers the output mosaic entirely. Among other applications, this simplifies the task of stereo reconstruction enormously: instead of treating multiple stereo pairs of small images, it is possible to reconstruct the entire area covered by the push-frame acquisition using a single pair of mosaics incorporating all the images. We test our method using strips of SkySat L1B scenes and denote the output images as L1B+. To evaluate the quality of the L1B+ images and their localization models, the stereo reconstructions obtained with L1B+ are compared with those obtained with L1B and with a lidar reference model.

We present a protocol to construct your own depth validation dataset for navigation. This protocol, called RDC for Rigid Depth Constructor, aims at being more accessible and cheaper than already existing techniques, requiring only a camera and a Lidar sensor to get started. We also develop a test suite to get insightful information from the evaluated algorithm. Finally, we take the example of UAV videos, on which we test two depth algorithms that were initially tested on KITTI and show that the drone context is dramatically different from in-car videos. This shows that a single context benchmark should not be considered reliable, and when developing a depth estimation algorithm, one should benchmark it on a dataset that best fits one's particular needs, which often means creating a brand new one. Along with this paper we provide the tool with an open source implementation and plan to make it as user-friendly as possible, to make depth dataset creation possible even for small teams. Our key contributions are the following: We propose a complete, open-source and almost fully automatic software application for creating validation datasets with densely annotated depth, adaptable to a wide variety of image, video and range data. It includes selection tools to adapt the dataset to specific validation needs, and conversion tools to other dataset formats. Using this application, we propose two new real datasets, outdoor and indoor, readily usable in UAV navigation context. Finally as examples, we show an evaluation of two depth prediction algorithms, using a collection of comprehensive (e.g. distribution based) metrics.

Video anomaly detection in surveillance systems with only video-level labels (i.e. weakly-supervised) is challenging. This is due to, (i) the complex integration of human and scene based anomalies comprising of subtle and sharp spatio-temporal cues in real-world scenarios, (ii) non-optimal optimization between normal and anomaly instances under weak supervision. In this paper, we propose a Human-Scene Network to learn discriminative representations by capturing both subtle and strong cues in a dissociative manner. In addition, a self-rectifying loss is also proposed that dynamically computes the pseudo temporal annotations from video-level labels for optimizing the Human-Scene Network effectively. The proposed Human-Scene Network optimized with self-rectifying loss is validated on three publicly available datasets i.e. UCF-Crime, ShanghaiTech and IITB-Corridor, outperforming recently reported state-of-the-art approaches on five out of the six scenarios considered.

This paper proposes a novel framework for recognizing industrial actions, in the perspective of human-robot collaboration. Given a one second long measure of the human's motion, the framework can determine his/her action. The originality lies in the use of joint angles, instead of Cartesian coordinates. This design choice makes the framework sensor agnostic and invariant to affine transformations and to anthropometric differences. On AnDy dataset, we outperform the state of art classifier. Furthermore, we show that our framework is effective with limited training data, that it is subject independent, and that it is compatible with robotic realtime constraints. In terms of methodology, the framework is an original synergy of two antithetical schools of thought: modelbased and data-based algorithms. Indeed, it is the cascade of an inverse kinematics estimator compliant with the International Society of Biomechanics recommendations, followed by a deep learning architecture based on Bidirectional Long Short Term Memory. We believe our work may pave the way to successful and fast action recognition with standard depth cameras, embedded on moving collaborative robots.

Deep-learning techniques have enabled a breakthrough in robustness and execution time in automated cell detection in live fluorescence microscopy datasets. However, the heterogeneity, dimensionality and ever-growing size of 3D+time datasets challenge the evaluation of measurements. Here we propose a quality score for the accuracy of cell segmentation maps that is detector-independent and does not need any groundtruth nor priors on object appearance. Our method learns the dynamic parameters of each cell to detect inconsistencies in local displacements induced by segmentation errors. Using simulations that approximate the dynamics of cellular aggregates, we demonstrate the score ability to rank the performance of detectors up to 40% of false positives. On live volumetric imaging of organoids, our score is able to appropriately rank two stateof-the-art pre-trained deep-learning detectors (Stardist3D and Cellpose).; Les techniques d'apprentissage profond ont permis une percée dans la robustesse et le temps d'exécution de la détection automatisée des cellules dans les ensembles de données de microscopie à fluorescence en direct. Cependant, l'hétérogénéité, la dimensionnalité et la taille toujours croissante des ensembles de données 3D+temps compliquent l'évaluation des mesures. Nous proposons ici un score de qualité pour la précision des cartes de segmentation cellulaire qui est indépendant du détecteur et ne nécessite aucune vérité de base ni aucun a priori sur l'apparence de l'objet. Notre méthode apprend les paramètres dynamiques de chaque cellule pour détecter les incohérences dans les déplacements locaux induits par les erreurs de segmentation. En utilisant des simulations qui se rapprochent de la dynamique des agrégats cellulaires, nous démontrons la capacité du score à classer la performance des détecteurs jusqu'à 40% de faux positifs. Sur l'imagerie volumétrique en direct d'organoïdes, notre score est capable de classer de manière appropriée deux détecteurs d'apprentissage profond pré-entraînés de l'état de l'art (Stardist3D et Cellpose).

We propose a state-of-the-art method for super-resolution with non-uniform blur. Single-image super-resolution methods seek to restore a high-resolution image from blurred, subsampled, and noisy measurements. Despite their impressive performance, existing techniques usually assume a uniform blur kernel. Hence, these techniques do not generalize well to the more general case of non-uniform blur. Instead, in this paper, we address the more realistic and computationally challenging case of spatially-varying blur. To this end, we first propose a fast deep plug-and-play algorithm, based on linearized ADMM splitting techniques, which can solve the super-resolution problem with spatially-varying blur. Second, we unfold our iterative algorithm into a single network and train it end-to-end. In this way, we overcome the intricacy of manually tuning the parameters involved in the optimization scheme. Our algorithm presents remarkable performance and generalizes well after a single training to a large family of spatially-varying blur kernels, noise levels and scale factors.

RGB-D 6D pose estimation has recently drawn great research attention thanks to the complementary depth information. Whereas, the depth and the color image are often noisy in real industrial scenarios. Therefore, it becomes challenging for many existing methods that fuse equally RGB and depth features. In this paper, we present a novel fusion design to adaptively merge RGB-D cues. Specifically, we created a Qualityassessment block that estimates the global quality of the input modalities. This quality represented as an α parameter is then used to reinforce the fusion. We have thus found a simple and effective way to improve the robustness to low-quality inputs in terms of Depth and RGB. Extensive experiments on 6D pose estimation demonstrate the efficiency of our method, especially when noise is present in the input.

International audience; Deep learning methods have become very popular for the processing of natural images, and were then successfully adapted to the neuroimaging field. As these methods are non-transparent, interpretability methods are needed to validate them and ensure their reliability. Indeed, it has been shown that deep learning models may obtain high performance even when using irrelevant features, by exploiting biases in the training set. Such undesirable situations can potentially be detected by using interpretability methods. Recently, many methods have been proposed to interpret neural networks. However, this domain is not mature yet. Machine learning users face two major issues when aiming to interpret their models: which method to choose, and how to assess its reliability? Here, we aim at providing answers to these questions by presenting the most common interpretability methods and metrics developed to assess their reliability, as well as their applications and benchmarks in the neuroimaging context. Note that this is not an exhaustive survey: we aimed to focus on the studies which we found to be the most representative and relevant.

In the context of mobile robotics, this work reports research addressing scene understanding using embedded multi-modal perception systems. Scene understanding remains a complex problem, essential for high society impact applications such as Intelligent Transportation Systems, involving in particular ADAS and Autonomous Vehicles. Results and outlook based on the conducted research are detailed and structured following two research perspectives., Dans le domaine de la robotique mobile, mes travaux de recherche portent sur l’analyse de scènes dynamiques par des systèmes de perception multimodale embarqués. Ceci demeure un verrou scientifique complexe, mais essentiel pour des applications avec un fort impact sociétal telles que les systèmes de transport intelligents où se situent les systèmes d’assistance à la conduite et les voitures à conduite automatisée. Une synthèse et des perspectives structurées sur deux axes thématiques sont présentés.

Even after decades of research, dynamic scene background reconstruction and foreground object segmentation are still considered as open problems due to various challenges such as illumination changes, camera movements, or background noise caused by air turbulence or moving trees. We propose in this paper to model the background of a frame sequence as a low dimensional manifold using an autoencoder and compare the reconstructed background provided by this autoencoder with the original image to compute the foreground/background segmentation masks. The main novelty of the proposed model is that the autoencoder is also trained to predict the background noise, which allows to compute for each frame a pixel-dependent threshold to perform the foreground segmentation. Although the proposed model does not use any temporal or motion information, it exceeds the state of the art for unsupervised background subtraction on the CDnet 2014 and LASIESTA datasets, with a significant improvement on videos where the camera is moving. It is also able to perform background reconstruction on some non-video image datasets.

Transformers have proved to be very effective for visual recognition tasks. In particular, vision transformers construct compressed global representations through self-attention and learnable class tokens. Multi-resolution transformers have shown recent successes in semantic segmentation but can only capture local interactions in high-resolution feature maps. This paper extends the notion of global tokens to build GLobal Attention Multi-resolution (GLAM) transformers. GLAM is a generic module that can be integrated into most existing transformer backbones. GLAM includes learnable global tokens, which unlike previous methods can model interactions between all image regions, and extracts powerful representations during training. Extensive experiments show that GLAM-Swin or GLAM-Swin-UNet exhibit substantially better performances than their vanilla counterparts on ADE20K and Cityscapes. Moreover, GLAM can be used to segment large 3D medical images, and GLAM-nnFormer achieves new state-of-the-art performance on the BCV dataset., Comment: Winter Conference on Applications of Computer Vision (WACV 2023)

Local feature extraction is a standard approach in computer vision for tackling important tasks such as image matching and retrieval. The core assumption of most methods is that images undergo affine transformations, disregarding more complicated effects such as non-rigid deformations. Furthermore, incipient works tailored for non-rigid correspondence still rely on keypoint detectors designed for rigid transformations, hindering performance due to the limitations of the detector. We propose DALF (Deformation-Aware Local Features), a novel deformation-aware network for jointly detecting and describing keypoints, to handle the challenging problem of matching deformable surfaces. All network components work cooperatively through a feature fusion approach that enforces the descriptors' distinctiveness and invariance. Experiments using real deforming objects showcase the superiority of our method, where it delivers 8% improvement in matching scores compared to the previous best results. Our approach also enhances the performance of two real-world applications: deformable object retrieval and non-rigid 3D surface registration. Code for training, inference, and applications are publicly available at https://verlab.dcc.ufmg.br/descriptors/dalf_cvpr23., Comment: CVPR 2023; Source code available at https://verlab.dcc.ufmg.br/descriptors/dalf_cvpr23

We propose a novel learning-based formulation for visual localization of vehicles that can operate in real-time in city-scale environments. Visual localization algorithms determine the position and orientation from which an image has been captured, using a set of geo-referenced images or a 3D scene representation. Our new localization paradigm, named Implicit Pose Encoding (ImPosing), embeds images and camera poses into a common latent representation with 2 separate neural networks, such that we can compute a similarity score for each image-pose pair. By evaluating candidates through the latent space in a hierarchical manner, the camera position and orientation are not directly regressed but incrementally refined. Very large environments force competitors to store gigabytes of map data, whereas our method is very compact independently of the reference database size. In this paper, we describe how to effectively optimize our learned modules, how to combine them to achieve real-time localization, and demonstrate results on diverse large scale scenarios that significantly outperform prior work in accuracy and computational efficiency., Accepted at WACV 2023

Neural style transfer is a deep learning technique that produces an unprecedentedly rich style transfer from a style image to a content image and is particularly impressive when it comes to transferring style from a painting to an image. It was originally achieved by solving an optimization problem to match the global style statistics of the style image while preserving the local geometric features of the content image. The two main drawbacks of this original approach is that it is computationally expensive and that the resolution of the output images is limited by high GPU memory requirements. Many solutions have been proposed to both accelerate neural style transfer and increase its resolution, but they all compromise the quality of the produced images. Indeed, transferring the style of a painting is a complex task involving features at different scales, from the color palette and compositional style to the fine brushstrokes and texture of the canvas. This paper provides a solution to solve the original global optimization for ultra-high resolution images, enabling multiscale style transfer at unprecedented image sizes. This is achieved by spatially localizing the computation of each forward and backward passes through the VGG network. Extensive qualitative and quantitative comparisons show that our method produces a style transfer of unmatched quality for such high resolution painting styles., 10 pages, 5 figures

Despite significant advances, the performance of state-of-the-art continual learning approaches hinges on the unrealistic scenario of fully labeled data. In this paper, we tackle this challenge and propose an approach for continual semi-supervised learning--a setting where not all the data samples are labeled. A primary issue in this scenario is the model forgetting representations of unlabeled data and overfitting the labeled samples. We leverage the power of nearest-neighbor classifiers to nonlinearly partition the feature space and flexibly model the underlying data distribution thanks to its non-parametric nature. This enables the model to learn a strong representation for the current task, and distill relevant information from previous tasks. We perform a thorough experimental evaluation and show that our method outperforms all the existing approaches by large margins, setting a solid state of the art on the continual semi-supervised learning paradigm. For example, on CIFAR-100 we surpass several others even when using at least 30 times less supervision (0.8% vs. 25% of annotations). Finally, our method works well on both low and high resolution images and scales seamlessly to more complex datasets such as ImageNet-100. The code is publicly available on https://github.com/kangzhiq/NNCSL, 13 pages

In this paper, we address the problem of registering multiple point clouds corrupted with high anisotropic localization noise. Our approach follows the widely used framework of Gaussian mixture model (GMM) reconstruction with an expectation-maximization (EM) algorithm. Existing methods are based on an implicit assumption of space-invariant isotropic Gaussian noise. However, this assumption is violated in practice in applications such as single molecule localization microscopy (SMLM). To address this issue, we propose to introduce an explicit localization noise model that decouples shape modeling with the GMM from noise handling. We design a stochastic EM algorithm that considers noise-free data as a latent variable, with closed-form solutions at each EM step. The first advantage of our approach is to handle space-variant and anisotropic Gaussian noise with arbitrary covariances. The second advantage is to leverage the explicit noise model to impose prior knowledge about the noise that may be available from physical sensors. We show on various simulated data that our noise handling strategy improves significantly the robustness to high levels of anisotropic noise. We also demonstrate the performance of our method on real SMLM data.

In the context of 3D object manipulation and 3D reconstruction, several methods are available to acquire 3D data (point clouds, volumes or meshes). The estimation of 3D mesh quality is a fundamental step in many applications, as for example, in a 3D reconstruction process performed in an iterative way. Some approaches allow to evaluate the quality of 3D models generated with the use of a reference model. However, a reference 3D model is not always provided., Dans le contexte de la manipulation d’objets 3D et de la reconstruction 3D, de nombreuses méthodes permettent d’acquérir des données 3D (nuages de points, volumes ou maillages). L’estimation de la qualité d’un maillage 3D est une étape fondamentale dans de nombreuses applications, comme par exemple, dans un processus de reconstruction 3D réalisé de manière itérative. Il existe des approches permettant d’évaluer la qualité des modèles 3D obtenus en utilisant un modèle de référence. Toutefois, un modèle 3D de référence n’est pas toujours disponible. Ainsi, dans un premier temps, nous présentons ces métriques utilisant un modèle de référence, nommées métriques extrinsèques et qui fournissent des résultats corrélés à l’évaluation subjective obtenue par des opérateurs humains. Dans un second temps, nous abordons les métriques extrinsèques, sans modèle de référence et faisant appel aux notions de courbure et de rugosité. Nous avons mis en place un protocole d’évaluation s’appuyant sur l’évaluation de modèles 3D dégradés de manière progressive (ajout de bruits et décimation). Les résultats montrent que les deux catégories de métriques mettent en évidence les défauts de reconstruction, c’est-à-dire qu’elles ont des réponses significatives sur les régions modifiées localement par du bruit

In the last ten years, Convolutional Neural Networks (CNNs) have formed the basis of deep-learning architectures for most computer vision tasks. However, they are not necessarily optimal. For example, mathematical morphology is known to be better suited to deal with binary images. In this work, we create a morphological neural network that handles binary inputs and outputs. We propose their construction inspired by CNNs to formulate layers adapted to such images by replacing convolutions with erosions and dilations. We give explainable theoretical results on whether or not the resulting learned networks are indeed morphological operators. We present promising experimental results designed to learn basic binary operators, and we have made our code publicly available online., Comment: Preprint of a submission to ICIP 2022. 7 pages. 4 figures

International audience; For best medical imaging application results, learning-based approaches such as deep learning necessitate specific, extensive and precise annotations. Outside well-curated public benchmarks, these are rarely available in practice, and so it becomes necessary to use less-than-perfect annotations. One way of compensating for this is the embedding of anatomical knowledge. Complementing this, there is the incremental semi-supervised learning technique, whereby a small amount of annotations can be used to derive more and superior labels.In this article, we illustrate this approach on a deep learning system to help radiologists and rheumatologists finely and interactively assess MRI scans of the sacro-iliac joint in order to correctly diagnose Axial Spondyloarthritis. Our model is trained initially on a relatively small set of images with promising results, on par with expert opinion and generalizable to new datasets.

International audience; Weaning a patient from mechanical ventilation is a critical task in Intensive Care Units, but it can be made safer by using Lung Ultrasound scoring. This scoring is currently done visually by specialists based on Lung Ultrasound artifacts among which are A-lines. Automating this scoring may help standardizing results and saving time for clinicians. In this paper, we propose a method to automatically detect A-lines from a manual delineation of the pleural line, and by using both the intensity profile of the LUS image and morphological operations. A score is then assigned to significant lines and represents the possibility of them being A-lines. The proposed method shows promising results in differentiating A-lines from other elements with an Area Under the Curve of 0.95; furthermore, using a threshold at 0.5 to detect A-lines leads to very good performances with an accuracy of 95% and a F0.5-score of 0.84.

Moving Object Detection (MOD) is a critical vision task for successfully achieving safe autonomous driving. Despite plausible results of deep learning methods, most existing approaches are only frame-based and may fail to reach reasonable performance when dealing with dynamic traffic participants. Recent advances in sensor technologies, especially the Event camera, can naturally complement the conventional camera approach to better model moving objects. However, event-based works often adopt a pre-defined time window for event representation, and simply integrate it to estimate image intensities from events, neglecting much of the rich temporal information from the available asynchronous events. Therefore, from a new perspective, we propose RENet, a novel RGB-Event fusion Network, that jointly exploits the two complementary modalities to achieve more robust MOD under challenging scenarios for autonomous driving. Specifically, we first design a temporal multi-scale aggregation module to fully leverage event frames from both the RGB exposure time and larger intervals. Then we introduce a bi-directional fusion module to attentively calibrate and fuse multi-modal features. To evaluate the performance of our network, we carefully select and annotate a sub-MOD dataset from the commonly used DSEC dataset. Extensive experiments demonstrate that our proposed method performs significantly better than the state-of-the-art RGB-Event fusion alternatives. The source code and dataset are publicly available at: https://github.com/ZZY-Zhou/RENet., ICRA'23

International audience; Humans learn to both visually recognize individual objects and categorize them at different levels of abstraction. Such multi-semantic representation is crucial to efficiently reason about the world. However, it is currently unclear how such representations could be learned with the very sparse labeling available to human learners. To answer this question we let an artificial agent play with objects while occasionally "hearing" their category label. Our agent assigns similar representations to a) similarly labelled and b) close-in-time visual inputs. We show that our agent learns a 2-level hierarchical representation that first aggregates different views of objects and then brings together different objects to form categories. Interestingly, we do not observe a trade-off between each semantic content. Our work suggests that the temporal structure of visual experience during object play together with occasional labeling suffice for learning a hierarchically structured object/category representation.

International audience; Recent time-contrastive learning approaches manage to learn invariant object representations without supervision. This is achieved by mapping successive views of an object onto close-by internal representations. When considering this learning approach as a model of the development of human object recognition, it is important to consider what visual input a toddler would typically observe while interacting with objects. First, human vision is highly foveated, with high resolution only available in the central region of the field of view. Second, objects may be seen against a blurry background due to toddlers' limited depth of field. Third, during object manipulation a toddler mostly observes close objects filling a large part of the field of view due to their rather short arms. Here, we study how these effects impact the quality of visual representations learnt through time-contrastive learning. To this end, we let a visually embodied agent "play" with objects in different locations of a near photo-realistic flat. During each play session the agent views an object in multiple orientations before turning its body to view another object. The resulting sequence of views feeds a time-contrastive learning algorithm. Our results show that visual statistics mimicking those of a toddler improve object recognition accuracy in both familiar and novel environments. We argue that this effect is caused by the reduction of features extracted in the background, a neural network bias for large features in the image and a greater similarity between novel and familiar background regions. The results of our model suggest that several influences on toddler's visual input statistics support their unsupervised learning of object representations.

This paper presents a novel method for restoring digital videos via a Deep Plug-and-Play (PnP) approach. Under a Bayesian formalism, the method consists in using a deep convolutional denoising network in place of the proximal operator of the prior in an alternating optimization scheme. We distinguish ourselves from prior PnP work by directly applying that method to restore a digital video from a degraded video observation. This way, a network trained once for denoising can be repurposed for other video restoration tasks. Our experiments in video deblurring, super-resolution, and interpolation of random missing pixels all show a clear benefit to using a network specifically designed for video denoising, as it yields better restoration performance and better temporal stability than a single image network with similar denoising performance using the same PnP formulation. Moreover, our method compares favorably to applying a different state-of-the-art PnP scheme separately on each frame of the sequence. This opens new perspectives in the field of video restoration., 10 pages + 4 pages supplementary; code at github.com/amonod/pnp-video

International audience; The de facto standard for storing human motion data on a computer involves a representation based on Euler angles. This representation, while effective, has several shortcomings. Triplets of Euler angles are not unique, and the same posture may be expressed using different combinations of angles. Furthermore, many possible Euler angle triplets correspond to unnatural positions for human joints. This means that, in general, a large part of the representational space remains unused. In this paper, we further investigate a recently proposed representation inspired by molecular representations. It uses only two (instead of three) degrees of freedom per joint: a vector and a torsion angle. Using the two key ingredients of this new representation, we present a complete analysis of the Graphics Lab Motion Capture Database. The data found in this analysis provide us with some powerful insights about natural and unnatural human postures in human motions. These insights can potentially lead to possible constraints on human motions which may be used to more effectively solve open problems in the computer graphics community, most notably the problem of (human) motion adaptation.

The generation of 3-dimensional geometric objects in the most efficient way is a thriving research topic with, for example, the development of geometric deep learning, extending classical machine learning concepts to non euclidean data such as graphs or meshes. In this short paper, we study the effect of a reparameterization on two popular mesh and point cloud neural networks in an auto-encoder mode: PointNet [QSMG16] and SpiralNet [BBP∗19]. Finally, we tested a modified version of PointNet that takes orientation into account (through coordinates of the normals) as a first step towards the construction of a geometric deep learning model built with a more flexible metric regarding the parameterization. The experimental results on standardized face datasets show that SpiralNet is more robust to the reparametrization than PointNet in this specific context with the proposed reparameterization., Short Papers, Emery Pierson, Thomas Besnier, Mohamed Daoudi, and Sylvain Arguillère

Fitting apparel and apparel in performing different activities is essential for the functional yet comfortable experience of the user. 4D scans, i.e. 3D scans in continuous timestamps, of the body (part) in performing those activities are the basis for the design of garments/apparel in 4D. In this paper, we proposed a semi-automatic workflow for constructing 4D scans of the body parts with the emphasis on registering noisy scans at a given timestamp. Continuous 3D scans regarding the moving body parts are captured first from different depth cameras from different view angles. In a given timestamp, the collected 3D scans are roughly aligned to a template using the rigid Iterative Closest Points (ICP) algorithm. Then these scans are further registered using a newly proposed non-rigid Iterative Closest-Farthest Points (ICFP) algorithm, in which correspondences between the source and the target are established by either closest or farthest points based on the newly defined logical distance concept and the probability theory. Experimental results indicated that the ICFP method is robust against noise and the scanning accuracy can be as high as 3.4 %. It also reveals that, for the human foot, the differences of ball width and ball angles between the loaded and the unloaded situation can be as large as 8 mm and 2 degrees, respectively. This highlights the importance of using 4D scan in designing garments and apparel.

We present MonteBoxFinder, a method that, given a noisy input point cloud, fits cuboids to the input scene. Our primary contribution is a discrete optimization algorithm that, from a dense set of initially detected cuboids, is able to efficiently filter good boxes from the noisy ones. Inspired by recent applications of MCTS to scene understanding problems, we develop a stochastic algorithm that is, by design, more efficient for our task. Indeed, the quality of a fit for a cuboid arrangement is invariant to the order in which the cuboids are added into the scene. We develop several search baselines for our problem and demonstrate, on the ScanNet dataset, that our approach is more efficient and precise. Finally, we strongly believe that our core algorithm is very general and that it could be extended to many other problems in 3D scene understanding., Accepted at ECCV 2022. Project page: https://michaelramamonjisoa.github.io/projects/MonteBoxFinder, Code: https://github.com/MichaelRamamonjisoa/MonteBoxFinder

Existing multimodal stress/pain recognition approaches generally extract features from different modalities independently and thus ignore cross-modality correlations. This paper proposes a novel geometric framework for multimodal stress/pain detection utilizing Symmetric Positive Definite (SPD) matrices as a representation that incorporates the correlation relationship of physiological and behavioural signals from covariance and cross-covariance. Considering the non-linearity of the Riemannian manifold of SPD matrices, well-known machine learning techniques are not suited to classify these matrices. Therefore, a tangent space mapping method is adopted to map the derived SPD matrix sequences to the vector sequences in the tangent space where the LSTM-based network can be applied for classification. The proposed framework has been evaluated on two public multimodal datasets, achieving both the state-of-the-art results for stress and pain detection tasks., International Conference on Systems, Man, and Cybernetics, IEEE SMC 2022, October 9-12, 2022

International audience; L'Étiquetage en Composantes Connexe (ECC) est une opération fondamentale de la vision numérique depuis des décennies. Bien que la majeure partie de la littérature traite des algorithmes en 2D, sur des applications telles que la vidéo-surveillance ou la conduite autonome, il y a de plus en plus besoin d'algorithmes pour traiter des images en 3D, notamment pour des applications médicales. Bien que les algorithmes d'ECC en 2D génèrent un nombre important d'accès mémoire et de comparaisons, le problème empire avec algorithmes en 3D. Il s'agit de la malédiction de la dimension. La mise en place d'un algorithme efficace doit donc résoudre ce problème. Ce papier introduit un algorithme utilisant des segments pour étiqueter des volumes en 3D. Les équivalences entre les étiquettes sont accélérées à l'aide d'une nouvelle stratégie atténuant l'impact de la dimension supérieure. Ce nouvel algorithme surpasse les algorithmes de l'État-de-l'art d'un facteur allant de ×1.5 à ×3.1 sur des images médicales et sur des images aléatoires.

This paper proves that robustness implies generalization via data-dependent generalization bounds. As a result, robustness and generalization are shown to be connected closely in a data-dependent manner. Our bounds improve previous bounds in two directions, to solve an open problem that has seen little development since 2010. The first is to reduce the dependence on the covering number. The second is to remove the dependence on the hypothesis space. We present several examples, including ones for lasso and deep learning, in which our bounds are provably preferable. The experiments on real-world data and theoretical models demonstrate near-exponential improvements in various situations. To achieve these improvements, we do not require additional assumptions on the unknown distribution; instead, we only incorporate an observable and computable property of the training samples. A key technical innovation is an improved concentration bound for multinomial random variables that is of independent interest beyond robustness and generalization., Accepted by ICML 2022, and selected for ICML long presentation (top 2% of submissions)

International audience; Protection against illicit drone intrusions is a matter of great concern. The relative stealthy nature of UAVs makes their detection difficult. To address this issue, the Deeplomatics project provides a multimodal and modular approach, which combines the advantages of different systems, while adapting to various topologies of the areas to be secured. The originality lies in the fact that acoustic and optronic devices feed independent AI to simultaneously localize and identify the targets using both spatial audio and visual signatures.Several microphone arrays are deployed on the area to be protected. Within its coverage area (about 15 hectares), each microphone array simultaneously localizes and identifies flying drones using a deep learning approach based on the BeamLearning network. Each array is attached to a local AI which processes spatial audio measurements in realtime (40 estimations per second), independently to the other units of the surveillance network.A data fusion system refines the estimates provided by each of the AI-enhanced microphone arrays. This detected position is shared in real-time with an optronic system. Once this system has hooked its target, a Deep Learning tracking algorithm is used to allow an autonomous visual tracking and identification.The optronic system is composed of various cameras (visible, thermal, and active imaging) mounted on a servo-turret. The active imaging system can capture scenes up to 1 km, and only captures objects within a given distance, which naturally excludes foreground and background from the image, and enhances the capabilities of computer vision.The Deeplomatics project combines benefits from acoustics and optronics to ensure real-time localization and identification of drones, with a high precision (less than 7° of absolute 3D error, more than 90 % detection accuracy). The modular approach also allows to consider in the long term the addition of new capture systems such as electromagnetic radars.

Supervised training has led to state-of-the-art results in image and video denoising. However, its application to real data is limited since it requires large datasets of noisy-clean pairs that are difficult to obtain. For this reason, networks are often trained on realistic synthetic data. More recently, some self-supervised frameworks have been proposed for training such denoising networks directly on the noisy data without requiring ground truth. On synthetic denoising problems supervised training outperforms self-supervised approaches, however in recent years the gap has become narrower, especially for video. In this paper, we propose a study aiming to determine which is the best approach to train denoising networks for real raw videos: supervision on synthetic realistic data or self-supervision on real data. A complete study with quantitative results in case of natural videos with real motion is impossible since no dataset with clean-noisy pairs exists. We address this issue by considering three independent experiments in which we compare the two frameworks. We found that self-supervision on the real data outperforms supervision on synthetic data, and that in normal illumination conditions the drop in performance is due to the synthetic ground truth generation, not the noise model., L'entraînement supervisé a conduit à l'état de l'art en débruitage d'images et de vidéos. Cependant, son application aux données réelles est limitée car il nécessite de grandes bases de données formées de paires donnée bruitée / donnée vérité-terrain et celles-ci sont difficiles à obtenir. Pour cette raison, les réseaux de neurones sont souvent entraînés sur des données certes réalistes mais synthétiques. Plus récemment, des entraînements auto-supervisés ont été proposés pour entraîner de tels réseaux de débruitage directement sur les données bruitées et sans nécessiter de vérités-terrain. Pour le problème de débruitage sur des données synthétiques, les entraînements supervisés conduisent à de meilleurs résultats que les entraînements auto-supervisés, toutefois depuis quelques temps l'écart entre les deux approches s'est réduit, et plus particulièrement dans le cas des vidéos. Dans cet article, nous proposons une étude visant à déterminer quelle approches est la plus adaptée pour l'entraînement de réseaux de débruitage de données brutes réelles : la supervision sur des bases de données synthétiques ou l'auto-supervision sur les données réelles. Une étude complète avec des résultats quantitatif dans le cas de vidéos naturelles avec mouvement réel est impossible car aucune base de données avec des paires de données bruitée / vérité-terrain n'existe. Nous contournons ce problème en considérant trois expériences indépendantes dans lesquelles nous comparons les deux approches. Nous avons trouvé que l'auto-supervision sur des données réelles conduit à de meilleurs résultats que la supervision sur des données synthétiques et que dans des conditions normales d'illumination, la baisse de performance est due à la synthèse des vérités-terrain synthétiques et non au modèle de bruit.

Contrary to RGB cameras, Dynamic Vision Sensor (DVS) output visual data in the form of an asynchronous events stream by recording pixel-wise luminance changes at microsecond resolution. While conventional computer vision approaches utilise frame-based input data, thus failing to take full advantage of the high temporal resolution, novel approaches use spiking neural networks Spiking Neural Networks (SNNs) which are more compatible to handle event-based data since these bioinspired neural models intrinsically encode information in a sparse manner using activation spikes trains. This paper presents an attentional mechanism which detects regions with higher event density by using inherent SNN dynamics combined with online weight and threshold adaptation. We implemented the network directly on Intel's research neuromorphic chip Loihi and evaluate our proposed method on the open DVS128 Gesture Dataset. Our system is able to process 1 ms of event-data in 6 ms and reject more than 50% of incoming unwanted events occurring only 20 ms after activity onset.

Deep network architectures struggle to continually learn new tasks without forgetting the previous tasks. A recent trend indicates that dynamic architectures based on an expansion of the parameters can reduce catastrophic forgetting efficiently in continual learning. However, existing approaches often require a task identifier at test-time, need complex tuning to balance the growing number of parameters, and barely share any information across tasks. As a result, they struggle to scale to a large number of tasks without significant overhead. In this paper, we propose a transformer architecture based on a dedicated encoder/decoder framework. Critically, the encoder and decoder are shared among all tasks. Through a dynamic expansion of special tokens, we specialize each forward of our decoder network on a task distribution. Our strategy scales to a large number of tasks while having negligible memory and time overheads due to strict control of the parameters expansion. Moreover, this efficient strategy doesn't need any hyperparameter tuning to control the network's expansion. Our model reaches excellent results on CIFAR100 and state-of-the-art performances on the large-scale ImageNet100 and ImageNet1000 while having less parameters than concurrent dynamic frameworks., Comment: CVPR 2022, Code at https://github.com/arthurdouillard/dytox

We propose a robust and accurate method for estimating the 3D poses of two hands in close interaction from a single color image. This is a very challenging problem, as large occlusions and many confusions between the joints may happen. State-of-the-art methods solve this problem by regressing a heatmap for each joint, which requires solving two problems simultaneously: localizing the joints and recognizing them. In this work, we propose to separate these tasks by relying on a CNN to first localize joints as 2D keypoints, and on self-attention between the CNN features at these keypoints to associate them with the corresponding hand joint. The resulting architecture, which we call "Keypoint Transformer", is highly efficient as it achieves state-of-the-art performance with roughly half the number of model parameters on the InterHand2.6M dataset. We also show it can be easily extended to estimate the 3D pose of an object manipulated by one or two hands with high performance. Moreover, we created a new dataset of more than 75,000 images of two hands manipulating an object fully annotated in 3D and will make it publicly available., Accepted at CVPR2022

Homologous Recombination DNA-repair deficiency (HRD) is a well-recognized marker of platinum-salt and PARP inhibitor chemotherapies in ovarian and breast cancers (BC). Causing high genomic instability, HRD is currently determined by BRCA1/2 sequencing or by genomic signatures, but its morphological manifestation is not well understood. Deep Learning (DL) is a powerful machine learning technique that has been recently shown to be capable of predicting genomic signatures from stained tissue slides. However, DL is known to be sensitive to dataset biases and lacks interpretability. Here, we present and evaluate a strategy to control for biases in retrospective cohorts. We train a deep-learning model to predict the HRD in a controlled cohort with unprecedented accuracy (AUC: 0.86) and we develop a new visualization technique that allows for automatic extraction of new morphological features related to HRD. We analyze in detail the extracted morphological patterns that open new hypotheses on the phenotypic impact of HRD.

In the human, the folding of the brain takes place mostly in utero, during the last trimester of pregnancy, resulting in a convoluted cortical surface at term, displaying furrows (sulci) and crests (gyri), whose patterns remain relatively stable throughout life. The same global folding organization is found between individuals, allowing to identify and compare matching sulci, while the more refined individual folding pattern is unique. The variability of sulcal folding pattern and its relations to functional specialization (both in health and pathology) are increasingly investigated in the mature brain, while studies in the developing brain are still scarce because of the many challenges of early-life neuroimaging. During my PhD, I have investigated the dynamics of folding pattern based on longitudinally acquired magnetic resonance imaging (MRI) images from a cohort of extremely preterm infants, imaged once around the equivalent of 7 months of pregnancy, and once at term-equivalent age (TEA). I have adapted a pipeline established for adult sulcal shape characterization to the context of developing folds and applied it to four sulcal objects: the central sulcus, the sylvian fissure, and the perisylvian region and the inferior frontal region (these last two were only investigated at TEA because of their late appearance in the brain).Throughout my works, I have described the shape variability observed in the central sulcus and the sylvian fissure at both ages. This led me to observe a consistent encoding of pattern between both ages, suggesting an early expression of sulcal shape features. I have incidentally tested if the central sulcus’s shape was able to predict later motor outcome: even though it did not achieve significant results, it still seemed to suggest some functional relevance in sulcal signatures. My observations on the central sulcus and sylvian fissure were concordant with the state of the art and detailed additional pattern specificities and the chronology of shape encoding. My focus in and around the sylvian fissure additionally led me to hypothesize a different dynamic in opercularization (the process leading to the burial of the insular cortex under the frontal, parietal and temporal lobes) from that in sulcation (the process leading to the formation of sulci): opercularization seemed to show an advance in the left hemisphere while sulcation has previously been reported to show an advance in the right hemisphere. Moreover, I have captured hemispheric asymmetries and detailed their shape characteristics in each fold-related object or region considered. These studies have therefore contributed to shedding more light on early-life asymmetries., which could be linked to the functional lateralization of the brain regarding handedness or language perception.In order to complement my contributions, additional investigations should be led in different directions, including the description of pattern variability and its links with brain function in later developmental stages in the child and the adult, the quantification of prematurity-induced sulcal alterations, and the further investigation of the relation between opercularization and sulcation processes, which might interact together during development., Chez l’homme, le cerveau se plisse principalement in utero, pendant le troisième trimestre de grossesse : la surface du cortex au moment de la naissance à terme présente alors des creux (sillons) et des bosses (gyri) dont la forme reste relativement stable au cours de la vie. Les motifs de plissement du cerveau, bien qu'uniques à chaque individu, présentent une organisation globalement cohérente au sein de l'espèce, permettant d'identifier et de comparer les sillons entre individus. La variabilité en forme des sillons et ses relations avec la spécialisation fonctionnelle (que le contexte soit pathologique ou non) sont de plus en plus étudiées dans le cerveau mature, mais les études dans le cerveau en développement restent rares à cause de nombreuses difficultés soulevées par la neuro-imagerie du tout-petit.Pendant mon doctorat, j’ai étudié les dynamiques de forme du plissement cortical sur la base de données d’imagerie par résonance magnétique (IRM) acquises longitudinalement dans une cohorte de nouveau-nés extrêmement prématurés, imagés vers l’âge équivalent à 7 mois de grossesse et à l’âge équivalent de la naissance à terme. J’ai adapté une méthode établie pour le cerveau adulte de caractérisation de la forme des sillons au contexte complexe du développement, et je l’ai appliquée à quatre régions corticales distinctes : le sillon central, la scissure sylvienne, et les régions périsylvienne et frontale inférieure, ces deux dernières n’ayant été explorées qu’à l’âge équivalent du terme du fait de leur apparition tardive dans le cerveau.À travers mes travaux, j’ai tout d’abord décrit la variabilité en forme du sillon central et de la scissure sylvienne. Ceci m’a amenée à découvrir que la forme de ces plis est cohérente entre les deux âges, ce qui suggère un encodage précoce. J’ai par ailleurs évalué si la forme du sillon central permettait de prédire le devenir moteur des sujets : faute d’être significative,elle a néanmoins suggéré la pertinence fonctionnelle de certaines signatures sulcales.Tout en étant concordantes avec l’état de l’art, mes observations sur le sillon central et la scissure sylvienne ont permis de détailler les motifs spécifiques et la chronologie d’encodage de leur forme précoce. En ciblant la scissure sylvienne et ses régions environnantes, j’ai émis l’hypothèse de dynamiques différentes entre l’opercularisation (processus menant au recouvrement du lobe insulaire par les lobes frontaux, pariétaux et temporaux) et la sulcation (processus menant à la formation des sillons) : mes observations ont alors suggéré une avance de l’hémisphère gauche dans le cadre de l’opercularisation, contre une avance de l’hémisphère droit pour la sulcation.Par ailleurs, j’ai capturé des asymétries conséquentes entre hémisphères cérébraux, en détaillant les caractéristiques de formes dans chacun des objets sulcaux considérés : ces travaux ont ainsi contribué à une meilleure compréhension des asymétries précoces qui pourraient être liées au développement de la latéralisation fonctionnelle du cerveau en liens avec la latéralité manuelle ou la perception du langage.De manière à compléter mes contributions, différentes directions devraient être explorées, dont la description de la variabilité des motifs des sillons et leurs relations avec le fonctionnement cérébral à des stades plus tardifs de développement de l’enfant et dans le cerveau de l’adulte, la quantification des altérations de plissement liées à la prématurité, et l’investigation plus approfondie des rapports entre opercularisation et sulcation, qui pourraient interagir ensemble au cours du développement.

Background and ObjectivesBrain amyloid deposition, a major risk factor for Alzheimer disease (AD), is currently estimated by measuring CSF or plasma amyloid peptide levels or by PET imaging. Assessing genetic risks relating to amyloid deposition before any accumulation has occurred would allow for earlier intervention in persons at increased risk for developing AD. Previous work linking amyloid burden and genetic risk relied almost exclusively on APOE, a major AD genetic risk factor. Here, we ask whether a polygenic risk score (PRS) that incorporates an optimized list of common variants linked to AD and excludes APOE is associated with brain amyloid load in cognitively unimpaired older adults.MethodsWe included 291 asymptomatic older participants from the INveStIGation of AlzHeimer's PredicTors (INSIGHT pre-AD) cohort who underwent amyloid imaging, including 83 amyloid-positive (+) participants. We used an Alzheimer's (A) PRS composed of 33 AD risk variants excluding APOE and selected the 17 variants that showed the strongest association with amyloid positivity to define an optimized (oA) PRS. Participants from the Alzheimer's Disease Neuroimaging Initiative (ADNI) study (228 participants, 90 amyloid [+]) were tested as a validation cohort. Finally, 2,300 patients with AD and 6,994 controls from the European Alzheimer's Disease Initiative (EADI) were evaluated.ResultsA-PRS was not significantly associated with amyloid burden in the INSIGHT or ADNI cohorts with or without correction for the APOE genotype. However, oA-PRS was significantly associated with amyloid status independently of APOE adjustment (INSIGHT odds ratio [OR]: 5.26 [1.71–16.88]; ADNI OR: 3.38 [1.02–11.63]). Of interest, oA-PRS accurately discriminated amyloid (+) and (−) APOE ε4 carriers (INSIGHT OR: 181.6 [7.53–10,674.6]; ADNI OR: 44.94 [3.03–1,277]). A-PRS and oA-PRS showed a significant association with disease status in the EADI cohort (OR: 1.68 [1.53–1.85] and 2.06 [1.73–2.45], respectively). Genes assigned to oA-PRS variants were enriched in ontologies related to β-amyloid metabolism and deposition.DiscussionPRSs relying on AD genetic risk factors excluding APOE may improve risk prediction for brain amyloid, allowing stratification of cognitively unimpaired individuals at risk of AD independent of their APOE status.

As a core component of intelligent video surveillance systems, person re-identification (ReID) targets at retrieving a person of interest across non-overlapping cameras. Despite significant improvements in supervised ReID, cumbersome annotation process makes it less scalable in real-world deployments. Moreover, as appearance representations can be affected by noisy factors, such as illumination level and camera properties, between different domains, person ReID models suffer a large performance drop in the presence of domain gaps. We are particularly interested in designing algorithms that can adapt a person ReID model to a target domain without human supervision. In such context, we mainly focus on designing unsupervised domain adaptation and unsupervised representation learning methods for person ReID.In this thesis, we first explore how to build robust representations by combining both global and local features under the supervised condition. Then, towards an unsupervised domain adaptive ReID system, we propose three unsupervised methods for person ReID, including 1) teacher-student knowledge distillation with asymmetric network structures for feature diversity encouragement, 2) joint generative and contrastive learning framework that generates augmented views with a generative adversarial network for contrastive learning, and 3) exploring inter-instance relations and designing relation-aware loss functions for better contrastive learning based person ReID.Our methods have been extensively evaluated on main-stream ReID datasets, such as Market-1501, DukeMTMC-reID and MSMT17. The proposed methods significantly outperform previous methods on the ReID datasets, significantly pushing person ReID to real-world deployments., En tant que composant central des systèmes de vidéo-surveillance intelligents, la ré-identification de personnes (ReID) vise à rechercher une personne d'intérêt à travers des caméras qui ne se chevauchent pas. Malgré des améliorations significatives de la ReID supervisée, le processus d'annotation encombrant le rend moins évolutif dans les déploiements réels. De plus, comme les représentations d'apparence peuvent être affectées par des facteurs bruyants, tels que le niveau d'éclairage et les propriétés de la caméra, entre différents domaines, les modèles ReID de personnes subissent une baisse de performances importante en présence d'écarts de domaine. Nous sommes particulièrement intéressés par la conception d'algorithmes capables d'adapter un modèle ReID de personnes à un domaine cible sans supervision humaine. Dans un tel contexte, nous nous concentrons principalement sur la conception de méthodes d'adaptation de domaine non-supervisée et d'apprentissage de représentation non-supervisée pour le ReID de personnes.Dans cette thèse, nous explorons d'abord comment construire des représentations robustes en combinant à la fois des caractéristiques globales et locales sous la condition supervisée. Ensuite, vers un système ReID adaptatif au domaine non-supervisé, nous proposons trois méthodes non-supervisées pour la ReID de personnes, notamment 1) la distillation des connaissances enseignant-étudiant avec des structures de réseau asymétriques pour encourager la diversité des caractéristiques, 2) un cadre d'apprentissage conjoint génératif et contrastif qui génère des vues augmentées avec un réseau génératif pour l'apprentissage contrastif, et 3) explorer les relations inter-instances et concevoir des fonctions de perte conscientes des relations pour une meilleure ReID de personnes basée sur l'apprentissage contrastif.Nos méthodes ont été largement évaluées sur des benchmarks de ReID, tels que Market-1501, DukeMTMC-reID et MSMT17. Les méthodes proposées surpassent considérablement les méthodes précédentes sur les benchmarks de ReID, poussant considérablement la ReID de personnes vers des déploiements dans le monde réel.

We present a method that can recognize new objects and estimate their 3D pose in RGB images even under partial occlusions. Our method requires neither a training phase on these objects nor real images depicting them, only their CAD models. It relies on a small set of training objects to learn local object representations, which allow us to locally match the input image to a set of "templates", rendered images of the CAD models for the new objects. In contrast with the state-of-the-art methods, the new objects on which our method is applied can be very different from the training objects. As a result, we are the first to show generalization without retraining on the LINEMOD and Occlusion-LINEMOD datasets. Our analysis of the failure modes of previous template-based approaches further confirms the benefits of local features for template matching. We outperform the state-of-the-art template matching methods on the LINEMOD, Occlusion-LINEMOD and T-LESS datasets. Our source code and data are publicly available at https://github.com/nv-nguyen/template-pose, CVPR 2022

Precisely detecting solar active regions (AR) from multi-spectral images is a challenging task yet important in understanding solar activity and its influence on space weather. A main challenge comes from each modality capturing a different location of these 3D objects, as opposed to more traditional multi-spectral imaging scenarios where all image bands observe the same scene. We present a multi-task deep learning framework that exploits the dependencies between image bands to produce 3D AR detection where different image bands (and physical locations) each have their own set of results. Different feature fusion strategies are investigated in this work, where information from different image modalities is aggregated at different semantic levels throughout the network. This allows the network to benefit from the joint analysis while preserving the band-specific information. We compare our detection method against baseline approaches for solar image analysis (multi-channel coronal hole detection, SPOCA for ARs (Verbeeck et al. Astron Astrophys 561:16, 2013)) and a state-of-the-art deep learning method (Faster RCNN) and show enhanced performances in detecting ARs jointly from multiple bands. We also evaluate our proposed approach on synthetic data of similar spatial configurations obtained from annotated multi-modal magnetic resonance images.

Statistical shape models (SSMs) are a popular tool to conduct morphological analysis of anatomical structures which is a crucial step in clinical practices. However, shape representations through SSMs are based on shape coefficients and lack an explicit one-to-one relationship with anatomical measures of clinical relevance. While a shape coefficient embeds a combination of anatomical measures, a formalized approach to find the relationship between them remains elusive in the literature. This limits the use of SSMs to subjective evaluations in clinical practices. We propose a novel SSM controlled by anatomical parameters derived from morphometric analysis. The proposed anatomically parameterized SSM (ANAT-SSM) is based on learning a linear mapping between shape coefficients and selected anatomical parameters. This mapping is learned from a synthetic population generated by the standard SSM. Determining the pseudo-inverse of the mapping allows us to build the ANAT-SSM. We further impose orthogonality constraints to the anatomical parameterization to obtain independent shape variation patterns. The proposed contribution was evaluated on two skeletal databases of femoral and scapular bone shapes using clinically relevant anatomical parameters. Anatomical measures of the synthetically generated shapes exhibited realistic statistics. The learned matrices corroborated well with the obtained statistical relationship, while the two SSMs achieved moderate to excellent performance in predicting anatomical parameters on unseen shapes. This study demonstrates the use of anatomical representation for creating anatomically parameterized SSM and as a result, removes the limited clinical interpretability of standard SSMs. The proposed models could help analyze differences in relevant bone morphometry between populations, and be integrated in patient-specific pre-surgery planning or in-surgery assessment., 12 pages, 5 figures, 3 tables, Accepted for publication at IEEE Transactions on Biomedical Engineering

Magnetic Resonance Imaging (MRI) is one of the most prominent imaging techniques in the world. Its main purpose is to probe soft tissues in a non-invasive and non-ionizing way. However, its wider adoption is hindered by an overall high scan time. In order to reduce this duration, several approaches have been proposed, among which Parallel Imaging (PI) and Compressed Sensing (CS) are the most important. Using these techniques, MR data can be acquired in a highly compressed way which allows the reduction of acquisition times. However, the algorithms typically used to reconstruct the MR images from these undersampled data are slow and underperform in highly accelerated scenarios. In order to address these issues, unrolled neural networks have been introduced. The core idea of these models is to unroll the iterations of classical reconstruction algorithms into a finite computation graph. The main objective of this PhD thesis is to propose new architecture designs for acquisition scenarios which deviate from the typical Cartesian 2D sampling. To this end, we first review a handful of neural networks for MRI reconstruction. After selecting the best performer, the PDNet, we extend it to two contexts: the fastMRI 2020 reconstruction challenge and the 3D non-Cartesian data problem. We also chose to adress the concerns of many regarding the clinical applicability of deep learning for medical imaging. We do so by proposing ways to build robust and inspectable models, but also by simply testing the trained networks in out-of-distribution settings. Finally, after noticing how the implicit deep learning framework can help implement deeper MRI reconstruction models, we introduce a new acceleration method (called SHINE) for the training of such models., L'imagerie par résonance magnétique (IRM) est l'une des modalités d'imagerie les plus utilisées au monde. Son objectif principal est de visualiser les tissus mous de manière non invasive et non ionisante. Cependant, son adoption générale est entravée par une durée d'examen globalement élevée. Afin de la raccourcir, plusieurs techniques ont été proposées, parmi lesquelles l'imagerie parallèle (PI) et l'échantillonnage compressif (CS) jouent une place prédominante. Grâce à ces techniques, les données en IRM peuvent être acquises de manière fortement compressée, réduisant ainsi significativement le temps d'acquisition. Cependant, les algorithmes généralement utilisés pour reconstruire les images IRM à partir de ces données sous-échantillonnées sont lents et peu performants dans des scénarios d'acquisition fortement accélérés. Afin de résoudre ces problèmes, les "réseaux de neurones déroulés" ont été introduits. L'idée centrale de ces modèles est de dérouler ou déplier les itérations des algorithmes de reconstruction classiques en un graphe de calcul fini. L'objectif principal de cette thèse est de proposer de nouvelles architectures pour des scénarios d'acquisition qui s'écartent de l'acquisition cartésienne 2D typique. À cette fin, nous passons d'abord en revue une poignée de réseaux neuronaux pour la reconstruction IRM. Après avoir sélectionné le plus performant, i.e. le PDNet, nous l'étendons à deux contextes : le challenge fastMRI 2020 et le problème des données 3D non cartésiennes. Nous avons également choisi de répondre aux préoccupations de beaucoup concernant l'applicabilité clinique de l'apprentissage profond pour l'imagerie médicale. Nous le faisons en proposant des moyens de construire des modèles robustes et inspectables, mais aussi en testant simplement les réseaux entraînés dans des contextes qui s'écartent de la distribution d'entraînement. Enfin, après avoir remarqué comment l'outil de l'apprentissage profond implicite peut aider à entraîner des modèles de reconstruction IRM plus profonds, nous introduisons une nouvelle méthode d'accélération (i.e. SHINE) pour l'entraînement de ces modèles.