Search

International audience; We consider a neural network architecture to solve inverse problems, which is built by unfolding a forwardbackward algorithm. This algorithm is based on the minimization of an objective function which corresponds to a penalized least squares problem. In this context, ensuring stability is consistent with inverse problem theory since it guarantees both the continuity of the inversion method and its insensitivity to small noise. The latter is a critical property as deep neural networks have been shown to be vulnerable to adversarial perturbations. The main novelty of our work is to analyze the robustness of this inversion method with respect to a perturbation of the bias parameter of the network. In our architecture, the bias accounts for the observed data in the inverse problem. The analysis is performed by using tools of fixed point theory. Our theoretical results are illustrated by numerical simulations on a problem of signal restoration.

International audience; Importance sampling (IS) is a powerful Monte Carlo methodology for the approximation of intractable integrals, very often involving a target probability distribution. The performance of IS heavily depends on the appropriate selection of the proposal distributions where the samples are simulated from. In this paper, we propose an adaptive importance sampler, called GRAMIS, that iteratively improves the set of proposals. The algorithm exploits geometric information of the target to adapt the location and scale parameters of those proposals. Moreover, in order to allow for a cooperative adaptation, a repulsion term is introduced that favors a coordinated exploration of the state space. This translates into a more diverse exploration and a better approximation of the target via the mixture of proposals. Moreover, we provide a theoretical justification of the repulsion term. We show the good performance of GRAMIS in two problems where the target has a challenging shape and cannot be easily approximated by a standard uni-modal proposal.

Denoising, detrending, deconvolution: usual restoration tasks, traditionally decoupled. Coupled formulations entail complex ill-posed inverse problems. We propose PENDANTSS for joint trend removal and blind deconvolution of sparse peak-like signals. It blends a parsimonious prior with the hypothesis that smooth trend and noise can somewhat be separated by low-pass filtering. We combine the generalized pseudo-norm ratio SOOT/SPOQ sparse penalties $\ell_p/\ell_q$ with the BEADS ternary assisted source separation algorithm. This results in a both convergent and efficient tool, with a novel Trust-Region block alternating variable metric forward-backward approach. It outperforms comparable methods, when applied to typically peaked analytical chemistry signals. Reproducible code is provided: https://github.com/paulzhengfr/PENDANTSS., Comment: in French language

International audience; Large datasets of geospatial satellite images are available online, exhibiting significant variations in both image quality and content. These variations in image quality stem from the image processing pipeline and image acquisition settings, resulting in subtle differences within datasets of images acquired with the same satellites. Recent progress in the field of image processing have considerably enhanced capabilities in noise and artifacts removal, as well as image super-resolution. Consequently, this opens up possibilities for homogenizing geospatial image datasets by reducing the intra-dataset variations in image quality. In this work, we show that conventional image detection and segmentation neural networks trained on geospatial data are robust neither to noise and artefact removal preprocessing, nor to mild resolution variations.

International audience; This paper presents an investigation into the zero-shot performance of pre-trained foundation models on remote sensing tasks. Recent advances in self-supervised learning suggest that these models, when trained on vast amounts of unsupervised data, could potentially improve generalization across a number of downstream tasks. Our study offers an empirical evaluation of these models on standard remote-sensing benchmarks such as EuroSAT and BigEarthNet-S2, with the intent to confirm whether these models have encountered satellite imagery during their training phase. Moreover, we examine the impact of adding a geospatial domain-specific textual description of classes, contrasting it with the standard class-based prompts. Our findings indicate that the fine-tuned BLIP models exhibit superior zeroshot performance on these benchmarks compared to their standard counterparts, signifying that fine-tuning on standard benchmarks enhances performance. Furthermore, the addition of geospatial context variably influences performance depending on the specific model and dataset. This work provides crucial insights into the applicability of foundation models in remote sensing tasks and lays the groundwork for further research.

National audience

International audience; Simulated annealing is a popular approach to solve nonconvex and black-box optimization problems. It consists in running a non-homogeneous Markov chain to sample from a sequence of Boltzmann probability distributions. This sequence is controlled by a cooling schedule, which governs the concentration of the mass of the Boltzmann distributions around the global minimizers. However, convergence is often slow, difficult to assess, and requires a fixed cooling schedule. We propose here a new simulated annealing algorithm with adaptive cooling schedule, which draws samples from variational approximations of the Boltzmann distributions. Our approach is theoretically sound and relies on an alternating Bregman proximal-gradient scheme minimizing a regularized Rényi divergence. Numerical experiments illustrate the performance of the method.

International audience; State-space models (SSMs) are a common tool for modeling multi-variate discrete-time signals. The linear-Gaussian (LG) SSM is widely applied as it allows for a closed-form solution at inference, if the model parameters are known. However, they are rarely available in real-world problems and must be estimated. Promoting sparsity of these parameters favours both interpretability and tractable inference. In this work, we propose GraphIT, a majorization-minimization (MM) algorithm for estimating the linear operator in the state equation of an LG-SSM under sparse prior. A versatile family of non-convex regularization potentials is proposed. The MM method relies on tools inherited from the expectationmaximization methodology and the iterated reweighted-l1 approach. In particular, we derive a suitable convex upper bound for the objective function, that we then minimize using a proximal splitting algorithm. Numerical experiments illustrate the benefits of the proposed inference technique.

Co-administration of two or more drugs simultaneously can result in adverse drug reactions. Identifying drug-drug interactions (DDIs) is necessary, especially for drug development and for repurposing old drugs. DDI prediction can be viewed as a matrix completion task, for which matrix factorization (MF) appears as a suitable solution. This paper presents a novel Graph Regularized Probabilistic Matrix Factorization (GRPMF) method, which incorporates expert knowledge through a novel graph-based regularization strategy within an MF framework. An efficient and sounded optimization algorithm is proposed to solve the resulting non-convex problem in an alternating fashion. The performance of the proposed method is evaluated through the DrugBank dataset, and comparisons are provided against state-of-the-art techniques. The results demonstrate the superior performance of GRPMF when compared to its counterparts.

International audience

International audience; We propose an adaptive, smooth, non-convex and sparsity-promoting variational model for singleimage super-resolution of real murine Optical Coherence Tomography (OCT) data. We follow a sparse-representation approach where sparsity is modelled with respect to a suitable dictionary generated from high-resolution OCT data. To do so, we employ pre-learned dictionaries tailored to model α-stable statistics in the non-Gaussian case, i.e. α < 2. The image reconstruction problem renders here particularly challenging due to the high level of noise degradation and to the heterogeneity of the data at hand. As a regulariser, we employ a smooth, non-convex and separable Cauchy-type penalty. To favour adaptivity to heterogeneous image contents, we propose a space-variant modelling by which the local degree of non-convexity encoded patch-wise within the local Cauchy shape parameter is estimated via maximum likelihood. For the solution of the patch-based smooth non-convex optimisation problems, we consider an extension of the cautious Broyden-Fletcher-Goldfarb-Shanno (BFGS) algorithm where the descent direction is suitably updated depending on the local convexity behaviour of the functional. Our numerical results show that the combination of a space-variant modelling with a fast optimisation strategy improves reconstruction results, maintaining tissue texture and suppressing background noise to a desirable amount at the same time. Furthermore, the proposed optimisation strategy significantly reduces the computational efforts which often represent a major limitation in the analysis of OCT data.

International audience; Denoising, detrending, deconvolution: usual restoration tasks, traditionally decoupled. Coupled formulations entail complex ill-posed inverse problems. We propose PENDANTSS for joint trend removal and blind deconvolution of sparse peak-like signals. It blends a parsimonious prior with the hypothesis that smooth trend and noise can somewhat be separated by low-pass filtering. We combine the generalized quasi-norm ratio SOOT/SPOQ sparse penalties $\ell_p/\ell_q$ with the BEADS ternary assisted source separation algorithm. This results in a both convergent and efficient tool, with a novel Trust-Region block alternating variable metric forward-backward approach. It outperforms comparable methods, when applied to typically peaked analytical chemistry signals. Reproducible code is provided.

International audience; Lifting-based wavelet transform has been extensively used for efficient compression of various types of visual data. Generally, the performance of such coding schemes strongly depends on the lifting operators used, namely the prediction and update filters. Unlike conventional schemes based on linear filters, we propose, in this paper, to learn these operators by exploiting neural networks. More precisely, a classical Fully Connected Neural Network (FCNN) architecture is firstly employed to perform the prediction and update. Then, we propose to improve this FCNN-based Lifting Scheme (LS) in order to better take into account the input image to be encoded. Thus, a novel dynamical FCNN model is developed, making the learning process adaptive to the input image contents for which two adaptive learning techniques are proposed. While the first one resorts to an iterative algorithm where the computation of two kinds of variables is performed in an alternating manner, the second learning method aims to learn the model parameters directly through a reformulation of the loss function. Experimental results carried out on various test images show the benefits of the proposed approaches in the context of lossy and lossless image compression.

International audience; This paper introduces a novel approach for building a robust Automatic Gesture Recognition system based on Surface Electromyographic (sEMG) signals, acquired at the forearm level. Our main contribution is to propose new constrained learning strategies that ensure robustness against adversarial perturbations by controlling the Lipschitz constant of the classifier. We focus on positive neural networks for which accurate Lipschitz bounds can be derived, and we propose different spectral norm constraints offering robustness guarantees from a theoretical viewpoint. Experimental results on two distinct datasets highlight that a good trade-off in terms of accuracy and performance is achieved. We then demonstrate the robustness of our models, compared to standard trained classifiers in three scenarios, considering both white-box and black-box attacks.

Stochastic differentiable approximation schemes are widely used for solving high dimensional problems. Most of existing methods satisfy some desirable properties, including conditional descent inequalities, and almost sure (a.s.) convergence guarantees on the objective function, or on the involved gradient. However, for non-convex objective functions, a.s. convergence of the iterates, i.e., the stochastic process, to a critical point is usually not guaranteed, and remains an important challenge. In this article, we develop a framework to bridge the gap between descent-type inequalities and a.s. convergence of the associated stochastic process. Leveraging a novel Kurdyka-Łojasiewicz property, we show convergence guarantees of stochastic processes under mild assumptions on the objective function. We also provide examples of stochastic algorithms benefiting from the proposed framework and derive a.s. convergence guarantees on the iterates.

Lipschitz bounded neural networks are certifiably robust and have a good trade-off between clean and certified accuracy. Existing Lipschitz bounding methods train from scratch and are limited to moderately sized networks (< 6M parameters). They require a fair amount of hyper-parameter tuning and are computationally prohibitive for large networks like Vision Transformers (5M to 660M parameters). Obtaining certified robustness of transformers is not feasible due to the non-scalability and inflexibility of the current methods. This work presents CertViT, a two-step proximal-projection method to achieve certified robustness from pre-trained weights. The proximal step tries to lower the Lipschitz bound and the projection step tries to maintain the clean accuracy of pre-trained weights. We show that CertViT networks have better certified accuracy than state-of-the-art Lipschitz trained networks. We apply CertViT on several variants of pre-trained vision transformers and show adversarial robustness using standard attacks. Code : https://github.com/sagarverma/transformer-lipschitz, Preprint work. 13 pages, 3 figures, https://github.com/sagarverma/transformer-lipschitz

We consider the minimization of a differentiable Lipschitz gradient but non necessarily convex, function F defined on R N. We propose an accelerated gradient descent approach which combines three strategies, namely (i) a variable metric derived from the majorization-minimization principle ; (ii) a subspace strategy incorporating information from the past iterates ; (iii) a block alternating update. Under the assumption that F satisfies the Kurdyka-Łojasiewicz property, we give conditions under which the sequence generated by the resulting block majorize-minimize subspace algorithm converges to a critical point of the objective function, and we exhibit convergence rates for its iterates.

International audience; Stochastic gradient optimization methods are broadly used to minimize non-convex smooth objective functions, for instance when training deep neural networks. However, theoretical guarantees on the asymptotic behaviour of these methods remain scarce. Especially, ensuring almost-sure convergence of the iterates to a stationary point is quite challenging. In this work, we introduce a new Kurdyka-Łojasiewicz theoretical framework to analyze asymptotic behavior of stochastic gradient descent (SGD) schemes when minimizing non-convex smooth objectives. In particular, our framework provides new almost-sure convergence results, on iterates generated by any SGD method satisfying mild conditional descent conditions. We illustrate the proposed framework by means of several toy simulation examples. We illustrate the role of the considered theoretical assumptions, and investigate how SGD iterates are impacted whether these assumptions are either fully or partially satisfied.

International audience; In this paper, a neural network approach is introduced to estimate non-noisy speed and torque from noisy measured currents and voltages in induction motors with Variable Speed Drives. The proposed estimation method is comprised of a neural speed-torque estimator and a neural signal denoiser. A new training strategy is introduced that combines large amount of simulated data and a small amount of real world data. The proposed denoiser does not require non-noisy ground truth data for training, and instead uses classification labels which are easily generated from real-world data. This approach improves upon existing noise removal techniques by learning to denoise as well as classify noisy signals into static and dynamic parts. The proposed neural network based denoiser generates clean estimates of currents and voltages which are then used as inputs to the neural network estimator of speed and torque. Extensive experiments show that the proposed joint denoising-estimation strategy performs very well on real data benchmarks. The proposed denoising method is shown to outperform several widely used denoising methods and a proper ablation study of the proposed method is conducted.

International audience; Current solutions for bacteria and viruses identification are based on time-consuming technics with complex preparation procedures. In the present work, we revealed label-free the presence of free viral particles and bacteria with a computational two-photon fluorescence (C-TPF) strategy. Six bacteria were tested: Escherichia coli, Staphylococcus epidermidis, Proteus vulgaris, Pseudomonas fluorescens, Bacillus subtilis and Clostridium perfringens. The two families of viral particles were the herpes virus with the cytomegalovirus (CMV, 300 nm of diameter) and the coronavirus with the SARS-CoV-2 (100 nm of diameter). The instrumental and computational pipeline FAMOUS optimized the produced 3D images. The origin of the fluorescence emission was discussed for bacteria regarding to their two-photon excitation spectra and attributed to the metabolic indicators (FAD and NADH). The optical and computational strategy constitute a new approach for imaging label-free viral particles and bacteria and paves the way to a new understanding of viral or bacterial ways of infection.

Modern image acquisition devices, from microscopes to medical imaging machines, require to deal with increasingly large amount of data. To limit the dependence of an optimization algorithm on the dimension of the problem, distributed algorithms have been developed. In these schemes, at each iteration only a subset of the variables are updated simultaneously allowing to distribute computations on different nodes (or machines). The implementation of distributed algorithms requires to pay careful attention to the cost of communication, which can be reduced and controlled by resorting to an asynchronous implementation. However, asynchronous implementation raises challenging questions, in terms of convergence analysis, as the communication delays may introduce instabilities in the algorithm behavior. In this work, we propose an asynchronous majoration-minimization (MM) algorithm for solving large scale differentiable non-convex optimization problems. The proposed algorithm runs efficient MM memory gradient updates on block of coordinates, in a parallel and possibly asynchronous manner. We establish the convergence of the resulting sequence of iterates under mild assumptions. The performance of the algorithm is illustrated on the restoration of 3D images degraded by depth-variant 3D blur, arising in multiphoton microscopy. Significant computational time reduction, scalability and robustness are observed on synthetic data, when compared to state-of-the-art methods. Experiments on the restoration of real acquisitions of a muscle structure illustrate the qualitative performance of our approach and its practical applicability.

We investigate inexact proximity operators for weakly convex functions. To this aim, we derive sum rules for proximal ε-subdifferentials, by incorporating the moduli of weak convexity of the functions into the respective formulas. This allows us to investigate inexact proximity operators for weakly convex functions in terms of proximal ε-subdifferentials.

We leverage path differentiability and a recent result on nonsmooth implicit differentiation calculus to give sufficient conditions ensuring that the solution to a monotone inclusion problem will be path differentiable, with formulas for computing its generalized gradient. A direct consequence of our result is that these solutions happen to be differentiable almost everywhere. Our approach is fully compatible with automatic differentiation and comes with assumptions which are easy to check, roughly speaking: semialgebraicity and strong monotonicity. We illustrate the scope of our results by considering three fundamental composite problem settings: strongly convex problems, dual solutions to convex minimization problems and primal-dual solutions to min-max problems.

In this paper, we introduce a variational Bayesian algorithm (VBA) for image blind deconvolution. Our generic framework incorporates smoothness priors on the unknown blur/image and possible affine constraints (e.g., sum to one) on the blur kernel. One of our main contributions is the integration of VBA within a neural network paradigm, following an unrolling methodology. The proposed architecture is trained in a supervised fashion, which allows us to optimally set two key hyperparameters of the VBA model and lead to further improvements in terms of resulting visual quality. Various experiments involving grayscale/color images and diverse kernel shapes, are performed. The numerical examples illustrate the high performance of our approach when compared to state-of-the-art techniques based on optimization, Bayesian estimation, or deep learning., Comment: 13 pages

Motivation: Over the centuries, there have been occurrences of various viral pandemics, one after another. There were major outbreaks caused by the Ebola virus, influenza virus, Mers-COV virus, Sars-COV virus, and most recently, the COVID-19 pandemic caused by Sars-COV-2. These pandemics caused havoc in the world, killed millions of humans and severely effected the world economy. Statistics indicate that COVID-19 will probably become endemic very soon. However, as the cases of monkeypox caused by the monkeypox virus (MPV) are seen to be increasing rapidly, there are fears that we might enter into another pandemic. The COVID-19 pandemic researchers faced numerous challenges in creating a viable vaccine, including a scarcity of time, high costs, and the potential for adverse effects due to lack of proper testing. In order to avoid these problems now use of existing drugs seem to be the best way to treat the infected individuals. This is called drug repositioning. This research aims to repurpose existing drugs for treating monkeypox. We aim to solve the problem of drug-virus interaction (DVA) through deep probabilistic matrix factorization with graph regularization. Here, we recast the curated database of drug-virus associations as a binary matrix, indicating whether the specific drug (from row) has been utilized for treating the associated virus (from column). This matrix is sparse. Prospective treatment regimes planning (using existing drugs) require to infer the missing entries of this matrix. We also utilize metadata for drugs and viruses, obtained from chemical and mechanism of action based similarities between drugs and genomic and symptomatic similarities between the viruses. We demonstrate that our proposed approach significantly outperforms stateof-the-art MF methods. Moreover, when our algorithm is used to make in silico predictions of anti-virals for monkeypox, it returns the drugs that are already being used to treat infected patients.This shows the high efficacy of our algorithm.

International audience; Standard few-shot benchmarks are often built upon simplifying assumptions on the query sets, which may not always hold in practice. In particular, for each task at testing time, the classes effectively present in the unlabeled query set are known a priori, and correspond exactly to the set of classes represented in the labeled support set. We relax these assumptions and extend current benchmarks, so that the query-set classes of a given task are unknown, but just belong to a much larger set of possible classes. Our setting could be viewed as an instance of the challenging yet practical problem of extremely imbalanced K-way classification, K being much larger than the values typically used in standard benchmarks, and with potentially irrelevant supervision from the support set. Expectedly, our setting incurs drops in the performances of state-of-the-art methods. Motivated by these observations, we introduce a PrimAl Dual Minimum Description LEngth (PADDLE) formulation, which balances data-fitting accuracy and model complexity for a given few-shot task, under supervision constraints from the support set. Our constrained MDL-like objective promotes competition among a large set of possible classes, preserving only effective classes that befit better the data of a few-shot task. It is hyperparameter free, and could be applied on top of any base-class training. Furthermore, we derive a fast block coordinate descent algorithm for optimizing our objective, with convergence guarantee, and a linear computational complexity at each iteration. Comprehensive experiments over the standard few-shot datasets and the more realistic and challenging i-Nat dataset show highly competitive performances of our method, more so when the numbers of possible classes in the tasks increase. Our code is publicly available at https://github.com/SegoleneMartin/PADDLE.

The Sphere handbook and its core premise of right to life with dignity have been broadly adopted, establishing a standard operating procedure for global humanitarian intervention. Plenty of machine learning methods aim to aid in disaster relief. While performing exceptionally on a machine learning task, these methods fail to deliver targeted effort to the victims of natural disasters. We argue that this is due to the misalignment of such methods with real-world relief practices. This paper presents the alignment of the Sphere guidelines with Geospatial AI solutions. We show several limitations in different machine learning methods proposed for disaster relief in recent years. We take the case of WASH requirements during flood disasters, extend these models to align with Sphere guidelines, and build a solution that has a much better potential to serve individuals stuck in disasters.

Publisher Copyright: © 2022 The Author(s) This paper presents a multimodal image fusion method using a novel decomposition model based on coupled dictionary learning. The proposed method is general and can be used for a variety of imaging modalities. In particular, the images to be fused are decomposed into correlated and uncorrelated components using sparse representations with identical supports and a Pearson correlation constraint, respectively. The resulting optimization problem is solved by an alternating minimization algorithm. Contrary to other learning-based fusion methods, the proposed approach does not require any training data, and the correlated features are extracted online from the data itself. By preserving the uncorrelated components in the fused images, the proposed fusion method significantly improves on current fusion approaches in terms of maintaining the texture details and modality-specific information. The maximum-absolute-value rule is used for the fusion of correlated components only. This leads to an enhanced contrast-resolution without causing intensity attenuation or loss of important information. Experimental results show that the proposed method achieves superior performance in terms of both visual and objective evaluations compared to state-of-the-art image fusion methods.

International audience; A wide class of problems involves the minimization of a coercive and differentiable function F on R N whose gradient cannot be evaluated in an exact manner. In such context, many existing convergence results from standard gradientbased optimization literature cannot be directly applied and robustness to errors in the gradient is not necessarily guaranteed. This work is dedicated to investigating the convergence of Majorization-Minimization (MM) schemes when stochastic errors affect the gradient terms. We introduce a general stochastic optimization framework, called SABRINA (StochAstic suBspace majoRIzation-miNimization Algorithm) that encompasses MM quadratic schemes possibly enhanced with a subspace acceleration strategy. New asymptotical results are built for the stochastic process generated by SABRINA. Two sets of numerical experiments in the field of machine learning and image processing are presented to support our theoretical results and illustrate the good performance of SABRINA with respect to state-of-the-art gradient-based stochastic optimization methods.

BackgroundAnti-PD-(L)1 treatment is indicated for patients with mismatch repair-deficient (MMRD) tumors, regardless of tumor origin. However, the response rate is highly heterogeneous across MMRD tumors. The objective of the study is to find a score that predicts anti-PD-(L)1 response in patients with MMRD tumors.MethodsSixty-one patients with various origin of MMRD tumors and treated with anti-PD-(L)1 were retrospectively included in this study. An expert radiologist annotated all tumors present at the baseline and first evaluation CT-scans for all the patients by circumscribing them on their largest axial axis (single slice), allowing us to compute an approximation of their tumor volume. In total, 2120 lesions were annotated, which led to the computation of the total tumor volume for each patient. The RECIST sum of target lesions’ diameters and neutrophile-to-lymphocyte (NLR) were also reported at both examinations. These parameters were determined at baseline and first evaluation and the variation between the first evaluation and baseline was calculated, to determine a comprehensive score for overall survival (OS) and progression-free survival (PFS).ResultsTotal tumor volume at baseline was found to be significantly correlated to the OS (p-value: 0.005) and to the PFS (p-value:ConclusionTotal tumor volume appears to be a prognostic biomarker of anti-PD-(L)1 response to immunotherapy in metastatic patients with MMRD tumors. Combining tumor volume and NLR with a simple concordance score stratifies patients well according to their survival and offers a good predictive measure of response to immunotherapy.

BackgroundGene expression is regulated at different molecular levels, including chromatin accessibility, transcription, RNA maturation, and transport. These regulatory mechanisms have strong connections with cellular metabolism. In order to study the cellular system and its functioning, omics data at each molecular level can be generated and efficiently integrated. Here, we proposeBRANEnet, a novel multi-omics integration framework for multilayer heterogeneous networks.BRANEnetis an expressive, scalable, and versatile method to learn node embeddings, leveraging random walk information within a matrix factorization framework. Our goal is to efficiently integrate multi-omics data to study different regulatory aspects of multilayered processes that occur in organisms. We evaluate our framework using multi-omics data ofSaccharomyces cerevisiae, a well-studied yeast model organism.ResultsWe testBRANEneton transcriptomics (RNA-seq) and targeted metabolomics (NMR) data for wild-type yeast strain during a heat-shock time course of 0, 20, and 120 min. Our framework learns features for differentially expressed bio-molecules showing heat stress response. We demonstrate the applicability of the learned features for targeted omics inference tasks: transcription factor (TF)-target prediction, integrated omics network (ION) inference, and module identification. The performance ofBRANEnetis compared to existing network integration methods. Our model outperforms baseline methods by achieving high prediction scores for a variety of downstream tasks.

International audience; Conventional Convolutional Neural Network (CNN) based clustering formulations are based on the encoder-decoder based framework, where the clustering loss is incorporated after the encoder network. The problem with this approach is that it requires training an additional decoder network; this, in turn, means learning additional weights which can lead to over-fitting in data constrained scenarios. This work introduces a Deep Convolutional Transform Learning (DCTL) based clustering framework. The advantage of our proposed formulation is that we do not require learning the additional decoder network. Therefore our formulation is less prone to over-fitting. Comparison with state-of-the-art deep learning based clustering solutions on benchmark image datasets shows that our proposed method improves over the rest in challenging scenarios where there are many clusters with limited samples.

In the last ten years, Convolutional Neural Networks (CNNs) have formed the basis of deep-learning architectures for most computer vision tasks. However, they are not necessarily optimal. For example, mathematical morphology is known to be better suited to deal with binary images. In this work, we create a morphological neural network that handles binary inputs and outputs. We propose their construction inspired by CNNs to formulate layers adapted to such images by replacing convolutions with erosions and dilations. We give explainable theoretical results on whether or not the resulting learned networks are indeed morphological operators. We present promising experimental results designed to learn basic binary operators, and we have made our code publicly available online., Comment: Preprint of a submission to ICIP 2022. 7 pages. 4 figures

International audience; Positron emission tomography (PET) is a quantitative imaging modality widely used in oncology, neurology, and pharmacology. The data acquired by a PET scanner correspond to projections of the concentration activity, assumed to follow a Poisson distribution. The reconstruction of images from tomographic projections corrupted by Poisson noise is a challenging ill-posed large-scale inverse problem. Several available solvers use the majorization-minimization (MM) principle, though relying on various construction strategies with a lack of unifying framework. This work fills the gap by introducing the concept of Bregman majorization. This leads to a unified view of MM-based methods for image reconstruction in the presence of Poisson noise. From this general approach, we exhibit three algorithmic solutions and compare their computational efficiency on a problem of dynamic PET image reconstruction, either using GPU or CPU processing.

International audience; For best medical imaging application results, learning-based approaches such as deep learning necessitate specific, extensive and precise annotations. Outside well-curated public benchmarks, these are rarely available in practice, and so it becomes necessary to use less-than-perfect annotations. One way of compensating for this is the embedding of anatomical knowledge. Complementing this, there is the incremental semi-supervised learning technique, whereby a small amount of annotations can be used to derive more and superior labels.In this article, we illustrate this approach on a deep learning system to help radiologists and rheumatologists finely and interactively assess MRI scans of the sacro-iliac joint in order to correctly diagnose Axial Spondyloarthritis. Our model is trained initially on a relatively small set of images with promising results, on par with expert opinion and generalizable to new datasets.

Artificial intelligence (AI) is increasingly being studied in musculoskeletal oncology imaging. AI has been applied to both primary and secondary bone tumors and assessed for various predictive tasks that include detection, segmentation, classification, and prognosis. Still, in the field of clinical research, further efforts are needed to improve AI reproducibility and reach an acceptable level of evidence in musculoskeletal oncology. This review describes the basic principles of the most common AI techniques, including machine learning, deep learning and radiomics. Then, recent developments and current results of AI in the field of musculoskeletal oncology are presented. Finally, limitations and future perspectives of AI in this field are discussed.

International audience; In whole slide imaging, commonly used staining techniques based on hematoxylin and eosin (H&E) and immunohistochemistry (IHC) stains accentuate different aspects of the tissue landscape. In the case of detecting metastases, IHC provides a distinct readout that is readily interpretable by pathologists. IHC, however, is a more expensive approach and not available at all medical centers. Virtually generating IHC images from H&E using deep neural networks thus becomes an attractive alternative. Deep generative models such as CycleGANs learn a semantically-consistent mapping between two image domains, while emulating the textural properties of each domain. They are therefore a suitable choice for stain transfer applications. However, they remain fully unsupervised, and possess no mechanism for enforcing biological consistency in stain transfer. In this paper, we propose an extension to CycleGANs in the form of a region of interest discriminator. This allows the CycleGAN to learn from unpaired datasets where, in addition, there is a partial annotation of objects for which one wishes to enforce consistency. We present a use case on whole slide images, where an IHC stain provides an experimentally generated signal for metastatic cells. We demonstrate the superiority of our approach over prior art in stain transfer on histopathology tiles over two datasets. Our code and model are available at https://github.com/jcboyd/miccai2022-roigan.

Histopathology whole slide images (WSIs) play a very important role in clinical studies and serve as the gold standard for many cancer diagnoses. However, generating automatic tools for processing WSIs is challenging due to their enormous sizes. Currently, to deal with this issue, conventional methods rely on a multiple instance learning (MIL) strategy to process a WSI at patch level. Although effective, such methods are computationally expensive, because tiling a WSI into patches takes time and does not explore the spatial relations between these tiles. To tackle these limitations, we propose a locally supervised learning framework which processes the entire slide by exploring the entire local and global information that it contains. This framework divides a pre-trained network into several modules and optimizes each module locally using an auxiliary model. We also introduce a random feature reconstruction unit (RFR) to preserve distinguishing features during training and improve the performance of our method by 1% to 3%. Extensive experiments on three publicly available WSI datasets: TCGA-NSCLC, TCGA-RCC and LKS, highlight the superiority of our method on different classification tasks. Our method outperforms the state-of-the-art MIL methods by 2% to 5% in accuracy, while being 7 to 10 times faster. Additionally, when dividing it into eight modules, our method requires as little as 20% of the total gpu memory required by end-to-end training. Our code is available at https://github.com/cvlab-stonybrook/local_learning_wsi., Accepted to MICCAI 2022 Oral

International audience; Renal transplantation appears as the most effective solution for end-stage renal disease. However, it may lead to renal allograft rejection or dysfunction within 15% − 27% of patients in the first 5 years post-transplantation. Resulting from a simple blood test, serum creatinine is the primary clinical indicator of kidney function by calculating the Glomerular Filtration Rate. These characteristics motivate the challenging task of predicting serum creatinine early post-transplantation while investigating and exploring its correlation with imaging data. In this paper, we propose a sequential architecture based on transformer encoders to predict the renal function 2-years post-transplantation. Our method uses features generated from Dynamic Contrast-Enhanced Magnetic Resonance Imaging from 4 follow-ups during the first year after the transplant surgery. To deal with missing data, a key mask tensor exploiting the dot product attention mechanism of the transformers is used. Moreover, different contrastive schemes based on cosine similarity distance are proposed to handle the limited amount of available data. Trained on 69 subjects, our best model achieves 96.3% F1 score and 98.9% ROC AUC in the prediction of serum creatinine threshold on a separated test set of 20 subjects. Thus, our experiments highlight the relevance of considering sequential imaging data for this task and therefore in the study of chronic dysfunction mechanisms in renal transplantation, setting the path for future research in this area. Our code is available at https://github.com/leomlck/renal_transplant_imaging.

International audience

International audience; State-space models (SSM) are central to describe time-varying complex systems in countless signal processing applications such as remote sensing, networks, biomedicine, and finance to name a few. Inference and prediction in SSMs are possible when the model parameters are known, which is rarely the case. The estimation of these parameters is crucial, not only for performing statistical analysis, but also for uncovering the underlying structure of complex phenomena. In this paper, we focus on the linear-Gaussian model, arguably the most celebrated SSM, and particularly in the challenging task of estimating the transition matrix that encodes the Markovian dependencies in the evolution of the multi-variate state. We introduce a novel perspective by relating this matrix to the adjacency matrix of a directed graph, also interpreted as the causal relationship among state dimensions in the Granger-causality sense. Under this perspective, we propose a new method called GraphEM based on the well sounded expectation-maximization (EM) methodology for inferring the transition matrix jointly with the smoothing/filtering of the observed data. We propose an advanced convex optimization solver relying on a consensus-based implementation of a proximal splitting strategy for solving the M-step. This approach enables an efficient and versatile processing of various sophisticated priors on the graph structure, such as parsimony constraints, while benefiting from convergence guarantees. We demonstrate the good performance and the interpretable results of GraphEM by means of two sets of numerical examples.

International audience

International audience; Optimization problems arising in signal and image processing involve an increasingly large number of variables. In addition to the curse of dimensionality, another difficulty to overcome is that the cost function usually reads as the sum of several loss/regularization terms, which are non-necessarily smooth and possibly composed with large-size linear operators. Proximal splitting approaches are fundamental tools to address such problems, with demonstrated efficiency in many applicative fields. In this paper, we present a new distributed algorithm for computing the proximity operator of a sum of non-necessarily smooth convex functions composed with arbitrary linear operators. Our algorithm relies on a primal-dual splitting strategy, and benefits from established convergence guaranties. Each involved function is associated with a node of a hypergraph, with the ability to communicate with neighboring nodes sharing the same hyperedge. Thanks to this structure, our method can be efficiently implemented on modern parallel computing architectures, distributing the computations on multiple nodes or machines, with controlled requirements for synchronization steps. Good numerical performance and scalability properties are demonstrated on a problem of video sequence denoising. Our code implemented in Julia is made available at \url{https://github.com/MarinENSTA/distributed_julia_denoising}.

International audience; ObjectivesThe aim of this study is to evaluate a deep learning method designed to increase the contrast-to-noise ratio in contrast-enhanced gradient echo T1-weighted brain MRI acquisitions. The processed images are quantitatively evaluated in terms of lesion detection performance.Materials and methodsA total of 250 multiparametric brain MRIs, acquired between November 2019 and March 2021 at Gustave Roussy Cancer Campus (Villejuif, France), were considered for inclusion in this retrospective monocentric study. Independent training (107 cases, age 55y±14, 58 women) and test (79 cases, age 59y±14, 41 women) samples were defined. Patients had glioma, brain metastasis, meningioma, or no enhancing lesion. Gradient echo and turbo spin echo with variable flip angles postcontrast T1 sequences were acquired in all cases. For the cases that formed the training sample, “low-dose” postcontrast gradient echo T1 images using 0.025mmol/kg injections of contrast agent were also acquired. A deep neural network was trained to synthetically enhance the low-dose T1 acquisitions, taking standard-dose T1 MRI as reference. Once trained, the contrast enhancement network was used to process the test gradient echo T1 images. A read was then performed by two experienced neuroradiologists to evaluate the original and processed T1 MRI sequences in terms of contrast enhancement and lesion detection performance, taking the turbo spin echo sequences as reference.ResultsThe processed images were superior to the original gradient echo and reference turbo spin echo T1 sequences in terms of contrast-to-noise ratio (44.5 versus 9.1 and 16.8, p.99). The same effect was observed when considering all lesions larger than 5mm: sensitivity increased from 70% to 85% (p

International audience

International audience; In interventional radiology, 3D reconstruction of devices such as needles can increase the precision of procedures. Doing so with CBCT is time-consuming and increases the X-ray dose. Needles being sparse, a compressed-sensing reconstruction approach seems viable. In this paper, we thus investigate the benefits of directional total variation as an adequate prior for anisotropic devices. We introduce a decomposition method that allows several a priori directions to be considered at once as well as excludes the anatomical background that is not sparse. The capacity of the method is illustrated in simulations of limited-angle acquisitions. It allows good reconstruction of the needles from a small angular coverage, even if it cannot recover the anatomical background.

International audience; Both theoretical and practical problems in deep learning classification require solutions for assessing uncertainty prediction but current state-of-the-art methods in this area are computationally expensive. In this paper, we propose a new confidence measure dubbed Over-MAP that utilizes a measure of overlap between structural attention mechanisms and segmentation methods, that is of particular interest in accurate fine-grained contexts. We show that this classification confidence increases with the degree of overlap. The associated confidence and identification tools are conceptually simple, efficient, and of high practical interest as they allow for weeding out misleading examples in training data. Our measure is currently deployed in the real-world on widely used platforms to annotate large-scale data efficiently.

International audience; The identification of parameters of spatially variant blurs given a clean image and its blurry noisy version is a challenging inverse problem of interest in many application fields, such as biological microscopy and astronomical imaging. In this paper, we consider a parametric model of the blur and introduce an 1D state-space model to describe the statistical dependence among the neighboring kernels. We apply a Bayesian approach to estimate the posterior distribution of the kernel parameters given the available data. Since this posterior is intractable for most realistic models, we propose to approximate it through a sequential Monte Carlo approach by processing all data in a sequential and efficient manner. Additionally, we propose a new sampling method to alleviate the particle degeneracy problem, which is present in approximate Bayesian filtering, particularly in challenging concentrated posterior distributions. The considered method allows us to process sequentially image patches at a reasonable computational and memory costs. Moreover, the probabilistic approach we adopt in this paper provides uncertainty quantification which is useful for image restoration. The practical experimental results illustrate the improved estimation performance of our novel approach, demonstrating also the benefits of exploiting the spatial structure the parametric blurs in the considered models.

The SARS-COV-2 pandemic has put pressure on intensive care units, so that identifying predictors of disease severity is a priority. We collect 58 clinical and biological variables, and chest CT scan data, from 1003 coronavirus-infected patients from two French hospitals. We train a deep learning model based on CT scans to predict severity. We then construct the multimodal AI-severity score that includes 5 clinical and biological variables (age, sex, oxygenation, urea, platelet) in addition to the deep learning model. We show that neural network analysis of CT-scans brings unique prognosis information, although it is correlated with other markers of severity (oxygenation, LDH, and CRP) explaining the measurable but limited 0.03 increase of AUC obtained when adding CT-scan information to clinical variables. Here, we show that when comparing AI-severity with 11 existing severity scores, we find significantly improved prognosis performance; AI-severity can therefore rapidly become a reference scoring approach., The SARS-COV-2 pandemic has put pressure on intensive care units, so that predicting severe deterioration early is a priority. Here, the authors develop a multimodal severity score including clinical and imaging features that has significantly improved prognostic performance in two validation datasets compared to previous scores.

International audience; We are presenting the application of an optical and computational pipeline FAMOUS for revealing the presence of free viral particles named "virions". The idea of such a protocol is to give rise to images of virions in their environment with a soft solution for recording the native image, contrary to the standard solution of imaging virions with electron microscopy (EM) for visualizing viral particles. The final aim of the current work is to observe free viral particles of SARS-CoV-2, the virions responsible for the worldwide pandemic of Covid-19. But such particles have diameters between 80 and 120 nm, a dimension smaller than the resolution limit of optical-only microscopy solutions. We have chosen to start with the biggest free virions, cytomegalovirus (CMV), a virus from the herpesvirus family also named "Human Herpes Virus 5". Two kinds of cultures were involved: a fluorescent culture (BAD) and a label-free one (VHLE), both being collected from infected cell culture. VHLE virions were first observed after secondary immunostaining and concentrated with magnetic nanoparticles and then without labelling. The optical protocol rests on a standard solution of multiphoton microscopy combined with a computational strategy based on the point-spread-function (PSF) recordings, its mathematical modeling and the restauration of the image resting on the PSF model. A test with free viral particles of SARS-CoV-2 is led, delivering an optical visualization of the free-viral particles. The visualization of objects aggregates obtained in both situations confirm the relevance of the pipeline FAMOUS for imaging free virions.