Search

This paper presents a rigorous but tractable study of sparsity. We postulate a definition of sparsity that is as broad as possible, so that it generates all the various measures that are useful in practice, but narrow enough that the fundamental properties of generalized sparsity still hold. As we work through the various ways of demonstrating the advantageous properties of sparsity, we illustrate its meaning from geometrical and operational perspectives. Thereafter, we construct specific measures of sparsity which are successfully qualified in complexity analysis and sparse optimization scenarios. Overall, our main objective is to construct measures of sparsity that will facilitate and enhance the design of the next innovative sensing technologies.

An effective visible light and infrared image fusion method using a deep learning framework is designed to obtain a fused image which contains all the features from infrared and visible images. First, the source images are decomposed into low frequency and high frequency sub bands using wavelet transform. Then the low frequency is fused by maximum fusion rule. For the high frequency sub bands a deep learning network is used to find activity level measurements and then fused using the maximum fusion rule. For reconstruction, the optimized orthogonal matching pursuit algorithm and inverse wavelet transform are used.

Surface Electromyography (sEMG) is a non-invasive measurement process that does not involve tools and instruments to break the skin or physically enter the body to investigate and evaluate the muscular activities produced by skeletal muscles. The main drawbacks of existing sEMG systems are: (1) they are not able to provide real-time monitoring, (2) they suffer from long processing time and low speed, (3) they are not effective for wireless healthcare systems because they consume huge power. In this work, we present an analog-based Compressed Sensing (CS) architecture, which consists of three novel algorithms for design and implementation of wearable wireless sEMG bio-sensor. At the transmitter side, two new algorithms are presented in order to apply the analog-CS theory before Analog to Digital Converter (ADC). At the receiver side, a robust reconstruction algorithm based on a combination of ℓ1-ℓ1-optimization and Block Sparse Bayesian Learning (BSBL) framework is presented to reconstruct the original bio-signals from the compressed bio-signals. The proposed architecture allows reducing the sampling rate to 25% of Nyquist Rate (NR). In addition, the proposed architecture reduces the power consumption to 40%, Percentage Residual Difference (PRD) to 24%, Root Mean Squared Error (RMSE) to 2%, and the computation time from 22 s to 9.01 s, which provide good background for establishing wearable wireless healthcare systems. The proposed architecture achieves robust performance in low Signal-to-Noise Ratio (SNR) for the reconstruction process.

International audience; This paper considers a particular parameter estimator for switched systems and analyzes its properties. The estimator in question is defined as the map from the data set to the solution set of an optimization problem where the to-be-optimized cost function is a sum of pointwise infima over a finite set of sub-functions. This is a hard nonconvex problem. The paper studies some fundamental properties of this problem such as uniqueness of the solution or boundedness of the estimation error regardless of computational considerations. The interest of the analysis is to lay out the main influential properties of the data on the performance of this (ideal) estimator.

Data analysis methods have been extensively used in active thermography for defect identification. Among them, principal component thermography (PCT) is popular for dimensionality reduction and feature extraction. PCT summarizes the thermal images with a small number of empirical orthogonal functions that better reflect the information of defects. However, PCT does not induce sparsity, which limits the interpretation of PCT results. Recently, sparse PCT (SPCT) has been proposed to provide more interpretable analysis results. However, SPCT does not consider the spatial connectivity between pixels, omitting the fact that a defective region is usually spatially connected. In this article, a novel thermographic data analysis method is proposed to overcome the shortcomings of the existing methods. The proposed method imposes both spatial connectivity and sparsity constraints in PCT. Finally, one case study on an ancient marquetry sample and another on a carbon fiber-reinforced polymer composite illustrate the feasibility of the proposed method.

Standard compressive sensing (CS) scenario assumes a single sparsifying basis used to reconstruct the signals from a small set of incoherent measurements. However, in many cases, the signal cannot be sparsely represented using a single transformation. Particularly, in ECG signal analysis, each signal segment is specific in nature and reflects different physical phenomena. Hence, using the same transformation for all segments may be inappropriate for efficient analysis and reconstruction. Moreover, in the CS scenario, it would be necessary to combine different transforms to achieve compact signal support and to provide successful reconstruction from randomly under-sampled data. This work proposes a hybrid CS reconstruction algorithm that combines different transform basis, based on the concept of orthogonal matching pursuit. The performance of the proposed approach is verified experimentally using the combination of the Fourier and the Hermite transform on the real ECG signals.

This paper presents a methodology to design a sparse multiband ranging signal with a large virtual bandwidth, from which time delay and carrier phase are estimated by a low complexity multivariate maximum likelihood (ML) method. In the estimation model for a multipath channel, not all reflected paths are considered, and time delay and carrier phase are estimated in a step-wise manner to further reduce the computational load. By introducing a measure of dependence and a measure of bias for a multipath reflection, we analyse the bias, precision and accuracy of time delay and carrier phase estimation. Since these two indicators are determined by the signal spectrum pattern, they are used to formulate an optimization for signal design. By solving the optimization problem, only a few bands from the available signal spectrum are selected for ranging. Consequently, the designed signal only occupies a small amount of signal spectrum but has a large virtual bandwidth and can thereby still offer a high ranging precision with only a small bias, based on the low-complexity simplified ML method. Numerical and laboratory experiments are carried out to evaluate the ranging performance of the proposed estimation method based on sparsely selected signal bands. Relative positioning, in which we only measure a change in position, based on either the time delay estimates or the carrier phase estimates, is presented as a proof-of-concept for precise positioning. The results show that positioning based on only 7 out of 16 signal bands, sparsely placed in the available spectrum, achieves a decimeter level accuracy when using time delay estimates, and a millimeter level accuracy when using carrier phase estimates. Compared with the case of using all available bands, and without largely decreasing the positioning performance, the computational complexity when using the sparse multiband signal can be reduced by about 80%.

Classification and regression trees, as well as their variants, are off-the-shelf methods in Machine Learning. In this paper, we review recent contributions within the Continuous Optimization and the Mixed-Integer Linear Optimization paradigms to develop novel formulations in this research area. We compare those in terms of the nature of the decision variables and the constraints required, as well as the optimization algorithms proposed. We illustrate how these powerful formulations enhance the flexibility of tree models, being better suited to incorporate desirable properties such as cost-sensitivity, explainability, and fairness, and to deal with complex data, such as functional data.

Given a $k$-node pattern graph $H$ and an $n$-node host graph $G$, the subgraph counting problem asks to compute the number of copies of $H$ in $G$. In this work we address the following question: can we count the copies of $H$ faster if $G$ is sparse? We answer in the affirmative by introducing a novel tree-like decomposition for directed acyclic graphs, inspired by the classic tree decomposition for undirected graphs. This decomposition gives a dynamic program for counting the homomorphisms of $H$ in $G$ by exploiting the degeneracy of $G$, which allows us to beat the state-of-the-art subgraph counting algorithms when $G$ is sparse enough. For example, we can count the induced copies of any $k$-node pattern $H$ in time $2^{O(k^2)} O(n^{0.25k + 2} \log n)$ if $G$ has bounded degeneracy, and in time $2^{O(k^2)} O(n^{0.625k + 1} \log n)$ if $G$ has bounded average degree. These bounds are instantiations of a more general result, parameterized by the degeneracy of $G$ and the structure of $H$, which generalizes classic bounds on counting cliques and complete bipartite graphs. We also give lower bounds based on the Exponential Time Hypothesis, showing that our results are actually a characterization of the complexity of subgraph counting in bounded-degeneracy graphs., Comment: Extended version of a work appeared at IPEC 2019

Complex-valued Magnetic Resonance Imaging (MRI) is widely used in clinical diagnosis. The magnitude images are mainly used for structure visualization, and the phase images reveal tissue properties, such as magnetic susceptibility and fluid flow information. While MRI is slow, compressed sensing (CS) can be used to reconstruct the accelerated acquisitions. Current CS-based MRI reconstruction algorithms mostly focus on magnitude image recovery, with less attention to the recovery of phase images. In this paper, we propose a novel CS algorithm to simultaneously recover the magnitude and phase MR images based on the sparsity of the trigonometric function. The CS method requires a sparse representation of the original images, and it is observed the trigonometric functions of phase images in the Wavelet domain promote sparsity. Therefore, rather than transforming the phase images directly into the Wavelet domain, we calculate the sine and cosine of the phase images whose Wavelet transforms are set as the L1-norm regularization term. The combination of the dual trigonometric functions captures a unique, faithful four-quadrant phase information, which also improves the reconstructed magnitude images through an alternating optimization procedure. Reconstructions of simulated images and in vivo images are studied, and both show the superiority of the proposed method over compared phase recovery algorithms.

Data sparsity is a major challenge for collaborative filtering recommender systems. A promising solution is to utilize feedback or ratings from multiple domains to improve the performance of recommendations in a collective way, known as the cross-domain recommendation. Cross-domain recommendation using heterogeneous feedback is a popular solution, which transfers knowledge from the more easily available auxiliary binary feedback to improve the prediction performance of the target domain. Most of the existing work focuses on the transfer of knowledge between different domains from the same website, where user behavior data in different domains can be fully shared. The existing work mainly assumes that data from different domains can be fully shared. However, due to the constraints of business privacy policies, it is difficult to directly share exactly the same user behavior data between different e-commerce websites. It results in that the user’s latent factors learned in the auxiliary domain cannot be directly transferred to the target domain, otherwise, it will cause a negative transfer issue. In this article, we consider that the auxiliary domain with numerical ratings and target domains with binary feedbacks only share overlapping items rather than users. We propose a Selective Knowledge Transfer for Cross-domain Collaborative Recommendation, called SKT. The proposed SKT framework not only transfers the item’s latent factors learned from the auxiliary domain to the target domain, but also selectively transfers the user’s latent factors learned from the auxiliary domain to the target domain. In addition, due to the introduction of co-graph regularization of user graphs and item graphs, SKT can maintain respective intrinsic geometric structure within each domain and thus avoid negative transfer issue. Extensive experiments conducted on two real-world datasets show that our SKT method is significantly better than all baseline methods at various density levels.

We propose an algorithm for the blind separation of single-channel audio signals. It is based on a parametric model that describes the spectral properties of the sounds of musical instruments independently of pitch. We develop a novel sparse pursuit algorithm that can match the discrete frequency spectra from the recorded signal with the continuous spectra delivered by the model. We first use this algorithm to convert an STFT spectrogram from the recording into a novel form of log-frequency spectrogram whose resolution exceeds that of the mel spectrogram. We then make use of the pitch-invariant properties of that representation in order to identify the sounds of the instruments via the same sparse pursuit method. As the model parameters which characterize the musical instruments are not known beforehand, we train a dictionary that contains them, using a modified version of Adam. Applying the algorithm on various audio samples, we find that it is capable of producing high-quality separation results when the model assumptions are satisfied and the instruments are clearly distinguishable, but combinations of instruments with similar spectral characteristics pose a conceptual difficulty. While a key feature of the model is that it explicitly models inharmonicity, its presence can also still impede performance of the sparse pursuit algorithm. In general, due to its pitch-invariance, our method is especially suitable for dealing with spectra from acoustic instruments, requiring only a minimal number of hyperparameters to be preset. Additionally, we demonstrate that the dictionary that is constructed for one recording can be applied to a different recording with similar instruments without additional training.

Herein, we propose a localization method for implant devices using electromagnetic scattering based on sparse reconstruction. We define the product of the total electric field and the complex relative permittivity as a new unknown variable, which is expected to be sparse. Computer simulations are conducted to validate the sparsity of the estimated variable, and localization errors are evaluated to confirm the performance improvement using the sparsity-based localization method. The result confirmed the excellent localization performance, particularly below 17 MHz, even under low signal-to-noise power ratios.

Regularization addresses the ill-posedness of the training problem in machine learning or the reconstruction of a signal from a limited number of measurements. The method is applicable whenever the problem is formulated as an optimization task. The standard strategy consists in augmenting the original cost functional by an energy that penalizes solutions with undesirable behavior. The effect of regularization is very well understood when the penalty involves a Hilbertian norm. Another popular configuration is the use of an $$\ell _1$$ ℓ 1 -norm (or some variant thereof) that favors sparse solutions. In this paper, we propose a higher-level formulation of regularization within the context of Banach spaces. We present a general representer theorem that characterizes the solutions of a remarkably broad class of optimization problems. We then use our theorem to retrieve a number of known results in the literature such as the celebrated representer theorem of machine leaning for RKHS, Tikhonov regularization, representer theorems for sparsity promoting functionals, the recovery of spikes, as well as a few new ones.

Compressive sensing (CS) provides a potential platform for acquiring slow and sequential data, as in magnetic resonance (MR) imaging. However, CS requires high computational time for reconstructing MR images from sparse k-space data, which restricts its usage for high speed online reconstruction and wireless communications. Another major challenge is removal of Rician noise from magnitude MR images which changes the image characteristics, and thus affects the clinical usefulness. The work carried out so far predominantly models MRI noise as a Gaussian type. The use of advanced noise models primarily Rician type in CS paradigm is less explored. In this work, we develop a novel framework to reconstruct MR images with high speed and visual quality from noisy sparse k-space data. The proposed algorithm employs a convolutional neural network (CNN) to denoise MR images corrupted with Rician noise. To extract local features, the algorithm exploits signal similarities by processing similar patches as a group. An imperative reduction in the run time has been achieved as the CNN has been trained on a GPU with Convolutional Architecture for Fast Feature Embedding framework making it suitable for online reconstruction. The CNN based reconstruction also eliminates the necessity of optimization and prediction of noise level while denoising, which is the major advantage over existing state-of-the-art-techniques. Analytical experiments have been carried out with various undersampling schemes and the experimental results demonstrate high accuracy and consistent peak signal to noise ratio even at 20-fold undersampling. High undersampling rates provide scope for wireless transmission of k-space data and high speed reconstruction provides applicability of our algorithm for remote health monitoring.

Multispectral image denoising is a basic problem whose results affect subsequent processes such as target detection and classification. Numerous approaches have been proposed, but there are still many challenges, particularly in using prior knowledge of multispectral images, which is crucial for solving the ill-posed problem of noise removal. This paper considers both non-local self-similarity in space and global correlation in spectrum. We propose a novel low-rank Tucker decomposition model for removing the noise, in which sparse and graph Laplacian regularization terms are employed to encode this prior knowledge. It can jointly learn a sparse and low-rank representation while preserving the local geometrical structure between spectral bands, so as to better capture simultaneously the correlation in spatial and spectral directions. We adopt the alternating direction method of multipliers to solve the resulting problem. Experiments demonstrate that the proposed method outperforms the state-of-the-art, such as cube-based and tensor-based methods, both quantitatively and qualitatively.

International audience; High Frequency Oscillations (HFOs) occurring in the range of [30–500 Hz] in epileptic intracranial ElectroEncephaloGraphic (iEEG) signals have recently proven to be good biomarkers for localizing the epileptogenic zone. Identifying these particular cerebral events and their discrimination from other transient events like interictal epileptic spikes is traditionally performed by experts through a visual inspection. However, this is laborious, very time-consuming and subjective. In this paper, a new classification approach of HFOs is proposed. This approach mainly relies on the combination of raw time frequency (TF) features, computed from a TF representation of HFOs using S-transform, with relevant image-based ones derived from a binarization of the corresponding TF grayscale image. The obtained feature vector is then used to learn a multi-class Radial Basis Function (RBF) based Support Vector Machine (SVM) classifier. The efficiency of the proposed approach, compared to conventional classification schemes based only on time, frequency or energy-based features, is confirmed, using both simulated and real iEEG signals. The proposed classification system has achieved, using simulated data and a signal to noise ratio (SNR) of 15 dB, a sensitivity, specificity, accuracy, area under the curve and F1-score around 0.990, 0.996, 0.995, 0.993 and 0.990 respectively. Besides, for real data, our proposed approach has attained the scores of 0.765, 0.941, 0.906, 0.929 and 0.768 for sensitivity, specificity, accuracy, area under the curve and F1-score respectively. These results confirm the relevance of coupling TF and image-related features, in the way proposed in this paper, for higher HFOs classification quality compared to already existing approaches. © 2021 Elsevier Ltd

We show that a new design criterion, i.e., the least squares on subband errors regularized by a weighted norm, can be used to generalize the proportionate-type normalized subband adaptive filtering (PtNSAF) framework. The new criterion directly penalizes subband errors and includes a sparsity penalty term which is minimized using the damped regularized Newton's method. The impact of the proposed generalized PtNSAF (GPtNSAF) is studied for the system identification problem via computer simulations. Specifically, we study the effects of using different numbers of subbands and various sparsity penalty terms for quasi-sparse, sparse, and dispersive systems. The results show that the benefit of increasing the number of subbands is larger than promoting sparsity of the estimated filter coefficients when the target system is quasi-sparse or dispersive. On the other hand, for sparse target systems, promoting sparsity becomes more important. More importantly, the two aspects provide complementary and additive benefits to the GPtNSAF for speeding up convergence., 5 pages. Presented at Asilomar Conference on Signals, Systems, and Computers (ACSSC) 2019

International audience; The problem of blind joint FIR-MIMO channel and data estimation is addressed in this paper. Based on a regularized DML (Deterministic Maximum Likelihood) formulation of the problem, a bilinear approach is used in order to estimate jointly the channel impulse responses and the input data. This regularization is introduced as a penalty function added to the classical DML criterion representing the a priori information about the problem in order to enhance the accuracy of the estimation. Two types of priors information are considered for the transmitted data: the finite alphabet simplicity or the sparsity. The sparsity prior was also considered for channel impulse responses. The key advantage of the proposed criteria is their convexity when optimized alternatively over the channel and the input data. The proposed approach allows to improve further the estimation accuracy of such a blind estimation problem but suffers from a relatively high computational cost. Hence, a reduced complexity implementation of the latter has been proposed at the end of the paper, in an adaptive scheme for high dimensional or streaming data situations.

The goal of sparse FIR filter design is to minimize the number of nonzero filter coefficients, while keeping its frequency response within specified boundaries. Such a design can be formally expressed via minimization of l0-norm of filter’s impulse response. Unfortunately, the corresponding minimization problem has combinatorial complexity. Therefore, many design methods are developed, which solve the problem approximately, or which solve the approximate problem exactly. In this paper, we propose an approach, which is based on the approximation of the l0-norm by an lp-norm with 0 < p < 1. We minimize the lp-norm using recently developed method for signomial programming (SGP). Our design starts with forming a SGP problem that describes filter specifications. The optimum solution of the problem is then found by using iterative procedure, which solves a geometric program in each iteration. The filters whose magnitude responses are constrained in minimax sense are considered. The design examples are provided illustrating that the proposed method, in most cases, results in filters with higher sparsity than those of the filters obtained by recently published methods.

We develop an efficient computational solution to train deep neural networks (DNN) with free-form activation functions. To make the problem well-posed, we augment the cost functional of the DNN by adding an appropriate shape regularization: the sum of the second-order total-variations of the trainable nonlinearities. The representer theorem for DNNs tells us that the optimal activation functions are adaptive piecewise-linear splines, which allows us to recast the problem as a parametric optimization. The challenging point is that the corresponding basis functions (ReLUs) are poorly conditioned and that the determination of their number and positioning is also part of the problem. We circumvent the difficulty by using an equivalent B-spline basis to encode the activation functions and by expressing the regularization as an ℓ1-penalty. This results in the specification of parametric activation function modules that can be implemented and optimized efficiently on standard development platforms. We present experimental results that demonstrate the benefit of our approach.

Fulcrum coding combines a high-field outer Random Linear Network Coding (RLNC) that generates outer coding expansion packets with a small-field inner RLNC that combines the source packets and the outer coding expansion packets. This two-layer Fulcrum coding allows flexible decoding in receivers with heterogeneous computational capabilities. Fulcrum coding has so far only been studied for conventional dense RLNC, which randomly selects all coding coefficients, and only for a statically fixed number of outer expansion packets. However, the probability that the coding coefficient row of a newly received packet is linearly independent of prior received coding coefficient rows (a prerequisite for successful decoding) is highly dynamic. We propose to exploit the dynamics of this probability to reduce the computational complexity of Fulcrum coding. In particular, we vary the density of non-zero coding coefficients, i.e., equivalently, the sparsity of coding coefficients, and the number of outer expansion packets to keep the complexity low while maintaining a reasonably high decoding probability. We introduce the general principles of dynamic sparsity and expansion packets (DSEP) for Fulcrum coding as well as two specific example DSEP policies. Our evaluations indicate that DSEP Fulcrum can increase the encoding throughput tenfold and increase the decoding throughput 1.4 to 4.3 fold while achieving decoding probabilities that are typically less than 1% lower than the conventional Fulcrum decoding probabilities. We also find that DSEP achieves somewhat higher encoding and decoding throughputs than the CodornicesRq (Release 2.1) implementation of RaptorQ block coding for small blocks (generations) of source packets, while RaptorQ is substantially faster for large generation sizes. Furthermore, we develop and evaluate an elementary DSEP recoding mechanism that achieves a recoding throughput more than double the decoding throughput.

Collaborative filtering (CF) is a technique used in recommender systems to provide meaningful suggestions based on known feedback obtained from like-minded users. The measure of similarity plays a critical role in the performance of neighborhood-based CF methods. However, conventional similarity measures suffer from limitations because they only consider the direction of the rating vectors. We propose a novel similarity measure that considers the semantic nuances of the ratings; in particular, it weights the contributions of ratings in proportion to the users’ degree of indifference towards the items. Additionally, to address the sparsity problem that affects the performance of CF techniques, we propose a switching hybrid method that predicts user ratings based on either our custom similarity measure or through user and item biases. We evaluated the proposed method on six different datasets and compared it with other CF methods. The results show that the proposed recommender consistently outperforms those using conventional similarity measures when the sparsity of the dataset is high.

We study one-dimensional continuous-domain inverse problems with multiple generalized total-variation regularization, which involves the joint use of several regularization operators. Our starting point is a new representer theorem that states that such inverse problems have hybrid-spline solutions with a total sparsity bounded by the number of measurements. We show that such continuous-domain problems can be discretized in an exact way by using a union of B-spline dictionary bases matched to the regularization operators. We then propose a multiresolution algorithm that selects an appropriate grid size that depends on the problem. Finally, we demonstrate the computational feasibility of our algorithm for multiple-order derivative regularization operators.

In this paper, we study properties of two wind speed forecasting schemes from mid-to-short term wind velocity measurements. The measurements are assumed to be collected over time intervals shorter than the ones previously studied by the authors. Two application examples illustrate the properties of these schemes. In the first example, historical data originating from five meteorological stations are considered. This example demonstrates that the first scheme outperforms persistence and artificial neural network (ANN) predictors by a large margin for all step sizes considered. This result enlarges domain of application of the first forecasting scheme from multi-step-ahead only to one and multi-step-ahead and from long-term observations only to long and mid-term observations. A local wildly fluctuating dense data set obtained from a renewable energy research home unit is studied in the second example to check the performance of the first scheme under non-standard operating conditions. The second scheme is a compressive subspace algorithm developed recently for innovation models.The second scheme complements the first scheme in that it uses short-term observations and outperforms the multi-step-ahead persistence and the ANN predictors.

International audience; In this paper, we consider a high-dimensional non-parametric regression model with fixed design and i.i.d. random errors. We propose a powerful estimator by exponential weighted aggregation (EWA) with a group-analysis sparsity promoting prior on the weights. We prove that our estimator satisfies a sharp group-analysis sparse oracle inequality with a small remainder term ensuring its good theoretical performances. We also propose a forward-backward proximal Langevin Monte-Carlo algorithm to sample from the target distribution (which is not smooth nor log-concave) and derive its guarantees. In turn, this allows us to implement our estimator and validate it on some numerical experiments.

International audience; A contrast source inversion (CSI) algorithm is developed in the wavelet domain in order to tackle nonlinear electro- magnetic inverse scattering with the benefit of sparsity. The soft-thresholding technique is applied here to sparsify the reconstruction. Numerical results show the potential of the proposed approach to improve the quality of reconstruction compared to the original CSI method.

We propose a Non-Intrusive (or reference-free) Audio Clarity index (NIAC), inspired from previous works on image sharpness and defined as the sensitivity of the spectrogram sparsity to a convolution of the audio signal with a white noise. A closed-form formula is provided, which only involves the signal itself and very little parameter setting. Tested in various noise and reverberation conditions, the NIAC exhibits a high correlation with the well-established Speech Transmission Index, both for speech and music. It can also be used as a clarity criterion to drive sound enhancement algorithms. We propose a NIAC-based source separation algorithm, and show that its performance is comparable to that of a state-of-the-art algorithm, FastICA.

Convolutional neural networks (CNN) have been hugely successful recently with superior accuracy and performance in various imaging applications, such as classification, object detection, and segmentation. However, a highly accurate CNN model requires millions of parameters to be trained and utilized. Even to increase its performance slightly would require significantly more parameters due to adding more layers and/or increasing the number of filters per layer. Apparently, many of these weight parameters turn out to be redundant and extraneous, so the original, dense model can be replaced by its compressed version attained by imposing inter- and intra-group sparsity onto the layer weights during training. In this paper, we propose a nonconvex family of sparse group lasso that blends nonconvex regularization (e.g., transformed ℓ1, ℓ1−ℓ2, and ℓ0) that induces sparsity onto the individual weights and ℓ2,1 regularization onto the output channels of a layer. We apply variable splitting onto the proposed regularization to develop an algorithm that consists of two steps per iteration: gradient descent and thresholding. Numerical experiments are demonstrated on various CNN architectures showcasing the effectiveness of the nonconvex family of sparse group lasso in network sparsification and test accuracy on par with the current state of the art.

Many studies have been conducted on recommender systems in both the academic and industrial fields, as they are currently broadly used in various digital platforms to make personalized suggestions. Despite the improvement in the accuracy of recommenders, the diversity of interest areas recommended to a user tends to be reduced, and the sparsity of explicit feedback from users has been an important issue for making progress in recommender systems. In this paper, we introduce a novel approach, namely re-enrichment learning, which effectively leverages the implicit logged feedback from users to enhance user retention in a platform by enriching their interest areas. The approach consists of (i) graph-based domain transfer and (ii) metadata saliency, which (i) find an adaptive and collaborative domain representing the relations among many users’ metadata and (ii) extract attentional features from a user’s implicit logged feedback, respectively. The experimental results show that our proposed approach has a better capacity to enrich the diversity of interests of a user by means of implicit feedback and to help recommender systems achieve more balanced personalization. Our approach, finally, helps recommenders improve user retention, i.e., encouraging users to click more items or dwell longer on the platform.

International audience; Convolutional neural networks (CNN) are known to learn an image representation that captures concepts relevant to the task, but do so in an implicit way that hampers model interpretability. However, one could argue that such a representation is hidden in the neurons and can be made explicit by teaching the model to recognize semantically interpretable attributes that are present in the scene. We call such an intermediate layer a \emph{semantic bottleneck}. Once the attributes are learned, they can be re-combined to reach the final decision and provide both an accurate prediction and an explicit reasoning behind the CNN decision. In this paper, we look into semantic bottlenecks that capture context: we want attributes to be in groups of a few meaningful elements and participate jointly to the final decision. We use a two-layer semantic bottleneck that gathers attributes into interpretable, sparse groups, allowing them contribute differently to the final output depending on the context. We test our contextual semantic interpretable bottleneck (CSIB) on the task of landscape scenicness estimation and train the semantic interpretable bottleneck using an auxiliary database (SUN Attributes). Our model yields in predictions as accurate as a non-interpretable baseline when applied to a real-world test set of Flickr images, all while providing clear and interpretable explanations for each prediction.

International audience; Safe screening rules are powerful tools to accelerate iterative solvers in sparse regression problems. They allow early identification of inactive coordinates (i.e., those not belonging to the support of the solution) which can thus be screened out in the course of iterations. In this paper, we extend the GAP Safe screening rule to the L1-regularized Kullback-Leibler divergence which does not fulfil the regularity assumptions made in previous works. The proposed approach is experimentally validated on synthetic and real count data sets.

In this paper, we consider an optimal control problem for the two-dimensional evolutionary Navier-Stokes system. Looking for sparsity, we take controls as functions of time taking values in a space of Borel measures. The cost functional does not involve directly the control but we assume some constraints on them. We prove the well-posedness of the control problem and derive necessary and sufficient conditions for local optimality of the controls. The first author was supported by Spanish Ministerio de Economía, Industria y Competitividad under research project MTM2017-83185-P. The second author was supported by the ERC advanced grant 668998 (OCLOC) under the EU's H2020 research program.

Recent advances in audio declipping have substantially improved the state of the art.% in certain saturation regimes. Yet, practitioners need guidelines to choose a method, and while existing benchmarks have been instrumental in advancing the field, larger-scale experiments are needed to guide such choices. First, we show that the clipping levels in existing small-scale benchmarks are moderate and call for benchmarks with more perceptually significant clipping levels. We then propose a general algorithmic framework for declipping that covers existing and new combinations of variants of state-of-the-art techniques exploiting time-frequency sparsity: synthesis vs. analysis sparsity, with plain or structured sparsity. Finally, we systematically compare these combinations and a selection of state-of-the-art methods. Using a large-scale numerical benchmark and a smaller scale formal listening test, we provide guidelines for various clipping levels, both for speech and various musical genres. The code is made publicly available for the purpose of reproducible research and benchmarking.

International audience; We propose original, accurate and computationally efficient procedures to calibrate fluorescence microscopes from micro-beads images. The designed algorithms present many singularities. First, they allow to estimate space-varying blurs, which is a critical feature for large fields of views. Second, we propose a novel approach for calibration: instead of describing an optical system through a single operator, we suggest to vary the imaging conditions (temperature, focus, active elements) to get indirect images of its different states. Our algorithms then allow to represent the microscope responses as a low-dimensional convex set of operators. This novel approach is shown to significantly improve the estimation on a wide-field microscope. It is deemed as an essential step towards the effective resolution of blind inverse problems. We illustrate the potential of the approach by designing an original procedure for blind image deblurring of point sources and show a massive improvement compared to commercial software.

If music is the language of the universe, musical note onsets may be the syllables for this language. Not only do note onsets define the temporal pattern of a musical piece, but their time-frequency characteristics also contain rich information about the identity of the musical instrument producing the notes. Note onset detection (NOD) is the basic component for many music information retrieval tasks and has attracted significant interest in audio signal processing research. In this paper, we propose an NOD method based on a novel feature coined as Normalized Identification of Note Onset based on Spectral Sparsity (NINOS2). The NINOS2 feature can be thought of as a spectral sparsity measure, aiming to exploit the difference in spectral sparsity between the different parts of a musical note. This spectral structure is revealed when focusing on low-magnitude spectral components that are traditionally filtered out when computing note onset features. We present an extensive set of NOD simulation results covering a wide range of instruments, playing styles, and mixing options. The proposed algorithm consistently outperforms the baseline Logarithmic Spectral Flux (LSF) feature for the most difficult group of instruments which are the sustained-strings instruments. It also shows better performance for challenging scenarios including polyphonic music and vibrato performances. ispartof: Eurasip Journal On Audio Speech And Music Processing vol:2021 issue:1 ispartof: location:Germany status: published

International audience; Over-parameterization of neural networks is a well known issue that comes along with their great performance. Among the many approaches proposed to tackle this problem, low-rank tensor decompositions are largely investigated to compress deep neural networks. Such techniques rely on a low-rank assumption of the layer weight tensors that does not always hold in practice. Following this observation, this paper studies sparsity inducing techniques to build new sparse matrix product layers for high-rate neural networks compression. Specifically, we explore recent advances in sparse optimization to replace each layer's weight matrix, either convolutional or fully connected, by a product of sparse matrices. Our experiments validate that our approach provides a better compression-accuracy trade-off than most popular low-rank-based compression techniques.

How to locate missing rods within a micro-structure composed of a grid-like, finite set of infinitely long circular cylindrical dielectric rods under the sub-wavelength condition is investigated. Sub-wavelength distances between adjacent rods and sub-wavelength rod diameters require super-resolution, beyond the Rayleigh criterion. Two different methods are proposed to achieve this: One builds upon the multiple scattering expansion method (MSM), and it enforces strong sparsity-prior information. The other is a data-driven method that combines convolutional neural networks (CNN) and recurrent neural networks (RNN), and it can be applied in effect with little knowledge of the wavefield interactions involved, in much contrast with the previous one. Comprehensive numerical simulations are proposed in terms of the missing rod number, shape, the frequency of observation, and the configuration of the tested structures. Both methods are shown to achieve suitable detection, yet under more or less stringent conditions as discussed.

Fluid simulation can be automatically interpolated by using data-driven fluid simulations based on a space-time deformation. In this paper, we propose a novel data-driven fluid simulation scheme with the L0 based optical flow deformation method by matching two fluid surfaces rather than the L2 regularization. The L0 gradient smooth regularization can result in prominent structure of the fluid in a sparsity-control manner, thus the misalignment of the deformation can be suppressed. We adopt the objective function using an alternating minimization with a half-quadratic splitting for solving the L0 based optical flow deformation model. Experiment results demonstrate that our proposed method can generate more realistic fluid surface with the optimal space-time deformation under the L0 gradient smooth constraint than the L2 one, and outperform the state-of-the-art methods in terms of both objective and subjective quality.

International audience; This paper proposes a novel approach called IDrISS that exploits electromagnetic (EM) side-channel signals to design non-protocol based Intrusion Detection System (IDS). EM emanations side-channels are captured on power lines of an infrastructure. They are used to identify the presence of any type of electronic devices onto a physical network. IDrISS can learn the structure of the EM unintentional emanations of the legit devices composing the infrastructure as a reference profile. In a second step, it records and analyses the current emanations to compare and detect any kind of unwanted emanations. IDrISS is used as a Intrusion Detection System (IDS) that can trig an alarm as soon as a intrusion is detected. The results show that intrusion can be detected in various scenarios whatever the activity of the legit computers of the network. Furthermore, the capture device used is based on inexpensive off-the-shelf components that makes the deployment onto real network easy. © 2021 European Signal Processing Conference, EUSIPCO. All rights reserved.

Channel estimation is a formidable challenge in mmWave Multiple Input Multiple Output (MIMO) systems due to the large number of antennas. Therefore, compressed sensing (CS) techniques are used to exploit channel sparsity at mmWave frequencies to calculate fewer dominant paths in mmWave channels. However, conventional CS techniques require a higher training overhead for efficient recovery. In this paper, an efficient extended alternation direction method of multipliers (Ex-ADMM) is proposed for mmWave channel estimation. In the proposed scheme, a joint optimization problem is formulated to exploit low rank and channel sparsity individually in the antenna domain. Moreover, a relaxation factor is introduced which improves the proposed algorithm&rsquo, s convergence. Simulation experiments illustrate that the proposed algorithm converges at lower Normalized Mean Squared Error (NMSE )with improved spectral efficiency. The proposed algorithm also ameliorates NMSE performance at low, mid and high Signal to Noise (SNR) ranges.

Neuromorphic systems are designed with careful consideration of the physical properties of the computational substrate they use. Neuromorphic engineers often exploit physical phenomena to directly implement a desired functionality, enabled by "the isomorphism between physical processes in different media" (Douglas et al., 1995). This bottom-up design methodology could be described as matching computational primitives to physical phenomena. In this paper, we propose a top-down counterpart to the bottom-up approach to neuromorphic design. Our top-down approach, termed "bias matching," is to match the inductive biases required in a learning system to the hardware constraints of its implementation; a well-known example is enforcing translation equivariance in a neural network by tying weights (replacing vector-matrix multiplications with convolutions), which reduces memory requirements. We give numerous examples from the literature and explain how they can be understood from this perspective. Furthermore, we propose novel network designs based on this approach in the context of collaborative filtering. Our simulation results underline our central conclusions: additional hardware constraints can improve the predictions of a Machine Learning system, and understanding the inductive biases that underlie these performance gains can be useful in finding applications for a given constraint.

International audience; In Nuclear Magnetic Resonance (NMR) spectroscopy, an efficient analysis and a relevant extraction of different molecule properties from a given chemical mixture are important tasks, especially when processing bidimensional NMR data. To that end, using a blind source separation approach based on a variational formulation seems to be a good strategy. However, the poor resolution of NMR spectra and their large dimension require a new and modern blind source separation method. In this work, we propose a new variational formulation for blind source separation (BSS) based on a β-divergence data fidelity term combined with sparsity promoting regularization functions. An application to 2D HSQC NMR experiments illustrates the interest and the effectiveness of the proposed method whether in simulated or real cases.

Premature ventricular contraction (PVC) can cause great harm to human health. Both invasive and non-invasive techniques for detecting electrical activity of PVC or locating ectopic pacemakers are used in clinical diagnosis. Among them, the electrocardiographic imaging is a popular method for non-invasive reconstruction of cardiac electrophysiology through body-surface potential (BSP). In this paper, we propose a novel framework based on low-rank and sparse decomposition (LSD) + total variation (TV) to solve the ill-posedness of the spatiotemporal ECG-inverse problem to reconstruct the cardiac electrical activity of PVC. The proposed framework considers the spatiotemporal distribution of multi-frame cardiac potential as a whole. The TV is used to filter out relatively smooth candidates from countless inverse solutions. In addition, LSD utilizes the low-rank characteristic of the potential background and the sparseness of the potential outliers to avoid the loss of potential details and improve the accuracy of potential reconstruction. This improves the quality of electrical activity retrieval and the accuracy of locating the PVC origin. The simulation experiments of ventricular pacing reconstruction and diagnostic experiments of real PVC patients prove that the proposed framework is superior to the conventional quadratic methods (Tikhonov-0 and Tikhonov-2) and the non-quadratic method TV.

Kernel principal component analysis (KPCA) has been widely used for nonlinear process monitoring. However, since the principal components are linear combinations of all kernel functions, traditional KPCA suffers from poor interpretation and high-computation cost. To address this problem, obtaining sparse coefficients in KPCA is of paramount importance, particularly for real-time process monitoring and large-scale processes. In this paper, a new sparse kernel principal component analysis via sequential approach, named SSKPCA is proposed for nonlinear process monitoring. We first incorporate elastic net regularization into the framework of KPCA to establish a modified optimization problem. Then, a sequential approach is employed to derive the solution. Different from the existing sparse KPCA method based on elastic net regularization, the extra computations associated with the kernel matrix, such as matrix inversion and matrix square root are avoided in the optimizing procedure for solving the modified optimization problem. Therefore, the proposed SSKPCA method is more efficient in numerical implementation. The SSKPCA-based T2 and squared prediction error (Q) statistics are constructed for fault detection. Furthermore, the sensitivity analysis principle is adopted for fault identification. A comparative study of Tennessee Eastman Process (TEP) is carried out to illustrate the ability and efficiency of the proposed SSKPCA-based nonlinear process monitoring method.

Unsupervised feature learning via auto-encoders results in low-dimensional representations in latent space that capture the patterns of input data. The auto-encoders with robust regularization learn qualified features that are less sensitive to small perturbations of inputs. However, the previous robust auto-encoders highly depend on pre-defined structure settings and often learn full-connected networks that are easily prone to over-fitting. To solve the above limitations, we propose in this paper an explicitly regularized framework which improves the sparsity and flexibility of robust auto-encoders. First, our model encourages the activation functions to automatically adjust themselves between linear and non-linear ones. Second, the mapping functions of the encoder are constrained by group sparsity and exclusive sparsity to reduce the redundancy of parameters. The proximal gradient method is used to optimize our model since the objective function contains non-smooth components. We conduct experiments in single-layer and multiple-layer auto-encoders in the classification task. The numerical results show that our model achieves better accuracy than baseline models. Our method also shows better performance in denoising task.

With the refinement of tasks in artificial intelligence, bringing in exponential level increments in computation cost and storage. Therefore, the augment of computation resource for complicated neural networks severely hinders their applications on limited-power devices in recent years. As a result, there is an impending necessity to compress and accelerate the deep networks by special ways. Considering the different peculiarities of weight quantization and sparse regularization, in this paper, we propose a low rank sparse quantization (LRSQ) method to quantize network weights and regularize the corresponding structures at the same time. Our LRSQ can: 1) obtain low-bit quantized networks to reduce memory and computation cost and 2) learn a compact structure from complex convolutional networks for subsequent channel pruning which has significant reduction on FLOPs. In experimental sections, we evaluate the proposed method on several popular models such as VGG-7/16/19 and ResNet-18/34/50, and results show that this method can dramatically reduce parameters and channels of the network with slight inference accuracy loss. Furthermore, we also visualize and analyze the four-dimensional weight tensors, which shows the low rank and group-sparsity structure of it. Finally, we try pruning unimportant channels which are zero-channels in our quantized model, and finding even a little better precision than the standard full-precision network.

In this paper, a novel interference management technique based on compressive sensing (CS) theory is investigated for downlink non-orthogonal multiple access (NOMA) heterogeneous networks (HetNets). We mathematically formulate the interference management problem in terms of power and resource blocks (RBs) allocation to maximize the overall sum rate while considering both co-tier and cross-tier interferences and then explain its non-convexity. In this paper, we exploit the sparsity of the allocated RBs to relax the non-convexity of the formulated problem by transforming it into a sparse $l_{1}$ -norm problem for a near-optimum solution. Then, based on the CS theory, an interference management technique with a restricted weighted fast iterative shrinkage-thresholding (R-WFISTA) algorithm is proposed to solve the equivalent sparse $l_{1}$ -norm problem. The simulation results verify that compared with the conventional orthogonal multiple access (OMA) HetNets and conventional NOMA HetNets, the proposed technique improves the system performance in terms of overall sum rate and the outage probability.

New approaches have been proposed to detect and exploit sparsity in adaptive systems. However, the sparsity is not always explicit among the system coefficients, thus requiring some tools to reveal it. By means of the so-called feature function, we propose the low-complexity feature stochastic gradient (LF-SG) algorithm to exploit hidden sparsity. The proposed algorithm aims at reducing the computational load of the learning process, as compared to the least-mean-square (LMS) algorithm. We focus on block-lowpass systems, but the proposed approach can easily be adapted to exploit other kinds of features of the unknown system, e.g., highpass and bandpass characteristics. Then, we analyze some properties of the LF-SG algorithm, namely its steady-state mean squared error (MSE), its bias, and the choice of the step-size parameter. Simulation results illustrate the competitive MSE performance of the LF-SG in comparison with the LMS, but the former algorithm requires much fewer multiplication operations to identify lowpass systems. For instance, to identify a measured room impulse response, the LF-SG algorithm realized less than half of the multiplication operations required by the LMS algorithm.

International audience; In this paper we develop several approaches to approximately solve the capacitated arc routing problem (CARP) on sparse graphs namely sparse CARP. First, we give a mathematical model for the sparse CARP and we present a brief survey about a transformation technique to transform the sparse CARP into a sparse capacitated vehicle routing problem namely sparse CVRP. Later, we propose several approaches to solve the sparse CARP by solving its equivalent obtained sparse CVRP. The first approach is a constructive heuristic (CH) used to construct an initial feasible solution. The second approach is an improving randomized procedure (IRP) used to improve the quality of the initial solution. Finally, we introduce the main adapted tabu search approach (TS) under a sparse dynamic graph. The algorithm starts by applying the first two procedures CH and IRP and attempts to compute a better solution for the sparse CARP. Extensive computational tests on randomly generated problem instances show the effectiveness of the proposed approach. The TS algorithm yields satisfactory results within reasonable computational time. The approach outperformed also the commercial solver Cplex v12.71 which was able to solve only small instances with relatively a big CPU time for medium size instances.