Search

As a novel similarity measure that is defined as the expectation of a kernel function between two random variables, correntropy has been successfully applied in robust machine learning and signal processing to combat large outliers. The kernel function in correntropy is usually a zero-mean Gaussian kernel. In a recent work, the concept of mixture correntropy (MC) was proposed to improve the learning performance, where the kernel function is a mixture Gaussian kernel, namely, a linear combination of several zero-mean Gaussian kernels with different widths. In both correntropy and MC, the center of the kernel function is, however, always located at zero. In the present work, to further improve the learning performance, we propose the concept of multikernel correntropy (MKC), in which each component of the mixture Gaussian kernel can be centered at a different location. The properties of the MKC are investigated and an efficient approach is proposed to determine the free parameters in MKC. Experimental results show that the learning algorithms under the maximum MKC criterion (MMKCC) can outperform those under the original maximum correntropy criterion (MCC) and the maximum MC criterion (MMCC). [ABSTRACT FROM AUTHOR]

In recent years, correntropy has been successfully applied to robust adaptive filtering to eliminate adverse effects of impulsive noises or outliers. Correntropy is generally defined as the expectation of a Gaussian kernel between two random variables. This definition is reasonable when the error between the two random variables is symmetrically distributed around zero. For the case of asymmetric error distribution, the symmetric Gaussian kernel is however inappropriate and cannot adapt to the error distribution well. To address this problem, in this brief we propose a new variant of correntropy, named asymmetric correntropy, which uses an asymmetric Gaussian model as the kernel function. In addition, a robust adaptive filtering algorithm based on asymmetric correntropy is developed and its steady-state convergence performance is analyzed. Simulations are provided to confirm the theoretical results and good performance of the proposed algorithm. [ABSTRACT FROM AUTHOR]

In recent years, the sparse system identification (SSI) has received increasing attention, and various sparsity-aware adaptive algorithms based on the minimum mean square error (MMSE) criterion have been developed, which are optimal under the assumption of Gaussian distributions. However, the Gaussian assumption does not always hold in real-world environments. The maximum correntropy criterion (MCC) is used to replace the MMSE criterion to suppress the heavy-tailed non-Gaussian noises. For some more complex non-Gaussian noises such as those from multimodal distributions, the minimum error entropy (MEE) criterion can outperform MCC although it is computationally somewhat more expensive. To improve the performance of SSI in non-Gaussian noises, in this brief we develop a class of sparsity-aware MEE algorithms with the fixed point iteration (MEE-FP) by incorporating the zero-attracting ($\ell _{1}$ -norm), reweighted zero-attracting (reweighted $\ell _{1}$ -norm) and correntropy induced metric (CIM) penalty terms into the cost function. The corresponding algorithms are termed as ZA-MEE-FP, RZA-MEE-FP, and CIM-MEE-FP, which can achieve better performance than the original MEE-FP algorithm and the MCC based sparsity-aware algorithms. Simulation results confirm the excellent performance of the new algorithms. [ABSTRACT FROM AUTHOR]

Recently, maximum correntropy criterion (MCC) has been widely and successfully used in robust signal processing and machine learning, in which the correntropy is maximized instead of minimizing the popular mean square error (MSE) to improve the robustness with respect to outliers or impulsive noises. A lot of efforts have been devoted to derive different adaptive algorithms under MCC, but to date, little insight has been gained as to how the MCC solution will be influenced by outliers. In this paper, we investigate this problem and our focus is mainly on the parameter estimation of a simple linear errors-in-variables (EIVs) model with scalar variables. Under some conditions, we derive an upper bound on the absolute value of the estimation error and show that the MCC solution can get very close to the true value of the unknown parameter even with arbitrarily large outliers in both the input and output variables. Illustrative examples are provided to verify and clarify the theory. [ABSTRACT FROM AUTHOR]

Multiview subspace clustering, which aims to cluster the given data points with information from multiple sources or features into their underlying subspaces, has a wide range of applications in the communities of data mining and pattern recognition. Compared with the single-view subspace clustering, it is challenging to efficiently learn the structure of the representation matrix from each view and make use of the extra information embedded in multiple views. To address the two problems, a novel correntropy-based multiview subspace clustering (CMVSC) method is proposed in this article. The objective function of our model mainly includes two parts. The first part utilizes the Frobenius norm to efficiently estimate the dense connections between the points lying in the same subspace instead of following the standard compressive sensing approach. In the second part, the correntropy-induced metric (CIM) is introduced to characterize the noise in each view and utilize the information embedded in different views from an information-theoretic perspective. Furthermore, an efficient iterative algorithm based on the half-quadratic technique (HQ) and the alternating direction method of multipliers (ADMM) is developed to optimize the proposed joint learning problem, and extensive experimental results on six real-world multiview benchmarks demonstrate that the proposed methods can outperform several state-of-the-art multiview subspace clustering methods. [ABSTRACT FROM AUTHOR]

The extended Kalman filter (EKF) is a method extensively applied in many areas, particularly, in nonlinear target tracking. The optimization criterion commonly used in EKF is the celebrated minimum mean square error (MMSE) criterion, which exhibits excellent performance under Gaussian noise assumption. However, its performance may degrade dramatically when the noises are heavy tailed. To cope with this problem, this paper proposes two new nonlinear filters, namely the linear regression maximum correntropy EKF (LRMCEKF) and nonlinear regression maximum correntropy EKF (NRMCEKF), by applying the maximum correntropy criterion (MCC) rather than the MMSE criterion to EKF. In both filters, a regression model is formulated, and a fixed-point iterative algorithm is utilized to obtain the posterior estimates. The effectiveness and robustness of the proposed algorithms in target tracking are confirmed by an illustrative example. [ABSTRACT FROM AUTHOR]

Robust point set registration is a challenging problem, especially in the cases of noise, outliers, and partial overlapping. Previous methods generally formulate their objective functions based on the mean-square error (MSE) loss and, hence, are only able to register point sets under predefined constraints (e.g., with Gaussian noise). This article proposes a novel objective function based on a bidirectional kernel mean ${p}$ -power error (KMPE) loss, to jointly deal with the above nonideal situations. KMPE is a nonsecond-order similarity measure in kernel space and shows a strong robustness against various noise and outliers. Moreover, a bidirectional measure is applied to judge the registration, which can avoid the ill-posed problem when a lot of points converges to the same point. In particular, we develop two effective optimization methods to deal with the point set registrations with the similarity and the affine transformations, respectively. The experimental results demonstrate the effectiveness of our methods. [ABSTRACT FROM AUTHOR]

Due to the effectiveness in learning the subspace structures, low-rank representation (LRR) and its variations have been widely applied in various fields, such as computer vision and pattern recognition. However, in real applications, it is a challenge to handle the complex noises. To address this problem, we propose a novel robust LRR method based on kernel risk-sensitive loss (KRSL) with high-order manifold constraint, called RHLRR, in which the KRSL is introduced to deal with the noises and the multiple hypergraph regularization term is used as a high order manifold constraint to effectively capture the locality, similarity and the intrinsic geometric information in data. Besides, an iterative algorithm based on the half-quadratic (HQ) and the accelerated block coordinate update (BCU) is developed. The experimental results demonstrate that the proposed method can outperform other state-of-the-art LRR variants. [ABSTRACT FROM AUTHOR]

This brief developed a novel sparsity constrained recursive adaptive filtering algorithm via the generalized correntropy criterion with variable center (GMCC-VC) for sparse system parameters estimation. In order to improve the performance of the adaptive filtering algorithm with GMCC under the non-Gaussian and non-zero mean noise environments, the GMCC is extended by adding a center which can be located at any position in the generalized Gaussian kernel function. Specifically, we proposed a novel adaptive online update method to optimize the kernel method and center. In addition, we mainly derived a recursive GMCC-VC algorithm based on the GMCC with the sparse constraint to exploit sparsity. Numerical simulation results are given to show that the proposed algorithm is efficient for sparse system parameter estimation problems in non-Gaussian and non-mean noise environments. [ABSTRACT FROM AUTHOR]

Robust matrix completion aims to recover a low-rank matrix from a subset of noisy entries perturbed by complex noises. Traditional matrix completion algorithms are always based on $l_2$ -norm minimization and are sensitive to non-Gaussian noise with outliers. In this paper, we propose a novel robust and fast matrix completion method based on the maximum correntropy criterion (MCC). The correntropy-based error measure is utilized instead of the $l_2$ -based error norm to improve robustness against noise. By using the half-quadratic optimization technique, the correntropy-based optimization can be transformed into a weighted matrix factorization problem. Two efficient algorithms are then derived: an alternating minimization-based algorithm and an alternating gradient descent-based algorithm. These algorithms do not require the singular value decomposition (SVD) to be calculated for each iteration. Furthermore, an adaptive kernel width selection strategy is proposed to accelerate the convergence speed as well as improve the performance. A comparison with existing robust matrix completion algorithms is provided by simulations and shows that the new methods can achieve better performance than the existing state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]

A blocked proportionate normalized maximum correntropy criterion (PNMCC) is presented to improve the estimation behavior of the traditional maximum correntropy criterion (MCC) algorithm for identifying the blocked sparse systems. The proposed blocked MCC is implemented by constructing a new cost function based on a hybrid-norm constraint (HNC) of the filter coefficient vector to adaptively utilize the cluster-sparse characteristic of unknown systems, denoting as hybrid-norm constrained PNMCC (HNC-PNMCC). The proposed HNC-PNMCC algorithm is achieved by using the basis pursuit. Various simulations are brought out to confirm the validity of the HNC-PNMCC. Simulation results indicate that the HNC-PNMCC is better than the PNMCC, MCC, and sparse MCC with respect to the estimation performance for the cluster-sparse system identification under the impulsive noises. [ABSTRACT FROM AUTHOR]

In order to efficiently utilize the information in the data and eliminate the negative effects of outliers in the principal component analysis (PCA) method, in this paper, we propose a novel robust sparse PCA method based on maximum correntropy criterion (MCC) with high-order manifold constraints called the RHSPCA. Compared with the traditional PCA methods, the proposed RHSPCA has the following benefits: 1) the MCC regression term is more robust to outliers than the MSE-based regression term; 2) thanks to the high-order manifold constraints, the low-dimensional representations can preserve the local relations of the data and greatly improve the clustering and classification performance for image processing tasks; and 3) in order to further counteract the adverse effects of outliers, the MCC-based samples’ mean is proposed to better centralize the data. We also propose a new solver based on the half-quadratic technique and accelerated block coordinate update strategy to solve the RHSPCA model. Extensive experimental results show that the proposed method can outperform the state-of-the-art robust PCA methods on a variety of image processing tasks, including reconstruction, clustering, and classification, on outliers contaminated datasets. [ABSTRACT FROM AUTHOR]

High-dimensional data contain not only redundancy but also noises produced by the sensors. These noises are usually non-Gaussian distributed. The metrics based on Euclidean distance are not suitable for these situations in general. In order to select the useful features and combat the adverse effects of the noises simultaneously, a robust sparse subspace learning method in unsupervised scenario is proposed in this paper based on the maximum correntropy criterion that shows strong robustness against outliers. Furthermore, an iterative strategy based on half quadratic and an accelerated block coordinate update is proposed. The convergence analysis of the proposed method is also carried out to ensure the convergence to a reliable solution. Extensive experiments are conducted on real-world data sets to show that the new method can filter out the outliers and outperform several state-of-the-art unsupervised feature selection methods. [ABSTRACT FROM AUTHOR]

Kernel adaptive filtering is implemented by evaluating the inner product between the kernel function-based vector and the coefficient vector. In this brief, the coefficient vector is decomposed into the direction vector and the scale, which are updated using the steepest descent method and thus generate a novel online learning method, namely kernel online learning algorithm with scale adaptation (KOL-SA). In addition, the convergence of KOL-SA is proved and an upper bound of steady-state mean square error is therefore obtained. Simulation results confirm that the proposed KOL-SA achieves desirable filtering performance from the aspects of the filtering accuracy and stability. [ABSTRACT FROM AUTHOR]

Correntropy is a second order statistical measure in kernel space, which has been successfully applied in robust learning and signal processing. In this paper, we define a nonsecond order statistical measure in kernel space, called the kernel mean- ${p}$ power error (KMPE), including the correntropic loss (C-Loss) as a special case. Some basic properties of KMPE are presented. In particular, we apply the KMPE to extreme learning machine (ELM) and principal component analysis (PCA), and develop two robust learning algorithms, namely ELM-KMPE and PCA-KMPE. Experimental results on synthetic and benchmark data show that the developed algorithms can achieve better performance when compared with some existing methods. [ABSTRACT FROM AUTHOR]

In hyperspectral images, some spectral bands suffer from low signal-to-noise ratio due to noisy acquisition and atmospheric effects, thus requiring robust techniques for the unmixing problem. This paper presents a robust supervised spectral unmixing approach for hyperspectral images. The robustness is achieved by writing the unmixing problem as the maximization of the correntropy criterion subject to the most commonly used constraints. Two unmixing problems are derived: the first problem considers the fully constrained unmixing, with both the nonnegativity and sum-to-one constraints, while the second one deals with the nonnegativity and the sparsity promoting of the abundances. The corresponding optimization problems are solved using an alternating direction method of multipliers (ADMM) approach. Experiments on synthetic and real hyperspectral images validate the performance of the proposed algorithms for different scenarios, demonstrating that the correntropy-based unmixing with ADMM is particularly robust against highly noisy outlier bands. [ABSTRACT FROM PUBLISHER]

Nonlinear similarity measures defined in kernel space, such as correntropy, can extract higher order statistics of data and offer potentially significant performance improvement over their linear counterparts especially in non Gaussian signal processing and machine learning. In this paper, we propose a new similarity measure in kernel space, called the kernel risk-sensitive loss (KRSL), and provide some important properties. We apply the KRSL to adaptive filtering and investigate the robustness, and then develop the MKRSL algorithm and analyze the mean square convergence performance. Compared with correntropy, the KRSL can offer a more efficient performance surface, thereby enabling a gradient-based method to achieve faster convergence speed and higher accuracy while still maintaining the robustness to outliers. Theoretical analysis results and superior performance of the new algorithm are confirmed by simulation. [ABSTRACT FROM PUBLISHER]

As a robust nonlinear similarity measure in kernel space, correntropy has received increasing attention in domains of machine learning and signal processing. In particular, the maximum correntropy criterion (MCC) has recently been successfully applied in robust regression and filtering. The default kernel function in correntropy is the Gaussian kernel, which is, of course, not always the best choice. In this paper, we propose a generalized correntropy that adopts the generalized Gaussian density (GGD) function as the kernel, and present some important properties. We further propose the generalized maximum correntropy criterion (GMCC) and apply it to adaptive filtering. An adaptive algorithm, called the GMCC algorithm, is derived, and the stability problem and steady-state performance are studied. We show that the proposed algorithm is very stable and can achieve zero probability of divergence (POD). Simulation results confirm the theoretical expectations and demonstrate the desirable performance of the new algorithm. [ABSTRACT FROM AUTHOR]

Compared with the least-mean-square algorithm, the least mean $p$th power algorithm shows a better robustness performance against impulsive noises such as the $\alpha$-stable noises. However, it still exhibits slow convergence rate and high kernel misadjustment. To overcome this drawback, a novel recursive logarithmic least mean $p$th (RLLMP) algorithm is proposed for the Volterra system identification under $\alpha$-stable noise environments. Instead of minimizing the $p$th power, the new algorithm aims to minimize the $p$th logarithmic cost, which makes it more robust against impulsive interferences. Furthermore, to enhance tracking performance, an improved variable forgetting factor (IVFF) algorithm (IVFF-RLLMP) is proposed, which is based on the robust estimation of outliers. Simulation results are presented to demonstrate the improved performance of the RLLMP and IVFF-RLLMP. [ABSTRACT FROM AUTHOR]

Recently, the information potential (IP) of order \alpha, defined as the argument of the log in the \alpha-order Renyi entropy, has been successfully used as an information theoretic criterion for supervised adaptive system training. In this paper, we use the survival function (or equivalently the distribution function) of an absolute value transformed random variable to define a new information potential, named the survival information potential (SIP). Compared with the IP, the SIP has some advantages, such as validity in a wide range of distributions, robustness, and the simplicity in computation. The properties of SIP and a simple formula for computing the empirical SIP are given in the paper. Finally, the SIP criterion is applied in adaptive system training, and simulation examples on FIR adaptive filtering, kernel adaptive filtering, and time delay neural networks (TDNNs) training are presented to demonstrate the performance. [ABSTRACT FROM PUBLISHER]