Search

Constrained adaptive filtering algorithms have been extensively studied in many applications. Most existing constrained adaptive filtering algorithms are developed under the mean square error (MSE) criterion, which is an ideal optimality criterion under Gaussian noises. This assumption however fails to model the behavior of non-Gaussian noises found in practice. Motivated by the robustness and simplicity of maximum correntropy criterion (MCC) for non-Gaussian impulsive noises, this paper proposes a new adaptive filtering algorithm called constrained maximum correntropy criterion (CMCC). Specifically, CMCC incorporates a linear constraint into a MCC filter to solve a constrained optimization problem explicitly. The proposed adaptive filtering algorithm is easy to implement, has low computational complexity, and can significantly outperform those MSE based constrained adaptive algorithms in heavy-tailed impulsive noises. Additionally, the mean square convergence behaviors are studied under energy conservation relation, and a sufficient condition to ensure the mean square convergence and the steady-state mean square deviation (MSD) of the CMCC algorithm are obtained. Simulation results confirm the theoretical predictions under both Gaussian and non-Gaussian noises, and demonstrate the excellent performance of the novel algorithm by comparing it with other conventional methods. [ABSTRACT FROM AUTHOR]

Abstract: This paper presents a quantized kernel least mean square algorithm with a fixed memory budget, named QKLMS-FB. In order to deal with the growing support inherent in online kernel methods, the proposed algorithm utilizes a pruning criterion, called significance measure, based on a weighted contribution of the existing data centers. The basic idea of the proposed methodology is to discard the center with the smallest influence on the whole system, when a new sample is included in the dictionary. The significance measure can be updated recursively at each step which is suitable for online operation. Furthermore, the proposed methodology does not need any a priori knowledge about the data and its computational complexity is linear with the center number. Experiments show that the proposed algorithm successfully prunes the least “significant” centers and preserves the important ones, resulting in a compact KLMS model with little loss in accuracy. [Copyright &y& Elsevier]

Abstract: In this paper, we study the mean square convergence of the kernel least mean square (KLMS). The fundamental energy conservation relation has been established in feature space. Starting from the energy conservation relation, we carry out the mean square convergence analysis and obtain several important theoretical results, including an upper bound on step size that guarantees the mean square convergence, the theoretical steady-state excess mean square error (EMSE), an optimal step size for the fastest convergence, and an optimal kernel size for the fastest initial convergence. Monte Carlo simulation results agree with the theoretical analysis very well. [Copyright &y& Elsevier]

Abstract: The minimum mean-square error (MMSE) and minimum error entropy (MEE) are two important criteria in the estimation related problems. The MMSE can be viewed as a robust MEE criterion in the minimax sense, as its minimization is equivalent to minimizing an upper bound (the maximum value) of the error entropy. This note gives a new and more meaningful interpretation on the robustness of MMSE for problems in which there exists uncertainty in the probability model. It is shown that the MMSE estimator imposes an upper bound on error entropy for the true model. The upper bound consists of two terms. The first term quantifies the “MMSE performance” under nominal conditions, and the second term measures the “distance” between the true and nominal models. This robustness property is parallel to that of the risk-sensitive estimation. Illustration examples are included to confirm the robustness of MMSE. [Copyright &y& Elsevier]

Recently, the Kalman filter algorithm based on the minimum error entropy criterion has been presented to outperform the conventional Kalman filter in the case of impulsive noise. In practice, it may be unstable in numerical calculation. This paper proposes a robust algorithm to solve the problem of instability in numerical calculation. The convergence and stability of the algorithm are verified by the performance analysis and simulations. In the mean square behavior analysis, we propose a method to estimate the steady-state errors. Simulations show that the experimental steady-state errors of the algorithms agreed with the theoretical values. [ABSTRACT FROM AUTHOR]

This paper presents a new normalized least mean square (NLMS) algorithm for sparse system identification where the input signal is corrupted by white measurement noise. The proposed algorithm, which is called bias-compensated zero attracting NLMS (BC-ZA-NLMS) algorithm, introduces the bias-compensation vector to get rid of the bias resulting from noisy input and introduces an l 1 -norm penalty in the cost function of the NLMS algorithm to make full use of the special property of the sparse system. In addition, to address the time variant sparsity, the bias-compensated reweight ZA-NLMS (BC-RZA-NLMS)) algorithm is also proposed, where the l 1 -norm penalty in the cost function of BC-ZA-NLMS algorithm is replaced by a log-sum function. Owing to the zero attractors in update equation, the proposed algorithms are superior to the conventional NLMS and bias-compensated NLMS (BC-NLMS) algorithms in the application of identifying the sparse system. A transient analysis of the proposed algorithms is also derived, which is able to accurately predict the behaviors of proposed algorithms. In addition, a stability analysis is introduced. Monte Carlo (MC) simulations are conducted to demonstrate the advantage of the proposed algorithms and to validate the theoretical results. [ABSTRACT FROM AUTHOR]

Total least squares (TLS) method has been widely used in errors-in-variables (EIV) modeling tasks in which both input and output data are disturbed by noises, and adaptive filtering algorithm using TLS has shown significantly superior performance to classical least squares (LS) method in EIV system. The TLS essentially extends the minimum mean square error (MMSE) criterion to EIV model, which, however, may work poorly when noise is non-Gaussian (especially heavy-tailed distribution). Recently, an information theoretic learning (ITL) based minimum total error entropy (MTEE) adaptive filtering algorithm has been proposed, which extends the minimum error entropy (MEE) criterion to EIV model and shows desirable performance in non-Gaussian noise environments. However, due to complex mathematical expression, MTEE is computationally expensive and difficult to carry out the theoretical analysis. In this paper, we propose a new ITL-based criterion called maximum total correntropy (MTC) and develop a gradient-based MTC adaptive filtering algorithm. We analyze theoretically the local stability and steady-state performance of the proposed algorithm. Simulation results confirm the theoretical analysis and show the superior performance of MTC in heavy-tailed noises. Further, simulation comparisons between MTC and MTEE are presented. Compared with the MTEE, the MTC is mathematically more tractable and computationally much simpler while achieving similar or even better performance. [ABSTRACT FROM AUTHOR]

Recently, the sign subband adaptive filter (SSAF) algorithm has obtained great attention, due to its robustness against impulsive noises and decorrelating property for correlated input signals. However, the performance of the algorithm in the steady-state is not analyzed. In this paper, we study the steady-state mean-square-deviation (MSD) behavior of the SSAF algorithm by using energy conservation relation, Price׳s theorem and some reasonable assumptions. Simulation results in different system identification scenarios (including the input signals, tap lengths, impulsive noises, number of subbands, and step sizes) are provided to support our theoretical analysis. [ABSTRACT FROM AUTHOR]

Recently, the kernel adaptive filtering algorithm with the generalized maximum correntropy criterion (GMCC) has gained attention due to its excellent nonlinear system modeling ability and robustness to non-Gaussian noise. However, since the default center of the GMCC is located at zero, it hampers its performance in the non-zero mean noise environment. To overcome the problem, this paper proposes a kernel adaptive filtering algorithm based on the GMCC with variable center (GMCC-VC) called generalized kernel maximum correntropy criterion with variable center (GKMCC-VC). In addition, the online vector quantization (VQ) method is introduced to the GKMCC-VC algorithm for suppressing the increasing network size. The probability of divergence, the mean-square steady-state behavior, and the convergence condition of the GKMCC-VC algorithm are also obtained under some assumptions. The potential relationship between the GMCC-VC and the generalized minimum error entropy (GMEE) criteria is discussed in the paper. Finally, the simulation results verify the superiority of the proposed algorithm in modeling nonlinear systems and the correctness of the theoretical model. • This paper proposes the GKMCC-VC algorithm for non-zero mean and non-Gaussian noise. • The steady-state performance of the proposed algorithm is analyzed. • Simulation results show the effectiveness and feasibility of the proposed algorithm. [ABSTRACT FROM AUTHOR]

• A variant of the quantized minimum error entropy (QMEE) algorithm is proposed. • The mean square convergence performance analysis for QMEE algorithm is given. • The theoretical values of the steady-state EMSE for QMEE algorithm are validated by simulations. • QMEE algorithm is applied in electroencephalo (EEG) de-noising task. Adaptive filtering algorithms have been widely used in many areas, among which the minimum error entropy (MEE) algorithm is a superior choice, due to its excellent performance in the non-Gaussian noise situations. However, the computational complexity of the MEE algorithm is expensive, which leads to the computational bottlenecks, especially for large-scale datasets. In order to address the problem, we propose an adaptive filtering algorithm based on the quantized minimum error entropy (QMEE) criterion with an online quantization method, named QMEE algorithm. Moreover, we analyze the transient behavior characteristic and derive an approximate analytical expression for the steady-state excess mean square error (EMSE) based on the Taylor expansion. The extensive simulation results in linear modeling and electroencephalogram (EEG) denoising task demonstrate that the proposed method can outperform other robust adaptive filtering algorithms. [ABSTRACT FROM AUTHOR]

• The kernel adaptive filter with generalized tanh function (KAF-GT) is proposed, which shows favourable performance in combating impulsive noise. • The convergence analysis of the proposed KAF-GT algorithm is conducted using energy conservation relation. • By using the pre-tuned dictionary strategy to model the unknown nonlinear system, we perform the EMSE analysis and arrive at a closed-form solution for predicting the steady-state behavior. • To further improve the performance of KAF-GT, an optimization scheme for scale factor is designed. Recently, online learning algorithms in machine learning have been imposed much attention. As a typical family, kernel adaptive filtering algorithms receive particular interest due to their sequential learning-based features. However, the kernel least mean square (KLMS) algorithm is not suitable for nonlinear tasks corrupted by non-Gaussian noise, especially impulsive noise. This is because the derivation of the KLMS algorithm is on the basis of the mean square error (MSE) criterion which only captures information of second-order statistics. In this paper, motivated by tanh function, we develop its generalized variant by introducing a scale factor for better representation capability; then we incorporate kernel adaptive filter with the generalized tanh function to propose a robust sequential learning algorithm. Based on establishing the energy conservation relation, we derive a sufficient condition for ensuring the algorithm convergence. In addition, to perform the steady-state excess mean square error (EMSE) analysis, we use the pre-tuned dictionary strategy to model the unknown nonlinear system in form of a finite-order combination; by Taylor expansion, we arrive at a closed-form solution for predicting the steady-state behavior. To further improve the algorithm performance, we design an optimization scheme for scale factor. Simulations for nonlinear time series prediction show that the designed schemes yield better performance than some state-of-art algorithms. The steady-state EMSE analysis is validated to provide accurate prediction results. [ABSTRACT FROM AUTHOR]

Highlights • We propose a bias-compensated normalized MCC algorithm which is efficient for the noisy input and impulsive output noise cases. • We use the unbiasedness criterion to derive the bias compensated term for the proposed algorithm. • The proposed BCNMCC algorithm performs better than the original NMCC algorithm in noisy input environment. Abstract This paper proposes a bias-compensated normalized maximum correntropy criterion (BCNMCC) algorithm charactered by its low steady-state misalignment for system identification with noisy input in an impulsive output noise environment. The normalized maximum correntropy criterion (NMCC) is derived from a correntropy based cost function, which is rather robust with respect to impulsive noises. To deal with the noisy input, we introduce a bias-compensated vector to the NMCC algorithm, and then an unbiasedness criterion and some reasonable assumptions are used to compute the bias-compensated vector. Taking advantage of the bias-compensated vector, the bias caused by the input noise can be effectively suppressed. System identification simulation results demonstrate that the proposed BCNMCC algorithm can outperform other related algorithms with noisy input especially in an impulsive output noise environment. [ABSTRACT FROM AUTHOR]

The adaptive variable step-size technique is an effective way to improve the convergence rate of an adaptive filtering algorithm. However, for most existing variable step-size sign algorithm against impulsive noise, either the performance is not good enough, or the calculation is complex. In this paper, a correntropy inspired variable step-size method for sign algorithm is proposed. The new variable step-size method is computationally simple and robust to impulsive noises. Theoretical analysis and simulation results demonstrate that the proposed algorithm can achieve desirable performance with low computational complexity in presence of impulsive noises. [ABSTRACT FROM AUTHOR]

For high background noise environment, an improved affine projection subband adaptive filter (IAPSAF) is proposed, which is derived by minimizing the sum of the squared Euclidean norms of the differences between the updated tap-weight vector and past tap-weight vectors. Due to the use of the past tap-weight vectors, the IAPSAF can achieve lower steady-state misalignment relative to the improved multiband-structured subband adaptive filter (IMSAF). Furthermore, a combined step size IAPSAF (CSS-IAPSAF) is developed to improve the performance of the IAPSAF. Simulation results in an acoustic echo cancellation (AEC) context show that the IAPSAF achieves lower steady-state misalignment than the existing algorithms, such as normalized subband adaptive filter (NSAF), IMSAF, and improved NSAF (INSAF). The improvement of the CSS-IAPSAF is also demonstrated in terms of convergence rate and steady-state misalignment. [ABSTRACT FROM AUTHOR]

The state space recursive least squares (SSRLS) filter is a new addition to the well-known recursive least squares (RLS) family filters, which can achieve an excellent tracking performance by overcoming some limitations of the standard RLS algorithm. However, when the underlying system is disturbed by some heavy-tailed non-Gaussian impulsive noises, the performance of SSRLS will deteriorate significantly. The main reason for this is that the SSRLS is derived under the minimum mean square error (MMSE) criterion, which is not well-suited to estimation problems under non-Gaussian noises. To overcome this issue, we propose in this paper a novel linear filter, called the state space maximum correntropy (SSMC) filter, which is derived under the maximum correntropy criterion (MCC) instead of the MMSE. Since MCC is very suited to non-Gaussian signal processing, the SSMC performs very well in non-Gaussian noises especially when the signals are corrupted by impulsive noises. A simple illustrative example is presented to demonstrate the desirable performance of the new algorithm. [ABSTRACT FROM AUTHOR]

In this brief, a robust and sparse recursive adaptive filtering algorithm, called convex regularized recursive maximum correntropy (CR-RMC), is derived by adding a general convex regularization penalty term to the maximum correntropy criterion (MCC). An approximate expression for automatically selecting the regularization parameter is also introduced. Simulation results show that the CR-RMC can significantly outperform the original recursive maximum correntropy (RMC) algorithm especially when the underlying system is very sparse. Compared with the convex regularized recursive least squares (CR-RLS) algorithm, the new algorithm also shows strong robustness against impulsive noise. The CR-RMC also performs much better than other LMS-type sparse adaptive filtering algorithms based on MCC. [ABSTRACT FROM AUTHOR]

This paper addresses the problem of robust target localization in distributed multiple-input multiple-output (MIMO) radar using possibly outlier range measurements. To achieve robustness against outliers, we construct an objective function for MIMO target localization via the maximum correntropy criterion. To deal with such a nonconvex and nonlinear function, we apply a half-quadratic optimization technique to determine the target position and auxiliary variables alternately. Especially, we derive a semidefinite relaxation formulation for the aforementioned position determination step. The robust performance of the developed approach is demonstrated by comparing with several conventional localization methods via computer simulation. [ABSTRACT FROM AUTHOR]

Proposed is a new subband adaptive filter (SAF) algorithm by minimizing Huber’s cost function that is robust to impulsive noises. Generally, this algorithm works in the mode of the normalized SAF (NSAF) algorithm, while it behaves like the sign SAF (SSAF) algorithm only when the impulsive noises appear. To further improve the robustness of this algorithm against impulsive noises, the subband cutoff parameters are updated in a recursive way. Moreover, the proposed algorithm can be interpreted as a variable step size NSAF algorithm, thus it exhibits faster convergence rate and lower steady-state error than the NSAF. Simulation results, using different colored input signals in both impulsive and free-impulsive noise environments, show that the proposed algorithm works better than some existing algorithms. [ABSTRACT FROM AUTHOR]

In this letter, a robust kernel adaptive algorithm, called the kernel recursive maximum correntropy (KRMC), is derived in kernel space and under the maximum correntropy criterion (MCC). The proposed algorithm is particularly useful for nonlinear and non-Gaussian signal processing, especially when data contain large outliers or disturbed by impulsive noises. The superior performance of KRMC is confirmed by simulation results about short-term chaotic time series prediction in alpha-stable noise environments. [ABSTRACT FROM AUTHOR]

Active noise control (ANC) is an effective way for reducing the noise level in electroacoustic or electromechanical systems. Since its first introduction in 1936, this approach has been greatly developed. This paper focuses on discussing the development of ANC techniques over the past decade. Linear ANC algorithms, including the celebrated filtered-x least-mean-square (FxLMS)-based algorithms and distributed ANC algorithms, are investigated and evaluated. Nonlinear ANC (NLANC) techniques, such as functional link artificial neural network (FLANN)-based algorithms, are pursued in Part II. Furthermore, some novel methods and applications of ANC emerging in the past decade are summarized. Finally, future research challenges regarding the ANC technique are discussed. [ABSTRACT FROM AUTHOR]

Part I of this paper reviewed the development of the linear active noise control (ANC) technique in the past decade. However, ANC systems might have to deal with some nonlinear components and the performance of linear ANC techniques may degrade in this scenario. To overcome this limitation, nonlinear ANC (NLANC) algorithms were developed. In Part II, we review the development of NLANC algorithms during the last decade. The contributions of heuristic ANC algorithms are outlined. Moreover, we emphasize recent advances of NLANC algorithms, such as spline ANC algorithms, kernel adaptive filters, and nonlinear distributed ANC algorithms. Then, we present recent applications of ANC technique including linear and nonlinear perspectives. Future research challenges regarding ANC techniques are also discussed. [ABSTRACT FROM AUTHOR]

• A novel diffusion adaptation framework and two simple yet efficient sparsity adaptive diffusion algorithms are proposed for CS reconstruction. • Novel analyses on convergence conditions and improvement of the step size upper bound are proposed. • Experimental results confirm the theoretical analysis, and show that the proposed diffusion algorithms can achieve desirable reconstruction accuracy as well as much faster convergence speed compared with stand-alone l 0 -LMS algorithm. Compressive sensing (CS) has drawn much attention in recent years due to its low sampling rate as well as high recovery accuracy. As an important procedure, reconstructing a sparse signal from a few measurement data has been intensively studied. In this paper, we propose a novel diffusion adaptation framework for CS reconstruction, where the reconstruction is performed in a distributed network. The data of the measurement matrix are partitioned into small parts and are stored in each node, which assigns the storage load in a decentralized manner. Information diffusion provides the reconstruction ability of each node. Then, a simple and efficient sparsity adaptive diffusion algorithm has been proposed to collaboratively recover the sparse signal over the network. The convergence of the proposed algorithm is analyzed. To further increase the convergence speed, a novel mini-batch based diffusion algorithm is also proposed. Simulation results show that the proposed framework and algorithms can achieve good reconstruction accuracy as well as a much faster convergence speed compared with stand-alone l 0 -LMS algorithm for CS. [ABSTRACT FROM AUTHOR]