Search

To date, most linear and nonlinear Kalman filters (KFs) have been developed under the Gaussian assumption and the well-known minimum mean square error (MMSE) criterion. In order to improve the robustness with respect to impulsive (or heavy-tailed) non-Gaussian noises, the maximum correntropy criterion (MCC) has recently been used to replace the MMSE criterion in developing several robust Kalman-type filters. To deal with more complicated non-Gaussian noises such as noises from multimodal distributions, in this article, we develop a new Kalman-type filter, called minimum error entropy KF (MEE-KF), by using the minimum error entropy (MEE) criterion instead of the MMSE or MCC. Similar to the MCC-based KFs, the proposed filter is also an online algorithm with the recursive process, in which the propagation equations are used to give prior estimates of the state and covariance matrix, and a fixed-point algorithm is used to update the posterior estimates. In addition, the MEE extended KF (MEE-EKF) is also developed for performance improvement in the nonlinear situations. The high accuracy and strong robustness of MEE-KF and MEE-EKF are confirmed by experimental results. [ABSTRACT FROM AUTHOR]

Traditional Kalman filter (KF) is derived under the well-known minimum mean square error (MMSE) criterion, which is optimal under Gaussian assumption. However, when the signals are non-Gaussian, especially when the system is disturbed by some heavy-tailed impulsive noises, the performance of KF will deteriorate seriously. To improve the robustness of KF against impulsive noises, we propose in this work a new Kalman filter, called the maximum correntropy Kalman filter (MCKF), which adopts the robust maximum correntropy criterion (MCC) as the optimality criterion, instead of using the MMSE. Similar to the traditional KF, the state mean vector and covariance matrix propagation equations are used to give prior estimations of the state and covariance matrix in MCKF. A novel fixed-point algorithm is then used to update the posterior estimations. A sufficient condition that guarantees the convergence of the fixed-point algorithm is also given. Illustration examples are presented to demonstrate the effectiveness and robustness of the new algorithm. [ABSTRACT FROM AUTHOR]

Kernel adaptive filters (KAF) are a class of powerful nonlinear filters developed in Reproducing Kernel Hilbert Space (RKHS). The Gaussian kernel is usually the default kernel in KAF algorithms, but selecting the proper kernel size (bandwidth) is still an open important issue especially for learning with small sample sizes. In previous research, the kernel size was set manually or estimated in advance by Silverman׳s rule based on the sample distribution. This study aims to develop an online technique for optimizing the kernel size of the kernel least mean square (KLMS) algorithm. A sequential optimization strategy is proposed, and a new algorithm is developed, in which the filter weights and the kernel size are both sequentially updated by stochastic gradient algorithms that minimize the mean square error (MSE). Theoretical results on convergence are also presented. The excellent performance of the new algorithm is confirmed by simulations on static function estimation, short term chaotic time series prediction and real world Internet traffic prediction. [ABSTRACT FROM AUTHOR]

Abstract: This paper presents a quantized kernel least mean square algorithm with a fixed memory budget, named QKLMS-FB. In order to deal with the growing support inherent in online kernel methods, the proposed algorithm utilizes a pruning criterion, called significance measure, based on a weighted contribution of the existing data centers. The basic idea of the proposed methodology is to discard the center with the smallest influence on the whole system, when a new sample is included in the dictionary. The significance measure can be updated recursively at each step which is suitable for online operation. Furthermore, the proposed methodology does not need any a priori knowledge about the data and its computational complexity is linear with the center number. Experiments show that the proposed algorithm successfully prunes the least “significant” centers and preserves the important ones, resulting in a compact KLMS model with little loss in accuracy. [Copyright &y& Elsevier]

Abstract: In this paper, we study the mean square convergence of the kernel least mean square (KLMS). The fundamental energy conservation relation has been established in feature space. Starting from the energy conservation relation, we carry out the mean square convergence analysis and obtain several important theoretical results, including an upper bound on step size that guarantees the mean square convergence, the theoretical steady-state excess mean square error (EMSE), an optimal step size for the fastest convergence, and an optimal kernel size for the fastest initial convergence. Monte Carlo simulation results agree with the theoretical analysis very well. [Copyright &y& Elsevier]

Recently, the information potential (IP) of order \alpha, defined as the argument of the log in the \alpha-order Renyi entropy, has been successfully used as an information theoretic criterion for supervised adaptive system training. In this paper, we use the survival function (or equivalently the distribution function) of an absolute value transformed random variable to define a new information potential, named the survival information potential (SIP). Compared with the IP, the SIP has some advantages, such as validity in a wide range of distributions, robustness, and the simplicity in computation. The properties of SIP and a simple formula for computing the empirical SIP are given in the paper. Finally, the SIP criterion is applied in adaptive system training, and simulation examples on FIR adaptive filtering, kernel adaptive filtering, and time delay neural networks (TDNNs) training are presented to demonstrate the performance. [ABSTRACT FROM PUBLISHER]

The normalized subband adaptive filtering algorithm provides fast convergence rate for colored input signals as compared to the normalized least mean square algorithm, but it suffers from a poor convergence issue in the α -stable noise. In light of this, the normalized subband p-norm (NSPN) algorithm, which is based on the least mean p-power error (MPE) criterion, is proposed in this study. This technique is not only robust against impulsive noise samples, but it also maintains a fast convergence rate when colored input signals are used. In addition to this, we develop both the steady-state and the transient models of the NSPN algorithm and provide some insights. Then, in order to solve the problem of making a choice regarding the order p in the NSPN algorithm, we design an autonomous system and come up with the NSPN algorithm with a variable p-norm (VP-NSPN). In addition, we offer the TFC-based VP-NSPN algorithm with a fast convergence rate and low steady-state misadjustment simultaneously by making use of the tap-weights feedback-based convex combination (TFC) scheme. In conclusion, simulation results on system identification and acoustic echo cancellation are undertaken in order to validate the superiority of the proposed algorithms and validate the usefulness of the theoretical analysis. [ABSTRACT FROM AUTHOR]

Total least squares (TLS) method has been widely used in errors-in-variables (EIV) modeling tasks in which both input and output data are disturbed by noises, and adaptive filtering algorithm using TLS has shown significantly superior performance to classical least squares (LS) method in EIV system. The TLS essentially extends the minimum mean square error (MMSE) criterion to EIV model, which, however, may work poorly when noise is non-Gaussian (especially heavy-tailed distribution). Recently, an information theoretic learning (ITL) based minimum total error entropy (MTEE) adaptive filtering algorithm has been proposed, which extends the minimum error entropy (MEE) criterion to EIV model and shows desirable performance in non-Gaussian noise environments. However, due to complex mathematical expression, MTEE is computationally expensive and difficult to carry out the theoretical analysis. In this paper, we propose a new ITL-based criterion called maximum total correntropy (MTC) and develop a gradient-based MTC adaptive filtering algorithm. We analyze theoretically the local stability and steady-state performance of the proposed algorithm. Simulation results confirm the theoretical analysis and show the superior performance of MTC in heavy-tailed noises. Further, simulation comparisons between MTC and MTEE are presented. Compared with the MTEE, the MTC is mathematically more tractable and computationally much simpler while achieving similar or even better performance. [ABSTRACT FROM AUTHOR]

Recently, the sign subband adaptive filter (SSAF) algorithm has obtained great attention, due to its robustness against impulsive noises and decorrelating property for correlated input signals. However, the performance of the algorithm in the steady-state is not analyzed. In this paper, we study the steady-state mean-square-deviation (MSD) behavior of the SSAF algorithm by using energy conservation relation, Price׳s theorem and some reasonable assumptions. Simulation results in different system identification scenarios (including the input signals, tap lengths, impulsive noises, number of subbands, and step sizes) are provided to support our theoretical analysis. [ABSTRACT FROM AUTHOR]

Recurrent broad learning system (RBLS) is an effective learning method for processing sequential data. By replacing the enhancement nodes of the broad learning system with recurrent structure, RBLS obtains the capacity to capture the dynamic characteristics of time series data. However, RBLS is derived under the minimum mean square error (MMSE) criterion, which is sensitive to outliers. Moreover, RBLS is insufficient for online sequential learning. To address these limitations, we propose a novel online robust echo state structure based RBLS (OR-ESBLS). In OR-ESBLS, kernel recursive maximum correntropy (KRMC) is introduced to both enhance the robustness and discover the nonlinear characteristics of feature nodes in an online manner. To reduce the heavy computational requirements caused by the kernel method, a Quasi-Monte Carlo (QMC) based Random Fourier Feature (RFF) is utilized for kernel approximation. Furthermore, we adopt the randomized sparse reservoir as the enhancement nodes of RBLS, which can much more efficiently capture dynamic information of the data in the sequential learning setting. Experiments on both synthetic and real-world datasets are reported. The results show that the proposed OR-ESBLS can provide superior performance in online sequential time series prediction. [ABSTRACT FROM AUTHOR]

The state space recursive least squares (SSRLS) filter is a new addition to the well-known recursive least squares (RLS) family filters, which can achieve an excellent tracking performance by overcoming some limitations of the standard RLS algorithm. However, when the underlying system is disturbed by some heavy-tailed non-Gaussian impulsive noises, the performance of SSRLS will deteriorate significantly. The main reason for this is that the SSRLS is derived under the minimum mean square error (MMSE) criterion, which is not well-suited to estimation problems under non-Gaussian noises. To overcome this issue, we propose in this paper a novel linear filter, called the state space maximum correntropy (SSMC) filter, which is derived under the maximum correntropy criterion (MCC) instead of the MMSE. Since MCC is very suited to non-Gaussian signal processing, the SSMC performs very well in non-Gaussian noises especially when the signals are corrupted by impulsive noises. A simple illustrative example is presented to demonstrate the desirable performance of the new algorithm. [ABSTRACT FROM AUTHOR]

Kalman filters (KFs) are widely used for state-of-charge (SOC) estimation of Li-ion batteries due to their excellent dynamic tracking capability. Especially the cubature KF (CKF), with the computational efficiency and nonlinear processing ability, is an outstanding candidate for SOC estimation. However, the actual working conditions are complex and changeable, and the measurement data is usually accompanied by non-Gaussian noise (outliers). Therefore, the performance of the original CKF with minimum mean square error (MMSE) criterion may be degraded seriously in these cases. In order to enhance the robustness of CKF, the MMSE in the CKF framework is substituted by the generalized maximum correntropy criterion (GMCC), and thus a robust CKF with GMCC (GMCC-CKF) is developed by fixed point iteration approach in this work. Furthermore, a SOC estimation model via the GMCC-CKF is proposed to improve estimation accuracy under non-Gaussian noise environments. The simulation results show that, compared with the traditional KFs, the proposed GMCC-CKF can accurately estimate the SOC of lithium batteries under different temperatures and operating conditions considering non-Gaussian noise interference. The results of mean absolute error (MAE) and root mean square error (RMSE) are less than 1%, which verifies the excellent performance of GMCC-CKF. • GMCC is introduced into the CKF framework to develop a novel robust CKF. • The proposed GMCC-CKF method is applied for robust SOC estimation under non-Gaussian noise interferences. • Some simulations under different conditions are performed to evaluate the performance of the proposed method. [ABSTRACT FROM AUTHOR]

Deep learning systems aim at using hierarchical models to learning high-level features from low-level features. The progress in deep learning is great in recent years. The robustness of the learning systems with deep architectures is however rarely studied and needs further investigation. In particular, the mean square error (MSE), a commonly used optimization cost function in deep learning, is rather sensitive to outliers (or impulsive noises). Robust methods are needed to improve the learning performance and immunize the harmful influences caused by outliers which are pervasive in real-world data. In this paper, we propose an efficient and robust deep learning model based on stacked auto-encoders and Correntropy-induced loss function (CLF), called CLF-based stacked auto-encoders (CSAE). CLF as a nonlinear measure of similarity is robust to outliers and can approximate different norms (from $$l_0$$ to $$l_2$$ ) of data. Essentially, CLF is an MSE in reproducing kernel Hilbert space. Different from conventional stacked auto-encoders, which use, in general, the MSE as the reconstruction loss and KL divergence as the sparsity penalty term, the reconstruction loss and sparsity penalty term in CSAE are both built with CLF. The fine-tuning procedure in CSAE is also based on CLF, which can further enhance the learning performance. The excellent and robust performance of the proposed model is confirmed by simulation experiments on MNIST benchmark dataset. [ABSTRACT FROM AUTHOR]

To improve the performance and reduce the computational complexity, a novel sign adaptation scheme for convex combination of adaptive filters is proposed. By using a sign adaptation scheme, the proposed scheme slightly reduces the computational complexity of the basic combination of mixing parameter, and improves the robustness of mixing parameter. Moreover, to obtain fast convergence property during the period of convergence transition in different environments, the instantaneous transfer scheme is applied to the proposed algorithm. The instantaneous transfer scheme has lower computational complexity than that of the basic combination of least mean square (basic-CLMS) algorithms. Simulation results in the context of system identification under time-invariant and time-variant systems demonstrate that the proposed algorithm based on the instantaneous transfer scheme has lower computational complexity and faster convergence rate than that of the basic-CLMS algorithm during period of convergence transitions. [ABSTRACT FROM AUTHOR]