Search

Kernel adaptive filters (KAF) are a class of powerful nonlinear filters developed in Reproducing Kernel Hilbert Space (RKHS). The Gaussian kernel is usually the default kernel in KAF algorithms, but selecting the proper kernel size (bandwidth) is still an open important issue especially for learning with small sample sizes. In previous research, the kernel size was set manually or estimated in advance by Silverman׳s rule based on the sample distribution. This study aims to develop an online technique for optimizing the kernel size of the kernel least mean square (KLMS) algorithm. A sequential optimization strategy is proposed, and a new algorithm is developed, in which the filter weights and the kernel size are both sequentially updated by stochastic gradient algorithms that minimize the mean square error (MSE). Theoretical results on convergence are also presented. The excellent performance of the new algorithm is confirmed by simulations on static function estimation, short term chaotic time series prediction and real world Internet traffic prediction. [ABSTRACT FROM AUTHOR]

A sequential extreme learning machine incorporating a noise compensation scheme via an information measure is developed. In this design, the computationally simple extreme learning machine architecture is maintained while survival error information potential function provides a mechanism for noise compensation. The error compensation is updated online via an error codebook design where an error tolerant and stable solution is obtained. The developed method is tested on chaotic time sequence as well as benchmark data sets. Experimental results show potential applications for the developed method. [ABSTRACT FROM AUTHOR]

Abstract: As an alternative adaptation criterion, the minimum error entropy (MEE) criterion has been receiving increasing attention due to its successful use in, especially, nonlinear and non-Gaussian signal processing. In this paper, we study the application of error entropy minimization to kernel adaptive filtering, a new and promising technique that implements the conventional linear adaptive filters in reproducing kernel Hilbert space (RKHS) and obtains the nonlinear adaptive filters in original input space. The kernel minimum error entropy (KMEE) algorithm is derived, which is essentially a generalized stochastic information gradient (SIG) algorithm in RKHS. The computational complexity of KMEE is just similar to the kernel affine projection algorithm (KAPA). We also utilize the quantization approach to constrain the network size growth, and develop the quantized KMEE (QKMEE) algorithm. Further, we analyze the mean square convergence of KMEE. The energy conservation relation is derived and a sufficient condition that ensures the mean square convergence is obtained. The performance of the new algorithm is demonstrated in nonlinear system identification and short-term chaotic time series prediction. [Copyright &y& Elsevier]

Abstract: The maximum mutual information (MaxMI) criterion is used as the adaptation cost for the adaptive filtering. This criterion is robust to measure distortions, and has strong connection with traditional mean-square error (MSE) criterion. Under Gaussian assumption, the closed-form solution of the finite impulse response (FIR) filter is obtained. Further, based on the kernel density estimation, the stochastic mutual information gradient (SMIG) algorithm is derived. Simulation results emphasize the robustness of this new algorithm. [Copyright &y& Elsevier]

Least-squares one-class support vector machine (LS-OCSVM) is one of the most popular methods to perform one-class classification tasks, in which only the data of a specific class are available to train the classification model. However, the learning performance of LS-OCSVM heavily relies on the effectiveness of a squared loss function, which is sensitive to outliers, resulting in the poor robustness of LS-OCSVM to deal with contaminated data. In this paper, the original optimization problem of LS-OCSVM is therefore reformulated with a recently proposed robust similarity measure, called multikernel correntropy, generating a multikernel correntropy based LS-OCSVM (MKCLS-OCSVM). To find the solution to the new optimization problem effectively, a dynamic optimization algorithm developed with the popular half-quadratic optimization technique is adopted to perform the optimization process. Meanwhile, the convergence and computational complexity of the developed optimization algorithm are analyzed from theoretical perspectives. To further facilitate the implementation of MKCLS-OCSVM, an operationally simple search strategy, inspired by the hunting behavior of humpback whales, is designed for parameters selection. Experimental results on various one-class classification tasks are reported to demonstrate the performance superiority of the proposed MKCLS-OCSVM in comparison with LS-OCSVM and other robust LS-OCSVM variants. [ABSTRACT FROM AUTHOR]

Abstract Concept factorization (CF) technique is one of the most powerful approaches for feature learning, and has been successfully adopted in a wide range of practical applications such as data mining, computer vision, and information retrieval. Most existing concept factorization methods mainly minimize the square of the Euclidean distance, which is seriously sensitive to non-Gaussian noises or outliers in the data. To alleviate the adverse influence of this limitation, in this paper, a robust graph regularized concept factorization method, called correntropy based graph regularized concept factorization (GCCF), is proposed for clustering tasks. Specifically, based on the maximum correntropy criterion (MCC), GCCF is derived by incorporating the graph structure information into our proposed objective function. A half-quadratic optimization technique is adopted to solve the non-convex objective function of the GCCF method effectively. In addition, algorithm analysis of GCCF is studied. Extensive experiments on real world datasets demonstrate that the proposed GCCF method outperforms seven competing methods for clustering applications. [ABSTRACT FROM AUTHOR]

The convergence of nonlinear Kalman filters has conventionally been analyzed in terms of the estimation error. In this paper, we present a new method for investigating the convergence performance of a class of nonlinear Kalman filters based on deterministic sampling. The systems considered here are described by nonlinear state equations with linear measurements. For this type of systems, our proposed convergence analysis is performed using “innovation”, which is defined as the error between the measurement and its prediction. Specifically, we obtain a linear relationship between the innovation and the estimation error, and derive a set of sufficient conditions that ensures the convergence of nonlinear Kalman filters. Compared with the conventional convergence analysis method based on the estimation error, the proposed innovation-based method can obtain sufficient conditions for convergence more directly and readily. Simulation results show that the convergence of innovation generates the convergence of nonlinear Kalman filters. [ABSTRACT FROM AUTHOR]

In this paper, a robust echo state network with correntropy induced loss function (CLF) is presented. CLF is robust to outliers through the mechanism of correntropy which is widely applied in information theoretic learning. The proposed method can improve the anti-noise capacity of echo state network and overcome its problem of being sensitive outliers which are prevalent in real-world tasks. The echo state network with CLF inherits the basic architecture of echo state network, but replaces the commonly used mean square error (MSE) criterion with CLF. The stochastic gradient descent method is adopted to optimize the objective function. The proposed method is subsequently verified in nonlinear system identification and chaotic time-series prediction. Experimental results demonstrate that our method is robust to outliers and outperforms the echo state networks with Bayesian regression and Huber loss function. [ABSTRACT FROM AUTHOR]

Multi-view clustering (MVC) often provides superior effectiveness to single-view clustering due to the integration of information from diverse views. Nonetheless, existing MVC methods are limited to large-scale real-world data by the drawbacks of low efficiency and poor robustness. To address these issues, we propose a novel robust anchor-based MVC model via spectral embedded concept factorization (RAMCSF). RAMCSF builds anchor graphs to approximate full-sample graphs and decomposes these anchor graphs by concept factorization (CF). To improve the clustering effectiveness, factor matrices of CF are constrained as orthogonal matrices to reduce the freedom of decomposition, and a novel small-scale anchor-based spectral embedding is designed to explore the high-order neighbor relationships. To restrain complex noises distributed in real-world data, we employ correntropy to measure the error between the original data and the learned representation. Moreover, RAMCSF can get a clustering indicator matrix directly, avoiding additional post-processing and ensuring that changes in data dimensions have a limited impact on efficiency. The model is then optimized by a novel fast half-quadratic-based optimization strategy that combines the orthogonal properties and the traces of matrices. Extensive experiments indicate that RAMCSF can achieve higher efficiency and robustness while maintaining comparable effectiveness to other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

• Distributed optimization problems of consensus are examined by a frequency-domain analysis approach. • An optimized distributed protocol is proposed such that the nonuniform time-delay FDMSs with SF have better consensus performance. • The double-integral dynamics of delayed FDMSs with SF is considered. This paper addresses distributed optimization problems concerning consensus in delayed fractional-order double-integrator multi-agent systems (FDMSs). To start with, an optimized distributed protocol with state-fractional-order-derivative feedback (SF) is presented for delayed FDMSs. Then, the consensus problems are studied for the two kinds of delayed FDMSs with SF in the presence of symmetric time-delays over undirected network topology and asymmetric time-delays over directed network topology. Next, by the means of graph theory, matrix theory and frequency-domain analysis method, the sufficient conditions to guarantee consensus of delayed FDMSs with SF are derived. Compared to the traditional distributed protocol without SF, the proposed distributed optimization protocol with SF are taken into account to enable better consensus performance in delayed FDMSs with SF. Finally, numerical experiments are carried out to verify the feasibility of our theoretical results. [ABSTRACT FROM AUTHOR]

Concept factorization (CF) has attracted widespread attention for its promising performance in document clustering. Among various CF variants, graph-regularized CF is the most impressive type, which can improve clustering effectiveness by exploring structural information. Nevertheless, their clustering efficiency is restricted by the following considerations: (1) the introduction of the full-sample graph is accompanied by an increase in computational complexity; (2) most of them require intensive multiplications in optimization, which impair the optimization efficiency. To address these issues, in this work, we propose an anchor-graph regularized orthogonal concept factorization (AROCF) method to enhance the clustering efficiency and effectiveness in document clustering tasks. Firstly, AROCF approximates the full-sample graph with a small-scale anchor graph to reduce the complexity of graph construction from quadratic to linear. Then, one of the factor matrices is constrained as the cluster indicator matrix in our method, which can avoid extra efficiency loss in K-means after optimization. Finally, an orthogonal constraint is employed to restrict the freedom of factorization to increase the clustering effectiveness. To optimize the AROCF model, we develop a fast optimization strategy by combining the trace and orthogonality of matrices. Extensive experiments on various document datasets demonstrate the effectiveness and efficiency of AROCF. [ABSTRACT FROM AUTHOR]

Recurrent broad learning system (RBLS) is an effective learning method for processing sequential data. By replacing the enhancement nodes of the broad learning system with recurrent structure, RBLS obtains the capacity to capture the dynamic characteristics of time series data. However, RBLS is derived under the minimum mean square error (MMSE) criterion, which is sensitive to outliers. Moreover, RBLS is insufficient for online sequential learning. To address these limitations, we propose a novel online robust echo state structure based RBLS (OR-ESBLS). In OR-ESBLS, kernel recursive maximum correntropy (KRMC) is introduced to both enhance the robustness and discover the nonlinear characteristics of feature nodes in an online manner. To reduce the heavy computational requirements caused by the kernel method, a Quasi-Monte Carlo (QMC) based Random Fourier Feature (RFF) is utilized for kernel approximation. Furthermore, we adopt the randomized sparse reservoir as the enhancement nodes of RBLS, which can much more efficiently capture dynamic information of the data in the sequential learning setting. Experiments on both synthetic and real-world datasets are reported. The results show that the proposed OR-ESBLS can provide superior performance in online sequential time series prediction. [ABSTRACT FROM AUTHOR]

The traditional image quality assessments, such as the mean squared error (MSE), the signal-to-noise ratio (SNR), and the Peak signal-to-noise ratio (PSNR), are all based on the absolute error of images. Structural similarity (SSIM) index is another important image quality assessment which has been shown to be more effective in the human vision system (HVS). Although there are many essential differences between MSE and SSIM, some important associations exist between them. In this paper, the associations between MSE and SSIM as cost functions in linear decomposition are investigated. Based on the associations, a bit-allocation algorithm for sparse coding is proposed by considering both the reconstructed image quality and the reconstructed image contrast. In the proposed algorithm, the space occupied by a linear coefficient of a basis in sparse coding is reduced to only 9 to 10 bits, in which 1 bit is used to save the sign of linear coefficient, 3 bits are used to save the number of powers of 10 in scientific notation, and only 5 to 6 bits are used to save the significance digits. The experimental results show that the proposed bit-allocation algorithm for sparse coding can maintain both the image quality and the image contrast well. [ABSTRACT FROM AUTHOR]

• An optimization problem about the state mean of multi-agent systems with impulsive noise based on MCC derived from information theoretic learning is investigated in this paper, where the half-quadratic(HQ) strategy is used to simplify the solving process of the optimization problem. • A problem interfered by impulsive noise on the states of multi-agent systems is studied in this paper, where the adverse effect of impulsive noise on the states is eliminated by the MCC algorithm. • An event-triggered consensus problem of discrete-time multi-agent systems with random impulsive noise is addressed in this paper, where an event-triggering condition is derived for the system to achieve consensus. An event-triggered consensus problem of discrete-time multi-agent systems with impulsive noise under directed switching topology is investigated in this paper, where the control input of each agent is determined by its own state information and the state information of its neighbors that may be corrupted by impulsive noise. Firstly, to reduce the adverse effect of impulsive noise on consensus performance of the discrete-time multi-agent systems, maximum correntropy criterion (MCC) derived from information theoretic learning is introduced to calculate the communication weights among the agents, and an event-triggered strategy is adopted to drive the state variables of each agent to eventually converge to a same value. Secondly, the condition to guarantee consensus of the discrete-time multi-agent systems is derived by graph theory and Lyapunov stability theorem. Finally, the effectiveness of the theory is verified by several numerical simulations. [ABSTRACT FROM AUTHOR]