Search

As real industrial processes have measurement samples with noises of different statistical characteristics and obtain the sample one by one usually, on-line sequential learning algorithms which can achieve better learning performance for systems with noises of various statistics are necessary. This paper proposes a new online Extreme Learning Machine (ELM, of Huang et al.) algorithm, namely recursive least mean p-power ELM (RLMP-ELM). In RLMP-ELM, a novel error criterion for cost function, namely the least mean p-power (LMP) error criterion, provides a mechanism to update the output weights sequentially. The LMP error criterion aims to minimize the mean p-power of the error that is the generalization of the mean square error criterion used in the ELM. The proposed on-line learning algorithm is able to provide on-line predictions of variables with noises of different statistics and obtains better performance than ELM and online sequential ELM (OS-ELM) while the non-Gaussian noises impact the processes. Simulations are reported to demonstrate the performance and effectiveness of the proposed methods., (Copyright © 2017 Elsevier Ltd. All rights reserved.)

Identification of Hammerstein systems with polynomial features and mean square error loss has received a lot of attention due to their simplicity in calculation and solid theoretical foundation. However, when the prior information of nonlinear subblock of a Hammerstein system is unknown or some outliers are involved, the performance of related methods may degenerate seriously. The main reason is that the used polynomial just has finite approximation capability to an unknown nonlinear function, and mean square error loss is sensitive to outliers. In this paper, a new identification method based on random Fourier features and kernel risk sensitive loss is therefore proposed. Since the linear combination of random Fourier features can well approximate any continuous nonlinear function, it is expected to be more powerful to characterize the nonlinear behavior of Hammerstein systems. Moreover, since the kernel risk sensitive loss is a similarity measure that is insensitive to outliers, it is expected to have excellent robustness. Based on the mean square convergence analysis, a sufficient condition to ensure the convergence and some theoretical values regarding the steady-state excess mean square error of the proposed method are also provided. Simulation results on the tasks of Hammerstein system identification and electroencephalogram noise removal show that the new method can outperform other popular and competitive methods in terms of accuracy and robustness. [ABSTRACT FROM AUTHOR]

As a novel similarity measure that is defined as the expectation of a kernel function between two random variables, correntropy has been successfully applied in robust machine learning and signal processing to combat large outliers. The kernel function in correntropy is usually a zero-mean Gaussian kernel. In a recent work, the concept of mixture correntropy (MC) was proposed to improve the learning performance, where the kernel function is a mixture Gaussian kernel, namely, a linear combination of several zero-mean Gaussian kernels with different widths. In both correntropy and MC, the center of the kernel function is, however, always located at zero. In the present work, to further improve the learning performance, we propose the concept of multikernel correntropy (MKC), in which each component of the mixture Gaussian kernel can be centered at a different location. The properties of the MKC are investigated and an efficient approach is proposed to determine the free parameters in MKC. Experimental results show that the learning algorithms under the maximum MKC criterion (MMKCC) can outperform those under the original maximum correntropy criterion (MCC) and the maximum MC criterion (MMCC). [ABSTRACT FROM AUTHOR]

Spiking neural networks (SNNs) are regarded as a promising candidate to deal with the major challenges of current machine learning techniques, including the high energy consumption induced by deep neural networks. However, there is still a great gap between SNNs and the few-shot learning performance of artificial neural networks. Importantly, existing spike-based few-shot learning models do not target robust learning based on spatiotemporal dynamics and superior machine learning theory. In this paper, we propose a novel spike-based framework with the entropy theory, namely, heterogeneous ensemble-based spike-driven few-shot online learning (HESFOL). The proposed HESFOL model uses the entropy theory to establish the gradient-based few-shot learning scheme in a recurrent SNN architecture. We examine the performance of the HESFOL model based on the few-shot classification tasks using spiking patterns and the Omniglot data set, as well as the few-shot motor control task using an end-effector. Experimental results show that the proposed HESFOL scheme can effectively improve the accuracy and robustness of spike-driven few-shot learning performance. More importantly, the proposed HESFOL model emphasizes the application of modern entropy-based machine learning methods in state-of-the-art spike-driven learning algorithms. Therefore, our study provides new perspectives for further integration of advanced entropy theory in machine learning to improve the learning performance of SNNs, which could be of great merit to applied developments with spike-based neuromorphic systems. [ABSTRACT FROM AUTHOR]

The spiking neural network (SNN) is regarded as a promising candidate to deal with the great challenges presented by current machine learning techniques, including the high energy consumption induced by deep neural networks. However, there is still a great gap between SNNs and the online meta-learning performance of artificial neural networks. Importantly, existing spike-based online meta-learning models do not target the robust learning based on spatio-temporal dynamics and superior machine learning theory. In this invited article, we propose a novel spike-based framework with minimum error entropy, called MeMEE, using the entropy theory to establish the gradient-based online meta-learning scheme in a recurrent SNN architecture. We examine the performance based on various types of tasks, including autonomous navigation and the working memory test. The experimental results show that the proposed MeMEE model can effectively improve the accuracy and the robustness of the spike-based meta-learning performance. More importantly, the proposed MeMEE model emphasizes the application of the modern information theoretic learning approach on the state-of-the-art spike-based learning algorithms. Therefore, in this invited paper, we provide new perspectives for further integration of advanced information theory in machine learning to improve the learning performance of SNNs, which could be of great merit to applied developments with spike-based neuromorphic systems. [ABSTRACT FROM AUTHOR]

Recently, maximum correntropy criterion (MCC) has been widely and successfully used in robust signal processing and machine learning, in which the correntropy is maximized instead of minimizing the popular mean square error (MSE) to improve the robustness with respect to outliers or impulsive noises. A lot of efforts have been devoted to derive different adaptive algorithms under MCC, but to date, little insight has been gained as to how the MCC solution will be influenced by outliers. In this paper, we investigate this problem and our focus is mainly on the parameter estimation of a simple linear errors-in-variables (EIVs) model with scalar variables. Under some conditions, we derive an upper bound on the absolute value of the estimation error and show that the MCC solution can get very close to the true value of the unknown parameter even with arbitrarily large outliers in both the input and output variables. Illustrative examples are provided to verify and clarify the theory. [ABSTRACT FROM AUTHOR]

Correntropy is a local similarity measure defined in kernel space, and the maximum correntropy criterion (mcc) has been successfully applied in many areas of signal processing and machine learning in recent years. The kernel function in correntropy is usually restricted to the Gaussian function with the center located at zero. However, the zero-mean Gaussian function may not be a good choice for many practical applications. In this letter, we propose an extended version of correntropy, whose center can be located at any position. Accordingly, we propose a new optimization criterion called maximum correntropy criterion with variable center (MCC-VC). We also propose an efficient approach to optimize the kernel width and center location in the MCC-VC. Simulation results of regression with linear-in-parameter (LIP) models confirm the desirable performance of the new method. [ABSTRACT FROM AUTHOR]

High-dimensional data contain not only redundancy but also noises produced by the sensors. These noises are usually non-Gaussian distributed. The metrics based on Euclidean distance are not suitable for these situations in general. In order to select the useful features and combat the adverse effects of the noises simultaneously, a robust sparse subspace learning method in unsupervised scenario is proposed in this paper based on the maximum correntropy criterion that shows strong robustness against outliers. Furthermore, an iterative strategy based on half quadratic and an accelerated block coordinate update is proposed. The convergence analysis of the proposed method is also carried out to ensure the convergence to a reliable solution. Extensive experiments are conducted on real-world data sets to show that the new method can filter out the outliers and outperform several state-of-the-art unsupervised feature selection methods. [ABSTRACT FROM AUTHOR]

Correntropy is a local similarity measure defined in kernel space, hence can combat large outliers in robust signal processing and machine learning. So far, many robust learning algorithms have been developed under the maximum correntropy criterion (MCC), among which, a Gaussian kernel is generally used in correntropy. To further improve the learning performance, in this paper we propose the concept of mixture correntropy, which uses the mixture of two Gaussian functions as the kernel function. Some important properties of the mixture correntropy are presented. Applications of the maximum mixture correntropy criterion (MMCC) to extreme learning machine (ELM) and kernel adaptive filtering (KAF) for function approximation and data regression are also studied. Experimental results show that the learning algorithms under MMCC can perform very well and achieve better performance than the conventional MCC based algorithms as well as several other state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]

Extreme learning machine (ELM) is a popular learning algorithm for single hidden layer feedforward networks (SLFNs). It was originally proposed with the inspiration from biological learning and has attracted massive attentions due to its adaptability to various tasks with a fast learning ability and efficient computation cost. As an effective sparse representation method, orthogonal matching pursuit (OMP) method can be embedded into ELM to overcome the singularity problem and improve the stability. Usually OMP recovers a sparse vector by minimizing a least squares (LS) loss, which is efficient for Gaussian distributed data, but may suffer performance deterioration in presence of non-Gaussian data. To address this problem, a robust matching pursuit method based on a novel kernel risk-sensitive loss (in short KRSLMP) is first proposed in this paper. The KRSLMP is then applied to ELM to solve the sparse output weight vector, and the new method named the KRSLMP-ELM is developed for SLFN learning. Experimental results on synthetic and real-world data sets confirm the effectiveness and superiority of the proposed method. [ABSTRACT FROM AUTHOR]

Nonlinear similarity measures defined in kernel space, such as correntropy, can extract higher order statistics of data and offer potentially significant performance improvement over their linear counterparts especially in non Gaussian signal processing and machine learning. In this paper, we propose a new similarity measure in kernel space, called the kernel risk-sensitive loss (KRSL), and provide some important properties. We apply the KRSL to adaptive filtering and investigate the robustness, and then develop the MKRSL algorithm and analyze the mean square convergence performance. Compared with correntropy, the KRSL can offer a more efficient performance surface, thereby enabling a gradient-based method to achieve faster convergence speed and higher accuracy while still maintaining the robustness to outliers. Theoretical analysis results and superior performance of the new algorithm are confirmed by simulation. [ABSTRACT FROM PUBLISHER]

As a robust nonlinear similarity measure in kernel space, correntropy has received increasing attention in domains of machine learning and signal processing. In particular, the maximum correntropy criterion (MCC) has recently been successfully applied in robust regression and filtering. The default kernel function in correntropy is the Gaussian kernel, which is, of course, not always the best choice. In this paper, we propose a generalized correntropy that adopts the generalized Gaussian density (GGD) function as the kernel, and present some important properties. We further propose the generalized maximum correntropy criterion (GMCC) and apply it to adaptive filtering. An adaptive algorithm, called the GMCC algorithm, is derived, and the stability problem and steady-state performance are studied. We show that the proposed algorithm is very stable and can achieve zero probability of divergence (POD). Simulation results confirm the theoretical expectations and demonstrate the desirable performance of the new algorithm. [ABSTRACT FROM AUTHOR]

A sequential extreme learning machine incorporating a noise compensation scheme via an information measure is developed. In this design, the computationally simple extreme learning machine architecture is maintained while survival error information potential function provides a mechanism for noise compensation. The error compensation is updated online via an error codebook design where an error tolerant and stable solution is obtained. The developed method is tested on chaotic time sequence as well as benchmark data sets. Experimental results show potential applications for the developed method. [ABSTRACT FROM AUTHOR]

Accurate cluster photovoltaic power prediction (CPPP) is crucial for the operation and control of renewable energy grid-connected power systems. The traditional modeling strategies for CPPP such as direct aggregation (DA) and statistical upscaling (SU) have limitations such as error accumulation and upscaling factor uncertainty. To address these issues, this paper proposed a novel hybrid approach for CPPP by combining machine learning models with an improved SU technique. Firstly, a robust broad learning system (BLS) model, in which the Generalized Maximum Correntropy Criterion (GMCC) is used to replace the original mean square error (MSE) loss in BLS, is proposed to solve the problem of multiple outliers affecting the prediction accuracy of regional cluster stations, and it is called GBLS. Then, the Relevance Vector Machine (RVM) as an effective nonlinear regression model is further utilized to compensate for the prediction errors obtained by the GBLS to form the hybrid prediction model, namely GBLS-RVM. Moreover, to mitigate the uncertainty associated with scaling factors in traditional SU strategy, a new SU strategy is developed to refine the relationship between the reference station and the cluster sub-region, enabling direct modeling for regional power prediction. Finally, data from two PV clusters in different regions of China are used to validate the effectiveness of the proposed model, and the results show that under the improved SU strategy, the GBLS-RVM model, reduced RMSE by approximately 33.7% compared to the traditional BLS model, and the RMSE decreased by 12.89% and 30.2% when compared to traditional DA and traditional SU strategies. • A novel hybrid method for CPPP is developed by combining ML and improved SU technique. • A BLS with GMCC (GMCC-BLS) as a prediction model is developed to handle the outliers in measurement data. • An innovative error correction strategy based on RVM is proposed to improve prediction reliability. • A new SU technique is used to establish direct relationships between reference stations and subregions. [ABSTRACT FROM AUTHOR]

• We propose a robot perception intelligence adaptation mechanism and three different data sampling styles of this mechanism. • We theoretically analyze and compare the learning effect of the proposed three different data sampling styles. • We determine the most suitable data sampling style based on the performance availability trade off. • Our work offers guidance for training data collection in the robot perception intelligence adaptation process. Robots need more intelligence to complete perception tasks in uncertain and unstructured environments. This paper presents a new self-evolving home service robot framework that learns new perception skills by using manually labeled data obtained in a new home environment. In this framework, a global model is trained which serves as the starting point of the robots' local model, and an adaptation mechanism is developed in the robot to adapt the initial local model to the new home environment. First, three different data sampling styles are proposed to carry out the adaptation process, and theoretical analysis is given to explain the difference between the proposed three data sampling styles. Second, the most suitable data sampling style for incremental learning for a home service robot is determined. Third, we present a case study of multi-style learning, and the experimental results validate our analysis. The theoretical analysis and experimental results lead us to propose a guideline of data collection and labeling for robots to adapt their perception intelligence in home environments. [ABSTRACT FROM AUTHOR]

Accurate estimation of the state of health (SOH) is crucial for the safe and stable operation of lithium-ion batteries. However, since the labeled values may contain non-Gaussian noise which may lead to a decrease in the effectiveness of traditional data-driven based estimation methods. Therefore, a novel robust stacking integrated learning model (ILM) is proposed to enable accurate SOH estimation under non-Gaussian noises (or outliers) conditions. The mixture correntropy loss (MCL) is used in original extreme learning machine (ELM) and the gated recurrent unit (GRU) frameworks to develop novel robust learning models, i.e MCL-ELM and MCL-GRU, and they are used as the sub-models of the proposed ILM, which takes into account the generalization performance of the model and the overall trend of SOH over the time series. In addition, the entropy weighting method that can well reflect the differences between the sub-models is utilized to reduce the complexity of the stacked ILM. Furthermore, the local tangent space alignment is used to capture the local relationships between different loops to enhance the effectiveness of the extracted health features. Several numerical experiments are performed under different cases to validate the estimation effect of the proposed model by using two publicly available datasets, and the experimental results demonstrate that the maximum RMSE, MAE, and MAPE values of the proposed model are 1.391%, 0.932%, and 1.480%, respectively, under non-Gaussian noises conditions. • Robust ELM and GRU models using MCL are developed. • Novel robust stacking ILM via the combination of the MCL-ELM and MCL-GRU is proposed for SOH estimation. • The EWM is used to obtain the output results of multiple base learners in the proposed ILM. [ABSTRACT FROM AUTHOR]

Along with the explosive growing of data, semi-supervised learning attracts increasing attention in the past years due to its powerful capability in labeling unlabeled data and knowledge mining. As an emerging method, the semi-supervised extreme learning machine (SSELM), that builds on ELM, has been developed for data classification and shown superiorities in learning efficiency and accuracy. However, the optimization of SSELM as well as most of the other ELMs is generally based on the mean square error (MSE) criterion, which has been shown less effective in dealing with non-Gaussian noises. In this paper, a robust regularized correntropy criterion based SSELM (RC-SSELM) is developed. The optimization of the output weight matrix of RC-SSELM is derived by the fixed-point iteration based approach. A convergent analysis of the proposed RC-SSELM is presented based on the half-quadratic optimization technique. Experimental results on 4 synthetic datasets and 13 benchmark UCI datasets are provided to show the superiorities of the proposed RC-SSELM over SSELM and other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

A fixed kernel width in MCC algorithm imposes a trade-off among robustness, convergence rate and steady-state accuracy. With a variable kernel width, the adaptive kernel width MCC (AMCC) algorithm can improve the learning speed of the MCC algorithm especially when the initial weight vector is far away from the optimal weight vector. In this paper, the steady-state excess mean square error (EMSE) of the AMCC algorithm is studied based on energy conservation relation. In addition, a novel convergence measure called initial convergence rate is introduced to evaluate the convergence speed at the beginning of the learning. Simulation experiments are carried out to verify the theoretical analysis and confirm the desirable performance of the AMCC algorithm in several different non-Gaussian noise environments. [ABSTRACT FROM AUTHOR]