Search

The generative autoencoders, such as the variational autoencoders or the adversarial autoencoders, have achieved great success in lots of real-world applications, including image generation and signal communication. However, little concern has been devoted to their robustness during practical deployment. Due to the probabilistic latent structure, variational autoencoders (VAEs) may confront problems such as a mismatch between the posterior distribution of the latent and real data manifold, or discontinuity in the posterior distribution of the latent. This leaves a back door for malicious attackers to collapse VAEs from the latent space, especially in scenarios where the encoder and decoder are used separately, such as communication and compressed sensing. In this work, we provide the first study on the adversarial robustness of generative autoencoders in the latent space. Specifically, we empirically demonstrate the latent vulnerability of popular generative autoencoders through attacks in the latent space. We also evaluate the difference between variational autoencoders and their deterministic variants and observe that the latter performs better in latent robustness. Meanwhile, we identify a potential trade-off between the adversarial robustness and the degree of the disentanglement of the latent codes. Additionally, we also verify the feasibility of improvement for the latent robustness of generative autoencoders through adversarial training. In summary, we suggest concerning the adversarial latent robustness of the generative autoencoders, analyze several robustness-relative issues, and give some insights into a series of key challenges. [ABSTRACT FROM AUTHOR]

The multivariate autoregressive (MVAR) model is widely used in describing the dynamics of nonlinear systems, in which the estimates of model parameters and underlying states can be achieved by dual extended Kalman filter (DEKF). However, when the measurements are corrupted by complicated non-Gaussian noises, the DEKF based on the minimum mean-square error (MMSE) criterion may provide biased estimates. In the present article, we develop a novel dual Kalman-type filter, referred to as DEKF under minimum error entropy (MEE) with fiducial points (MEEFs-DEKF) to deal with the non-Gaussian noises. First, the equivalent state-space model and parameter-space model are presented based on the MVAR model. Then, an optimality criterion based on MEE with fiducial points (MEEFs) is applied in the batch-mode regression equation to improve the robustness. Finally, a fixed-point iteration algorithm gives the posterior estimates of state and parameter. Simulation results confirm that the proposed MEEF-DEKF can achieve excellent performance in various noises with different distributions, especially in heavy-tailed and multimodal noises. [ABSTRACT FROM AUTHOR]

To date, most linear and nonlinear Kalman filters (KFs) have been developed under the Gaussian assumption and the well-known minimum mean square error (MMSE) criterion. In order to improve the robustness with respect to impulsive (or heavy-tailed) non-Gaussian noises, the maximum correntropy criterion (MCC) has recently been used to replace the MMSE criterion in developing several robust Kalman-type filters. To deal with more complicated non-Gaussian noises such as noises from multimodal distributions, in this article, we develop a new Kalman-type filter, called minimum error entropy KF (MEE-KF), by using the minimum error entropy (MEE) criterion instead of the MMSE or MCC. Similar to the MCC-based KFs, the proposed filter is also an online algorithm with the recursive process, in which the propagation equations are used to give prior estimates of the state and covariance matrix, and a fixed-point algorithm is used to update the posterior estimates. In addition, the MEE extended KF (MEE-EKF) is also developed for performance improvement in the nonlinear situations. The high accuracy and strong robustness of MEE-KF and MEE-EKF are confirmed by experimental results. [ABSTRACT FROM AUTHOR]

Recently, maximum correntropy criterion (MCC) has been widely and successfully used in robust signal processing and machine learning, in which the correntropy is maximized instead of minimizing the popular mean square error (MSE) to improve the robustness with respect to outliers or impulsive noises. A lot of efforts have been devoted to derive different adaptive algorithms under MCC, but to date, little insight has been gained as to how the MCC solution will be influenced by outliers. In this paper, we investigate this problem and our focus is mainly on the parameter estimation of a simple linear errors-in-variables (EIVs) model with scalar variables. Under some conditions, we derive an upper bound on the absolute value of the estimation error and show that the MCC solution can get very close to the true value of the unknown parameter even with arbitrarily large outliers in both the input and output variables. Illustrative examples are provided to verify and clarify the theory. [ABSTRACT FROM AUTHOR]

Due to the effectiveness in learning the subspace structures, low-rank representation (LRR) and its variations have been widely applied in various fields, such as computer vision and pattern recognition. However, in real applications, it is a challenge to handle the complex noises. To address this problem, we propose a novel robust LRR method based on kernel risk-sensitive loss (KRSL) with high-order manifold constraint, called RHLRR, in which the KRSL is introduced to deal with the noises and the multiple hypergraph regularization term is used as a high order manifold constraint to effectively capture the locality, similarity and the intrinsic geometric information in data. Besides, an iterative algorithm based on the half-quadratic (HQ) and the accelerated block coordinate update (BCU) is developed. The experimental results demonstrate that the proposed method can outperform other state-of-the-art LRR variants. [ABSTRACT FROM AUTHOR]

The unscented Kalman filter (UKF) provides a powerful tool for power system forecasting-aided state estimation (FASE). However, when the power systems are affected by the abnormal operating situations, i.e., the non-Gaussian communication noises, sudden loads or state changes, and instrument failures, the original UKF based on the minimum mean square error (MMSE) criterion may suffer from performance degradation. In contrast to the MMSE criterion, the minimum error entropy (MEE) exhibits the robustness with respect to complex non-Gaussian disturbances. In this article, we develop a new unscented Kalman-type filter based on the MEE criterion, termed MEE-UKF. To derive the MEE-UKF, a statistical linearization approach is adopted in the augmented model such that the state and measurement errors are combined in the MEE cost function simultaneously. Then, a fixed-point iteration algorithm is used to recursively update the posterior estimates and covariance matrix. Apart from the impulsive noises, the MEE-UKF can deal with complex multimodal distribution noises in both process and measurement. The high accuracy and strong robustness of MEE-UKF are confirmed by the simulation results on IEEE 14, 30, and 57 bus test systems under different non-Gaussian disturbances. [ABSTRACT FROM AUTHOR]

Common spatial pattern (CSP) is a classic method commonly used in multichannel electroencephalogram (EEG) signal processing, which aims to extract effective features for binary classification by solving spatial filters that maximize the ratio of filtered dispersion between two classes. The aim of this paper is to improve the performance of the conventional CSP method, which will be badly influenced by noises. The recently proposed quantized minimum error entropy (QMEE) criterion is applied to structure a new objective function instead of the ${L} _{2}$ -norm in the conventional CSP. Quantization is utilized to reduce the computational complexity. The new objective function is optimized by a gradient-based iterative algorithm. The desirable performance of the QMEE-based CSP method, namely CSP-QMEE, is demonstrated with a toy example and two real EEG datasets, including Dataset IIb of the brain–computer interfaces (BCIs) Competition IV (three channels) and Dataset IIIa of the BCI Competition III (60 channels). The new method can achieve satisfactory performance compared to existing methods on all datasets. The promising results in this paper suggest that the CSP-QMEE may become a powerful tool for BCIs. [ABSTRACT FROM AUTHOR]

Minimum error entropy with fiducial points (MEEF) has received a lot of attention, due to its outstanding performance to curb the negative influence caused by non-Gaussian noises in the fields of machine learning and signal processing. However, the estimate of the information potential of MEEF involves a double summation operator based on all available error samples, which can result in large computational burden in many practical scenarios. In this paper, an efficient quantization method is therefore adopted to represent the primary set of error samples with a smaller subset, generating a quantized MEEF (QMEEF). Some basic properties of QMEEF are presented and proved from theoretical perspectives. In addition, we have applied this new criterion to train a class of linear-in-parameters models, including the commonly used linear regression model, random vector functional link network, and broad learning system as special cases. Experimental results on various datasets are reported to demonstrate the desirable performance of the proposed methods to perform regression tasks with contaminated data. [ABSTRACT FROM AUTHOR]

Correntropy is a local similarity measure defined in kernel space, hence can combat large outliers in robust signal processing and machine learning. So far, many robust learning algorithms have been developed under the maximum correntropy criterion (MCC), among which, a Gaussian kernel is generally used in correntropy. To further improve the learning performance, in this paper we propose the concept of mixture correntropy, which uses the mixture of two Gaussian functions as the kernel function. Some important properties of the mixture correntropy are presented. Applications of the maximum mixture correntropy criterion (MMCC) to extreme learning machine (ELM) and kernel adaptive filtering (KAF) for function approximation and data regression are also studied. Experimental results show that the learning algorithms under MMCC can perform very well and achieve better performance than the conventional MCC based algorithms as well as several other state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]

Constrained adaptive filtering algorithms have been extensively studied in many applications. Most existing constrained adaptive filtering algorithms are developed under the mean square error (MSE) criterion, which is an ideal optimality criterion under Gaussian noises. This assumption however fails to model the behavior of non-Gaussian noises found in practice. Motivated by the robustness and simplicity of maximum correntropy criterion (MCC) for non-Gaussian impulsive noises, this paper proposes a new adaptive filtering algorithm called constrained maximum correntropy criterion (CMCC). Specifically, CMCC incorporates a linear constraint into a MCC filter to solve a constrained optimization problem explicitly. The proposed adaptive filtering algorithm is easy to implement, has low computational complexity, and can significantly outperform those MSE based constrained adaptive algorithms in heavy-tailed impulsive noises. Additionally, the mean square convergence behaviors are studied under energy conservation relation, and a sufficient condition to ensure the mean square convergence and the steady-state mean square deviation (MSD) of the CMCC algorithm are obtained. Simulation results confirm the theoretical predictions under both Gaussian and non-Gaussian noises, and demonstrate the excellent performance of the novel algorithm by comparing it with other conventional methods. [ABSTRACT FROM AUTHOR]

Nonlinear similarity measures defined in kernel space, such as correntropy, can extract higher order statistics of data and offer potentially significant performance improvement over their linear counterparts especially in non Gaussian signal processing and machine learning. In this paper, we propose a new similarity measure in kernel space, called the kernel risk-sensitive loss (KRSL), and provide some important properties. We apply the KRSL to adaptive filtering and investigate the robustness, and then develop the MKRSL algorithm and analyze the mean square convergence performance. Compared with correntropy, the KRSL can offer a more efficient performance surface, thereby enabling a gradient-based method to achieve faster convergence speed and higher accuracy while still maintaining the robustness to outliers. Theoretical analysis results and superior performance of the new algorithm are confirmed by simulation. [ABSTRACT FROM PUBLISHER]

The unscented transformation (UT) is an efficient method to solve the state estimation problem for a non-linear dynamic system, utilising a derivative-free higher-order approximation by approximating a Gaussian distribution rather than approximating a non-linear function. Applying the UT to a Kalman filter type estimator leads to the well-known unscented Kalman filter (UKF). Although the UKF works very well in Gaussian noises, its performance may deteriorate significantly when the noises are non-Gaussian, especially when the system is disturbed by some heavy-tailed impulsive noises. To improve the robustness of the UKF against impulsive noises, a new filter for non-linear systems is proposed in this work, namely the maximum correntropy unscented filter (MCUF). In MCUF, the UT is applied to obtain the prior estimates of the state and covariance matrix, and a robust statistical linearisation regression based on the maximum correntropy criterion is then used to obtain the posterior estimates of the state and covariance matrix. The satisfying performance of the new algorithm is confirmed by two illustrative examples. [ABSTRACT FROM AUTHOR]

Traditional Kalman filter (KF) is derived under the well-known minimum mean square error (MMSE) criterion, which is optimal under Gaussian assumption. However, when the signals are non-Gaussian, especially when the system is disturbed by some heavy-tailed impulsive noises, the performance of KF will deteriorate seriously. To improve the robustness of KF against impulsive noises, we propose in this work a new Kalman filter, called the maximum correntropy Kalman filter (MCKF), which adopts the robust maximum correntropy criterion (MCC) as the optimality criterion, instead of using the MMSE. Similar to the traditional KF, the state mean vector and covariance matrix propagation equations are used to give prior estimations of the state and covariance matrix in MCKF. A novel fixed-point algorithm is then used to update the posterior estimations. A sufficient condition that guarantees the convergence of the fixed-point algorithm is also given. Illustration examples are presented to demonstrate the effectiveness and robustness of the new algorithm. [ABSTRACT FROM AUTHOR]

As a robust nonlinear similarity measure in kernel space, correntropy has received increasing attention in domains of machine learning and signal processing. In particular, the maximum correntropy criterion (MCC) has recently been successfully applied in robust regression and filtering. The default kernel function in correntropy is the Gaussian kernel, which is, of course, not always the best choice. In this paper, we propose a generalized correntropy that adopts the generalized Gaussian density (GGD) function as the kernel, and present some important properties. We further propose the generalized maximum correntropy criterion (GMCC) and apply it to adaptive filtering. An adaptive algorithm, called the GMCC algorithm, is derived, and the stability problem and steady-state performance are studied. We show that the proposed algorithm is very stable and can achieve zero probability of divergence (POD). Simulation results confirm the theoretical expectations and demonstrate the desirable performance of the new algorithm. [ABSTRACT FROM AUTHOR]

Kernel adaptive filters (KAF) are a class of powerful nonlinear filters developed in Reproducing Kernel Hilbert Space (RKHS). The Gaussian kernel is usually the default kernel in KAF algorithms, but selecting the proper kernel size (bandwidth) is still an open important issue especially for learning with small sample sizes. In previous research, the kernel size was set manually or estimated in advance by Silverman׳s rule based on the sample distribution. This study aims to develop an online technique for optimizing the kernel size of the kernel least mean square (KLMS) algorithm. A sequential optimization strategy is proposed, and a new algorithm is developed, in which the filter weights and the kernel size are both sequentially updated by stochastic gradient algorithms that minimize the mean square error (MSE). Theoretical results on convergence are also presented. The excellent performance of the new algorithm is confirmed by simulations on static function estimation, short term chaotic time series prediction and real world Internet traffic prediction. [ABSTRACT FROM AUTHOR]

Sparse adaptive filtering algorithms are utilized to exploit system sparsity as well as to mitigate interferences in many applications such as channel estimation and system identification. In order to improve the robustness of the sparse adaptive filtering, a novel adaptive filter is developed in this work by incorporating a correntropy-induced metric (CIM) constraint into the least logarithmic absolute difference (LLAD) algorithm. The CIM as an $$l_{0}$$ -norm approximation exerts a zero attraction, and hence, the LLAD algorithm performs well with robustness against impulsive noises. Numerical simulation results show that the proposed algorithm may achieve much better performance than other robust and sparse adaptive filtering algorithms such as the least mean p-power algorithm with $$l_{1}$$ -norm or reweighted $$l_{1}$$ -norm constraints. [ABSTRACT FROM AUTHOR]

A sequential extreme learning machine incorporating a noise compensation scheme via an information measure is developed. In this design, the computationally simple extreme learning machine architecture is maintained while survival error information potential function provides a mechanism for noise compensation. The error compensation is updated online via an error codebook design where an error tolerant and stable solution is obtained. The developed method is tested on chaotic time sequence as well as benchmark data sets. Experimental results show potential applications for the developed method. [ABSTRACT FROM AUTHOR]

Abstract: As an alternative adaptation criterion, the minimum error entropy (MEE) criterion has been receiving increasing attention due to its successful use in, especially, nonlinear and non-Gaussian signal processing. In this paper, we study the application of error entropy minimization to kernel adaptive filtering, a new and promising technique that implements the conventional linear adaptive filters in reproducing kernel Hilbert space (RKHS) and obtains the nonlinear adaptive filters in original input space. The kernel minimum error entropy (KMEE) algorithm is derived, which is essentially a generalized stochastic information gradient (SIG) algorithm in RKHS. The computational complexity of KMEE is just similar to the kernel affine projection algorithm (KAPA). We also utilize the quantization approach to constrain the network size growth, and develop the quantized KMEE (QKMEE) algorithm. Further, we analyze the mean square convergence of KMEE. The energy conservation relation is derived and a sufficient condition that ensures the mean square convergence is obtained. The performance of the new algorithm is demonstrated in nonlinear system identification and short-term chaotic time series prediction. [Copyright &y& Elsevier]

Abstract: This paper presents a quantized kernel least mean square algorithm with a fixed memory budget, named QKLMS-FB. In order to deal with the growing support inherent in online kernel methods, the proposed algorithm utilizes a pruning criterion, called significance measure, based on a weighted contribution of the existing data centers. The basic idea of the proposed methodology is to discard the center with the smallest influence on the whole system, when a new sample is included in the dictionary. The significance measure can be updated recursively at each step which is suitable for online operation. Furthermore, the proposed methodology does not need any a priori knowledge about the data and its computational complexity is linear with the center number. Experiments show that the proposed algorithm successfully prunes the least “significant” centers and preserves the important ones, resulting in a compact KLMS model with little loss in accuracy. [Copyright &y& Elsevier]

Permutation entropy (PE) has been recently suggested as a novel measure to characterize the complexity of nonlinear time series. In this paper, we propose a simple method to address some of PE's limitations, mainly its inability to differentiate between distinct patterns of a certain motif and the sensitivity of patterns close to the noise floor. The method relies on the fact that patterns may be too disparate in amplitudes and variances and proceeds by assigning weights for each extracted vector when computing the relative frequencies associated with every motif. Simulations were conducted over synthetic and real data for a weighting scheme inspired by the variance of each pattern. Results show better robustness and stability in the presence of higher levels of noise, in addition to a distinctive ability to extract complexity information from data with spiky features or having abrupt changes in magnitude. [ABSTRACT FROM AUTHOR]

Recent studies suggest that the minimum error entropy (MEE) criterion can outperform the traditional mean square error criterion in supervised machine learning, especially in nonlinear and non-Gaussian situations. In practice, however, one has to estimate the error entropy from the samples since in general the analytical evaluation of error entropy is not possible. By the Parzen windowing approach, the estimated error entropy converges asymptotically to the entropy of the error plus an independent random variable whose probability density function (PDF) corresponds to the kernel function in the Parzen method. This quantity of entropy is called the smoothed error entropy, and the corresponding optimality criterion is named the smoothed MEE (SMEE) criterion. In this paper, we study theoretically the SMEE criterion in supervised machine learning where the learning machine is assumed to be nonparametric and universal. Some basic properties are presented. In particular, we show that when the smoothing factor is very small, the smoothed error entropy equals approximately the true error entropy plus a scaled version of the Fisher information of error. We also investigate how the smoothing factor affects the optimal solution. In some special situations, the optimal solution under the SMEE criterion does not change with increasing smoothing factor. In general cases, when the smoothing factor tends to infinity, minimizing the smoothed error entropy will be approximately equivalent to minimizing error variance, regardless of the conditional PDF and the kernel. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, we study the mean square convergence of the kernel least mean square (KLMS). The fundamental energy conservation relation has been established in feature space. Starting from the energy conservation relation, we carry out the mean square convergence analysis and obtain several important theoretical results, including an upper bound on step size that guarantees the mean square convergence, the theoretical steady-state excess mean square error (EMSE), an optimal step size for the fastest convergence, and an optimal kernel size for the fastest initial convergence. Monte Carlo simulation results agree with the theoretical analysis very well. [Copyright &y& Elsevier]

Abstract: In this paper, a novel extended kernel recursive least squares algorithm is proposed combining the kernel recursive least squares algorithm and the Kalman filter or its extensions to estimate or predict signals. Unlike the extended kernel recursive least squares (Ex-KRLS) algorithm proposed by Liu, the state model of our algorithm is still constructed in the original state space and the hidden state is estimated using the Kalman filter. The measurement model used in hidden state estimation is learned by the kernel recursive least squares algorithm (KRLS) in reproducing kernel Hilbert space (RKHS). The novel algorithm has more flexible state and noise models. We apply this algorithm to vehicle tracking and the nonlinear Rayleigh fading channel tracking, and compare the tracking performances with other existing algorithms. [Copyright &y& Elsevier]

In this article, a novel normal-skew mixture (NSM) distribution is presented to model the normal and/or heavy-tailed and/or skew nonstationary distributed noises. The NSM distribution can be formulated as a hierarchically Gaussian presentation by leveraging a Bernoulli distributed random variable. Based on this, a novel robust Kalman filtering framework can be developed utilizing the variational Bayesian method, where the one-step prediction and measurement-likelihood densities are modeled as NSM distributions. For implementation, several exemplary robust Kalman filters (KFs) are derived based on some specific cases of NSM distribution. The relationships between some existing robust KFs and the presented framework are also revealed. The superiority of the proposed robust Kalman filtering framework is validated by a target tracking simulation example. [ABSTRACT FROM AUTHOR]

The minimum error entropy (MEE) criterion has been receiving increasing attention due to its promising perspectives for applications in signal processing and machine learning. In the context of Bayesian estimation, the MEE criterion is concerned with the estimation of a certain random variable based on another random variable, so that the error's entropy is minimized. Several theoretical results on this topic have been reported. In this work, we present some further results on the MEE estimation. The contributions are twofold: (1) we extend a recent result on the minimum entropy of a mixture of unimodal and symmetric distributions to a more general case, and prove that if the conditional distributions are generalized uniformly dominated (GUD), the dominant alignment will be the MEE estimator; (2) we show by examples that the MEE estimator (not limited to singular cases) may be non-unique even if the error distribution is restricted to zero-mean (unbiased). [ABSTRACT FROM AUTHOR]

Recently, the information potential (IP) of order \alpha, defined as the argument of the log in the \alpha-order Renyi entropy, has been successfully used as an information theoretic criterion for supervised adaptive system training. In this paper, we use the survival function (or equivalently the distribution function) of an absolute value transformed random variable to define a new information potential, named the survival information potential (SIP). Compared with the IP, the SIP has some advantages, such as validity in a wide range of distributions, robustness, and the simplicity in computation. The properties of SIP and a simple formula for computing the empirical SIP are given in the paper. Finally, the SIP criterion is applied in adaptive system training, and simulation examples on FIR adaptive filtering, kernel adaptive filtering, and time delay neural networks (TDNNs) training are presented to demonstrate the performance. [ABSTRACT FROM PUBLISHER]

This paper presents a parameterized version of the stochastic information gradient (SIG) algorithm, in which the error distribution is modeled by generalized Gaussian density (GGD), with location, shape, and dispersion parameters. Compared with the kernel-based SIG (SIG-Kernel) algorithm, the GGD-based SIG (SIG-GGD) algorithm does not involve kernel width selection. If the error is zero-mean, the SIG-GGD algorithm will become the least mean p-power (LMP) algorithm with adaptive order and variable step-size. Due to its well matched density estimation and automatic switching capability, the proposed algorithm is favorably in line with existing algorithms. [ABSTRACT FROM AUTHOR]

Abstract: The minimum mean-square error (MMSE) and minimum error entropy (MEE) are two important criteria in the estimation related problems. The MMSE can be viewed as a robust MEE criterion in the minimax sense, as its minimization is equivalent to minimizing an upper bound (the maximum value) of the error entropy. This note gives a new and more meaningful interpretation on the robustness of MMSE for problems in which there exists uncertainty in the probability model. It is shown that the MMSE estimator imposes an upper bound on error entropy for the true model. The upper bound consists of two terms. The first term quantifies the “MMSE performance” under nominal conditions, and the second term measures the “distance” between the true and nominal models. This robustness property is parallel to that of the risk-sensitive estimation. Illustration examples are included to confirm the robustness of MMSE. [Copyright &y& Elsevier]

Abstract: The maximum mutual information (MaxMI) criterion is used as the adaptation cost for the adaptive filtering. This criterion is robust to measure distortions, and has strong connection with traditional mean-square error (MSE) criterion. Under Gaussian assumption, the closed-form solution of the finite impulse response (FIR) filter is obtained. Further, based on the kernel density estimation, the stochastic mutual information gradient (SMIG) algorithm is derived. Simulation results emphasize the robustness of this new algorithm. [Copyright &y& Elsevier]

Recently, the Kalman filter algorithm based on the minimum error entropy criterion has been presented to outperform the conventional Kalman filter in the case of impulsive noise. In practice, it may be unstable in numerical calculation. This paper proposes a robust algorithm to solve the problem of instability in numerical calculation. The convergence and stability of the algorithm are verified by the performance analysis and simulations. In the mean square behavior analysis, we propose a method to estimate the steady-state errors. Simulations show that the experimental steady-state errors of the algorithms agreed with the theoretical values. [ABSTRACT FROM AUTHOR]

The spiking neural network (SNN) is regarded as a promising candidate to deal with the great challenges presented by current machine learning techniques, including the high energy consumption induced by deep neural networks. However, there is still a great gap between SNNs and the online meta-learning performance of artificial neural networks. Importantly, existing spike-based online meta-learning models do not target the robust learning based on spatio-temporal dynamics and superior machine learning theory. In this invited article, we propose a novel spike-based framework with minimum error entropy, called MeMEE, using the entropy theory to establish the gradient-based online meta-learning scheme in a recurrent SNN architecture. We examine the performance based on various types of tasks, including autonomous navigation and the working memory test. The experimental results show that the proposed MeMEE model can effectively improve the accuracy and the robustness of the spike-based meta-learning performance. More importantly, the proposed MeMEE model emphasizes the application of the modern information theoretic learning approach on the state-of-the-art spike-based learning algorithms. Therefore, in this invited paper, we provide new perspectives for further integration of advanced information theory in machine learning to improve the learning performance of SNNs, which could be of great merit to applied developments with spike-based neuromorphic systems. [ABSTRACT FROM AUTHOR]

Although multi-view clustering has received widespread attention due to its far superior performance to single-view clustering, it still faces the following issues: (1) high computational cost, considering the introduction of multi-view information, reduces the clustering efficiency greatly; (2) complex noises and outliers, existed in real-world data, pose a huge challenge to the robustness of clustering algorithms. Currently, how to increase the efficiency and robustness has become two important issues of multi-view clustering. To cope with the above issues, an efficient correntropy-based multi-view clustering algorithm (ECMC) is proposed in this paper, which can not only improve clustering efficiency by constructing embedded anchor graph and utilizing nonnegative matrix factorization (NMF), but also enhance the robustness by exploring correntropy to suppress various noises and outliers. To further improve clustering efficiency, one of the factors of NMF is constrained to be an indicator matrix instead of a traditional non-negative matrix, so that the categories of samples can be obtained directly without any extra operation. Subsequently, a novel half-quadratic-based strategy is proposed to optimize the non-convex objective function of ECMC. Finally, extensive experiments on eight real-world datasets and eighteen noisy datasets show that ECMC can guarantee faster speed and better robustness than other state-of-the-art multi-view clustering algorithms. [ABSTRACT FROM AUTHOR]

There is a recent trend to leverage the power of graph neural networks (GNNs) for brain-network based psychiatric diagnosis, which, in turn, also motivates an urgent need for psychiatrists to fully understand the decision behavior of the used GNNs. However, most of the existing GNN explainers are either post-hoc in which another interpretive model needs to be created to explain a well-trained GNN, or do not consider the causal relationship between the extracted explanation and the decision, such that the explanation itself contains spurious correlations and suffers from weak faithfulness. In this work, we propose a granger causality-inspired graph neural network (CI-GNN), a built-in interpretable model that is able to identify the most influential subgraph (i.e., functional connectivity within brain regions) that is causally related to the decision (e.g., major depressive disorder patients or healthy controls), without the training of an auxillary interpretive network. CI-GNN learns disentangled subgraph-level representations α and β that encode, respectively, the causal and non-causal aspects of original graph under a graph variational autoencoder framework, regularized by a conditional mutual information (CMI) constraint. We theoretically justify the validity of the CMI regulation in capturing the causal relationship. We also empirically evaluate the performance of CI-GNN against three baseline GNNs and four state-of-the-art GNN explainers on synthetic data and three large-scale brain disease datasets. We observe that CI-GNN achieves the best performance in a wide range of metrics and provides more reliable and concise explanations which have clinical evidence. The source code and implementation details of CI-GNN are freely available at GitHub repository (https://github.com/ZKZ-Brain/CI-GNN/). [ABSTRACT FROM AUTHOR]

The normalized subband adaptive filtering algorithm provides fast convergence rate for colored input signals as compared to the normalized least mean square algorithm, but it suffers from a poor convergence issue in the α -stable noise. In light of this, the normalized subband p-norm (NSPN) algorithm, which is based on the least mean p-power error (MPE) criterion, is proposed in this study. This technique is not only robust against impulsive noise samples, but it also maintains a fast convergence rate when colored input signals are used. In addition to this, we develop both the steady-state and the transient models of the NSPN algorithm and provide some insights. Then, in order to solve the problem of making a choice regarding the order p in the NSPN algorithm, we design an autonomous system and come up with the NSPN algorithm with a variable p-norm (VP-NSPN). In addition, we offer the TFC-based VP-NSPN algorithm with a fast convergence rate and low steady-state misadjustment simultaneously by making use of the tap-weights feedback-based convex combination (TFC) scheme. In conclusion, simulation results on system identification and acoustic echo cancellation are undertaken in order to validate the superiority of the proposed algorithms and validate the usefulness of the theoretical analysis. [ABSTRACT FROM AUTHOR]

In this paper, robust first and second-order divided difference filtering algorithms based on correntropy are proposed, which not only retain the advantages of divided difference filters, but also exhibit robustness in the presence of non-Gaussian noises, especially when the measurements are contaminated by heavy-tailed noises. The proposed filters are then applied to the problem of ship positioning. In order to improve the accuracy and reliability of ship positioning, the positioning method combines the Dead Reckoning (DR) algorithm and the Global Positioning System (GPS). Experimental results of an illustrative example show the superior performance of the new algorithms when applied to ship positioning. [ABSTRACT FROM AUTHOR]

Pedestrian detection with high detection and localization accuracy is increasingly important for many practical applications. Due to the flexible structure of the human body, it is hard to train a template-based pedestrian detector that achieves a high detection rate and a good localization accuracy simultaneously. In this paper, we utilize human pose estimation to improve the detection and localization accuracy of pedestrian detection. We design two kinds of pose-indexed features that can considerably improve the discriminability of the detector. In addition to employing a two-stage pipeline to carry out these two tasks, we unify pose estimation and pedestrian detection into a cascaded decision forest in which they can cooperate sufficiently. To prevent irregular positive examples, such as truncated ones, from distracting the pedestrian detection and the pose regression, we clean the positive training data by realigning the bounding boxes and rejecting the wrong positive samples. Experimental results on the Caltech test dataset demonstrate the effectiveness of our proposed method. Our detector achieves 11.1% MR−2, outperforming all existing detectors without using the convolutional neural network (CNN). Moreover, our method can be assembled with other detectors based on CNNs to improve detection and localization performance. By collaborating with the recent CNN-based method, our detector achieves 5.5% MR−2 on the Caltech test dataset, outperforming the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Least-squares one-class support vector machine (LS-OCSVM) is one of the most popular methods to perform one-class classification tasks, in which only the data of a specific class are available to train the classification model. However, the learning performance of LS-OCSVM heavily relies on the effectiveness of a squared loss function, which is sensitive to outliers, resulting in the poor robustness of LS-OCSVM to deal with contaminated data. In this paper, the original optimization problem of LS-OCSVM is therefore reformulated with a recently proposed robust similarity measure, called multikernel correntropy, generating a multikernel correntropy based LS-OCSVM (MKCLS-OCSVM). To find the solution to the new optimization problem effectively, a dynamic optimization algorithm developed with the popular half-quadratic optimization technique is adopted to perform the optimization process. Meanwhile, the convergence and computational complexity of the developed optimization algorithm are analyzed from theoretical perspectives. To further facilitate the implementation of MKCLS-OCSVM, an operationally simple search strategy, inspired by the hunting behavior of humpback whales, is designed for parameters selection. Experimental results on various one-class classification tasks are reported to demonstrate the performance superiority of the proposed MKCLS-OCSVM in comparison with LS-OCSVM and other robust LS-OCSVM variants. [ABSTRACT FROM AUTHOR]

• Build feature layers with higher resolution, which can extract local fine information for small defects. • Attention mechanism is introduced into the FPN to improve the representation ability for small defects. • The features extracted on the small perceptual field are aggregated by the dilated residual block. Although object detection methods have got surprising performance in simple natural scenes, it is still a challenging task to apply them to complex scenes, especially when the detected objects are small which is common in defect detection tasks of industrial scenarios. Motivated by the higher resolution feature maps, the better detection performance on small objects, we increase the resolution of the feature layers in this paper, and introduce convolutional block attention module (CBAM) into the feature aggregation network to better integrate features at different scales and make better use of the information of small defects. For solving the problem of global context information loss caused by limiting the perceptual field by increasing the resolution of the feature layers, we use dilated convolution blocks to expand the perceptual field. Experimental results show that our method improves the mAP(0.5) on the Industrial Surface Defect Dataset from 88.3% in Yolov5 to 89.2%, mAP(0.75) on the DAGM 2007 increases from 76.56% in Yolov5 to 77.65%, mAP on small object increases by 2.6% on Industrial Surface Defect Dataset and 5.8% on DAGM 2007, which proves the effectiveness of our method for small defect detection. [ABSTRACT FROM AUTHOR]

Abstract Concept factorization (CF) technique is one of the most powerful approaches for feature learning, and has been successfully adopted in a wide range of practical applications such as data mining, computer vision, and information retrieval. Most existing concept factorization methods mainly minimize the square of the Euclidean distance, which is seriously sensitive to non-Gaussian noises or outliers in the data. To alleviate the adverse influence of this limitation, in this paper, a robust graph regularized concept factorization method, called correntropy based graph regularized concept factorization (GCCF), is proposed for clustering tasks. Specifically, based on the maximum correntropy criterion (MCC), GCCF is derived by incorporating the graph structure information into our proposed objective function. A half-quadratic optimization technique is adopted to solve the non-convex objective function of the GCCF method effectively. In addition, algorithm analysis of GCCF is studied. Extensive experiments on real world datasets demonstrate that the proposed GCCF method outperforms seven competing methods for clustering applications. [ABSTRACT FROM AUTHOR]

Support vector data description (SVDD) has received a lot of attention due to its outstanding performance to perform one-class classification or novelty detection tasks. However, the same weight is directly imposed on all slack variables in the process of modeling, which may result in degraded learning performance when training data are contaminated by some outliers or mislabeled observations. In this paper, an extended SVDD model is therefore proposed by reformulating the original optimization problem of SVDD with a mixed exponential loss function. Since this loss function can emphasize the importance of the samples that tend to be the target class, and weaken the influence of those tending to be outliers, it can be viewed as a weighted SVDD. However, the weights in the new model are automatically calculated rather than being calculated in advance using some specific manners. To solve the optimization problem of the proposed model effectively, the half-quadratic optimization technique has been adopted to perform the optimization, generating a dynamic optimization algorithm. Meanwhile, the convergence and computational complexity of this dynamic optimization algorithm are analyzed from a theoretical respective. Experimental results on a synthetic data set and some publicly available real world data sets are reported to demonstrate the performance superiority of the new method in comparison with SVDD and other competitive SVDD variants. [ABSTRACT FROM AUTHOR]

The convergence of nonlinear Kalman filters has conventionally been analyzed in terms of the estimation error. In this paper, we present a new method for investigating the convergence performance of a class of nonlinear Kalman filters based on deterministic sampling. The systems considered here are described by nonlinear state equations with linear measurements. For this type of systems, our proposed convergence analysis is performed using “innovation”, which is defined as the error between the measurement and its prediction. Specifically, we obtain a linear relationship between the innovation and the estimation error, and derive a set of sufficient conditions that ensures the convergence of nonlinear Kalman filters. Compared with the conventional convergence analysis method based on the estimation error, the proposed innovation-based method can obtain sufficient conditions for convergence more directly and readily. Simulation results show that the convergence of innovation generates the convergence of nonlinear Kalman filters. [ABSTRACT FROM AUTHOR]

This paper presents a new normalized least mean square (NLMS) algorithm for sparse system identification where the input signal is corrupted by white measurement noise. The proposed algorithm, which is called bias-compensated zero attracting NLMS (BC-ZA-NLMS) algorithm, introduces the bias-compensation vector to get rid of the bias resulting from noisy input and introduces an l 1 -norm penalty in the cost function of the NLMS algorithm to make full use of the special property of the sparse system. In addition, to address the time variant sparsity, the bias-compensated reweight ZA-NLMS (BC-RZA-NLMS)) algorithm is also proposed, where the l 1 -norm penalty in the cost function of BC-ZA-NLMS algorithm is replaced by a log-sum function. Owing to the zero attractors in update equation, the proposed algorithms are superior to the conventional NLMS and bias-compensated NLMS (BC-NLMS) algorithms in the application of identifying the sparse system. A transient analysis of the proposed algorithms is also derived, which is able to accurately predict the behaviors of proposed algorithms. In addition, a stability analysis is introduced. Monte Carlo (MC) simulations are conducted to demonstrate the advantage of the proposed algorithms and to validate the theoretical results. [ABSTRACT FROM AUTHOR]

In this paper, a robust echo state network with correntropy induced loss function (CLF) is presented. CLF is robust to outliers through the mechanism of correntropy which is widely applied in information theoretic learning. The proposed method can improve the anti-noise capacity of echo state network and overcome its problem of being sensitive outliers which are prevalent in real-world tasks. The echo state network with CLF inherits the basic architecture of echo state network, but replaces the commonly used mean square error (MSE) criterion with CLF. The stochastic gradient descent method is adopted to optimize the objective function. The proposed method is subsequently verified in nonlinear system identification and chaotic time-series prediction. Experimental results demonstrate that our method is robust to outliers and outperforms the echo state networks with Bayesian regression and Huber loss function. [ABSTRACT FROM AUTHOR]

Total least squares (TLS) method has been widely used in errors-in-variables (EIV) modeling tasks in which both input and output data are disturbed by noises, and adaptive filtering algorithm using TLS has shown significantly superior performance to classical least squares (LS) method in EIV system. The TLS essentially extends the minimum mean square error (MMSE) criterion to EIV model, which, however, may work poorly when noise is non-Gaussian (especially heavy-tailed distribution). Recently, an information theoretic learning (ITL) based minimum total error entropy (MTEE) adaptive filtering algorithm has been proposed, which extends the minimum error entropy (MEE) criterion to EIV model and shows desirable performance in non-Gaussian noise environments. However, due to complex mathematical expression, MTEE is computationally expensive and difficult to carry out the theoretical analysis. In this paper, we propose a new ITL-based criterion called maximum total correntropy (MTC) and develop a gradient-based MTC adaptive filtering algorithm. We analyze theoretically the local stability and steady-state performance of the proposed algorithm. Simulation results confirm the theoretical analysis and show the superior performance of MTC in heavy-tailed noises. Further, simulation comparisons between MTC and MTEE are presented. Compared with the MTEE, the MTC is mathematically more tractable and computationally much simpler while achieving similar or even better performance. [ABSTRACT FROM AUTHOR]

As a robust adaptation criterion, the maximum correntropy criterion (MCC) has gained increased attention due to its successful application in the adaptation, especially in nonlinear and non-Gaussian situations. In this paper, the second-order Volterra (SOV) filter based on MCC is derived, which is called MCC-SOV. It combines the advantages of the SOV filter and MCC. However, similar to the conventional adaptive algorithm, the proposed MCC-SOV filter has a tradeoff between convergence rate and steady-state error. In order to solve this problem, we present a combination of the MCC-SOV filter by combining Volterra kernels, which is called CK-MCC-SOV. In addition, a weight transfer method is applied to improve the performance of the proposed filter in terms of the convergence rate and the steady-state error. Finally, simulations are carried out to demonstrate the advantages of the proposed filters. [ABSTRACT FROM AUTHOR]

Multi-view clustering (MVC) often provides superior effectiveness to single-view clustering due to the integration of information from diverse views. Nonetheless, existing MVC methods are limited to large-scale real-world data by the drawbacks of low efficiency and poor robustness. To address these issues, we propose a novel robust anchor-based MVC model via spectral embedded concept factorization (RAMCSF). RAMCSF builds anchor graphs to approximate full-sample graphs and decomposes these anchor graphs by concept factorization (CF). To improve the clustering effectiveness, factor matrices of CF are constrained as orthogonal matrices to reduce the freedom of decomposition, and a novel small-scale anchor-based spectral embedding is designed to explore the high-order neighbor relationships. To restrain complex noises distributed in real-world data, we employ correntropy to measure the error between the original data and the learned representation. Moreover, RAMCSF can get a clustering indicator matrix directly, avoiding additional post-processing and ensuring that changes in data dimensions have a limited impact on efficiency. The model is then optimized by a novel fast half-quadratic-based optimization strategy that combines the orthogonal properties and the traces of matrices. Extensive experiments indicate that RAMCSF can achieve higher efficiency and robustness while maintaining comparable effectiveness to other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Single-cell profiling is key to uncover the cellular heterogeneity and drives deep understanding of cell fate. In recent years, microfluidics has become an ideal tool for single-cell profiling owing to its benefits of high throughput and automation. Among various microfluidic platforms, microwell has the advantages of simple operation and easy integration with in situ analysis ability, making it an ideal technique for single-cell studies. Herein, recent advances of single-cell analysis based on microwell array chips are summarized. We first introduce the design and preparation of different microwell chips. Then microwell-based cell capture and lysis strategies are discussed. We finally focus on advanced microwell-based analysis of single-cell proteins, nucleic acids, and metabolites. The challenges and opportunities for the development of microwell-based single-cell analysis are also presented. [ABSTRACT FROM AUTHOR]

• Distributed optimization problems of consensus are examined by a frequency-domain analysis approach. • An optimized distributed protocol is proposed such that the nonuniform time-delay FDMSs with SF have better consensus performance. • The double-integral dynamics of delayed FDMSs with SF is considered. This paper addresses distributed optimization problems concerning consensus in delayed fractional-order double-integrator multi-agent systems (FDMSs). To start with, an optimized distributed protocol with state-fractional-order-derivative feedback (SF) is presented for delayed FDMSs. Then, the consensus problems are studied for the two kinds of delayed FDMSs with SF in the presence of symmetric time-delays over undirected network topology and asymmetric time-delays over directed network topology. Next, by the means of graph theory, matrix theory and frequency-domain analysis method, the sufficient conditions to guarantee consensus of delayed FDMSs with SF are derived. Compared to the traditional distributed protocol without SF, the proposed distributed optimization protocol with SF are taken into account to enable better consensus performance in delayed FDMSs with SF. Finally, numerical experiments are carried out to verify the feasibility of our theoretical results. [ABSTRACT FROM AUTHOR]

Accumulating evidence suggests that the brain is highly dynamic; thus, investigation of brain dynamics especially in brain connectivity would provide crucial information that stationary functional connectivity could miss. This study investigated temporal expressions of spatial modes within the default mode network (DMN), salience network (SN) and cognitive control network (CCN) using a reliable data-driven co-activation pattern (CAP) analysis in two independent data sets. We found enhanced CAP-to-CAP transitions of the SN in patients with MDD. Results suggested enhanced flexibility of this network in the patients. By contrast, we also found reduced spatial consistency and persistence of the DMN in the patients, indicating reduced variability and stability in individuals with MDD. In addition, the patients were characterized by prominent activation of mPFC. Moreover, further correlation analysis revealed that persistence and transitions of RCCN were associated with the severity of depression. Our findings suggest that functional connectivity in the patients may not be simply attenuated or potentiated, but just alternating faster or slower among more complex patterns. The aberrant temporal–spatial complexity of intrinsic fluctuations reflects functional diaschisis of resting-state networks as characteristic of patients with MDD. [ABSTRACT FROM AUTHOR]

To improve the robustness of subband adaptive filter (SAF) against impulsive interferences, we propose two modified SAF algorithms with an individual scale function for each subband, which are derived by maximizing correntropy-based cost function and minimizing logarithm-based cost function, respectively, called MCC-SAF and LC-SAF. Whenever the impulsive interference happens, the subband scale functions can sharply drop the step size, which eliminate the influence of outliers on the tap-weight vector update. Therefore, the proposed algorithms are robust against impulsive interferences and exhibit the faster convergence rate and better tracking capability than the sign SAF (SSAF) algorithm. Besides, in impulse-free interference environments, the proposed algorithms achieve similar convergence performance as the normalized SAF (NSAF) algorithm. Simulation results have demonstrated the performance of our proposed algorithms. [ABSTRACT FROM AUTHOR]