Search

The generative autoencoders, such as the variational autoencoders or the adversarial autoencoders, have achieved great success in lots of real-world applications, including image generation and signal communication. However, little concern has been devoted to their robustness during practical deployment. Due to the probabilistic latent structure, variational autoencoders (VAEs) may confront problems such as a mismatch between the posterior distribution of the latent and real data manifold, or discontinuity in the posterior distribution of the latent. This leaves a back door for malicious attackers to collapse VAEs from the latent space, especially in scenarios where the encoder and decoder are used separately, such as communication and compressed sensing. In this work, we provide the first study on the adversarial robustness of generative autoencoders in the latent space. Specifically, we empirically demonstrate the latent vulnerability of popular generative autoencoders through attacks in the latent space. We also evaluate the difference between variational autoencoders and their deterministic variants and observe that the latter performs better in latent robustness. Meanwhile, we identify a potential trade-off between the adversarial robustness and the degree of the disentanglement of the latent codes. Additionally, we also verify the feasibility of improvement for the latent robustness of generative autoencoders through adversarial training. In summary, we suggest concerning the adversarial latent robustness of the generative autoencoders, analyze several robustness-relative issues, and give some insights into a series of key challenges. [ABSTRACT FROM AUTHOR]

The multivariate autoregressive (MVAR) model is widely used in describing the dynamics of nonlinear systems, in which the estimates of model parameters and underlying states can be achieved by dual extended Kalman filter (DEKF). However, when the measurements are corrupted by complicated non-Gaussian noises, the DEKF based on the minimum mean-square error (MMSE) criterion may provide biased estimates. In the present article, we develop a novel dual Kalman-type filter, referred to as DEKF under minimum error entropy (MEE) with fiducial points (MEEFs-DEKF) to deal with the non-Gaussian noises. First, the equivalent state-space model and parameter-space model are presented based on the MVAR model. Then, an optimality criterion based on MEE with fiducial points (MEEFs) is applied in the batch-mode regression equation to improve the robustness. Finally, a fixed-point iteration algorithm gives the posterior estimates of state and parameter. Simulation results confirm that the proposed MEEF-DEKF can achieve excellent performance in various noises with different distributions, especially in heavy-tailed and multimodal noises. [ABSTRACT FROM AUTHOR]

To date, most linear and nonlinear Kalman filters (KFs) have been developed under the Gaussian assumption and the well-known minimum mean square error (MMSE) criterion. In order to improve the robustness with respect to impulsive (or heavy-tailed) non-Gaussian noises, the maximum correntropy criterion (MCC) has recently been used to replace the MMSE criterion in developing several robust Kalman-type filters. To deal with more complicated non-Gaussian noises such as noises from multimodal distributions, in this article, we develop a new Kalman-type filter, called minimum error entropy KF (MEE-KF), by using the minimum error entropy (MEE) criterion instead of the MMSE or MCC. Similar to the MCC-based KFs, the proposed filter is also an online algorithm with the recursive process, in which the propagation equations are used to give prior estimates of the state and covariance matrix, and a fixed-point algorithm is used to update the posterior estimates. In addition, the MEE extended KF (MEE-EKF) is also developed for performance improvement in the nonlinear situations. The high accuracy and strong robustness of MEE-KF and MEE-EKF are confirmed by experimental results. [ABSTRACT FROM AUTHOR]

Recently, maximum correntropy criterion (MCC) has been widely and successfully used in robust signal processing and machine learning, in which the correntropy is maximized instead of minimizing the popular mean square error (MSE) to improve the robustness with respect to outliers or impulsive noises. A lot of efforts have been devoted to derive different adaptive algorithms under MCC, but to date, little insight has been gained as to how the MCC solution will be influenced by outliers. In this paper, we investigate this problem and our focus is mainly on the parameter estimation of a simple linear errors-in-variables (EIVs) model with scalar variables. Under some conditions, we derive an upper bound on the absolute value of the estimation error and show that the MCC solution can get very close to the true value of the unknown parameter even with arbitrarily large outliers in both the input and output variables. Illustrative examples are provided to verify and clarify the theory. [ABSTRACT FROM AUTHOR]

Due to the effectiveness in learning the subspace structures, low-rank representation (LRR) and its variations have been widely applied in various fields, such as computer vision and pattern recognition. However, in real applications, it is a challenge to handle the complex noises. To address this problem, we propose a novel robust LRR method based on kernel risk-sensitive loss (KRSL) with high-order manifold constraint, called RHLRR, in which the KRSL is introduced to deal with the noises and the multiple hypergraph regularization term is used as a high order manifold constraint to effectively capture the locality, similarity and the intrinsic geometric information in data. Besides, an iterative algorithm based on the half-quadratic (HQ) and the accelerated block coordinate update (BCU) is developed. The experimental results demonstrate that the proposed method can outperform other state-of-the-art LRR variants. [ABSTRACT FROM AUTHOR]

The unscented Kalman filter (UKF) provides a powerful tool for power system forecasting-aided state estimation (FASE). However, when the power systems are affected by the abnormal operating situations, i.e., the non-Gaussian communication noises, sudden loads or state changes, and instrument failures, the original UKF based on the minimum mean square error (MMSE) criterion may suffer from performance degradation. In contrast to the MMSE criterion, the minimum error entropy (MEE) exhibits the robustness with respect to complex non-Gaussian disturbances. In this article, we develop a new unscented Kalman-type filter based on the MEE criterion, termed MEE-UKF. To derive the MEE-UKF, a statistical linearization approach is adopted in the augmented model such that the state and measurement errors are combined in the MEE cost function simultaneously. Then, a fixed-point iteration algorithm is used to recursively update the posterior estimates and covariance matrix. Apart from the impulsive noises, the MEE-UKF can deal with complex multimodal distribution noises in both process and measurement. The high accuracy and strong robustness of MEE-UKF are confirmed by the simulation results on IEEE 14, 30, and 57 bus test systems under different non-Gaussian disturbances. [ABSTRACT FROM AUTHOR]

Common spatial pattern (CSP) is a classic method commonly used in multichannel electroencephalogram (EEG) signal processing, which aims to extract effective features for binary classification by solving spatial filters that maximize the ratio of filtered dispersion between two classes. The aim of this paper is to improve the performance of the conventional CSP method, which will be badly influenced by noises. The recently proposed quantized minimum error entropy (QMEE) criterion is applied to structure a new objective function instead of the ${L} _{2}$ -norm in the conventional CSP. Quantization is utilized to reduce the computational complexity. The new objective function is optimized by a gradient-based iterative algorithm. The desirable performance of the QMEE-based CSP method, namely CSP-QMEE, is demonstrated with a toy example and two real EEG datasets, including Dataset IIb of the brain–computer interfaces (BCIs) Competition IV (three channels) and Dataset IIIa of the BCI Competition III (60 channels). The new method can achieve satisfactory performance compared to existing methods on all datasets. The promising results in this paper suggest that the CSP-QMEE may become a powerful tool for BCIs. [ABSTRACT FROM AUTHOR]

Minimum error entropy with fiducial points (MEEF) has received a lot of attention, due to its outstanding performance to curb the negative influence caused by non-Gaussian noises in the fields of machine learning and signal processing. However, the estimate of the information potential of MEEF involves a double summation operator based on all available error samples, which can result in large computational burden in many practical scenarios. In this paper, an efficient quantization method is therefore adopted to represent the primary set of error samples with a smaller subset, generating a quantized MEEF (QMEEF). Some basic properties of QMEEF are presented and proved from theoretical perspectives. In addition, we have applied this new criterion to train a class of linear-in-parameters models, including the commonly used linear regression model, random vector functional link network, and broad learning system as special cases. Experimental results on various datasets are reported to demonstrate the desirable performance of the proposed methods to perform regression tasks with contaminated data. [ABSTRACT FROM AUTHOR]

Correntropy is a local similarity measure defined in kernel space, hence can combat large outliers in robust signal processing and machine learning. So far, many robust learning algorithms have been developed under the maximum correntropy criterion (MCC), among which, a Gaussian kernel is generally used in correntropy. To further improve the learning performance, in this paper we propose the concept of mixture correntropy, which uses the mixture of two Gaussian functions as the kernel function. Some important properties of the mixture correntropy are presented. Applications of the maximum mixture correntropy criterion (MMCC) to extreme learning machine (ELM) and kernel adaptive filtering (KAF) for function approximation and data regression are also studied. Experimental results show that the learning algorithms under MMCC can perform very well and achieve better performance than the conventional MCC based algorithms as well as several other state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]

Constrained adaptive filtering algorithms have been extensively studied in many applications. Most existing constrained adaptive filtering algorithms are developed under the mean square error (MSE) criterion, which is an ideal optimality criterion under Gaussian noises. This assumption however fails to model the behavior of non-Gaussian noises found in practice. Motivated by the robustness and simplicity of maximum correntropy criterion (MCC) for non-Gaussian impulsive noises, this paper proposes a new adaptive filtering algorithm called constrained maximum correntropy criterion (CMCC). Specifically, CMCC incorporates a linear constraint into a MCC filter to solve a constrained optimization problem explicitly. The proposed adaptive filtering algorithm is easy to implement, has low computational complexity, and can significantly outperform those MSE based constrained adaptive algorithms in heavy-tailed impulsive noises. Additionally, the mean square convergence behaviors are studied under energy conservation relation, and a sufficient condition to ensure the mean square convergence and the steady-state mean square deviation (MSD) of the CMCC algorithm are obtained. Simulation results confirm the theoretical predictions under both Gaussian and non-Gaussian noises, and demonstrate the excellent performance of the novel algorithm by comparing it with other conventional methods. [ABSTRACT FROM AUTHOR]

Nonlinear similarity measures defined in kernel space, such as correntropy, can extract higher order statistics of data and offer potentially significant performance improvement over their linear counterparts especially in non Gaussian signal processing and machine learning. In this paper, we propose a new similarity measure in kernel space, called the kernel risk-sensitive loss (KRSL), and provide some important properties. We apply the KRSL to adaptive filtering and investigate the robustness, and then develop the MKRSL algorithm and analyze the mean square convergence performance. Compared with correntropy, the KRSL can offer a more efficient performance surface, thereby enabling a gradient-based method to achieve faster convergence speed and higher accuracy while still maintaining the robustness to outliers. Theoretical analysis results and superior performance of the new algorithm are confirmed by simulation. [ABSTRACT FROM PUBLISHER]

The unscented transformation (UT) is an efficient method to solve the state estimation problem for a non-linear dynamic system, utilising a derivative-free higher-order approximation by approximating a Gaussian distribution rather than approximating a non-linear function. Applying the UT to a Kalman filter type estimator leads to the well-known unscented Kalman filter (UKF). Although the UKF works very well in Gaussian noises, its performance may deteriorate significantly when the noises are non-Gaussian, especially when the system is disturbed by some heavy-tailed impulsive noises. To improve the robustness of the UKF against impulsive noises, a new filter for non-linear systems is proposed in this work, namely the maximum correntropy unscented filter (MCUF). In MCUF, the UT is applied to obtain the prior estimates of the state and covariance matrix, and a robust statistical linearisation regression based on the maximum correntropy criterion is then used to obtain the posterior estimates of the state and covariance matrix. The satisfying performance of the new algorithm is confirmed by two illustrative examples. [ABSTRACT FROM AUTHOR]

Traditional Kalman filter (KF) is derived under the well-known minimum mean square error (MMSE) criterion, which is optimal under Gaussian assumption. However, when the signals are non-Gaussian, especially when the system is disturbed by some heavy-tailed impulsive noises, the performance of KF will deteriorate seriously. To improve the robustness of KF against impulsive noises, we propose in this work a new Kalman filter, called the maximum correntropy Kalman filter (MCKF), which adopts the robust maximum correntropy criterion (MCC) as the optimality criterion, instead of using the MMSE. Similar to the traditional KF, the state mean vector and covariance matrix propagation equations are used to give prior estimations of the state and covariance matrix in MCKF. A novel fixed-point algorithm is then used to update the posterior estimations. A sufficient condition that guarantees the convergence of the fixed-point algorithm is also given. Illustration examples are presented to demonstrate the effectiveness and robustness of the new algorithm. [ABSTRACT FROM AUTHOR]

As a robust nonlinear similarity measure in kernel space, correntropy has received increasing attention in domains of machine learning and signal processing. In particular, the maximum correntropy criterion (MCC) has recently been successfully applied in robust regression and filtering. The default kernel function in correntropy is the Gaussian kernel, which is, of course, not always the best choice. In this paper, we propose a generalized correntropy that adopts the generalized Gaussian density (GGD) function as the kernel, and present some important properties. We further propose the generalized maximum correntropy criterion (GMCC) and apply it to adaptive filtering. An adaptive algorithm, called the GMCC algorithm, is derived, and the stability problem and steady-state performance are studied. We show that the proposed algorithm is very stable and can achieve zero probability of divergence (POD). Simulation results confirm the theoretical expectations and demonstrate the desirable performance of the new algorithm. [ABSTRACT FROM AUTHOR]

Kernel adaptive filters (KAF) are a class of powerful nonlinear filters developed in Reproducing Kernel Hilbert Space (RKHS). The Gaussian kernel is usually the default kernel in KAF algorithms, but selecting the proper kernel size (bandwidth) is still an open important issue especially for learning with small sample sizes. In previous research, the kernel size was set manually or estimated in advance by Silverman׳s rule based on the sample distribution. This study aims to develop an online technique for optimizing the kernel size of the kernel least mean square (KLMS) algorithm. A sequential optimization strategy is proposed, and a new algorithm is developed, in which the filter weights and the kernel size are both sequentially updated by stochastic gradient algorithms that minimize the mean square error (MSE). Theoretical results on convergence are also presented. The excellent performance of the new algorithm is confirmed by simulations on static function estimation, short term chaotic time series prediction and real world Internet traffic prediction. [ABSTRACT FROM AUTHOR]

Sparse adaptive filtering algorithms are utilized to exploit system sparsity as well as to mitigate interferences in many applications such as channel estimation and system identification. In order to improve the robustness of the sparse adaptive filtering, a novel adaptive filter is developed in this work by incorporating a correntropy-induced metric (CIM) constraint into the least logarithmic absolute difference (LLAD) algorithm. The CIM as an $$l_{0}$$ -norm approximation exerts a zero attraction, and hence, the LLAD algorithm performs well with robustness against impulsive noises. Numerical simulation results show that the proposed algorithm may achieve much better performance than other robust and sparse adaptive filtering algorithms such as the least mean p-power algorithm with $$l_{1}$$ -norm or reweighted $$l_{1}$$ -norm constraints. [ABSTRACT FROM AUTHOR]

A sequential extreme learning machine incorporating a noise compensation scheme via an information measure is developed. In this design, the computationally simple extreme learning machine architecture is maintained while survival error information potential function provides a mechanism for noise compensation. The error compensation is updated online via an error codebook design where an error tolerant and stable solution is obtained. The developed method is tested on chaotic time sequence as well as benchmark data sets. Experimental results show potential applications for the developed method. [ABSTRACT FROM AUTHOR]

Abstract: As an alternative adaptation criterion, the minimum error entropy (MEE) criterion has been receiving increasing attention due to its successful use in, especially, nonlinear and non-Gaussian signal processing. In this paper, we study the application of error entropy minimization to kernel adaptive filtering, a new and promising technique that implements the conventional linear adaptive filters in reproducing kernel Hilbert space (RKHS) and obtains the nonlinear adaptive filters in original input space. The kernel minimum error entropy (KMEE) algorithm is derived, which is essentially a generalized stochastic information gradient (SIG) algorithm in RKHS. The computational complexity of KMEE is just similar to the kernel affine projection algorithm (KAPA). We also utilize the quantization approach to constrain the network size growth, and develop the quantized KMEE (QKMEE) algorithm. Further, we analyze the mean square convergence of KMEE. The energy conservation relation is derived and a sufficient condition that ensures the mean square convergence is obtained. The performance of the new algorithm is demonstrated in nonlinear system identification and short-term chaotic time series prediction. [Copyright &y& Elsevier]

Abstract: This paper presents a quantized kernel least mean square algorithm with a fixed memory budget, named QKLMS-FB. In order to deal with the growing support inherent in online kernel methods, the proposed algorithm utilizes a pruning criterion, called significance measure, based on a weighted contribution of the existing data centers. The basic idea of the proposed methodology is to discard the center with the smallest influence on the whole system, when a new sample is included in the dictionary. The significance measure can be updated recursively at each step which is suitable for online operation. Furthermore, the proposed methodology does not need any a priori knowledge about the data and its computational complexity is linear with the center number. Experiments show that the proposed algorithm successfully prunes the least “significant” centers and preserves the important ones, resulting in a compact KLMS model with little loss in accuracy. [Copyright &y& Elsevier]

Permutation entropy (PE) has been recently suggested as a novel measure to characterize the complexity of nonlinear time series. In this paper, we propose a simple method to address some of PE's limitations, mainly its inability to differentiate between distinct patterns of a certain motif and the sensitivity of patterns close to the noise floor. The method relies on the fact that patterns may be too disparate in amplitudes and variances and proceeds by assigning weights for each extracted vector when computing the relative frequencies associated with every motif. Simulations were conducted over synthetic and real data for a weighting scheme inspired by the variance of each pattern. Results show better robustness and stability in the presence of higher levels of noise, in addition to a distinctive ability to extract complexity information from data with spiky features or having abrupt changes in magnitude. [ABSTRACT FROM AUTHOR]

Recent studies suggest that the minimum error entropy (MEE) criterion can outperform the traditional mean square error criterion in supervised machine learning, especially in nonlinear and non-Gaussian situations. In practice, however, one has to estimate the error entropy from the samples since in general the analytical evaluation of error entropy is not possible. By the Parzen windowing approach, the estimated error entropy converges asymptotically to the entropy of the error plus an independent random variable whose probability density function (PDF) corresponds to the kernel function in the Parzen method. This quantity of entropy is called the smoothed error entropy, and the corresponding optimality criterion is named the smoothed MEE (SMEE) criterion. In this paper, we study theoretically the SMEE criterion in supervised machine learning where the learning machine is assumed to be nonparametric and universal. Some basic properties are presented. In particular, we show that when the smoothing factor is very small, the smoothed error entropy equals approximately the true error entropy plus a scaled version of the Fisher information of error. We also investigate how the smoothing factor affects the optimal solution. In some special situations, the optimal solution under the SMEE criterion does not change with increasing smoothing factor. In general cases, when the smoothing factor tends to infinity, minimizing the smoothed error entropy will be approximately equivalent to minimizing error variance, regardless of the conditional PDF and the kernel. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, we study the mean square convergence of the kernel least mean square (KLMS). The fundamental energy conservation relation has been established in feature space. Starting from the energy conservation relation, we carry out the mean square convergence analysis and obtain several important theoretical results, including an upper bound on step size that guarantees the mean square convergence, the theoretical steady-state excess mean square error (EMSE), an optimal step size for the fastest convergence, and an optimal kernel size for the fastest initial convergence. Monte Carlo simulation results agree with the theoretical analysis very well. [Copyright &y& Elsevier]

Abstract: In this paper, a novel extended kernel recursive least squares algorithm is proposed combining the kernel recursive least squares algorithm and the Kalman filter or its extensions to estimate or predict signals. Unlike the extended kernel recursive least squares (Ex-KRLS) algorithm proposed by Liu, the state model of our algorithm is still constructed in the original state space and the hidden state is estimated using the Kalman filter. The measurement model used in hidden state estimation is learned by the kernel recursive least squares algorithm (KRLS) in reproducing kernel Hilbert space (RKHS). The novel algorithm has more flexible state and noise models. We apply this algorithm to vehicle tracking and the nonlinear Rayleigh fading channel tracking, and compare the tracking performances with other existing algorithms. [Copyright &y& Elsevier]

In this article, a novel normal-skew mixture (NSM) distribution is presented to model the normal and/or heavy-tailed and/or skew nonstationary distributed noises. The NSM distribution can be formulated as a hierarchically Gaussian presentation by leveraging a Bernoulli distributed random variable. Based on this, a novel robust Kalman filtering framework can be developed utilizing the variational Bayesian method, where the one-step prediction and measurement-likelihood densities are modeled as NSM distributions. For implementation, several exemplary robust Kalman filters (KFs) are derived based on some specific cases of NSM distribution. The relationships between some existing robust KFs and the presented framework are also revealed. The superiority of the proposed robust Kalman filtering framework is validated by a target tracking simulation example. [ABSTRACT FROM AUTHOR]

The minimum error entropy (MEE) criterion has been receiving increasing attention due to its promising perspectives for applications in signal processing and machine learning. In the context of Bayesian estimation, the MEE criterion is concerned with the estimation of a certain random variable based on another random variable, so that the error's entropy is minimized. Several theoretical results on this topic have been reported. In this work, we present some further results on the MEE estimation. The contributions are twofold: (1) we extend a recent result on the minimum entropy of a mixture of unimodal and symmetric distributions to a more general case, and prove that if the conditional distributions are generalized uniformly dominated (GUD), the dominant alignment will be the MEE estimator; (2) we show by examples that the MEE estimator (not limited to singular cases) may be non-unique even if the error distribution is restricted to zero-mean (unbiased). [ABSTRACT FROM AUTHOR]

Recently, the information potential (IP) of order \alpha, defined as the argument of the log in the \alpha-order Renyi entropy, has been successfully used as an information theoretic criterion for supervised adaptive system training. In this paper, we use the survival function (or equivalently the distribution function) of an absolute value transformed random variable to define a new information potential, named the survival information potential (SIP). Compared with the IP, the SIP has some advantages, such as validity in a wide range of distributions, robustness, and the simplicity in computation. The properties of SIP and a simple formula for computing the empirical SIP are given in the paper. Finally, the SIP criterion is applied in adaptive system training, and simulation examples on FIR adaptive filtering, kernel adaptive filtering, and time delay neural networks (TDNNs) training are presented to demonstrate the performance. [ABSTRACT FROM PUBLISHER]

This paper presents a parameterized version of the stochastic information gradient (SIG) algorithm, in which the error distribution is modeled by generalized Gaussian density (GGD), with location, shape, and dispersion parameters. Compared with the kernel-based SIG (SIG-Kernel) algorithm, the GGD-based SIG (SIG-GGD) algorithm does not involve kernel width selection. If the error is zero-mean, the SIG-GGD algorithm will become the least mean p-power (LMP) algorithm with adaptive order and variable step-size. Due to its well matched density estimation and automatic switching capability, the proposed algorithm is favorably in line with existing algorithms. [ABSTRACT FROM AUTHOR]

Abstract: The minimum mean-square error (MMSE) and minimum error entropy (MEE) are two important criteria in the estimation related problems. The MMSE can be viewed as a robust MEE criterion in the minimax sense, as its minimization is equivalent to minimizing an upper bound (the maximum value) of the error entropy. This note gives a new and more meaningful interpretation on the robustness of MMSE for problems in which there exists uncertainty in the probability model. It is shown that the MMSE estimator imposes an upper bound on error entropy for the true model. The upper bound consists of two terms. The first term quantifies the “MMSE performance” under nominal conditions, and the second term measures the “distance” between the true and nominal models. This robustness property is parallel to that of the risk-sensitive estimation. Illustration examples are included to confirm the robustness of MMSE. [Copyright &y& Elsevier]

Abstract: The maximum mutual information (MaxMI) criterion is used as the adaptation cost for the adaptive filtering. This criterion is robust to measure distortions, and has strong connection with traditional mean-square error (MSE) criterion. Under Gaussian assumption, the closed-form solution of the finite impulse response (FIR) filter is obtained. Further, based on the kernel density estimation, the stochastic mutual information gradient (SMIG) algorithm is derived. Simulation results emphasize the robustness of this new algorithm. [Copyright &y& Elsevier]

Recently, the Kalman filter algorithm based on the minimum error entropy criterion has been presented to outperform the conventional Kalman filter in the case of impulsive noise. In practice, it may be unstable in numerical calculation. This paper proposes a robust algorithm to solve the problem of instability in numerical calculation. The convergence and stability of the algorithm are verified by the performance analysis and simulations. In the mean square behavior analysis, we propose a method to estimate the steady-state errors. Simulations show that the experimental steady-state errors of the algorithms agreed with the theoretical values. [ABSTRACT FROM AUTHOR]

The spiking neural network (SNN) is regarded as a promising candidate to deal with the great challenges presented by current machine learning techniques, including the high energy consumption induced by deep neural networks. However, there is still a great gap between SNNs and the online meta-learning performance of artificial neural networks. Importantly, existing spike-based online meta-learning models do not target the robust learning based on spatio-temporal dynamics and superior machine learning theory. In this invited article, we propose a novel spike-based framework with minimum error entropy, called MeMEE, using the entropy theory to establish the gradient-based online meta-learning scheme in a recurrent SNN architecture. We examine the performance based on various types of tasks, including autonomous navigation and the working memory test. The experimental results show that the proposed MeMEE model can effectively improve the accuracy and the robustness of the spike-based meta-learning performance. More importantly, the proposed MeMEE model emphasizes the application of the modern information theoretic learning approach on the state-of-the-art spike-based learning algorithms. Therefore, in this invited paper, we provide new perspectives for further integration of advanced information theory in machine learning to improve the learning performance of SNNs, which could be of great merit to applied developments with spike-based neuromorphic systems. [ABSTRACT FROM AUTHOR]

Although multi-view clustering has received widespread attention due to its far superior performance to single-view clustering, it still faces the following issues: (1) high computational cost, considering the introduction of multi-view information, reduces the clustering efficiency greatly; (2) complex noises and outliers, existed in real-world data, pose a huge challenge to the robustness of clustering algorithms. Currently, how to increase the efficiency and robustness has become two important issues of multi-view clustering. To cope with the above issues, an efficient correntropy-based multi-view clustering algorithm (ECMC) is proposed in this paper, which can not only improve clustering efficiency by constructing embedded anchor graph and utilizing nonnegative matrix factorization (NMF), but also enhance the robustness by exploring correntropy to suppress various noises and outliers. To further improve clustering efficiency, one of the factors of NMF is constrained to be an indicator matrix instead of a traditional non-negative matrix, so that the categories of samples can be obtained directly without any extra operation. Subsequently, a novel half-quadratic-based strategy is proposed to optimize the non-convex objective function of ECMC. Finally, extensive experiments on eight real-world datasets and eighteen noisy datasets show that ECMC can guarantee faster speed and better robustness than other state-of-the-art multi-view clustering algorithms. [ABSTRACT FROM AUTHOR]

There is a recent trend to leverage the power of graph neural networks (GNNs) for brain-network based psychiatric diagnosis, which, in turn, also motivates an urgent need for psychiatrists to fully understand the decision behavior of the used GNNs. However, most of the existing GNN explainers are either post-hoc in which another interpretive model needs to be created to explain a well-trained GNN, or do not consider the causal relationship between the extracted explanation and the decision, such that the explanation itself contains spurious correlations and suffers from weak faithfulness. In this work, we propose a granger causality-inspired graph neural network (CI-GNN), a built-in interpretable model that is able to identify the most influential subgraph (i.e., functional connectivity within brain regions) that is causally related to the decision (e.g., major depressive disorder patients or healthy controls), without the training of an auxillary interpretive network. CI-GNN learns disentangled subgraph-level representations α and β that encode, respectively, the causal and non-causal aspects of original graph under a graph variational autoencoder framework, regularized by a conditional mutual information (CMI) constraint. We theoretically justify the validity of the CMI regulation in capturing the causal relationship. We also empirically evaluate the performance of CI-GNN against three baseline GNNs and four state-of-the-art GNN explainers on synthetic data and three large-scale brain disease datasets. We observe that CI-GNN achieves the best performance in a wide range of metrics and provides more reliable and concise explanations which have clinical evidence. The source code and implementation details of CI-GNN are freely available at GitHub repository (https://github.com/ZKZ-Brain/CI-GNN/). [ABSTRACT FROM AUTHOR]

The normalized subband adaptive filtering algorithm provides fast convergence rate for colored input signals as compared to the normalized least mean square algorithm, but it suffers from a poor convergence issue in the α -stable noise. In light of this, the normalized subband p-norm (NSPN) algorithm, which is based on the least mean p-power error (MPE) criterion, is proposed in this study. This technique is not only robust against impulsive noise samples, but it also maintains a fast convergence rate when colored input signals are used. In addition to this, we develop both the steady-state and the transient models of the NSPN algorithm and provide some insights. Then, in order to solve the problem of making a choice regarding the order p in the NSPN algorithm, we design an autonomous system and come up with the NSPN algorithm with a variable p-norm (VP-NSPN). In addition, we offer the TFC-based VP-NSPN algorithm with a fast convergence rate and low steady-state misadjustment simultaneously by making use of the tap-weights feedback-based convex combination (TFC) scheme. In conclusion, simulation results on system identification and acoustic echo cancellation are undertaken in order to validate the superiority of the proposed algorithms and validate the usefulness of the theoretical analysis. [ABSTRACT FROM AUTHOR]