Search

The generative autoencoders, such as the variational autoencoders or the adversarial autoencoders, have achieved great success in lots of real-world applications, including image generation and signal communication. However, little concern has been devoted to their robustness during practical deployment. Due to the probabilistic latent structure, variational autoencoders (VAEs) may confront problems such as a mismatch between the posterior distribution of the latent and real data manifold, or discontinuity in the posterior distribution of the latent. This leaves a back door for malicious attackers to collapse VAEs from the latent space, especially in scenarios where the encoder and decoder are used separately, such as communication and compressed sensing. In this work, we provide the first study on the adversarial robustness of generative autoencoders in the latent space. Specifically, we empirically demonstrate the latent vulnerability of popular generative autoencoders through attacks in the latent space. We also evaluate the difference between variational autoencoders and their deterministic variants and observe that the latter performs better in latent robustness. Meanwhile, we identify a potential trade-off between the adversarial robustness and the degree of the disentanglement of the latent codes. Additionally, we also verify the feasibility of improvement for the latent robustness of generative autoencoders through adversarial training. In summary, we suggest concerning the adversarial latent robustness of the generative autoencoders, analyze several robustness-relative issues, and give some insights into a series of key challenges. [ABSTRACT FROM AUTHOR]

The multivariate autoregressive (MVAR) model is widely used in describing the dynamics of nonlinear systems, in which the estimates of model parameters and underlying states can be achieved by dual extended Kalman filter (DEKF). However, when the measurements are corrupted by complicated non-Gaussian noises, the DEKF based on the minimum mean-square error (MMSE) criterion may provide biased estimates. In the present article, we develop a novel dual Kalman-type filter, referred to as DEKF under minimum error entropy (MEE) with fiducial points (MEEFs-DEKF) to deal with the non-Gaussian noises. First, the equivalent state-space model and parameter-space model are presented based on the MVAR model. Then, an optimality criterion based on MEE with fiducial points (MEEFs) is applied in the batch-mode regression equation to improve the robustness. Finally, a fixed-point iteration algorithm gives the posterior estimates of state and parameter. Simulation results confirm that the proposed MEEF-DEKF can achieve excellent performance in various noises with different distributions, especially in heavy-tailed and multimodal noises. [ABSTRACT FROM AUTHOR]

To date, most linear and nonlinear Kalman filters (KFs) have been developed under the Gaussian assumption and the well-known minimum mean square error (MMSE) criterion. In order to improve the robustness with respect to impulsive (or heavy-tailed) non-Gaussian noises, the maximum correntropy criterion (MCC) has recently been used to replace the MMSE criterion in developing several robust Kalman-type filters. To deal with more complicated non-Gaussian noises such as noises from multimodal distributions, in this article, we develop a new Kalman-type filter, called minimum error entropy KF (MEE-KF), by using the minimum error entropy (MEE) criterion instead of the MMSE or MCC. Similar to the MCC-based KFs, the proposed filter is also an online algorithm with the recursive process, in which the propagation equations are used to give prior estimates of the state and covariance matrix, and a fixed-point algorithm is used to update the posterior estimates. In addition, the MEE extended KF (MEE-EKF) is also developed for performance improvement in the nonlinear situations. The high accuracy and strong robustness of MEE-KF and MEE-EKF are confirmed by experimental results. [ABSTRACT FROM AUTHOR]

Minimum error entropy with fiducial points (MEEF) has received a lot of attention, due to its outstanding performance to curb the negative influence caused by non-Gaussian noises in the fields of machine learning and signal processing. However, the estimate of the information potential of MEEF involves a double summation operator based on all available error samples, which can result in large computational burden in many practical scenarios. In this paper, an efficient quantization method is therefore adopted to represent the primary set of error samples with a smaller subset, generating a quantized MEEF (QMEEF). Some basic properties of QMEEF are presented and proved from theoretical perspectives. In addition, we have applied this new criterion to train a class of linear-in-parameters models, including the commonly used linear regression model, random vector functional link network, and broad learning system as special cases. Experimental results on various datasets are reported to demonstrate the desirable performance of the proposed methods to perform regression tasks with contaminated data. [ABSTRACT FROM AUTHOR]

In this article, a novel normal-skew mixture (NSM) distribution is presented to model the normal and/or heavy-tailed and/or skew nonstationary distributed noises. The NSM distribution can be formulated as a hierarchically Gaussian presentation by leveraging a Bernoulli distributed random variable. Based on this, a novel robust Kalman filtering framework can be developed utilizing the variational Bayesian method, where the one-step prediction and measurement-likelihood densities are modeled as NSM distributions. For implementation, several exemplary robust Kalman filters (KFs) are derived based on some specific cases of NSM distribution. The relationships between some existing robust KFs and the presented framework are also revealed. The superiority of the proposed robust Kalman filtering framework is validated by a target tracking simulation example. [ABSTRACT FROM AUTHOR]

The spiking neural network (SNN) is regarded as a promising candidate to deal with the great challenges presented by current machine learning techniques, including the high energy consumption induced by deep neural networks. However, there is still a great gap between SNNs and the online meta-learning performance of artificial neural networks. Importantly, existing spike-based online meta-learning models do not target the robust learning based on spatio-temporal dynamics and superior machine learning theory. In this invited article, we propose a novel spike-based framework with minimum error entropy, called MeMEE, using the entropy theory to establish the gradient-based online meta-learning scheme in a recurrent SNN architecture. We examine the performance based on various types of tasks, including autonomous navigation and the working memory test. The experimental results show that the proposed MeMEE model can effectively improve the accuracy and the robustness of the spike-based meta-learning performance. More importantly, the proposed MeMEE model emphasizes the application of the modern information theoretic learning approach on the state-of-the-art spike-based learning algorithms. Therefore, in this invited paper, we provide new perspectives for further integration of advanced information theory in machine learning to improve the learning performance of SNNs, which could be of great merit to applied developments with spike-based neuromorphic systems. [ABSTRACT FROM AUTHOR]

Although multi-view clustering has received widespread attention due to its far superior performance to single-view clustering, it still faces the following issues: (1) high computational cost, considering the introduction of multi-view information, reduces the clustering efficiency greatly; (2) complex noises and outliers, existed in real-world data, pose a huge challenge to the robustness of clustering algorithms. Currently, how to increase the efficiency and robustness has become two important issues of multi-view clustering. To cope with the above issues, an efficient correntropy-based multi-view clustering algorithm (ECMC) is proposed in this paper, which can not only improve clustering efficiency by constructing embedded anchor graph and utilizing nonnegative matrix factorization (NMF), but also enhance the robustness by exploring correntropy to suppress various noises and outliers. To further improve clustering efficiency, one of the factors of NMF is constrained to be an indicator matrix instead of a traditional non-negative matrix, so that the categories of samples can be obtained directly without any extra operation. Subsequently, a novel half-quadratic-based strategy is proposed to optimize the non-convex objective function of ECMC. Finally, extensive experiments on eight real-world datasets and eighteen noisy datasets show that ECMC can guarantee faster speed and better robustness than other state-of-the-art multi-view clustering algorithms. [ABSTRACT FROM AUTHOR]

There is a recent trend to leverage the power of graph neural networks (GNNs) for brain-network based psychiatric diagnosis, which, in turn, also motivates an urgent need for psychiatrists to fully understand the decision behavior of the used GNNs. However, most of the existing GNN explainers are either post-hoc in which another interpretive model needs to be created to explain a well-trained GNN, or do not consider the causal relationship between the extracted explanation and the decision, such that the explanation itself contains spurious correlations and suffers from weak faithfulness. In this work, we propose a granger causality-inspired graph neural network (CI-GNN), a built-in interpretable model that is able to identify the most influential subgraph (i.e., functional connectivity within brain regions) that is causally related to the decision (e.g., major depressive disorder patients or healthy controls), without the training of an auxillary interpretive network. CI-GNN learns disentangled subgraph-level representations α and β that encode, respectively, the causal and non-causal aspects of original graph under a graph variational autoencoder framework, regularized by a conditional mutual information (CMI) constraint. We theoretically justify the validity of the CMI regulation in capturing the causal relationship. We also empirically evaluate the performance of CI-GNN against three baseline GNNs and four state-of-the-art GNN explainers on synthetic data and three large-scale brain disease datasets. We observe that CI-GNN achieves the best performance in a wide range of metrics and provides more reliable and concise explanations which have clinical evidence. The source code and implementation details of CI-GNN are freely available at GitHub repository (https://github.com/ZKZ-Brain/CI-GNN/). [ABSTRACT FROM AUTHOR]

The normalized subband adaptive filtering algorithm provides fast convergence rate for colored input signals as compared to the normalized least mean square algorithm, but it suffers from a poor convergence issue in the α -stable noise. In light of this, the normalized subband p-norm (NSPN) algorithm, which is based on the least mean p-power error (MPE) criterion, is proposed in this study. This technique is not only robust against impulsive noise samples, but it also maintains a fast convergence rate when colored input signals are used. In addition to this, we develop both the steady-state and the transient models of the NSPN algorithm and provide some insights. Then, in order to solve the problem of making a choice regarding the order p in the NSPN algorithm, we design an autonomous system and come up with the NSPN algorithm with a variable p-norm (VP-NSPN). In addition, we offer the TFC-based VP-NSPN algorithm with a fast convergence rate and low steady-state misadjustment simultaneously by making use of the tap-weights feedback-based convex combination (TFC) scheme. In conclusion, simulation results on system identification and acoustic echo cancellation are undertaken in order to validate the superiority of the proposed algorithms and validate the usefulness of the theoretical analysis. [ABSTRACT FROM AUTHOR]

Least-squares one-class support vector machine (LS-OCSVM) is one of the most popular methods to perform one-class classification tasks, in which only the data of a specific class are available to train the classification model. However, the learning performance of LS-OCSVM heavily relies on the effectiveness of a squared loss function, which is sensitive to outliers, resulting in the poor robustness of LS-OCSVM to deal with contaminated data. In this paper, the original optimization problem of LS-OCSVM is therefore reformulated with a recently proposed robust similarity measure, called multikernel correntropy, generating a multikernel correntropy based LS-OCSVM (MKCLS-OCSVM). To find the solution to the new optimization problem effectively, a dynamic optimization algorithm developed with the popular half-quadratic optimization technique is adopted to perform the optimization process. Meanwhile, the convergence and computational complexity of the developed optimization algorithm are analyzed from theoretical perspectives. To further facilitate the implementation of MKCLS-OCSVM, an operationally simple search strategy, inspired by the hunting behavior of humpback whales, is designed for parameters selection. Experimental results on various one-class classification tasks are reported to demonstrate the performance superiority of the proposed MKCLS-OCSVM in comparison with LS-OCSVM and other robust LS-OCSVM variants. [ABSTRACT FROM AUTHOR]

• Build feature layers with higher resolution, which can extract local fine information for small defects. • Attention mechanism is introduced into the FPN to improve the representation ability for small defects. • The features extracted on the small perceptual field are aggregated by the dilated residual block. Although object detection methods have got surprising performance in simple natural scenes, it is still a challenging task to apply them to complex scenes, especially when the detected objects are small which is common in defect detection tasks of industrial scenarios. Motivated by the higher resolution feature maps, the better detection performance on small objects, we increase the resolution of the feature layers in this paper, and introduce convolutional block attention module (CBAM) into the feature aggregation network to better integrate features at different scales and make better use of the information of small defects. For solving the problem of global context information loss caused by limiting the perceptual field by increasing the resolution of the feature layers, we use dilated convolution blocks to expand the perceptual field. Experimental results show that our method improves the mAP(0.5) on the Industrial Surface Defect Dataset from 88.3% in Yolov5 to 89.2%, mAP(0.75) on the DAGM 2007 increases from 76.56% in Yolov5 to 77.65%, mAP on small object increases by 2.6% on Industrial Surface Defect Dataset and 5.8% on DAGM 2007, which proves the effectiveness of our method for small defect detection. [ABSTRACT FROM AUTHOR]

Support vector data description (SVDD) has received a lot of attention due to its outstanding performance to perform one-class classification or novelty detection tasks. However, the same weight is directly imposed on all slack variables in the process of modeling, which may result in degraded learning performance when training data are contaminated by some outliers or mislabeled observations. In this paper, an extended SVDD model is therefore proposed by reformulating the original optimization problem of SVDD with a mixed exponential loss function. Since this loss function can emphasize the importance of the samples that tend to be the target class, and weaken the influence of those tending to be outliers, it can be viewed as a weighted SVDD. However, the weights in the new model are automatically calculated rather than being calculated in advance using some specific manners. To solve the optimization problem of the proposed model effectively, the half-quadratic optimization technique has been adopted to perform the optimization, generating a dynamic optimization algorithm. Meanwhile, the convergence and computational complexity of this dynamic optimization algorithm are analyzed from a theoretical respective. Experimental results on a synthetic data set and some publicly available real world data sets are reported to demonstrate the performance superiority of the new method in comparison with SVDD and other competitive SVDD variants. [ABSTRACT FROM AUTHOR]

Multi-view clustering (MVC) often provides superior effectiveness to single-view clustering due to the integration of information from diverse views. Nonetheless, existing MVC methods are limited to large-scale real-world data by the drawbacks of low efficiency and poor robustness. To address these issues, we propose a novel robust anchor-based MVC model via spectral embedded concept factorization (RAMCSF). RAMCSF builds anchor graphs to approximate full-sample graphs and decomposes these anchor graphs by concept factorization (CF). To improve the clustering effectiveness, factor matrices of CF are constrained as orthogonal matrices to reduce the freedom of decomposition, and a novel small-scale anchor-based spectral embedding is designed to explore the high-order neighbor relationships. To restrain complex noises distributed in real-world data, we employ correntropy to measure the error between the original data and the learned representation. Moreover, RAMCSF can get a clustering indicator matrix directly, avoiding additional post-processing and ensuring that changes in data dimensions have a limited impact on efficiency. The model is then optimized by a novel fast half-quadratic-based optimization strategy that combines the orthogonal properties and the traces of matrices. Extensive experiments indicate that RAMCSF can achieve higher efficiency and robustness while maintaining comparable effectiveness to other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Single-cell profiling is key to uncover the cellular heterogeneity and drives deep understanding of cell fate. In recent years, microfluidics has become an ideal tool for single-cell profiling owing to its benefits of high throughput and automation. Among various microfluidic platforms, microwell has the advantages of simple operation and easy integration with in situ analysis ability, making it an ideal technique for single-cell studies. Herein, recent advances of single-cell analysis based on microwell array chips are summarized. We first introduce the design and preparation of different microwell chips. Then microwell-based cell capture and lysis strategies are discussed. We finally focus on advanced microwell-based analysis of single-cell proteins, nucleic acids, and metabolites. The challenges and opportunities for the development of microwell-based single-cell analysis are also presented. [ABSTRACT FROM AUTHOR]

• Distributed optimization problems of consensus are examined by a frequency-domain analysis approach. • An optimized distributed protocol is proposed such that the nonuniform time-delay FDMSs with SF have better consensus performance. • The double-integral dynamics of delayed FDMSs with SF is considered. This paper addresses distributed optimization problems concerning consensus in delayed fractional-order double-integrator multi-agent systems (FDMSs). To start with, an optimized distributed protocol with state-fractional-order-derivative feedback (SF) is presented for delayed FDMSs. Then, the consensus problems are studied for the two kinds of delayed FDMSs with SF in the presence of symmetric time-delays over undirected network topology and asymmetric time-delays over directed network topology. Next, by the means of graph theory, matrix theory and frequency-domain analysis method, the sufficient conditions to guarantee consensus of delayed FDMSs with SF are derived. Compared to the traditional distributed protocol without SF, the proposed distributed optimization protocol with SF are taken into account to enable better consensus performance in delayed FDMSs with SF. Finally, numerical experiments are carried out to verify the feasibility of our theoretical results. [ABSTRACT FROM AUTHOR]

Accumulating evidence suggests that the brain is highly dynamic; thus, investigation of brain dynamics especially in brain connectivity would provide crucial information that stationary functional connectivity could miss. This study investigated temporal expressions of spatial modes within the default mode network (DMN), salience network (SN) and cognitive control network (CCN) using a reliable data-driven co-activation pattern (CAP) analysis in two independent data sets. We found enhanced CAP-to-CAP transitions of the SN in patients with MDD. Results suggested enhanced flexibility of this network in the patients. By contrast, we also found reduced spatial consistency and persistence of the DMN in the patients, indicating reduced variability and stability in individuals with MDD. In addition, the patients were characterized by prominent activation of mPFC. Moreover, further correlation analysis revealed that persistence and transitions of RCCN were associated with the severity of depression. Our findings suggest that functional connectivity in the patients may not be simply attenuated or potentiated, but just alternating faster or slower among more complex patterns. The aberrant temporal–spatial complexity of intrinsic fluctuations reflects functional diaschisis of resting-state networks as characteristic of patients with MDD. [ABSTRACT FROM AUTHOR]

The Unscented Kalman filter (UKF) based on maximum correntropy criterion (MCC) is robust to heavy-tailed non-Gaussian noise. However, the approximate linear measurement equation obtained by statistical linearization technique may be not accurate enough since it only uses prior information. In this paper, a robust stable iterative maximum correntropy criterion UKF (RS-IMCC-UKF) is proposed by using nonlinear measurement function directly and numerical stability methods. Different from the existing UKF algorithms, we only need to perform one-time unscented transformation in each filtering cycle, reducing the execution time of algorithm. Then a nonlinear enhancement model is constructed to handle predictions and observations simultaneously, which will be included in the cost function of robust MCC. In the process of iterative solution, thanks to the latest iteration values which are used to update the measurement information, RS-IMCC-UKF has more accurate results compared with traditional filters. In order to avoid non-positive definite characteristic in covariance matrix and the inverse operation in ill-conditioned matrix, the hyperbolic QR decomposition and Moore–Penrose pseudo-inversion are introduced to improve the numerical stability of the algorithm. Finally, a target tracking is modeled to verify the effectiveness of the algorithm. [ABSTRACT FROM AUTHOR]

Concept factorization (CF) has shown the effectiveness in the field of data clustering. In this paper, a novel and robust semi-supervised CF method, called correntropy based semi-supervised concept factorization with adaptive neighbors (CSCF), is proposed with improved performance in clustering applications. Specifically, on the one hand, the CSCF method adopts correntropy as the cost function to increase the robustness for non-Gaussian noise and outliers, and combines two different types of supervised information simultaneously for obtaining a compact low-dimensional representation of the original data. On the other hand, CSCF assigns the adaptive neighbors for each data point to construct a good data similarity matrix for reducing the sensitiveness of data. Moreover, a generalized version of CSCF is derived for enlarging the clustering application ranges. Analysis is also presented for the relationship of CSCF with several typical CF methods. Experimental results have shown that CSCF has better clustering performance than several state-of-the-art CF methods. [ABSTRACT FROM AUTHOR]

Multi-view clustering has received widespread attention owing to its effectiveness by integrating multi-view data appropriately, but traditional algorithms have limited applicability to large-scale real-world data due to their high computational complexity and low robustness. Focusing on the aforementioned issues, we propose an efficient and robust multi-view clustering algorithm with anchor graph regularization (ERMC-AGR). In this work, a novel anchor graph regularization (ARG) is designed to improve the quality of the learned embedded anchor graph (EAG), and the obtained EAG is decomposed by nonnegative matrix factorization (NMF) under correntropy criterion to acquire clustering results directly. Different from the traditional graph regularization that needs to construct a large-scale Laplacian matrix pertaining to the all-sample graph, our lightweight AGR, constructed from the perspective of anchors, can reduce the computational complexity significantly while improving the EAG quality. Moreover, a factor matrix of NMF is constrained to be the cluster indicator matrix to omit additional k-means after optimization. Subsequently, correntropy is utilized to improve the effectiveness and robustness of ERMC-AGR owing to its promising performance to complex noises and outliers. Extensive experiments on real-world datasets and noisy datasets show that ERMC-ARG can improve the clustering efficiency and robustness while ensuring comparable or even better effectiveness. [ABSTRACT FROM AUTHOR]

Multiview clustering (MVC) has attracted considerable attention owing to its remarkable capacity to reconcile diverse information from multiple perspectives. However, traditional MVC generally has a narrow scope of application owing to its limited efficiency. Consequently, various efficient MVC (EMVC) methods have emerged recently. Despite their promising performance, these EMVC methods still have several unresolved issues: (1) They suffer from reduced effectiveness caused by representation non-alignment across views and information mismatch between stages, and (2) they fail to efficiently resist complex noises and outliers. To address these issues, we propose an efficient correntropy-based multiview clustering method with alignment discretization (ECMCAD). Specifically, a correntropy-based multipartition learning model was developed to efficiently learn view-specific robust partition-level representations. Additionally, a novel alignment discretization strategy was designed to align the learned cross-view representations into a consensus discrete indicator to integrate representation learning, representation alignment, and discrete label acquisition into a unified framework. Furthermore, an efficient alternating optimization method was developed to solve the model. Numerous experiments illustrated the superiority of the proposed method over state-of-the-art baselines. [ABSTRACT FROM AUTHOR]

Concept factorization (CF), unlike nonnegative matrix factorization (NMF), can handle data with negative values by approximating the original data with two low-dimensional nonnegative matrices and itself. Nevertheless, existing CF-based methods continue to suffer from the two issues specified as follows: (1) Their effectiveness is reduced by the high degree of factorization freedom and the two-stage mismatch between factorization and category acquisition, and (2) their robustness drops significantly when dealing with complex noise. In response to the aforementioned issues, we propose a robust spectral-embedded bilateral orthogonal concept factorization (RSOCF) model for clustering. It constrains the factor matrices as orthogonal matrices to decrease the freedom and obtain samples' categories directly after factorization, which can significantly improve clustering effectiveness. Moreover, correntropy is introduced into RSOCF to improve its robustness to complex noise. To optimize the non-convex RSOCF model, a half-quadratic-based algorithm is devised. Numerous experiments demonstrate that RSOCF surpasses other state-of-the-art methods in terms of clustering effectiveness and robustness. • A novel robust concept factorization-based clustering model is proposed. • A fast algorithm is proposed to optimize the non-convex model. • Numerous experiments validate its clustering effectiveness and robustness. [ABSTRACT FROM AUTHOR]

Converging evidence increasingly suggests that psychiatric disorders, such as major depressive disorder (MDD) and autism spectrum disorder (ASD), are not unitary diseases, but rather heterogeneous syndromes that involve diverse, co-occurring symptoms and divergent responses to treatment. This clinical heterogeneity has hindered the progress of precision diagnosis and treatment effectiveness in psychiatric disorders. In this study, we propose BPI-GNN, a new interpretable graph neural network (GNN) framework for analyzing functional magnetic resonance images (fMRI), by leveraging the famed prototype learning. In addition, we introduce a novel generation process of prototype subgraph to discover essential edges of distinct prototypes and employ total correlation (TC) to ensure the independence of distinct prototype subgraph patterns. BPI-GNN can effectively discriminate psychiatric patients and healthy controls (HC), and identify biological meaningful subtypes of psychiatric disorders. We evaluate the performance of BPI-GNN against 11 popular brain network classification methods on three psychiatric datasets and observe that our BPI-GNN always achieves the highest diagnosis accuracy. More importantly, we examine differences in clinical symptom profiles and gene expression profiles among identified subtypes and observe that our identified brain-based subtypes have the clinical relevance. It also discovers the subtype biomarkers that align with current neuro-scientific knowledge. • We propose a new GNN architecture with prototype learning for psychiatric diagnosis. • We successfully identify biological meaningful subtypes of psychiatric disorders. • We observe subtype biomarkers that closely correspond to neuroimaging findings. • Our framework achieves the best performance on three psychiatric datasets. [ABSTRACT FROM AUTHOR]

Owing to its ability to handle negative data and promising clustering performance, concept factorization (CF), an improved version of non-negative matrix factorization, has been incorporated into multi-view clustering recently. Nevertheless, existing CF-based multi-view clustering methods still have the following issues: (1) they directly conduct factorization in the original data space, which means its efficiency is sensitive to the feature dimension; (2) they ignore the high degree of factorization freedom of standard CF, which may lead to non-uniqueness factorization thereby causing reduced effectiveness; (3) traditional robust norms they used are unable to handle complex noises, significantly challenging their robustness. To address these issues, we establish a fast multi-view clustering via correntropy-based orthogonal concept factorization (FMVCCF). Specifically, FMVCCF executes factorization on a learned consensus anchor graph rather than directly decomposing the original data, lessening the dimensionality sensitivity. Then, a lightweight graph regularization term is incorporated to refine the factorization process with a low computational burden. Moreover, an improved multi-view correntropy-based orthogonal CF model is developed, which can enhance the effectiveness and robustness under the orthogonal constraint and correntropy criterion, respectively. Extensive experiments demonstrate that FMVCCF can achieve promising effectiveness and robustness on various real-world datasets with high efficiency. [ABSTRACT FROM AUTHOR]

Least squares (LS) estimation is simple yet effective for parameter estimation. Most real-world problems are nonlinear. In practice, nonlinear LS problems are linearized first and then solved by a linear LS estimator. However, since these methods solve a nonlinear LS problem by a linear way, there is room for improvement only if estimators being nonlinear in the original measurement. In this article, an LS approach is proposed for nonlinear LS problems, along with theoretical analysis and justification. First, for a general nonlinear measurement model with any form of noise, a unified linearization (UL) approach is proposed under certain assumptions and a linearized least squares (LLS) estimator is defined based on the UL model. Second, an augmented nonlinear least squares (ANLS) estimator is proposed based on LLS estimation. In the ANLS estimator, the original measurement is augmented by its nonlinear conversion intended to further utilize the nonlinear measurement. Moreover, we prove that the ANLS estimator outperforms LLS estimation-based estimators. Finally, the UL and the ANLS estimator are applied to source localization with time difference of arrival and/or frequency difference of arrival measurement, and a two-step ANLS localization algorithm is developed. The performance of the proposed algorithm is evaluated via simulation of some typical localization scenarios. [ABSTRACT FROM AUTHOR]

Background noise and scale variation are common problems that have been long recognized in crowd counting. Humans glance at a crowd image and instantly know the approximate number of human and where they are through attention the crowd regions and the congestion degree of crowd regions with a global receptive field. Hence, in this paper, we propose a novel feedback network with Region-Aware block called RANet by modeling human's Top-Down visual perception mechanism. Firstly, we introduce a feedback architecture to generate priority maps that provide prior about candidate crowd regions in input images. The prior enables the RANet pay more attention to crowd regions. Then we design Region-Aware block that could adaptively encode the contextual information into input images through global receptive field. More specifically, we scan the whole input images and its priority maps in the form of column vector to obtain a relevance matrix estimating their similarity. The relevance matrix obtained would be utilized to build global relationships between pixels. Our method outperforms state-of-the-art crowd counting methods on several public datasets. [ABSTRACT FROM AUTHOR]

Accurate cluster photovoltaic power prediction (CPPP) is crucial for the operation and control of renewable energy grid-connected power systems. The traditional modeling strategies for CPPP such as direct aggregation (DA) and statistical upscaling (SU) have limitations such as error accumulation and upscaling factor uncertainty. To address these issues, this paper proposed a novel hybrid approach for CPPP by combining machine learning models with an improved SU technique. Firstly, a robust broad learning system (BLS) model, in which the Generalized Maximum Correntropy Criterion (GMCC) is used to replace the original mean square error (MSE) loss in BLS, is proposed to solve the problem of multiple outliers affecting the prediction accuracy of regional cluster stations, and it is called GBLS. Then, the Relevance Vector Machine (RVM) as an effective nonlinear regression model is further utilized to compensate for the prediction errors obtained by the GBLS to form the hybrid prediction model, namely GBLS-RVM. Moreover, to mitigate the uncertainty associated with scaling factors in traditional SU strategy, a new SU strategy is developed to refine the relationship between the reference station and the cluster sub-region, enabling direct modeling for regional power prediction. Finally, data from two PV clusters in different regions of China are used to validate the effectiveness of the proposed model, and the results show that under the improved SU strategy, the GBLS-RVM model, reduced RMSE by approximately 33.7% compared to the traditional BLS model, and the RMSE decreased by 12.89% and 30.2% when compared to traditional DA and traditional SU strategies. • A novel hybrid method for CPPP is developed by combining ML and improved SU technique. • A BLS with GMCC (GMCC-BLS) as a prediction model is developed to handle the outliers in measurement data. • An innovative error correction strategy based on RVM is proposed to improve prediction reliability. • A new SU technique is used to establish direct relationships between reference stations and subregions. [ABSTRACT FROM AUTHOR]

Concept factorization (CF) has attracted widespread attention for its promising performance in document clustering. Among various CF variants, graph-regularized CF is the most impressive type, which can improve clustering effectiveness by exploring structural information. Nevertheless, their clustering efficiency is restricted by the following considerations: (1) the introduction of the full-sample graph is accompanied by an increase in computational complexity; (2) most of them require intensive multiplications in optimization, which impair the optimization efficiency. To address these issues, in this work, we propose an anchor-graph regularized orthogonal concept factorization (AROCF) method to enhance the clustering efficiency and effectiveness in document clustering tasks. Firstly, AROCF approximates the full-sample graph with a small-scale anchor graph to reduce the complexity of graph construction from quadratic to linear. Then, one of the factor matrices is constrained as the cluster indicator matrix in our method, which can avoid extra efficiency loss in K-means after optimization. Finally, an orthogonal constraint is employed to restrict the freedom of factorization to increase the clustering effectiveness. To optimize the AROCF model, we develop a fast optimization strategy by combining the trace and orthogonality of matrices. Extensive experiments on various document datasets demonstrate the effectiveness and efficiency of AROCF. [ABSTRACT FROM AUTHOR]

Graph-based clustering commonly provides promising clustering effectiveness as it can preserve samples' local geometric information. Inspired by it, multi-view graph clustering was developed to integrate complementary information among graphs of diverse views and it has received intensive attention recently. Nevertheless, on the one hand, most existing methods require extra k-means after obtaining embedding representations to generate a discrete cluster indicator, which reduces effectiveness due to the two-stage mismatch. On the other hand, numerous complex noises in real-world multi-view data challenge the robustness of existing clustering methods. In this paper, we established a discrete correntropy-based multi-view anchor-graph clustering (DCMAC) model that not only emphasizes the aforementioned issues but also makes use of anchor graphs to improve the efficiency of the graph construction stage. To optimize this non-convex model, we propose a fast half-quadratic-based coordinate descent strategy to acquire the discrete cluster indicator directly without extra k-means. Furthermore, we extend the DCMAC model to a single-view form and provide optimization strategies for it. Extensive experiments illustrate that the proposed method is effective and robust compared to those advanced baselines. • A new discrete correntropy-based multi-view anchor-graph clustering (DCMAC) model is proposed. • A novel half-quadratic-based coordinate descent algorithm is proposed to solve the DCMAC model. • To handle single-view clustering instances, DCMAC is extended to a single-view form named DCAC. • Numerous experiments demonstrate the outstanding effectiveness and robustness of DCMAC. [ABSTRACT FROM AUTHOR]

Recently, the kernel adaptive filtering algorithm with the generalized maximum correntropy criterion (GMCC) has gained attention due to its excellent nonlinear system modeling ability and robustness to non-Gaussian noise. However, since the default center of the GMCC is located at zero, it hampers its performance in the non-zero mean noise environment. To overcome the problem, this paper proposes a kernel adaptive filtering algorithm based on the GMCC with variable center (GMCC-VC) called generalized kernel maximum correntropy criterion with variable center (GKMCC-VC). In addition, the online vector quantization (VQ) method is introduced to the GKMCC-VC algorithm for suppressing the increasing network size. The probability of divergence, the mean-square steady-state behavior, and the convergence condition of the GKMCC-VC algorithm are also obtained under some assumptions. The potential relationship between the GMCC-VC and the generalized minimum error entropy (GMEE) criteria is discussed in the paper. Finally, the simulation results verify the superiority of the proposed algorithm in modeling nonlinear systems and the correctness of the theoretical model. • This paper proposes the GKMCC-VC algorithm for non-zero mean and non-Gaussian noise. • The steady-state performance of the proposed algorithm is analyzed. • Simulation results show the effectiveness and feasibility of the proposed algorithm. [ABSTRACT FROM AUTHOR]

Randomized autoencoder (RAE) has attracted much attention due to its strong capability of representation with fast learning speed. However, the mainstream RAEs are still designed for scalar/vector data, which inevitably destroys the structure information of tensor data. To alleviate this deficiency, a novel convolutions based matrix randomized autoencoder (MRAE) is developed for two-dimensional (2D) data in this paper, including a one-side MRAE (OMRAE) exploiting the row or column information and a double-side MRAE (DMRAE) that simultaneously extracts the row and column information by 2 parallel OMRAEs. To reduce meaningless encoded features, the within-class scatter matrix (WSI) and within-class interaction distance (WID) constraints are added into OMRAE resulting WSI-OMRAE and WID-OMRAE, respectively. To demonstrate the superiority, stacked MRAEs are embedded into hierarchical regularized least squares for one-class classification and comparisons with several state-of-the-art methods are provided. The source code would be available at https://github.com/ML-HDU/MRAE. • MRAE can learn from 2D data directly without breaking structure information. • WSI-OMRAE minimizes both reconstruction error and within-class scatter. • WID-OMRAE can learn intrinsic manifold structure with compact representation. • Adopting two-sides OMRAEs can perform row/column feature learning in parallel. [ABSTRACT FROM AUTHOR]

Recurrent broad learning system (RBLS) is an effective learning method for processing sequential data. By replacing the enhancement nodes of the broad learning system with recurrent structure, RBLS obtains the capacity to capture the dynamic characteristics of time series data. However, RBLS is derived under the minimum mean square error (MMSE) criterion, which is sensitive to outliers. Moreover, RBLS is insufficient for online sequential learning. To address these limitations, we propose a novel online robust echo state structure based RBLS (OR-ESBLS). In OR-ESBLS, kernel recursive maximum correntropy (KRMC) is introduced to both enhance the robustness and discover the nonlinear characteristics of feature nodes in an online manner. To reduce the heavy computational requirements caused by the kernel method, a Quasi-Monte Carlo (QMC) based Random Fourier Feature (RFF) is utilized for kernel approximation. Furthermore, we adopt the randomized sparse reservoir as the enhancement nodes of RBLS, which can much more efficiently capture dynamic information of the data in the sequential learning setting. Experiments on both synthetic and real-world datasets are reported. The results show that the proposed OR-ESBLS can provide superior performance in online sequential time series prediction. [ABSTRACT FROM AUTHOR]

• We propose a robot perception intelligence adaptation mechanism and three different data sampling styles of this mechanism. • We theoretically analyze and compare the learning effect of the proposed three different data sampling styles. • We determine the most suitable data sampling style based on the performance availability trade off. • Our work offers guidance for training data collection in the robot perception intelligence adaptation process. Robots need more intelligence to complete perception tasks in uncertain and unstructured environments. This paper presents a new self-evolving home service robot framework that learns new perception skills by using manually labeled data obtained in a new home environment. In this framework, a global model is trained which serves as the starting point of the robots' local model, and an adaptation mechanism is developed in the robot to adapt the initial local model to the new home environment. First, three different data sampling styles are proposed to carry out the adaptation process, and theoretical analysis is given to explain the difference between the proposed three data sampling styles. Second, the most suitable data sampling style for incremental learning for a home service robot is determined. Third, we present a case study of multi-style learning, and the experimental results validate our analysis. The theoretical analysis and experimental results lead us to propose a guideline of data collection and labeling for robots to adapt their perception intelligence in home environments. [ABSTRACT FROM AUTHOR]

Partial discharge (PD) detection plays a vital role in on-line condition monitoring of electrical apparatus in the power systems. However, the noise of PD measurements significantly degrades the performance of detection algorithms. In this paper, we focus on developing an adaptive filtering technique for the PD denoising problem. Heretofore, there are just a few literature reviews addressing the PD denoising based on such a method. The proposed recursive continuous S-shaped (RCSS) algorithm integrates the advantages of recursive strategy and continuous S-shaped function into adaptive noise cancellation (ANC) system, yielding enhanced filtering performance. The proposed algorithm can tackle PD noises in both Gaussian and impulsive scenarios, which is easy to implement in practical applications. The convergence behavior is also analyzed. Extensive simulation and experimental results confirm that the proposed algorithm can address polluted PD pulses, even in excessively noisy conditions, resulting in smaller mean square error (MSE) as compared to other state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]

Accurate estimation of the state of health (SOH) is crucial for the safe and stable operation of lithium-ion batteries. However, since the labeled values may contain non-Gaussian noise which may lead to a decrease in the effectiveness of traditional data-driven based estimation methods. Therefore, a novel robust stacking integrated learning model (ILM) is proposed to enable accurate SOH estimation under non-Gaussian noises (or outliers) conditions. The mixture correntropy loss (MCL) is used in original extreme learning machine (ELM) and the gated recurrent unit (GRU) frameworks to develop novel robust learning models, i.e MCL-ELM and MCL-GRU, and they are used as the sub-models of the proposed ILM, which takes into account the generalization performance of the model and the overall trend of SOH over the time series. In addition, the entropy weighting method that can well reflect the differences between the sub-models is utilized to reduce the complexity of the stacked ILM. Furthermore, the local tangent space alignment is used to capture the local relationships between different loops to enhance the effectiveness of the extracted health features. Several numerical experiments are performed under different cases to validate the estimation effect of the proposed model by using two publicly available datasets, and the experimental results demonstrate that the maximum RMSE, MAE, and MAPE values of the proposed model are 1.391%, 0.932%, and 1.480%, respectively, under non-Gaussian noises conditions. • Robust ELM and GRU models using MCL are developed. • Novel robust stacking ILM via the combination of the MCL-ELM and MCL-GRU is proposed for SOH estimation. • The EWM is used to obtain the output results of multiple base learners in the proposed ILM. [ABSTRACT FROM AUTHOR]

The brain aggregates meaningless local sensory elements to form meaningful global patterns in a process called perceptual grouping. Current brain imaging studies have found that neural activities in V1 are modulated during visual grouping. However, how grouping is represented in each of the early visual areas, and how attention alters these representations, is still unknown. We adopted MVPA to decode the specific content of perceptual grouping by comparing neural activity patterns between gratings and dot lattice stimuli which can be grouped with proximity law. Furthermore, we quantified the grouping effect by defining the strength of grouping, and assessed the effect of attention on grouping. We found that activity patterns to proximity grouped stimuli in early visual areas resemble these to grating stimuli with the same orientations. This similarity exists even when there is no attention focused on the stimuli. The results also showed a progressive increase of representational strength of grouping from V1 to V3, and attention modulation to grouping is only significant in V3 among all the visual areas. Most of the previous work on perceptual grouping has focused on how activity amplitudes are modulated by grouping. Using MVPA, the present work successfully decoded the contents of neural activity patterns corresponding to proximity grouping stimuli, thus shed light on the availability of content-decoding approach in the research on perceptual grouping. Our work found that the content of the neural activity patterns during perceptual grouping can be decoded in the early visual areas under both attended and unattended task, and provide novel evidence that there is a cascade processing for proximity grouping through V1 to V3. The strength of grouping was larger in V3 than in any other visual areas, and the attention modulation to the strength of grouping was only significant in V3 among all the visual areas, implying that V3 plays an important role in proximity grouping. [ABSTRACT FROM AUTHOR]

• The kernel adaptive filter with generalized tanh function (KAF-GT) is proposed, which shows favourable performance in combating impulsive noise. • The convergence analysis of the proposed KAF-GT algorithm is conducted using energy conservation relation. • By using the pre-tuned dictionary strategy to model the unknown nonlinear system, we perform the EMSE analysis and arrive at a closed-form solution for predicting the steady-state behavior. • To further improve the performance of KAF-GT, an optimization scheme for scale factor is designed. Recently, online learning algorithms in machine learning have been imposed much attention. As a typical family, kernel adaptive filtering algorithms receive particular interest due to their sequential learning-based features. However, the kernel least mean square (KLMS) algorithm is not suitable for nonlinear tasks corrupted by non-Gaussian noise, especially impulsive noise. This is because the derivation of the KLMS algorithm is on the basis of the mean square error (MSE) criterion which only captures information of second-order statistics. In this paper, motivated by tanh function, we develop its generalized variant by introducing a scale factor for better representation capability; then we incorporate kernel adaptive filter with the generalized tanh function to propose a robust sequential learning algorithm. Based on establishing the energy conservation relation, we derive a sufficient condition for ensuring the algorithm convergence. In addition, to perform the steady-state excess mean square error (EMSE) analysis, we use the pre-tuned dictionary strategy to model the unknown nonlinear system in form of a finite-order combination; by Taylor expansion, we arrive at a closed-form solution for predicting the steady-state behavior. To further improve the algorithm performance, we design an optimization scheme for scale factor. Simulations for nonlinear time series prediction show that the designed schemes yield better performance than some state-of-art algorithms. The steady-state EMSE analysis is validated to provide accurate prediction results. [ABSTRACT FROM AUTHOR]

• An optimization problem about the state mean of multi-agent systems with impulsive noise based on MCC derived from information theoretic learning is investigated in this paper, where the half-quadratic(HQ) strategy is used to simplify the solving process of the optimization problem. • A problem interfered by impulsive noise on the states of multi-agent systems is studied in this paper, where the adverse effect of impulsive noise on the states is eliminated by the MCC algorithm. • An event-triggered consensus problem of discrete-time multi-agent systems with random impulsive noise is addressed in this paper, where an event-triggering condition is derived for the system to achieve consensus. An event-triggered consensus problem of discrete-time multi-agent systems with impulsive noise under directed switching topology is investigated in this paper, where the control input of each agent is determined by its own state information and the state information of its neighbors that may be corrupted by impulsive noise. Firstly, to reduce the adverse effect of impulsive noise on consensus performance of the discrete-time multi-agent systems, maximum correntropy criterion (MCC) derived from information theoretic learning is introduced to calculate the communication weights among the agents, and an event-triggered strategy is adopted to drive the state variables of each agent to eventually converge to a same value. Secondly, the condition to guarantee consensus of the discrete-time multi-agent systems is derived by graph theory and Lyapunov stability theorem. Finally, the effectiveness of the theory is verified by several numerical simulations. [ABSTRACT FROM AUTHOR]

• A novel spatial attention encoder is proposed, which builds global feature insight with long-range dependence to enhance perception. • We propose a context block decoder to aggregate contextual information as channel-specific bias of the features. • A feature fusion module with parallel branches considering expectation and residual property is proposed to realize adaptive information fusion. • We build an ESD dataset with 89.70% small defects collected from real industry an evaluation benchmark. Despite the promising development of Automatic Visual Inspection (AVI) in the manufacturing industry, detecting small-sized defects with fewer pixels coverage remains a challenging problem due to its insufficient attention and lack of semantic information. Most exsiting convolutional inspection methods overlook the long-range dependence of context and lack adaptive fusion strategies to exploit heterogeneous features. To address these issues in AVI, this paper proposes a novel contextual information and spatial attention based network (CANet), which consists of two steps, namely CAblock and LaplacianFPN, for effective perception and exploitation of small defect features. Specifically, CAblock extracts semantic information with rich context by encoding spatial long-range dependence and decoding contextual information as channel-specific bias through a Spatial Attention Encoder (SAE) and a Context Block Decoder (CBD), respectively. LaplacianFPN further performs adaptive feature fusion considering both feature consistency and heterogeneity via two parallel branches. As a benchmark, a self-built Engine Surface Defects (ESD) dataset collected in real industry containing 89.70% small defects is constructed. Experimental results show that CANet achieves mAP-50 improvements of 1.5% and 4.3% compared to state-of-the-art methods on NEU-DET and ESD, which demonstrates the effectiveness of the proposed method. The code is now available at https://github.com/xiuqhou/CANet. [ABSTRACT FROM AUTHOR]

• The UKF with EKRSL is developed by using the fixed-point iteration; • An enhanced EKRSL-UKF is proposed via an exponential function of innovation; • The proposed EnEKRSL-UKF is used for robust DSE. The traditional unscented Kalman filter (UKF) with mean square error (MSE) criterion for dynamic state estimation (DSE) is sensitive for unknown non-Gaussian noise and outliers. Leading to biased state estimates. This paper proposes a novel robust UKF with extended kernel risk-sensitive loss (EKRSL) for DSE considering unknown non-Gaussian process and measurement noises. Instead of MSE criterion, a novel robust EKRSL via the generalized Gaussian density is defined in KRSL framework, and we further develop a new robust UKF using the EnKRSL(called EKRSL-UKF). To obtain the recursive form of EKRSL-UKF, the statistical linear regression model is used and the fixed-point iteration is further utilized to iteratively get the optimal state estimate. An error constrained method is also introduced to restrict the error to address the numerical instability problem caused by large outliers. Furthermore, an enhanced EKRSL-UKF is established by using an exponential function of innovation to improve the estimation accuracy in the presence of noise uncertainties. Numerical results carried out on the IEEE 39-bus test system demonstrate that the proposed method can achieve desired robustness without loss of estimation accuracy under various conditions. [ABSTRACT FROM AUTHOR]

• A novel hypergraph based semi-supervised SNMF method is proposed for image clustering. • A new hypergraph based pairwise constraints propagation algorithm is presented. • The proposed method is analysed in terms of convergence, supervisory information, and computational complexity. • Extensive experiments confirm the effectiveness of the proposed method in clustering applications. Semi-supervised symmetric nonnegative matrix factorization (SNMF) has been shown to be a significant method for both linear and nonlinear data clustering applications. Nevertheless, existing SNMF-based methods only adopt a simple graph to construct the similarity matrix, and cannot fully use the limited supervised information for the construction of the similarity matrix. To overcome the drawbacks of previous SNMF-based methods, a new semi-supervised SNMF-based method called hypergraph based semi-supervised SNMF (HSSNMF), is proposed in this paper for image clustering. Specifically, HSSNMF adopts a predefined hypergraph to build a similarity matrix for capturing the high-order relationships of samples. By exploiting a new hypergraph based pairwise constraints propagation (HPCP) algorithm, HSSNMF propagates the pairwise constraints of the limited data points to the entire data points, which can make full use of the limited supervised information and construct a more informative similarity matrix. Using the multiplicative updating algorithm, a discriminative assignment matrix can then be obtained by solving the optimization problem of HSSNMF. Moreover, analyses of the convergence, supervisory information, and computational complexity of HSSNMF are presented. Finally, extensive clustering experiments have been conducted on six real-world image datasets, and the experimental results have demonstrated the superiority of HSSNMF while compared with several state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Multi-view clustering has been a significant research problem in unsupervised clustering in recent years and has important applications in computer vision, data mining and other fields. To explore more clustering information from different views and improve performance, researchers have developed a variety of multi-view clustering algorithms by applying the same L 1 or L 2 , 1 norm constraints for different views, which essentially assumes that the errors satisfy the characteristics described by the corresponding norms. It is clear that such assumption is not always satisfied in the real-world datasets. Therefore, the performance of existing algorithms can decrease under the complex noise that may be contained in the data. To solve the above problem, we introduce an information theoretic learning-based metric into a multi-view model to address noise. First, we propose a new mixture correntropy (MC) metric. Then, we propose a MC based robust multi-view K-means clustering (MC-RMVKM) method by applying the MC to an existing model. Furthermore, an efficient iterative algorithm is derived to solve the joint MC-RMVKM learning problem. Extensive results on real-world multi-view data show that the MC-based algorithm outperforms other state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]

Kalman filters (KFs) are widely used for state-of-charge (SOC) estimation of Li-ion batteries due to their excellent dynamic tracking capability. Especially the cubature KF (CKF), with the computational efficiency and nonlinear processing ability, is an outstanding candidate for SOC estimation. However, the actual working conditions are complex and changeable, and the measurement data is usually accompanied by non-Gaussian noise (outliers). Therefore, the performance of the original CKF with minimum mean square error (MMSE) criterion may be degraded seriously in these cases. In order to enhance the robustness of CKF, the MMSE in the CKF framework is substituted by the generalized maximum correntropy criterion (GMCC), and thus a robust CKF with GMCC (GMCC-CKF) is developed by fixed point iteration approach in this work. Furthermore, a SOC estimation model via the GMCC-CKF is proposed to improve estimation accuracy under non-Gaussian noise environments. The simulation results show that, compared with the traditional KFs, the proposed GMCC-CKF can accurately estimate the SOC of lithium batteries under different temperatures and operating conditions considering non-Gaussian noise interference. The results of mean absolute error (MAE) and root mean square error (RMSE) are less than 1%, which verifies the excellent performance of GMCC-CKF. • GMCC is introduced into the CKF framework to develop a novel robust CKF. • The proposed GMCC-CKF method is applied for robust SOC estimation under non-Gaussian noise interferences. • Some simulations under different conditions are performed to evaluate the performance of the proposed method. [ABSTRACT FROM AUTHOR]