Search

This paper focuses on the event-triggered guaranteed cost control design of nonlinear systems via a self-learning technique. In brief, an event-based guaranteed cost control strategy of nonlinear systems subjects to matched uncertainties is developed, thereby balancing the performance of guaranteed cost and the actuality of limited communication resource. The original control design is transformed into an optimal control problem with an event-based mechanism, where the relationship of guaranteed cost performance compared to the time-based formulation is discussed. A critic neural network is constructed for implementing the event-based optimal control design with stability guarantee. Simulation experiments are carried out to verify the theoretical results in detail. [ABSTRACT FROM AUTHOR]

For accurate prognostics, users have to determine the current health of the system and predict future degradation pattern of the system. An increasingly popular approach toward tackling prognostic problems involves the use of switching models to represent various degradation phases, which the system undergoes. Such approaches have the advantage of determining the exact degradation phase of the system and being able to handle nonlinear degradation models through piecewise linear approximation. However, limitations of such existing methods include, limited applicability due to the discretization of predicted remaining useful life, insufficient robustness due to the use of single models and others. This paper circumvents these limitations by proposing a hybrid of ensemble methods with switching methods. The proposed method first implements a switching Kalman filter (SKF) to classify between various linear degradation phases, then predict the future propagation of fault dimension using appropriate Kalman filters for each phase. This proposed method achieves both continuous and discrete prediction values representing the remaining life and degradation phase of the system, respectively. The proposed framework is shown via a case study on benchmark simulated aeroengine data sets. The evaluation of the proposed framework shows that the proposed method achieves better accuracy and robustness against noise compared with other methods reported in the literature. The results also indicate the effectiveness of the SKF in detecting the switching point between various degradation modes. [ABSTRACT FROM PUBLISHER]

In this paper, a differential neural network (DNN) implemented as a robust observer estimates the dynamics of perturbed uncertain nonlinear systems affected by exogenous unknown inputs. In the first stage, the identification error converges into a neighborhood around the origin. Then, the second-order sliding mode supertwisting algorithm implemented as a robust exact differentiator reconstructed the unknown inputs. The approach proposed in this paper can be applied in the case of full access to the state vector (identification problem) and in the case of partial access to the state vector (estimation problem). In the second case, the nonlinear system under study must have well-defined full relative degree with respect to the unknown input. Numerical examples showed the effectiveness of the proposed algorithm. The first example tested the DNN working as an identifier into a mathematical model describing the dynamics of a spatial minisatellite. The second example (with a DNN implemented as an observer) tested the methodology of this paper over a single link flexible robot manipulator represented in a canonical (Brunovsky) form. In both examples, the mathematical models served as data generators in the testing of the neural networks. Even when not exact mathematical description of both models was used in the input estimation, the accuracy obtained with the DNN is comparable with the case of applying a high-order differentiator with complete knowledge of the plant. [ABSTRACT FROM AUTHOR]

High-dimensional data present in the real world is often corrupted by noise and gross outliers. Principal component analysis (PCA) fails to learn the true low-dimensional subspace in such cases. This is the reason why robust versions of PCA, which put a penalty on arbitrarily large outlying entries, are preferred to perform dimension reduction. In this paper, we argue that it is necessary to study the presence of outliers not only in the observed data matrix but also in the orthogonal complement subspace of the authentic principal subspace. In fact, the latter can seriously skew the estimation of the principal components. A reinforced robustification of principal component pursuit is designed in order to cater to the problem of finding out both types of outliers and eliminate their influence on the final subspace estimation. Simulation results under different design situations clearly show the superiority of our proposed method as compared with other popular implementations of robust PCA. This paper also showcases possible applications of our method in critically tough scenarios of face recognition and video background subtraction. Along with approximating a usable low-dimensional subspace from real-world data sets, the technique can capture semantically meaningful outliers. [ABSTRACT FROM AUTHOR]

This paper is concerned with robust finite-time stabilization for a class of fractional-order neural networks (FNNs) with two types of activation functions (i.e., discontinuous and continuous activation function) under uncertainty. It is worth noting that there exist few results about FNNs with discontinuous activation functions, which is mainly because classical solutions and theories of differential equations cannot be applied in this case. Especially, there is no relevant finite-time stabilization research for such system, and this paper makes up for the gap. The existence of global solution under the framework of Filippov for such system is guaranteed by limiting discontinuous activation functions. According to set-valued analysis and Kakutani’s fixed point theorem, we obtain the existence of equilibrium point. In particular, based on differential inclusion theory and fractional Lyapunov stability theory, several new sufficient conditions are given to ensure finite-time stabilization via a novel discontinuous controller, and the upper bound of the settling time for stabilization is estimated. In addition, we analyze the finite-time stabilization of FNNs with Lipschitz-continuous activation functions under uncertainty. The results of this paper improve corresponding ones of integer-order neural networks with discontinuous and continuous activation functions. Finally, three numerical examples are given to show the effectiveness of the theoretical results. [ABSTRACT FROM PUBLISHER]

Cluster validation, which is the process of evaluating the quality of clustering results, plays an important role for practical machine learning systems. Categorical sequences, such as biological sequences in computational biology, have become common in real-world applications. Different from previous studies, which mainly focused on attribute-value data, in this paper, we work on the cluster validation problem for categorical sequences. The evaluation of sequences clustering is currently difficult due to the lack of an internal validation criterion defined with regard to the structural features hidden in sequences. To solve this problem, in this paper, a novel cluster validity index (CVI) is proposed as a function of clustering, with the intracluster structural compactness and intercluster structural separation linearly combined to measure the quality of sequence clusters. A partition-based algorithm for robust clustering of categorical sequences is also proposed, which provides the new measure with high-quality clustering results by the deterministic initialization and the elimination of noise clusters using an information theoretic method. The new clustering algorithm and the CVI are then assembled within the common model selection procedure to determine the number of clusters in categorical sequence sets. A case study on commonly used protein sequences and the experimental results on some real-world sequence sets from different domains are given to demonstrate the performance of the proposed method. [ABSTRACT FROM PUBLISHER]

In this paper, we propose a biologically inspired appearance model for robust visual tracking. Motivated in part by the success of the hierarchical organization of the primary visual cortex (area V1), we establish an architecture consisting of five layers: whitening, rectification, normalization, coding, and pooling. The first three layers stem from the models developed for object recognition. In this paper, our attention focuses on the coding and pooling layers. In particular, we use a discriminative sparse coding method in the coding layer along with spatial pyramid representation in the pooling layer, which makes it easier to distinguish the target to be tracked from its background in the presence of appearance variations. An extensive experimental study shows that the proposed method has higher tracking accuracy than several state-of-the-art trackers. [ABSTRACT FROM AUTHOR]

Sparse representation has shown an attractive performance in a number of applications. However, the available sparse representation methods still suffer from some problems, and it is necessary to design more efficient methods. Particularly, to design a computationally inexpensive, easily solvable, and robust sparse representation method is a significant task. In this paper, we explore the issue of designing the simple, robust, and powerfully efficient sparse representation methods for image classification. The contributions of this paper are as follows. First, a novel discriminative sparse representation method is proposed and its noticeable performance in image classification is demonstrated by the experimental results. More importantly, the proposed method outperforms the existing state-of-the-art sparse representation methods. Second, the proposed method is not only very computationally efficient but also has an intuitive and easily understandable idea. It exploits a simple algorithm to obtain a closed-form solution and discriminative representation of the test sample. Third, the feasibility, computational efficiency, and remarkable classification accuracy of the proposed l2 regularization-based representation are comprehensively shown by extensive experiments and analysis. The code of the proposed method is available at http://www.yongxu.org/lunwen.html. [ABSTRACT FROM AUTHOR]

A locally linear K Nearest Neighbor (LLK) method is presented in this paper with applications to robust visual recognition. Specifically, the concept of an ideal representation is first presented, which improves upon the traditional sparse representation in many ways. The objective function based on a host of criteria for sparsity, locality, and reconstruction is then optimized to derive a novel representation, which is an approximation to the ideal representation. The novel representation is further processed by two classifiers, namely, an LLK-based classifier and a locally linear nearest mean-based classifier, for visual recognition. The proposed classifiers are shown to connect to the Bayes decision rule for minimum error. Additional new theoretical analysis is presented, such as the nonnegative constraint, the group regularization, and the computational efficiency of the proposed LLK method. New methods such as a shifted power transformation for improving reliability, a coefficients’ truncating method for enhancing generalization, and an improved marginal Fisher analysis method for feature extraction are proposed to further improve visual recognition performance. Extensive experiments are implemented to evaluate the proposed LLK method for robust visual recognition. In particular, eight representative data sets are applied for assessing the performance of the LLK method for various visual recognition applications, such as action recognition, scene recognition, object recognition, and face recognition. [ABSTRACT FROM PUBLISHER]

The existing, semisupervised, spectral clustering approaches have two major drawbacks, i.e., either they cannot cope with multiple categories of supervision or they sometimes exhibit unstable effectiveness. To address these issues, two normalized affinity and penalty jointly constrained spectral clustering frameworks as well as their corresponding algorithms, referred to as type-I affinity and penalty jointly constrained spectral clustering (TI-APJCSC) and type-II affinity and penalty jointly constrained spectral clustering (TII-APJCSC), respectively, are proposed in this paper. TI refers to type-I and TII to type-II. The significance of this paper is fourfold. First, benefiting from the distinctive affinity and penalty jointly constrained strategies, both TI-APJCSC and TII-APJCSC are substantially more effective than the existing methods. Second, both TI-APJCSC and TII-APJCSC are fully compatible with the three well-known categories of supervision, i.e., class labels, pairwise constraints, and grouping information. Third, owing to the delicate framework normalization, both TI-APJCSC and TII-APJCSC are quite flexible. With a simple tradeoff factor varying in the small fixed interval (0, 1], they can self-adapt to any semisupervised scenario. Finally, both TI-APJCSC and TII-APJCSC demonstrate strong robustness, not only to the number of pairwise constraints but also to the parameter for affinity measurement. As such, the novel TI-APJCSC and TII-APJCSC algorithms are very practical for medium- and small-scale semisupervised data sets. The experimental studies thoroughly evaluated and demonstrated these advantages on both synthetic and real-life semisupervised data sets. [ABSTRACT FROM AUTHOR]

The large scale and complex nature of social media data raises the need to scale clustering techniques to big data and make them capable of automatically identifying data clusters with few empirical settings. In this paper, we present our investigation and three algorithms based on the fuzzy adaptive resonance theory (Fuzzy ART) that have linear computational complexity, use a single parameter, i.e., the vigilance parameter to identify data clusters, and are robust to modest parameter settings. The contribution of this paper lies in two aspects. First, we theoretically demonstrate how complement coding, commonly known as a normalization method, changes the clustering mechanism of Fuzzy ART, and discover the vigilance region (VR) that essentially determines how a cluster in the Fuzzy ART system recognizes similar patterns in the feature space. The VR gives an intrinsic interpretation of the clustering mechanism and limitations of Fuzzy ART. Second, we introduce the idea of allowing different clusters in the Fuzzy ART system to have different vigilance levels in order to meet the diverse nature of the pattern distribution of social media data. To this end, we propose three vigilance adaptation methods, namely, the activation maximization (AM) rule, the confliction minimization (CM) rule, and the hybrid integration (HI) rule. With an initial vigilance value, the resulting clustering algorithms, namely, the AM-ART, CM-ART, and HI-ART, can automatically adapt the vigilance values of all clusters during the learning epochs in order to produce better cluster boundaries. Experiments on four social media data sets show that AM-ART, CM-ART, and HI-ART are more robust than Fuzzy ART to the initial vigilance value, and they usually achieve better or comparable performance and much faster speed than the state-of-the-art clustering algorithms that also do not require a predefined number of clusters. [ABSTRACT FROM PUBLISHER]

This paper examines the robust stabilization problem of continuous-time delayed neural networks via the dissipativity-learning approach. A new learning algorithm is established to guarantee the asymptotic stability as well as the $(Q,S,R)$ - $\alpha$ -dissipativity of the considered neural networks. The developed result encompasses some existing results, such as $H_{\infty }$ and passivity performances, in a unified framework. With the introduction of a Lyapunov–Krasovskii functional together with the Legendre polynomial, a novel delay-dependent linear matrix inequality (LMI) condition and a learning algorithm for robust stabilization are presented. Demonstrative examples are given to show the usefulness of the established learning algorithm. [ABSTRACT FROM AUTHOR]

Low-rank matrix estimation arises in a number of statistical and machine learning tasks. In particular, the coefficient matrix is considered to have a low-rank structure in multivariate linear regression and multivariate quantile regression. In this paper, we propose a method called penalized matrix least squares approximation (PMLSA) toward a unified yet simple low-rank matrix estimate. Specifically, PMLSA can transform many different types of low-rank matrix estimation problems into their asymptotically equivalent least-squares forms, which can be efficiently solved by a popular matrix fast iterative shrinkage-thresholding algorithm. Furthermore, we derive analytic degrees of freedom for PMLSA, with which a Bayesian information criterion (BIC)-type criterion is developed to select the tuning parameters. The estimated rank based on the BIC-type criterion is verified to be asymptotically consistent with the true rank under mild conditions. Extensive experimental studies are performed to confirm our assertion. [ABSTRACT FROM AUTHOR]

Improving universality and robustness of the control method is one of the most challenging problems in the field of complex networks (CNs) synchronization. In this paper, a special unilateral coupling finite-time synchronization (UCFTS) method for uncertain CNs is proposed for this challenging problem. Multiple influencing factors are considered, so that the proposed method can be applied to a variety of situations. First, two kinds of drive-response CNs with different sizes are introduced, each of which contains two types of nonidentical nodes and time-varying coupling delay. In addition, the node parameters and topological structure are unknown in drive network. Then, an effective UCFTS control technique is proposed to realize the synchronization of drive-response CNs and identify the unknown parameters and topological structure. Second, the UCFTS of uncertain CNs with four types of nonidentical nodes is further studied. Moreover, both the networks are of unknown parameters, time-varying coupling delay and uncertain topological structure. Through designing corresponding adaptive updating laws, the unknown parameters are estimated successfully and the weight of uncertain topology can be automatically adapted to the appropriate value with the proposed UCFTS. Finally, two experimental examples show the correctness of the proposed scheme. Furthermore, the method is compared with the other three synchronization methods, which shows that our method has a better control performance. [ABSTRACT FROM AUTHOR]

Most feature selection methods first compute a similarity matrix by assigning a fixed value to pairs of objects in the whole data or to pairs of objects in a class or by computing the similarity between two objects from the original data. The similarity matrix is fixed as a constant in the subsequent feature selection process. However, the similarities computed from the original data may be unreliable, because they are affected by noise features. Moreover, the local structure within classes cannot be recovered if the similarities between the pairs of objects in a class are equal. In this paper, we propose a novel local adaptive projection (LAP) framework. Instead of computing fixed similarities before performing feature selection, LAP simultaneously learns an adaptive similarity matrix $\mathbf{S}$ and a projection matrix $\mathbf{W}$ with an iterative method. In each iteration, $\mathbf{S}$ is computed from the projected distance with the learned $\mathbf{W}$ and W is computed with the learned $\mathbf{S}$. Therefore, LAP can learn better projection matrix $\mathbf{W}$ by weakening the effect of noise features with the adaptive similarity matrix. A supervised feature selection with LAP (SLAP) method and an unsupervised feature selection with LAP (ULAP) method are proposed. Experimental results on eight data sets show the superiority of SLAP compared with seven supervised feature selection methods and the superiority of ULAP compared with five unsupervised feature selection methods. [ABSTRACT FROM AUTHOR]

This paper proposes a novel robust zeroing neural-dynamics (RZND) approach as well as its associated model for solving the inverse kinematics problem of mobile robot manipulators. Unlike existing works based on the assumption that neural network models are free of external disturbances, four common forms of time-varying disturbances suppressed by the proposed RZND model are investigated in this paper. In addition, theoretical analyses on the antidisturbance performance are presented in detail to prove the effectiveness and robustness of the proposed RZND model with time-varying disturbances suppressed for solving the inverse kinematics problem of mobile robot manipulators. That is, the RZND model converges toward the exact solution of the inverse kinematics problem of mobile robot manipulators with bounded or zero-oriented steady-state position error. Moreover, simulation studies and comprehensive comparisons with existing neural network models, e.g., the conventional Zhang neural network model and the gradient-based recurrent neural network model, together with extensive tests with four common forms of time-varying disturbances substantiate the efficacy, robustness, and superiority of the proposed RZND approach as well as its time-varying disturbances suppression model for solving the inverse kinematics problem of mobile robot manipulators. [ABSTRACT FROM AUTHOR]

In multiple instance learning, objects are sets (bags) of feature vectors (instances) rather than individual feature vectors. In this paper, we address the problem of how these bags can best be represented. Two standard approaches are to use (dis)similarities between bags and prototype bags, or between bags and prototype instances. The first approach results in a relatively low-dimensional representation, determined by the number of training bags, whereas the second approach results in a relatively high-dimensional representation, determined by the total number of instances in the training set. However, an advantage of the latter representation is that the informativeness of the prototype instances can be inferred. In this paper, a third, intermediate approach is proposed, which links the two approaches and combines their strengths. Our classifier is inspired by a random subspace ensemble, and considers subspaces of the dissimilarity space, defined by subsets of instances, as prototypes. We provide insight into the structure of some popular multiple instance problems and show state-of-the-art performances on these data sets. [ABSTRACT FROM PUBLISHER]

Feature learning plays a central role in pattern recognition. In recent years, many representation-based feature learning methods have been proposed and have achieved great success in many applications. However, these methods perform feature learning and subsequent classification in two separate steps, which may not be optimal for recognition tasks. In this paper, we present a supervised low-rank-based approach for learning discriminative features. By integrating latent low-rank representation (LatLRR) with a ridge regression-based classifier, our approach combines feature learning with classification, so that the regulated classification error is minimized. In this way, the extracted features are more discriminative for the recognition tasks. Our approach benefits from a recent discovery on the closed-form solutions to noiseless LatLRR. When there is noise, a robust Principal Component Analysis (PCA)-based denoising step can be added as preprocessing. When the scale of a problem is large, we utilize a fast randomized algorithm to speed up the computation of robust PCA. Extensive experimental results demonstrate the effectiveness and robustness of our method. [ABSTRACT FROM AUTHOR]

This paper addresses the robust gradient learning (RGL) problem. Gradient learning models aim at learning the gradient vector of some target functions in supervised learning problems, which can be further used to applications, such as variable selection, coordinate covariance estimation, and supervised dimension reduction. However, existing GL models are not robust to outliers or heavy-tailed noise. This paper provides an RGL framework to address this problem in both regression and classification. This is achieved by introducing a robust regression loss function and proposing a robust classification loss. Moreover, our RGL algorithm works in an instance-based kernelized dictionary instead of some fixed reproducing kernel Hilbert space, which may provide more flexibility. To solve the proposed nonconvex model, a simple computational algorithm based on gradient descent is provided and the convergence of the proposed method is also analyzed. We then apply the proposed RGL model to applications, such as nonlinear variable selection and coordinate covariance estimation. The efficiency of our proposed model is verified on both synthetic and real data sets. [ABSTRACT FROM AUTHOR]

This paper studies the zero-error tracking control problem of Euler-Lagrange systems subject to full-state constraints and nonparametric uncertainties. By blending an error transformation with barrier Lyapunov function, a neural adaptive tracking control scheme is developed, resulting in a solution with several salient features: 1) the control action is continuous and $\mathscr C^{1}$ smooth; 2) the full-state tracking error converges to a prescribed compact set around origin within a given finite time at a controllable rate of convergence that can be uniformly prespecified; 3) with Nussbaum gain in the loop, the tracking error further shrinks to zero as $t\to \infty $ ; and 4) the neural network (NN) unit can be safely included in the loop during the entire system operational envelope without the danger of violating the compact set precondition imposed on the NN training inputs. Furthermore, by using the Lyapunov analysis, it is proven that all the signals of the closed-loop systems are semiglobally uniformly ultimately bounded. The effectiveness and benefits of the proposed control method are validated via computer simulation. [ABSTRACT FROM AUTHOR]

Person reidentification usually refers to matching people in different camera views in nonoverlapping multicamera networks. Many existing methods learn a similarity measure by projecting the raw feature to a latent subspace to make the same target’s distance smaller than different targets’ distances. However, the same targets captured in different camera views should hold the same intrinsic attributes while different targets should hold different intrinsic attributes. Projecting all the data to the same subspace would cause loss of such an information and comparably poor discriminability. To address this problem, in this paper, a method based on elastic projections is proposed to learn a pairwise similarity measure for person reidentification. The proposed model learns two projections, positive projection and negative projection, which are both representative and discriminative. The representability refers to: for the same targets captured in two camera views, the positive projection can bridge the corresponding appearance variation and represent the intrinsic attributes of the same targets, while for the different targets captured in two camera views, the negative projection can explore and utilize the different attributes of different targets. The discriminability means that the intraclass distance should become smaller than its original distance after projection, while the interclass distance becomes larger on the contrary, which is the elastic property of the proposed model. In this case, prior information of the original data space is used to give guidance for the learning phase; more importantly, similar targets (but not the same) are effectively reduced by forcing the same targets to become more similar and different targets to become more distinct. The proposed model is evaluated on three benchmark data sets, including VIPeR, GRID, and CUHK, and achieves better performance than other methods. [ABSTRACT FROM PUBLISHER]

This paper is concerned with the robust optimal tracking control strategy for a class of nonlinear multi-input multi-output discrete-time systems with unknown uncertainty via adaptive critic design (ACD) scheme. The main purpose is to establish an adaptive actor-critic control method, so that the cost function in the procedure of dealing with uncertainty is minimum and the closed-loop system is stable. Based on the neural network approximator, an action network is applied to generate the optimal control signal and a critic network is used to approximate the cost function, respectively. In contrast to the previous methods, the main features of this paper are: 1) the ACD scheme is integrated into the controllers to cope with the uncertainty and 2) a novel cost function, which is not in quadric form, is proposed so that the total cost in the design procedure is reduced. It is proved that the optimal control signals and the tracking errors are uniformly ultimately bounded even when the uncertainty exists. Finally, a numerical simulation is developed to show the effectiveness of the present approach. [ABSTRACT FROM AUTHOR]

This paper studies the cooperative global robust output regulation problem for a class of heterogeneous second-order nonlinear uncertain multiagent systems with jointly connected switching networks. The main contributions consist of the following three aspects. First, we generalize the result of the adaptive distributed observer from undirected jointly connected switching networks to directed jointly connected switching networks. Second, by performing a new coordinate and input transformation, we convert our problem into the cooperative global robust stabilization problem of a more complex augmented system via the distributed internal model principle. Third, we solve the stabilization problem by a distributed state feedback control law. Our result is illustrated by the leader-following consensus problem for a group of Van der Pol oscillators. [ABSTRACT FROM PUBLISHER]

In this paper, pinning synchronization on complex networks of networks is investigated, where there are many subnetworks with the interactions among them. The subnetworks and their connections can be regarded as the nodes and interactions of the networks, respectively, which form the networks of networks. In this new setting, the aim is to design pinning controllers on the chosen nodes of each subnetwork so as to reach synchronization behavior. Some synchronization criteria are established for reaching pinning control on networks of networks. Furthermore, the pinning scheme is designed, which shows that the nodes with very low degrees and large degrees are good candidates for applying pinning controllers. Then, the attack and robustness of the pinning scheme are discussed. Finally, a simulation example is presented to verify the theoretical analysis in this paper. [ABSTRACT FROM PUBLISHER]

As a type of indispensable oceanic transportation tools, ship-mounted crane systems are widely employed to transport cargoes and containers on vessels due to their extraordinary flexibility. However, various working requirements and the oceanic environment may cause some uncertain and unfavorable factors for ship-mounted crane control. In particular, to accomplish different control tasks, some plant parameters (e.g., boom lengths, payload masses, and so on) frequently change; hence, most existing model-based controllers cannot ensure satisfactory control performance any longer. For example, inaccurate gravity compensation may result in positioning errors. Additionally, due to ship roll motions caused by sea waves, residual payload swing generally exists, which may result in safety risks in practice. To solve the above-mentioned issues, this paper designs a neural network-based adaptive control method that can provide effective control for both actuated and unactuated state variables based on the original nonlinear ship-mounted crane dynamics without any linearizing operations. In particular, the proposed update law availably compensates parameter/structure uncertainties for ship-mounted crane systems. Based on a 2-D sliding surface, the boom and rope can arrive at their preset positions in finite time, and the payload swing can be completely suppressed. Furthermore, the problem of nonlinear input dead zones is also taken into account. The stability of the equilibrium point of all state variables in ship-mounted crane systems is theoretically proven by a rigorous Lyapunov-based analysis. The hardware experimental results verify the practicability and robustness of the presented control approach. [ABSTRACT FROM AUTHOR]

Clustering aims at naturally grouping the data according to the underlying data distribution. The data distribution is often estimated using a parametric or nonparametric model, e.g., Gaussian mixture or kernel density estimation. Compared with nonparametric models, parametric models are statistically stable, i.e., a small perturbation of data points leads to a small change in the estimated density. However, parametric models are highly sensitive to outliers because the data distribution is far away from the parametric assumptions in the presence of outliers. Given a parametric clustering algorithm, this paper shows how to turn this algorithm into a robust one. The idea is to modify the original parametric density into a semiparametric one. The high-density data that form the core of each cluster are modeled with the original parametric density. The low-density data are often far away from the cluster cores and may have an arbitrary shape, thus are modeled using a nonparametric density. A combination of parametric and nonparametric clustering algorithms is used to group the data modeled as a semiparametric density. From the robust statistical point of view, the proposed method has good robustness properties. We test the proposed algorithm on several synthetic and 70 UCI data sets. The results indicate that the semiparametric method could significantly improve the clustering performance. [ABSTRACT FROM AUTHOR]

Feature selection (FS) is an important component of many pattern recognition tasks. In these tasks, one is often confronted with very high-dimensional data. FS algorithms are designed to identify the relevant feature subset from the original features, which can facilitate subsequent analysis, such as clustering and classification. Structured sparsity-inducing feature selection (SSFS) methods have been widely studied in the last few years, and a number of algorithms have been proposed. However, there is no comprehensive study concerning the connections between different SSFS methods, and how they have evolved. In this paper, we attempt to provide a survey on various SSFS methods, including their motivations and mathematical representations. We then explore the relationship among different formulations and propose a taxonomy to elucidate their evolution. We group the existing SSFS methods into two categories, i.e., vector-based feature selection (feature selection based on lasso) and matrix-based feature selection (feature selection based on lr,p -norm). Furthermore, FS has been combined with other machine learning algorithms for specific applications, such as multitask learning, multilabel learning, multiview learning, classification, and clustering. This paper not only compares the differences and commonalities of these methods based on regression and regularization strategies, but also provides useful guidelines to practitioners working in related fields to guide them how to do feature selection. [ABSTRACT FROM AUTHOR]

This paper addresses the robust low-rank tensor recovery problems. Tensor recovery aims at reconstructing a low-rank tensor from some linear measurements, which finds applications in image processing, pattern recognition, multitask learning, and so on. In real-world applications, data might be contaminated by sparse gross errors. However, the existing approaches may not be very robust to outliers. To resolve this problem, this paper proposes approaches based on the regularized redescending M-estimators, which have been introduced in robust statistics. The robustness of the proposed approaches is achieved by the regularized redescending M-estimators. However, the nonconvexity also leads to a computational difficulty. To handle this problem, we develop algorithms based on proximal and linearized block coordinate descent methods. By explicitly deriving the Lipschitz constant of the gradient of the data-fitting risk, the descent property of the algorithms is present. Moreover, we verify that the objective functions of the proposed approaches satisfy the Kurdyka–Łojasiewicz property, which establishes the global convergence of the algorithms. The numerical experiments on synthetic data as well as real data verify that our approaches are robust in the presence of outliers and still effective in the absence of outliers. [ABSTRACT FROM PUBLISHER]

Learning from saved measurement and control data to refine the performance of output tracking is the core feature of iterative learning control (ILC). Even though this implementation process of ILC does not need any model knowledge, ILC typically requires the strict repetitiveness of the control systems, especially on the plant models of them. The questions of interest in this paper are: 1) whether and how can robust ILC problems be solved with respect to the nonrepetitive (or iteration-dependent) model uncertainties and 2) can convergence conditions be developed with the effective contraction mapping (CM)-based approach to ILC? The answers to these questions are affirmative, and the CM-based approach is applicable to robust ILC that accommodates certain nonrepetitive uncertainties, especially in the plant models. In particular, an $H_{\infty }$ -norm condition is proposed to ensure the robust ILC convergence, which can be solved to determine learning gain matrices. Simulation tests are performed to illustrate the validity of our presented $H_{\infty }$ -based analysis results. [ABSTRACT FROM AUTHOR]

Tensor data (i.e., the data having multiple dimensions) are quickly growing in scale in many practical applications, which poses new challenges for data modeling and analysis approaches, such as high-order relations of large complexity, gross noise, and varying data scale. Existing low-rank data analysis methods, which are effective at analyzing matrix data, may fail in the regime of tensor data due to these challenges. A robust and scalable low-rank tensor modeling method is heavily desired. In this paper, we develop an online robust low-rank tensor modeling (ORLTM) method to address these challenges. The ORLTM method leverages the high-order correlations among all tensor modes to model an intrinsic low-rank structure of streaming tensor data online and can effectively analyze data residing in a mixture of multiple subspaces by virtue of dictionary learning. ORLTM consumes a very limited memory space that remains constant regardless of the increase of tensor data size, which facilitates processing tensor data at a large scale. More concretely, it models each mode unfolding of streaming tensor data using the bilinear formulation of tensor nuclear norms. With this reformulation, ORLTM employs a stochastic optimization algorithm to learn the tensor low-rank structure alternatively for online updating. To capture the final tensors, ORLTM uses an average pooling operation on folded tensors in all modes. We also provide the analysis regarding computational complexity, memory cost, and convergence. Moreover, we extend ORLTM to the image alignment scenario by incorporating the geometrical transformations and linearizing the constraints. Extensive empirical studies on synthetic database and three practical vision tasks, including video background subtraction, image alignment, and visual tracking, have demonstrated the superiority of the proposed method. [ABSTRACT FROM AUTHOR]

In pattern recognition and data mining, clustering is a classical technique to group matters of interest and has been widely employed to numerous applications. Among various clustering algorithms, K-means (KM) clustering is most popular for its simplicity and efficiency. However, with the rapid development of the social network, high-dimensional data are frequently generated, which poses a considerable challenge to the traditional KM clustering as the curse of dimensionality. In such scenarios, it is difficult to directly cluster such high-dimensional data that always contain redundant features and noises. Although the existing approaches try to solve this problem using joint subspace learning and KM clustering, there are still the following limitations: 1) the discriminative information in low-dimensional subspace is not well captured; 2) the intrinsic geometric information is seldom considered; and 3) the optimizing procedure of a discrete cluster indicator matrix is vulnerable to noises. In this paper, we propose a novel clustering model to cope with the above-mentioned challenges. Within the proposed model, discriminative information is adaptively explored by unifying local adaptive subspace learning and KM clustering. We extend the proposed model using a robust $l_{2,1}$ -norm loss function, where the robust cluster centroid can be calculated in a weighted iterative procedure. We also explore and discuss the relationships between the proposed algorithm and several related studies. Extensive experiments on kinds of benchmark data sets demonstrate the advantage of the proposed model compared with the state-of-the-art clustering approaches. [ABSTRACT FROM AUTHOR]

This paper proposes a novel and simple multilayer feature learning method for image classification by employing the extreme learning machine (ELM). The proposed algorithm is composed of two stages: the multilayer ELM (ML-ELM) feature mapping stage and the ELM learning stage. The ML-ELM feature mapping stage is recursively built by alternating between feature map construction and maximum pooling operation. In particular, the input weights for constructing feature maps are randomly generated and hence need not be trained or tuned, which makes the algorithm highly efficient. Moreover, the maximum pooling operation enables the algorithm to be invariant to certain transformations. During the ELM learning stage, elastic-net regularization is proposed to learn the output weight. Elastic-net regularization helps to learn more compact and meaningful output weight. In addition, we preprocess the input data with the dense scale-invariant feature transform operation to improve both the robustness and invariance of the algorithm. To evaluate the effectiveness of the proposed method, several experiments are conducted on three challenging databases. Compared with the conventional deep learning methods and other related ones, the proposed method achieves the best classification results with high computational efficiency. [ABSTRACT FROM AUTHOR]

This paper deals with sparse signal reconstruction by designing a discrete-time projection neural network. Sparse signal reconstruction can be converted into an $L_{1}$ -minimization problem, which can also be changed into the unconstrained basis pursuit denoising problem. To solve the $L_{1}$ -minimization problem, an iterative algorithm is proposed based on the discrete-time projection neural network, and the global convergence of the algorithm is analyzed by using Lyapunov method. Experiments on sparse signal reconstruction and several popular face data sets are organized to illustrate the effectiveness and performance of the proposed algorithm. The experimental results show that the proposed algorithm is not only robust to different levels of sparsity and amplitude of signals and the noise pixels but also insensitive to the diverse values of scalar weight. Moreover, the value of the step size of the proposed algorithm is close to 1/2, thus a fast convergence rate is potentially possible. Furthermore, the proposed algorithm achieves better classification performance compared with some other algorithms for face recognition. [ABSTRACT FROM AUTHOR]

Mean square error (MSE) is the most prominent criterion in training neural networks and has been employed in numerous learning problems. In this paper, we suggest a group of novel robust information theoretic backpropagation (BP) methods, as correntropy-based conjugate gradient BP (CCG-BP). CCG-BP algorithms converge faster than the common correntropy-based BP algorithms and have better performance than the common CG-BP algorithms based on MSE, especially in nonGaussian environments and in cases with impulsive noise or heavy-tailed distributions noise. In addition, a convergence analysis of this new type of method is particularly considered. Numerical results for several samples of function approximation, synthetic function estimation, and chaotic time series prediction illustrate that our new BP method is more robust than the MSE-based method in the sense of impulsive noise, especially when SNR is low. [ABSTRACT FROM AUTHOR]

The robust Huber’s M-estimator is widely used in signal and image processing, classification, and regression. From an optimization point of view, Huber’s M-estimation problem is often formulated as a large-sized quadratic programming (QP) problem in view of its nonsmooth cost function. This paper presents a generalized regression estimator which minimizes a reduced-sized QP problem. The generalized regression estimator may be viewed as a significant generalization of several robust regression estimators including Huber’s M-estimator. The performance of the generalized regression estimator is analyzed in terms of robustness and approximation accuracy. Furthermore, two low-dimensional recurrent neural networks (RNNs) are introduced for robust estimation. The two RNNs have low model complexity and enhanced computational efficiency. Finally, the experimental results of two examples and an application to image restoration are presented to substantiate superior performance of the proposed method over conventional algorithms for robust regression estimation in terms of approximation accuracy and convergence rate. [ABSTRACT FROM AUTHOR]

The independent component analysis (ICA) is a widely used method for solving blind separation problems. The ICA assumes that the sources are independent of each other and extracts them by maximizing their non-Gaussianity as the objective function. There are the two types of non-Gaussianity of the sources (the super-Gaussian type with the positive kurtosis and the sub-Gaussian one with the negative kurtosis). In this paper, we propose a new objective function unifying the two types of non-Gaussianity naturally, which is derived by applying the Gaussian approximation to the distribution of sources in the second-order polynomial feature space. The proposed objective function [called the adaptive ICA function (AIF)] is a simple form given as a summation of weighted fourth-order statistics, where the weights are adaptively estimated by the current kurtoses. The first practical advantage of the AIF is that it can extract the sources one by one in the descending order of the criterion of non-Gaussianity. It can solve the permutation ambiguity problem. The second and more important advantage is that it can estimate the number of non-Gaussian sources by the Akaike information criterion irrespective of the specific form of their distributions. In order to utilize the above-mentioned advantages of the AIF, we construct a new algorithm named the ordering ICA by extending the fast ICA. Experimental results verify that the ordering ICA can estimate the number of non-Gaussian sources correctly in both artificial and real data sets. [ABSTRACT FROM AUTHOR]

In this paper, we study the cooperative global robust practical output regulation problem for a class of second-order uncertain nonlinear multiagent systems via a distributed event-triggered state feedback control strategy. Compared with the existing work, one of the main challenges is that we need to design two distributed internal models to learn both the desired steady-state state and steady-state input for each agent. Moreover, to obtain a directly implementable digital control law, the two distributed internal models of each agent only depend on the sampled states of the neighboring agents and itself. As a result, the resulting augmented system is more complicated, and the control law needs to be recursively designed. To overcome the difficulty, we propose a novel distributed event-triggered control law and a novel distributed event-triggered mechanism to deal with our problem. By adjusting a design parameter in the proposed event-triggered mechanism, we show that the Zeno behavior does not happen and the ultimate bound of the tracking error can be made arbitrarily small. Our design will be illustrated by two examples. [ABSTRACT FROM AUTHOR]

Sparse Bayesian learning has emerged as a powerful tool to tackle various image classification tasks. The existing sparse Bayesian models usually use independent Gaussian distribution as the prior knowledge for the noise. However, this assumption often contradicts to the practical observations in which the noise is long tail and pixels containing noise are spatially correlated. To handle the practical noise, this paper proposes to partition the noise image into several 2-D groups and adopt the long-tail distribution, i.e., the scale mixture of the matrix Gaussian distribution, to model each group to capture the intragroup correlation of the noise. Under the nonparametric Bayesian estimation, the low-rank-induced prior and the matrix Gamma distribution prior are imposed on the covariance matrix of each group, respectively, to induce two Bayesian correlated group regression (BCGR) methods. Moreover, the proposed methods are extended to the case with unknown group structure. Our BCGR method provides an effective way to automatically fit the noise distribution and integrates the long-tail attribute and structure information of the practical noise into model. Therefore, the estimated coefficients are better for reconstructing the desired data. We apply BCGR to address image classification task and utilize the learned covariance matrices to construct a grouped Mahalanobis distance to measure the reconstruction residual of each class in the design of a classifier. Experimental results demonstrate the effectiveness of our new BCGR model. [ABSTRACT FROM AUTHOR]

The low-rank tensor factorization (LRTF) technique has received increasing attention in many computer vision applications. Compared with the traditional matrix factorization technique, it can better preserve the intrinsic structure information and thus has a better low-dimensional subspace recovery performance. Basically, the desired low-rank tensor is recovered by minimizing the least square loss between the input data and its factorized representation. Since the least square loss is most optimal when the noise follows a Gaussian distribution, $L_{1}$ -norm-based methods are designed to deal with outliers. Unfortunately, they may lose their effectiveness when dealing with real data, which are often contaminated by complex noise. In this paper, we consider integrating the noise modeling technique into a generalized weighted LRTF (GWLRTF) procedure. This procedure treats the original issue as an LRTF problem and models the noise using a mixture of Gaussians (MoG), a procedure called MoG GWLRTF. To extend the applicability of the model, two typical tensor factorization operations, i.e., CANDECOMP/PARAFAC factorization and Tucker factorization, are incorporated into the LRTF procedure. Its parameters are updated under the expectation–maximization framework. Extensive experiments indicate the respective advantages of these two versions of MoG GWLRTF in various applications and also demonstrate their effectiveness compared with other competing methods. [ABSTRACT FROM AUTHOR]

Facial expression recognition is a challenging task that involves detection and interpretation of complex and subtle changes in facial muscles. Recent advances in feed-forward deep neural networks (DNNs) have offered improved object recognition performance. Sparse feature learning in feed-forward DNN models offers further improvement in performance when compared to the earlier handcrafted techniques. However, the depth of the feed-forward DNNs and the computational complexity of the models increase proportional to the challenges posed by the facial expression recognition problem. The feed-forward DNN architectures do not exploit another important learning paradigm, known as recurrency, which is ubiquitous in the human visual system. Consequently, this paper proposes a novel biologically relevant sparse-deep simultaneous recurrent network (S-DSRN) for robust facial expression recognition. The feature sparsity is obtained by adopting dropout learning in the proposed DSRN as opposed to usual handcrafting of additional penalty terms for the sparse representation of data. Theoretical analysis of S-DSRN shows that the dropout learning offers desirable properties such as sparsity, and prevents the model from overfitting. Experimental results also suggest that the proposed method yields better performance accuracy, requires reduced number of parameters, and offers reduced computational complexity than that of the previously reported state-of-the-art feed-forward DNNs using two of the most widely used publicly available facial expression data sets. Furthermore, we show that by combining the proposed neural architecture with a state-of-the-art metric learning technique significantly improves the overall recognition performance. Finally, a graphical processing unit (GPU)-based implementation of S-DSRN is obtained for real-time applications. [ABSTRACT FROM AUTHOR]

Parameter in learning problems (usually arising from the tradeoff between training error minimization and regularization) is often tuned by cross validation (CV). A solution path provides a compact representation of all optimal solutions, which can be used to determine the parameter with the global minimum CV error, without solving original optimization problems multiple times based on grid search. However, existing solution path algorithms do not provide a unified implementation to various learning problems. In this paper, we first introduce a general parametric quadratic programming (PQP) problem that can be instantiated to an extensive number of learning problems. Then, we propose a generalized solution path (GSP) for the general PQP problem. Particularly, we use the $QR$ decomposition to handle singularities in GSP. Finally, we analyze the finite convergence and the time complexity of GSP. Our experimental results on a variety of data sets not only confirm the identicality between GSP and several existing solution path algorithms but also show the superiority of our GSP over the existing solution path algorithms on both generalization and robustness. Finally, we provide a practical guild of using the GSP to solve two important learning problems, i.e., generalized error path and Ivanov SVM. [ABSTRACT FROM AUTHOR]

Neural networks (NNs) have emerged as a powerful illustrative diagram for the brain. Unveiling the mechanism of neural-dynamic evolution is one of the crucial steps toward understanding how the brain works and evolves. Inspired by the universal existence of impulses in many real systems, this paper formulates a type of hybrid NNs (HNNs) with impulses, time delays, and interval uncertainties, and studies its global dynamic evolution by a robust interval analysis. The HNNs incorporate both continuous-time implementation and impulsive jump in mutual activations, where time delays and interval uncertainties are represented simultaneously. By constructing a Banach contraction mapping, the existence and uniqueness of the equilibrium of the HNN model are proved and analyzed in detail. Based on nonsmooth Lyapunov functions and delayed impulsive differential equations, new criteria are derived for ensuring the global robust exponential stability of the HNNs. Convergence analysis together with illustrative examples show the effectiveness of the theoretical results. [ABSTRACT FROM AUTHOR]

Recently, an adaptive exponential trigonometric functional link neural network (AETFLN) architecture has been introduced to enhance the nonlinear processing capability of the trigonometric functional link neural network (TFLN). However, it suffers from slow convergence speed, heavy computational burden, and poor robustness to noise in nonlinear acoustic echo cancellation, especially in the double-talk scenario. To reduce its computational complexity and improve its robustness against impulsive noise, this paper develops a recursive adaptive sparse exponential TFLN (RASETFLN). Based on sparse representations of functional links, the robust proportionate adaptive algorithm is deduced from the robust cost function over the RASETFLN in impulsive noise environments. Theoretical analysis shows that the proposed RASETFLN is stable under certain conditions. Finally, computer simulations illustrate that the proposed RASETFLN achieves much improved performance over the AETFLN in several nonlinear scenarios in terms of convergence rate, steady-state error, and robustness against noise. [ABSTRACT FROM AUTHOR]

Optimal operation of an industrial blast furnace (BF) ironmaking process largely depends on a reliable measurement of molten iron quality (MIQ) indices, which are not feasible using the conventional sensors. This paper proposes a novel data-driven robust modeling method for the online estimation and control of MIQ indices. First, a nonlinear autoregressive exogenous (NARX) model is constructed for the MIQ indices to completely capture the nonlinear dynamics of the BF process. Then, considering that the standard least-squares support vector regression (LS-SVR) cannot directly cope with the multioutput problem, a multitask transfer learning is proposed to design a novel multioutput LS-SVR (M-LS-SVR) for the learning of the NARX model. Furthermore, a novel M-estimator is proposed to reduce the interference of outliers and improve the robustness of the M-LS-SVR model. Since the weights of different outlier data are properly given by the weight function, their corresponding contributions on modeling can properly be distinguished, thus a robust modeling result can be achieved. Finally, a novel multiobjective evaluation index on the modeling performance is developed by comprehensively considering the root-mean-square error of modeling and the correlation coefficient on trend fitting, based on which the nondominated sorting genetic algorithm II is used to globally optimize the model parameters. Both experiments using industrial data and industrial applications illustrate that the proposed method can eliminate the adverse effect caused by the fluctuation of data in BF process efficiently. This indicates its stronger robustness and higher accuracy. Moreover, control testing shows that the developed model can be well applied to realize data-driven control of the BF process. [ABSTRACT FROM AUTHOR]

This paper presents a novel adaptive controller for controlling an autonomous helicopter with unknown inertial matrix to asymptotically track the desired trajectory. To identify the unknown inertial matrix included in the attitude dynamic model, this paper proposes a new structural identifier that differs from those previously proposed in that it additionally contains a neural networks (NNs) mechanism and a robust adaptive mechanism, respectively. Using the NNs to compensate the unknown aerodynamic forces online and the robust adaptive mechanism to cancel the combination of the overlarge NNs compensation error and the external disturbances, the new robust neural identifier exhibits a better identification performance in the complex flight environment. Moreover, an optimized algorithm is included in the NNs mechanism to alleviate the burdensome online computation. By the strict Lyapunov argument, the asymptotic convergence of the inertial matrix identification error, position tracking error, and attitude tracking error to arbitrarily small neighborhood of the origin is proved. The simulation and implementation results are provided to evaluate the performance of the proposed controller. [ABSTRACT FROM PUBLISHER]

In this paper, a neurodynamic optimization approach is proposed for synthesizing high-order descriptor linear systems with state feedback control via robust pole assignment. With a new robustness measure serving as the objective function, the robust eigenstructure assignment problem is formulated as a pseudoconvex optimization problem. A neurodynamic optimization approach is applied and shown to be capable of maximizing the robust stability margin for high-order singular systems with guaranteed optimality and exact pole assignment. Two numerical examples and vehicle vibration control application are discussed to substantiate the efficacy of the proposed approach. [ABSTRACT FROM PUBLISHER]

Novelty detection models aim to find the minimum volume set covering a given probability mass. This paper proposes a robust single-class support vector machine (SSVM) for novelty detection, which is mainly based on the worst case conditional value-at-risk minimization. By assuming that every input is subject to an uncertainty with a specified symmetric support, this robust formulation results in a maximization term that is similar to the regularization term in the classical SSVM. When the uncertainty set is \ell 1 -norm, $\ell _\infty $ -norm or box, its training can be reformulated to a linear program; while the uncertainty set is \ell _{2} -norm or ellipsoidal, its training is a tractable second-order cone program. The proposed method has a nice consistent statistical property. As the training size goes to infinity, the estimated normal region converges to the true provided that the magnitude of the uncertainty set decreases in a systematic way. The experimental results on three data sets clearly demonstrate its superiority over three benchmark models. [ABSTRACT FROM AUTHOR]

Scalable causal discovery is an essential technology to a wide spectrum of applications, including biomedical studies and social network evolution analysis. To tackle the difficulty of high dimensionality, a number of solutions are proposed in the literature, generally dividing the original variable domain into smaller subdomains by computation intensive partitioning strategies. These approaches usually suffer significant structural errors when the partitioning strategies fail to recognize true causal edges across the output subdomains. Such a structural error accumulates quickly with the growing depth of recursive partitioning, due to the lack of correction mechanism over causally connected variables when they are wrongly divided into two subdomains, finally jeopardizing the robustness of the integrated results. This paper proposes a completely different strategy to solve the problem, powered by a lightweight random partitioning scheme together with a carefully designed merging algorithm over results from the random partitions. Based on the randomness properties of the partitioning scheme, we design a suite of tricks for the merging algorithm, in order to support propagation-based significance enhancement, maximal acyclic subgraph causal ordering, and order-sensitive redundancy elimination. Theoretical studies as well as empirical evaluations verify the genericity, effectiveness, and scalability of our proposal on both simulated and real-world causal structures when the scheme is used in combination with a variety of causal solvers known effective on smaller domains. [ABSTRACT FROM AUTHOR]

Deep neural networks (NNs) are the state-of-the-art models for understanding the content of images and videos. However, implementing deep NNs in embedded systems is a challenging task, e.g., a typical deep belief network could exhaust gigabytes of memory and result in bandwidth and computational bottlenecks. To address this challenge, this paper presents an algorithm and hardware codesign for efficient deep neural computation. A hardware-oriented deep learning algorithm, named the deep adaptive network, is proposed to explore the sparsity of neural connections. By adaptively removing the majority of neural connections and robustly representing the reserved connections using binary integers, the proposed algorithm could save up to 99.9% memory utility and computational resources without undermining classification accuracy. An efficient sparse-mapping-memory-based hardware architecture is proposed to fully take advantage of the algorithmic optimization. Different from traditional Von Neumann architecture, the deep-adaptive network on chip (DANoC) brings communication and computation in close proximity to avoid power-hungry parameter transfers between on-board memory and on-chip computational units. Experiments over different image classification benchmarks show that the DANoC system achieves competitively high accuracy and efficiency comparing with the state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Stability is a key issue during spiking neural network training using SpikeProp. The inherent nonlinearity of Spiking Neuron means that the learning manifold changes abruptly; therefore, we need to carefully choose the learning steps at every instance. Other sources of instability are the external disturbances that come along with training sample as well as the internal disturbances that arise due to modeling imperfection. The unstable learning scenario can be indirectly observed in the form of surges, which are sudden increases in the learning cost and are a common occurrence during SpikeProp training. Research in the past has shown that proper learning step size is crucial to minimize surges during training process. To determine proper learning step in order to avoid steep learning manifolds, we perform weight convergence analysis of SpikeProp learning in the presence of disturbance signals. The weight convergence analysis is further extended to robust stability analysis linked with overall system error. This ensures boundedness of the total learning error with minimal assumption of bounded disturbance signals. These analyses result in the learning rate normalization scheme, which are the key results of this paper. The performance of learning using this scheme has been compared with the prevailing methods for different benchmark data sets and the results show that this method has stable learning reflected by minimal surges during learning, higher success in training instances, and faster learning as well. [ABSTRACT FROM AUTHOR]