Search

The distributed online convex optimization problem with time-varying constraints for multi-agent networks is addressed in this article. The purpose is to optimize a sequence of time-varying global cost functions defined as the accumulated values of local cost functions, also attempt to meet the requirement of a sequence of time-varying coupled constraint functions which denote the sum of local constraint functions. Cost functions and constraint functions are unknown to agents beforehand. It is supposed that each agent in the network communicates with its neighbours through a uniformly strongly connected sequence of time-varying directed communication topologies. This paper proposes the bandit distributed primal-dual mirror descent push-sum (BDPDMDPS) algorithm constructed by bandit primal-dual, mirror descent and push-sum methods. Operational performance of the presented algorithm is measured by expected regret and expected constraint violation, both of which are proved to be sublinear with respect to the total iteration span T in this paper. Finally, a numerical simulation example is shown, which confirms the results for expected regret and expected constraint violation of BDPDMDPS algorithm. [ABSTRACT FROM AUTHOR]

A continuous-time distributed algorithm based on multi-agent systems is proposed to solve the convex nonsmooth optimization with globally coupled constraints in this paper. Firstly, the relationship of equivalence between the equilibrium and the optimal solution of the system is proved by using the properties of projection operator and saddle point. In addition, the stability of the algorithm is analyzed by means of Lie derivative and set-valued LaSalle's invariance principle. In particular, the trajectory of the proposed algorithm from any initial point can converge to the global optimal solution. Finally, the effectiveness of the designed algorithm is verified by numerical simulations. [ABSTRACT FROM AUTHOR]

This paper studies the differential privacy-preserving problem for multi-agent systems (MASs) in the presence of antagonistic information over signed digraph. As for the structurally balanced case, an ε -differential privacy algorithm is proposed, upon which some sufficient conditions guaranteeing almost sure bipartite consensus are given. Based on the above scheme, the tradeoff between the system performance and the privacy guarantee is elaborated, and the optimal noise is also devised. Moreover, the proposed privacy preserving scheme is further applied to the scenario of structurally unbalanced graph, where a criterion with respect to almost sure stability of the considered system is derived, as well as the privacy preserving condition. This extends the balanced interaction scenario, and consequently the cooperative multi-agent systems. Finally, numerical simulations are presented to demonstrate the effectiveness of our results. [ABSTRACT FROM AUTHOR]

Abstract Dealing with high-dimensional dataset has always been an important problem and feature selection is one of useful tools. In this paper, we develop a new filter based supervised feature selection method by combining Trace Ratio Criterion of Linear Discriminant Analysis (TRC-LDA) and group sparsity regularization. The filter based supervised feature selection method is a classifier-independent method while the TRC-LDA criterion is a recently developed criterion for dimensionality reduction that can well preserve discriminative information of dataset. However, there are seldom methods by utilizing TRC-LDA criterion for feature selection. On the other hand, imposing the l2,0-norm to the projection matrix of TRC-LDA will force some rows in it to be zero while keep other rows nonzero making the index of nonzero rows to be the selected features, however, l 2,0 -nom minimizing problem is NP-hard and intractable. To solve the above problem, in this paper, we develop a new method, namely, Trace Ratio Criterion Discriminative Feature Selection (TRC-DFS), for feature selection. The proposed TRC-DFS has imposed l 2,1 -norm, i.e. an approximation of l 2,0 -norm, to the projection matrix W of TRC-LDA to achieve feature selection. As a result, the proposed TRC-DFS can both achieve feature selection as well as capture the discriminative structure of data. We also extend the proposed method with l 2, p -norm (0 < p < 1) regularization to grasp more sparse property and develop an iterative approach to calculate the optimal solution, which is rigorously proved to be converged. Extensive simulations based on several real-world datasets verify the effectiveness of the proposed methods. [ABSTRACT FROM AUTHOR]

Abstract Twin extreme learning machine (TELM) is an efficient and effective method for pattern classification, based on widely known extreme learning machine (ELM). However, TELM is mainly used to deal with supervised learning problems. In this paper, we extend TELM to handle semi-supervised learning problems and propose a novel Laplacian twin extreme learning machine (LapTELM), which simultaneously trains two related and paired semi-supervised ELMs with two nonparallel separating planes for the final classification. The proposed method exploits the geometry structure property of the unlabeled samples and incorporates it as a manifold regularization term. This allows LapTELM to reap the benefits of fully exploring the plentiful unlabeled samples while retaining the learning ability and efficiency of TELM. Moreover, the paper shows that semi-supervised and supervised TELM can form an unified learning framework. Compared with several mainstream semi-supervised learning methods, the experimental results on the synthetic and several real-world data sets verify the effectiveness and efficiency of LapTELM. [ABSTRACT FROM AUTHOR]

This paper examines a private distributed online convex optimization problem in which each agent strives to minimize the sum of objective functions while also tending to keep their individual objective functions confidential. We use differential privacy as the metric to safeguard each agent's privacy and offer a distributed online mirror descent technique that is differentially private. We demonstrate that for strongly convex objective functions, our proposed algorithm satisfies the expected regret bound of O ln T while maintaining differential privacy, where T is the number of iterations. The established expected regret bound matches the optimal theoretical regret bound with respect to T. We further illuminate the trade-off between the extent of privacy-preserving and the expected regret bound through numerical simulations. [ABSTRACT FROM AUTHOR]

Abstract This paper deals with the decentralized optimal tracking control problem of large-scale interconnected systems with constrained-input. The large-scale interconnected systems are firstly transformed to several nominal isolated subsystems. Then, nominal isolated subsystems tracking problem is solved via integral reinforcement learning (IRL) method. It is proved that the solved optimal controllers ensure the boundedness of the original systems tracking error. The actor-critic neural network (NN) technique is used to approximate the critic cost and control policy to implement the IRL algorithm. The least squares approach is employed to solve the weights of actor-critic NN by using only system data. A simulation example is provided to verify the effectiveness of the controllers by comparing with the controllers without considering constrained-input. [ABSTRACT FROM AUTHOR]

Abstract This paper investigates the adaptive consensus problem of multi-agent nonlinear systems with odd impulsive sequence. To reduce the required information transmitted among the nodes, the distributed impulsive control protocol is designed. Considering the unknown system parameters in real system, the adaptive control technique is also used and some novel sufficient consensus conditions are derived to achieve the consensus of multi-agent nonlinear systems. The proposed odd impulsive control can obtain larger control region than the common impulsive consensus method with fixed impulsive instants, which is more flexible and reliable in practical application. Finally, two simulations are given to illustrate the effectiveness of the theoretical results. [ABSTRACT FROM AUTHOR]

Abstract Multi-Instance learning (MIL) aims to predict labels of unlabeled bags by training a model with labeled bags. The usual assumption of existing MIL methods is that the underlying distribution of training data is the same as that of the testing data. However, this assumption may not be valid in practice, especially when training data from a source domain and testing data from a target domain are drawn from different distributions. In this paper, we put forward a novel algorithm Multi-Instance Transfer Metric Learning (MITML). Specially, MITML first attempts to bridge the distributions of different domains by using the bag weighting method. Then a consistent maximum likelihood estimation method is learned to construct an optimal distance metric and exploited to classify testing bags. Comprehensive experimental results on benchmark datasets have demonstrated that the learning performance of the proposed MITML algorithm is better than those of other state-of-the-art MIL algorithms. [ABSTRACT FROM AUTHOR]

The geometric measure of entanglement of a multipartite pure state is defined it terms of its geometric distance from the set of separable pure states. The quantum eigenvalue problem is derived to compute the separable pure state nearest to the given multipartite pure state. Computing the modulus largest quantum eigenvalue for a multipartite pure state is equivalent to finding the best complex rank-one approximation of the complex unit tensors, associated with the multipartite pure states. This paper is devoted to present a complex-valued neural networks approach for the computation of the quantum eigenvalue problem for multipartite pure states. We design the neural networks for computing the best rank-one tensor approximation of complex tensors, and prove that the solution of the networks is locally asymptotically stable in the sense of Lyapunov stability theory. This solution also converges to the local optimal solutions of the best complex rank-one tensor approximation. We illustrate our theoretical results via numerical simulations. [ABSTRACT FROM AUTHOR]

In this paper, a self-organizing fuzzy neural network with adaptive gradient algorithm (SOFNN-AGA) is proposed for nonlinear systems modeling. First, a potentiality of fuzzy rules (PFR) method is introduced by using the output of normalized layer and the error reduction ratio (ERR) in the training process. And a structure learning approach is developed to determine the network size based on PFR. Second, a novel adaptive gradient algorithm (AGA) with adaptive learning rate is designed to adjust the parameters of SOFNN-AGA. Moreover, a theoretical analysis on the convergence of SOFNN-AGA is given to show the efficiency in both fixed structure and self-organizing structure cases. Finally, to demonstrate the merits of SOFNN-AGA, simulation and experimental results of several benchmark problems and a real world application are examined for nonlinear systems modeling with comparisons against other existing methods. Some promising results are reported in this study, indicating that the proposed SOFNN-AGA performs better favorably in terms of both convergence speed and modeling accuracy. [ABSTRACT FROM AUTHOR]

ℓ p regularization has been a popular pruning method for neural networks. The parameter p was usually set as 0 < p ≤ 2 in the literature, and practical training algorithms with ℓ 0 regularization are lacking due to the NP-hard nature of the ℓ 0 regularization problem; however, the ℓ 0 regularization tends to produce the sparsest solution, corresponding to the most parsimonious network structure which is desirable in view of the generalization ability. To this end, this paper considers an online gradient training algorithm with smoothing ℓ 0 regularization (OGTSL0) for feedforward neural networks, where the ℓ 0 regularizer is approximated by a series of smoothing functions. The underlying principle for the sparsity of OGTSL0 is provided, and the convergence of the algorithm is also theoretically analyzed. Simulation examples support the theoretical analysis and illustrate the superiority of the proposed algorithm. [ABSTRACT FROM AUTHOR]

Abstract: Blind source extraction (BSE) is a special class of blind source separation (BSS) method. Due to its low computation load and fast processing speed, BSE has become one of the promising methods in signal processing and analysis. This paper addresses BSE problem when a desired source signal has temporal structures. Based on the generalized autocorrelations of the desired signal and the non-Gaussianity of its innovations, we develop an objective function. Maximizing this objective function, we present a BSE algorithm and further give its stability analysis in this paper. Simulations on image data and electrocardiogram (ECG) data indicate its better performance and the better property of tolerance to the estimation error of the time delay. [Copyright &y& Elsevier]

This paper focuses on studying the H ∞ state estimation of static neural networks with interval time-varying delays via augmented Lyapunov–Krasovskii functional. By constructing a suitable augmented Lyapunov–Krasovskii functional with triple integral terms and linear matrix inequality technique, the delay-dependent criteria are conferred so that the error system is globally asymptotically stable with H ∞ performance. The activation functions are assumed to satisfy sector-like nonlinearities. The desired estimator gain matrix can be characterized in terms of the solution to linear matrix inequalities, which can be easily solved by some standard numerical algorithms. Numerical simulation is given to demonstrate the effectiveness and superiority of the proposed method comparing with some existing results. [ABSTRACT FROM AUTHOR]

This paper studies the coordination of nonlinear multi-agent systems with multiple leaders and stochastic disturbance. Two impulsive control algorithms are proposed to make all agents to track the convex set determined by the multiple leaders under both fixed and switching topologies. Based on Lyapunov function and comparison principle of impulsive systems, some results on the convergence of the controlled systems are obtained. Finally, some numerical simulations demonstrate the validity of the proposed control algorithms. [ABSTRACT FROM AUTHOR]

Abstract: Traffic sign recognition is an important and active research topic of intelligent transport system. With a constant increasing of the training database size, not only the recognition accuracy, but also the computation complexity should be considered in designing a feasible recognition approach. In this paper, an effective and efficient algorithm based on a relatively new artificial neural network, extreme learning machine (ELM), is proposed for traffic sign recognition. In the proposed algorithm, the locally normalized histograms of the oriented gradient (HOG) descriptors, which are extracted from the traffic sign images, are used as the features and the inputs of the ELM classification model. Moreover, the ratio of feature's between-category to within-category sums of squares (BW) is designed as a feature selection criterion to improve the recognition accuracy and to decrease the computation burden. Application on a well known database, German traffic sign recognition benchmark (GTSRB) dataset, demonstrates the feasibility and efficiency of the proposed BW-ELM model. [Copyright &y& Elsevier]

Abstract: This paper investigates a flocking problem of multiple agents with heterogeneous nonlinear dynamics. In order to avoid fragmentation, we construct a potential function and a connectivity-preserving flocking algorithm to enable the multiple agents to move with the same velocity while preserving the connectivity of underlying networks with a mild assumption that the initial network is connected and the coupling strength of the initial network of the nonlinear velocity consensus term is larger than a threshold value. Furthermore the proposed flocking algorithm is extended to solve the problem of multi-agent systems with a nonlinear dynamical virtual leader. The result is that all agents' velocities asymptotically approach to the velocity of the virtual leader, and the distance between any two agents is asymptotically stabilized to avoid collisions among agents. Finally, some numerical simulations are presented to illustrate the effectiveness of the theoretical results. [Copyright &y& Elsevier]

Abstract: In presence of obstacles, inter-agent pulling actions must be bounded. In this case, to remain connected to the group, the leader-agent (LA) must perform an active leading strategy. In this paper, an active leading algorithm is proposed which monitors the neighborhood of the LA and adjusts its velocity. The algorithm is based on the ant colony optimization (ACO) technique. As a real time optimization package, the ACO algorithm maximizes influence of the LA on the group, leading to fast flocking. Comparison with another optimization method is provided as well. Simulations show that the algorithm is successful and cost effective. [Copyright &y& Elsevier]

Abstract: Feature statistics normalization in the cepstral domain is one of the most performing approaches for robust automaticspeech and speaker recognition in noisy acoustic scenarios: feature coefficients are normalized by using suitable linear or nonlinear transformations in order to match the noisy speech statistics to the clean speech one. Histogram equalization (HEQ) belongs to such a category of algorithms and has proved to be effective on purpose and therefore taken here as reference. In this paper the presence of multi-channel acoustic channels is used to enhance the statistics modeling capabilities of the HEQ algorithm, by exploiting the availability of multiple noisy speech occurrences, with the aim of maximizing the effectiveness of the cepstra normalization process. Computer simulations based on the Aurora 2 database in speech and speaker recognition scenarios have shown that a significant recognition improvement with respect to the single-channel counterpart and other multi-channel techniques can be achieved confirming the effectiveness of the idea. The proposed algorithmic configuration has also been combined with the kernel estimation technique in order to further improve the speech recognition performances. [Copyright &y& Elsevier]

Abstract: Semi-supervised learning techniques have gained increasing attention in the machine learning community, as a result of two main factors: (1) the available data is exponentially increasing; (2) the task of data labeling is cumbersome and expensive, involving human experts in the process. In this paper, we propose a network-based semi-supervised learning method inspired by the modularity greedy algorithm, which was originally applied for unsupervised learning. Changes have been made in the process of modularity maximization in a way to adapt the model to propagate labels throughout the network. Furthermore, a network reduction technique is introduced, as well as an extensive analysis of its impact on the network. Computer simulations are performed for artificial and real-world databases, providing a numerical quantitative basis for the performance of the proposed method. [Copyright &y& Elsevier]

Abstract: We present in this paper a variation of the self-organising map algorithm where the original time-dependent (learning rate and neighbourhood) learning function is replaced by a time-invariant one. This allows for on-line and continuous learning on both static and dynamic data distributions. One of the property of the newly proposed algorithm is that it does not fit the magnification law and the achieved vector density is not directly proportional to the density of the distribution as found in most vector quantisation algorithms. From a biological point of view, this algorithm sheds light on cortical plasticity seen as a dynamic and tight coupling between the environment and the model. [Copyright &y& Elsevier]

Abstract: Neural networks have some applications in computerized tomography, in particular to reconstruct an image from projections. The presented paper describes a new practical approach to the reconstruction problem using a Hopfield-type neural network. The methodology of this reconstruction algorithm resembles a transformation formula—the so-called ρ-filtered layergram method. The method proposed in this work is adapted for discrete fan beam projections, already used in practice. Performed computer simulations show that the neural network reconstruction algorithm designed to work in this way outperforms conventional methods in obtained image quality, and in perspective of hardware implementation in the speed of the reconstruction process. [Copyright &y& Elsevier]

Abstract: In this paper, we propose a null space discriminant locality preserving projections (NDLPP) method for facial feature extraction and recognition. Based on locality preserving projections (LPP) and discriminant locality preserving projections (DLPP) methods, NDLPP comes into the characteristics of DLPP that encodes both the geometrical and discriminant structure of the data manifold, and addresses the small sample size problem by solving an eigenvalue problem in null space. Experiments on synthetic data and ORL, Yale, and FERET face databases are performed to test and evaluate the proposed algorithm. The results demonstrate the effectiveness of NDLPP. [Copyright &y& Elsevier]

Abstract: A unifying model that combines three properties is proposed by Hyvärinen, and a gradient ascent algorithm for independent component analysis (ICA) is performed by maximum likelihood estimation. In this paper, we consider the estimation of the data model of ICA when Gaussian noise is present and the independent components are time dependent. Firstly, according to the useful property of Gaussian moments, we introduce Gaussian moments algorithm to estimation of the noisy unifying model when the noise covariance matrix is known. Next, when the noise covariance is unknown, a new Gaussian moments algorithm is developed. Finally, the validity and performance of our algorithms are demonstrated by computer simulations. [Copyright &y& Elsevier]

Abstract: This paper proposes an improved learning algorithm of analog compound orthogonal networks and a novel tracking control approach for nonholonomic mobile robots by integrating the neural network into the backstepping technique. The adaptive control is derived from continuously tuning parameters using the analog neural network in the backstepping control law. The proposed control approach for the mobile robot has the properties to quickly drive the position error to zero and to indicate better smooth movement in the tracking control process. These features are due to continuous online learning and adaptive capability of analog neural networks. Simulation results demonstrate that the proposed control approach is effective and has better control performance. [Copyright &y& Elsevier]

Abstract: This paper proposes blind source extraction methods based on several time-delay autocorrelations of primary sources, called MACBSE. The MACBSE approaches are batch fixed-point learning algorithms for extraction of source signals with linear autocorrelations. The fixed-point algorithms are very simple and do not need to choose any learning step sizes. Furthermore, the convergence properties of the algorithms are analyzed. Their efficiencies are demonstrated by extensive computer simulations. [Copyright &y& Elsevier]

Abstract: Previous papers have noted the difficulty in obtaining neural models which are stable under simulation when trained using prediction-error-based methods. Here the differences between series–parallel and parallel identification structures for training neural models are investigated. The effect of the error surface shape on training convergence and simulation performance is analysed using a standard algorithm operating in both training modes. A combined series–parallel/parallel training scheme is proposed, aiming to provide a more effective means of obtaining accurate neural simulation models. Simulation examples show the combined scheme is advantageous in circumstances where the solution space is known or suspected to be complex. [Copyright &y& Elsevier]

Abstract: A neural approach for the parameter estimation of adaptive FIR filters for linear system identification is presented in the paper. It is based on a linear neuron with a modified gradient algorithm, capable of resolving the total least squares (TLS) problem present in this kind of estimation, where noisy errors affect not only the observation vector but also the data matrix. The learning rule is analyzed mathematically. The results of computer simulations are given to illustrate that the neural approach considerably outperforms the existing TLS methods when a larger learning factor is used or the signal-noise-ratio (SNR) is lower. [Copyright &y& Elsevier]

Abstract: This paper addresses the problem of blind source separation and presents a fixed-point nonlinear principal component analysis (NPCA) algorithm. It is a block-wise batch algorithm and gives an alternative perspective on existing adaptive online NPCA algorithms. Utilizing new activation functions that automatically satisfy a stability condition, the proposed algorithm can separate mixed signals with sub- and super-Gaussian source distributions. The efficiency is confirmed by extensive computer simulations on man-made sources as well as practical speech signals. [Copyright &y& Elsevier]