Search

In this paper, the measured input–output data sequences with pinning gain are proposed via the topology of multi-agent systems (MAS), where the requirement of reinforcement learning algorithm on internal state is avoided by pinning gain and the measured data of leader and neighbors. Besides, a data-based tracking state is given, which can be applied to MAS with different control matrices. According to the sequences and tracking state, a distributed control policy and corresponding reinforcement learning algorithm are proposed for the output synchronization. The proposed algorithm overcomes the shortcoming that previous algorithms can not be applied to MAS with different control matrices in the absence of model information and full-state vector. Finally, the effectiveness of proposed algorithm is verified by simulation examples. [ABSTRACT FROM AUTHOR]

The target net method has been the foundation of deep reinforcement learning since Deepmind first proposed it in 2015. Almost all the current popular reinforcement learning algorithms include target net. However, while the slowly updated target network improves the stability of the algorithm, it also reduces the performance of the algorithm. In this paper, the authors design a novel triple-network algorithm(TPN). TPN combines the temporal-difference(TD) algorithm and policy gradient(PG) theorem. Using three networks to estimate the state value(v), action value (q) , and policy(π). These networks have no primary or secondary distinction but are trained synchronously and influence each other. The author found that through this TPN architecture, the convergence and stability of the algorithm can be greatly improved without increasing the amount of calculation. Although it is only a basic framework at present. The calculation process of TPN is simple and easy to implement. Experiments prove that the convergence speed and stability of TPN in discrete cases are better than PPO. • Proposed a new deep reinforcement learning algorithm that does not rely on target networks. • Presented the complete derivation process of the TPN algorithm. • Verified the effectiveness of proposed algorithm through experiments. [ABSTRACT FROM AUTHOR]

The Wasserstein distance in the optimal transport (OT) model is being widely used in machine learning applications due to its mathematical and geometric properties. Since the entropy regularized OT is a convex problem, and for its large-scale problems, many algorithms have been proposed. Existing methods mainly adopt iterative random projection techniques to approximate the Wasserstein distance in the OT model. In this paper, we employ the entropic domain decomposition algorithm with mini-batch learning to calculate the entropy regularized OT problem and investigate the stability of this algorithm. The algorithm performs on different cells of the domain instead of using the projection technique and it is based on a mini-batch loss function. The corresponding solution in each iteration is derived from the dual problem of entropy regularized OT on each decomposed cell. Once the solutions to the dual problem on all decomposed cells are obtained, the corresponding transport plan in each iteration can be obtained. We show that the transport plan computed by its dual variables is admissible, and we also give the sub-gradient approximation result. In addition, we exploit the statistical concentration result of the subsampled mini-batch Wasserstein distance on decomposed cells. Finally, the numerical experiments validate that the proposed algorithm can save computational time further. [ABSTRACT FROM AUTHOR]

The increasing digitization and datification of all aspects of people's daily life, and the consequent growth in the use of personal data, are increasingly challenging the current development and adoption of Machine Learning (ML). First, the sheer complexity and amount of data available in these applications strongly demands for ML algorithms that can be trained directly on complex structures, which can be naturally described by graphs. In fact, graphs inherently capture information about entities, their attributes, and relationships between them. Directly applying ML to graphs relieves domain experts and data scientists from the challenging and time-consuming problem of designing a suitable vector-based data representation used by classical ML techniques. Second, ML algorithms should not only be designed to achieve high technical and functional standards; as the automated decisions provided by these algorithms can have a relevant impact on people's lives, their behavior has to be aligned with the values and principles of individuals and society. This demands for designing automated algorithms that we, as humans, can trust, fulfilling the requirements of fairness, robustness, privacy, and explainability. Third, designing effective ML algorithms requires skills and expertise developed at different levels. This substantially hinders the democratization and widespread availability of such technology for society at large, which in turn demands for improving the level of automatization and systematization of their design process, while also providing guarantees on their performance. For this reason, this paper provides an overview of the current works focused towards learning trustworthily, automatically, and with guarantees on graphs. [ABSTRACT FROM AUTHOR]

This paper proposes an online reinforcement learning(RL) for solving Stackelberg games with completely unknown dynamic systems. To deal with the hierarchical optimal control problem, the key is to find the solution of a two-level optimal control with the leader's optimal problem being constrained. Firstly, the leader–follower coupled Hamiltonian-Jacobi(HJ) equations subject to the follower's costate equation is derived. Secondly, an off-policy scheme is designed following the policy iteration(PI) algorithm based on model. The improved off-policy algorithm is given for obtaining the solution to the leader–follower coupled HJ equations. The algorithm is built without requiring any knowledge of the dynamic system, and the leader–follower optimal control policies are directly constructed by the extracted data. Meanwhile, the existence of the Stackelberg equilibrium is demonstrated. Lastly, NNs for each player are built, and NN learning is accomplished online with Kronecker product technique and vectorization, which simples the NN form and could decrease computation burden. Simulation examples are presented to demonstrate the proposed learning algorithm. [ABSTRACT FROM AUTHOR]

Communication in parallel systems imposes significant overhead which often turns out to be a bottleneck in parallel machine learning. To relieve some of this overhead, in this paper, we present EventGraD - an algorithm with event-triggered communication for stochastic gradient descent in parallel machine learning. The main idea of this algorithm is to modify the requirement of communication at every iteration in standard implementations of stochastic gradient descent in parallel machine learning to communicating only when necessary at certain iterations. We provide theoretical analysis of convergence of our proposed algorithm. We also implement the proposed algorithm for data-parallel training of a popular residual neural network used for training the CIFAR-10 dataset and show that EventGraD can reduce the communication load by up to 60% while retaining the same level of accuracy. In addition, EventGraD can be combined with other approaches such as Top-K sparsification to decrease communication further while maintaining accuracy. [ABSTRACT FROM AUTHOR]

Deep Q-Networks algorithm (DQN) was the first reinforcement learning algorithm using deep neural network to successfully surpass human level performance in a number of Atari learning environments. However, divergent and unstable behaviour have been long standing issues in DQNs. The unstable behaviour is often characterised by overestimation in the Q -values, commonly referred to as the overestimation bias. To address the overestimation bias and the divergent behaviour, a number of heuristic extensions have been proposed. Notably, multi-step updates have been shown to drastically reduce unstable behaviour while improving agent's training performance. However, agents are often highly sensitive to the selection of the multi-step update horizon (n), and our empirical experiments show that a poorly chosen static value for n can in many cases lead to worse performance than single-step DQN. Inspired by the success of n -step DQN and the effects that multi-step updates have on overestimation bias, this paper proposes a new algorithm that we call 'Elastic Step DQN' (ES-DQN) to alleviate overestimation bias in DQNs. ES-DQN dynamically varies the step size horizon in multi-step updates based on the similarity between states visited. Our empirical evaluation shows that ES-DQN out-performs n -step with fixed n updates, Double DQN and Average DQN in several OpenAI Gym environments while at the same time alleviating the overestimation bias. [ABSTRACT FROM AUTHOR]

Recently, the Extreme Learning Machine (ELM) algorithm has been applied to various fields due to its rapidity and significant generalization performance. Traditionally, deep learning (DL) and wavelet neural networks (WNN) methods reach a high classification accuracy in machine learning applications. As a result, a new structure based on WNN, deep architecture and ELM is proposed in this paper. The proposed method is based on Extreme Learning Machine Auto-Encoder with DL structure and a composite wavelet activation function used in the hidden nodes. To evaluate the performance of our approach, we used standard benchmark data-sets, namely COIL-20, Pima Indian Diabetes (PID), MNIST and EMNIST. Experimental results show that our method offers satisfactory results and performance compared to other approaches. [ABSTRACT FROM AUTHOR]

Gradient-based algorithms are commonly used for training radial basis function neural network (RBFNN). However, it is still difficult to avoid vanishing gradient to improve the learning performance in the training process. For this reason, in this paper, an accelerated second-order learning (ASOL) algorithm is developed to train RBFNN. First, an adaptive expansion and pruning mechanism (AEPM) of gradient space, based on the integrity and orthogonality of hidden neurons, is designed. Then, the effective gradient information is constantly added to gradient space and the redundant gradient information is eliminated from gradient space. Second, with AEPM, the neurons are generated or pruned accordingly. In this way, a self-organizing RBFNN (SORBFNN) which reduces the structure complexity and improves the generalization ability is obtained. Then, the structure and parameters in the learning process can be optimized by the proposed ASOL-based SORBFNN (ASOL-SORBFNN). Third, some theoretical analyses including the efficiency of the proposed AEPM on avoiding the vanishing gradient and the stability of SORBFNN in the process of structural adjustment are given, then the successful application of the proposed ASOL-SORBFNN is guaranteed. Finally, to illustrate the advantages of the proposed ASOL-SORBFNN, several experimental studies are examined. By comparing with other existing approaches, the results show that ASOL-SORBFNN performs well in terms of both learning speed and prediction accuracy. [ABSTRACT FROM AUTHOR]

Fuzzy rough set is a theoretical framework of fuzzy uncertainty management, and discernibility matrix offers a mathematical foundation for algorithm construction of feature learning. The approaches of fuzzy rough set and discernibility matrix have been successfully applied in single-label learning. However, few works have been done on investigating the foundation of fuzzy rough discernibility matrix on multi-label data. There will be two pivotal problems to be addressed when using fuzzy rough discernibility matrix for multi-label data analysis. One is how to extract sample-level and label-level correlations; and the other is how to utilize the discernibility matrix for algorithm construction. For this reason, in this paper the fuzzy rough discrimination matrix is introduced to deal with the problem of multi-label feature selection. First, the significance of labels in the label space is captured based on the label correlation. Labels with different significances contribute to different weights for measuring the similarity between samples. Hence, a sample similarity matrix in the label space is computed based on the label weighting strategy. Then, a framework of a fuzzy decision system is formalized, in which the discernibility matrix of fuzzy rough sets is introduced as a foundation to evaluate the sample discrimination ability of features. Under the discernibility matrix criterion, a multi-label learning algorithm is developed to select discriminative features from multi-label data. A series of experimental analysis verifies the effectiveness and efficiency of the proposed method. [ABSTRACT FROM AUTHOR]

Sparse Bayesian learning (SBL) provides state-of-the-art performance in accuracy, sparsity and probabilistic prediction for classification. In SBL, the regularization priors are automatically determined that avoids an exhaustive hyperparameter selection by cross-validation. However, scalability to large problems is a drawback of SBL because of the inversion of a potentially enormous covariance matrix for updating the regularization priors in every iteration. This paper develops an approximate SBL algorithm called ARP-SBL, where the regularization priors are approximated without inverting the covariance matrix. Therefore, our approach can easily scale up to problems with large data size or feature dimension. It alleviates the long training time and high memory complexity in SBL. Based on ARP-SBL, two scalable nonlinear SBL models: scalable relevance vector machine (ARP-RVM) and scalable sparse Bayesian extreme learning machine (ARP-SBELM) are developed for problems of large data size and large feature size respectively. Experiments on a variety of benchmarks have shown that the proposed models are with competitive accuracy compared to existing methods while i) converging faster; ii) requiring thousands of times less memory; iii) without exhaustive regularized hyperparameter selection; and iv) easily scaling up to large data size and high dimensional features. [ABSTRACT FROM AUTHOR]

A novel sparse tensor dictionary learning algorithm and a convolutional neural network (CNN) classification method based on this algorithm are proposed for hyperspectral image (HSI) in this paper. The HSI is intuitively represented as a three-dimensional (3-D) cube, and utilizing the joint spatial-spectral information can significantly improve the accuracy of HSI classification. Therefore, the proposed atom-substituted tensor dictionary learning (ASTDL) algorithm utilizes tensor techniques to extract 3-D joint spatial-spectral features from HSI cubes directly. Sparsity constraint is enforced on the coefficient tensors, which obeys the sparsity attribute of HSI. The proposed ASTDL enhanced CNN (ASTDL-CNN) classification method utilizes a two-dimensional (2-D) CNN to extract deep features from the feature tensors obtained by the ASTDL, and to perform the pixel-wise classification. The use of ASTDL, which extracts intrinsic tensor features before CNN, alleviates within-class spatial-spectral variation and reduces the requirement of CNN for the labeled data. Whereas, the CNN works as a 3-D classifier, and provides the final classification results. Besides, we perform the majority vote on the classification map obtained by CNN to refine the classification. The performance of the proposed method is evaluated on three real HSI data sets. The competitive results to the compared state-of-the-art methods demonstrate that the ASTDL-CNN can provide accurate and robust classification results. [ABSTRACT FROM AUTHOR]

This paper investigates a new bacterial colony-based feature selection algorithm to improve the classification accuracy of customers for personalized products recommendation. An attribute learning strategy is developed in this study to update the feature related population. Specifically, the features can be weighted according to their historic contributions to both the individual- and group-based subsets. Additionally, the frequency of appearance is also recorded for the feature candidates to improve the diversity of feature distribution and avoid the over-fitting. Based on the weight-based feature indexes and frequency of appearance records, the performance of feature subsets are enhanced by replacing the features being repeatedly appeared in a same vector. To explore the feasibility of the proposed method for the missing feature problems, the objective of the optimization is to minimize the classification error using the acceptable number of features. K-Nearest Neighbor is employed as the learning technique to cooperate with the proposed feature selection method. The effectiveness of the proposed feature selection method is demonstrated by performing test on the datasets from UCI machine learning repository and real-world data from Amazon customer reviews of products. Compared with other seven feature selections methods, the proposed feature selection algorithm outperforms the other algorithms by achieving higher classification accuracy rate using smaller features. [ABSTRACT FROM AUTHOR]

In this paper we propose and discuss different Deep Learning-based ensemble algorithms for a problem of low-visibility events prediction due to fog. Specifically, seven different Deep Learning (DL) architectures have been considered, from which multiple individual learners are generated. Hyperparameters of the models, including parameters concerning data preprocessing, models architecture and training procedure, are randomly selected for each model within a pre-defined discrete range. Also, every model is trained with slightly different data sampled randomly, assuring that every models introduce variety in the ensemble. Then, three different information fusion techniques are employed to build the ensemble models. The influence of the filtering process and the elitism level (the percentage of the individual models entering the ensemble) is also assessed. The performance of the proposed methodology have been tested in two real problems of low-visibility events prediction due to orographical and radiation fog, at the north of Spain. Comparison with different Machine Learning, alternative DL algorithms and meteorological-based methods show the good performance of the proposed deep learning ensembles in this problem. [ABSTRACT FROM AUTHOR]