Search

In this paper, the measured input–output data sequences with pinning gain are proposed via the topology of multi-agent systems (MAS), where the requirement of reinforcement learning algorithm on internal state is avoided by pinning gain and the measured data of leader and neighbors. Besides, a data-based tracking state is given, which can be applied to MAS with different control matrices. According to the sequences and tracking state, a distributed control policy and corresponding reinforcement learning algorithm are proposed for the output synchronization. The proposed algorithm overcomes the shortcoming that previous algorithms can not be applied to MAS with different control matrices in the absence of model information and full-state vector. Finally, the effectiveness of proposed algorithm is verified by simulation examples. [ABSTRACT FROM AUTHOR]

The increasing digitization and datification of all aspects of people's daily life, and the consequent growth in the use of personal data, are increasingly challenging the current development and adoption of Machine Learning (ML). First, the sheer complexity and amount of data available in these applications strongly demands for ML algorithms that can be trained directly on complex structures, which can be naturally described by graphs. In fact, graphs inherently capture information about entities, their attributes, and relationships between them. Directly applying ML to graphs relieves domain experts and data scientists from the challenging and time-consuming problem of designing a suitable vector-based data representation used by classical ML techniques. Second, ML algorithms should not only be designed to achieve high technical and functional standards; as the automated decisions provided by these algorithms can have a relevant impact on people's lives, their behavior has to be aligned with the values and principles of individuals and society. This demands for designing automated algorithms that we, as humans, can trust, fulfilling the requirements of fairness, robustness, privacy, and explainability. Third, designing effective ML algorithms requires skills and expertise developed at different levels. This substantially hinders the democratization and widespread availability of such technology for society at large, which in turn demands for improving the level of automatization and systematization of their design process, while also providing guarantees on their performance. For this reason, this paper provides an overview of the current works focused towards learning trustworthily, automatically, and with guarantees on graphs. [ABSTRACT FROM AUTHOR]

The Wasserstein distance in the optimal transport (OT) model is being widely used in machine learning applications due to its mathematical and geometric properties. Since the entropy regularized OT is a convex problem, and for its large-scale problems, many algorithms have been proposed. Existing methods mainly adopt iterative random projection techniques to approximate the Wasserstein distance in the OT model. In this paper, we employ the entropic domain decomposition algorithm with mini-batch learning to calculate the entropy regularized OT problem and investigate the stability of this algorithm. The algorithm performs on different cells of the domain instead of using the projection technique and it is based on a mini-batch loss function. The corresponding solution in each iteration is derived from the dual problem of entropy regularized OT on each decomposed cell. Once the solutions to the dual problem on all decomposed cells are obtained, the corresponding transport plan in each iteration can be obtained. We show that the transport plan computed by its dual variables is admissible, and we also give the sub-gradient approximation result. In addition, we exploit the statistical concentration result of the subsampled mini-batch Wasserstein distance on decomposed cells. Finally, the numerical experiments validate that the proposed algorithm can save computational time further. [ABSTRACT FROM AUTHOR]

Deep Q-Networks algorithm (DQN) was the first reinforcement learning algorithm using deep neural network to successfully surpass human level performance in a number of Atari learning environments. However, divergent and unstable behaviour have been long standing issues in DQNs. The unstable behaviour is often characterised by overestimation in the Q -values, commonly referred to as the overestimation bias. To address the overestimation bias and the divergent behaviour, a number of heuristic extensions have been proposed. Notably, multi-step updates have been shown to drastically reduce unstable behaviour while improving agent's training performance. However, agents are often highly sensitive to the selection of the multi-step update horizon (n), and our empirical experiments show that a poorly chosen static value for n can in many cases lead to worse performance than single-step DQN. Inspired by the success of n -step DQN and the effects that multi-step updates have on overestimation bias, this paper proposes a new algorithm that we call 'Elastic Step DQN' (ES-DQN) to alleviate overestimation bias in DQNs. ES-DQN dynamically varies the step size horizon in multi-step updates based on the similarity between states visited. Our empirical evaluation shows that ES-DQN out-performs n -step with fixed n updates, Double DQN and Average DQN in several OpenAI Gym environments while at the same time alleviating the overestimation bias. [ABSTRACT FROM AUTHOR]

Analyzing the runtime behaviors of Android apps is crucial for malware detection. In this paper, we attempt to learn the behavior level features of an app from function calls. The challenges of this task are twofold. First, the absence of function attributes hinders the understanding of app behaviors. Second, the graphical representation of function calls cannot be directly processed by classical machine learning algorithms. In this paper, we develop two methods to overcome these challenges. Based on function embedding, we first propose the concept of enhanced function call graphs (E-FCGs) to characterize app runtime behaviors. We then develop a Graph Convolutional Network (GCN) based algorithm to obtain vector representations of E-FCGs. Extensive experiments show that the features learned by our method can achieve surprisingly high detection performance on a variety of classifiers (e.g., LR, DT, SVM, KNN, RF, MLP and CNN), significantly outperforming the traditional static features. [ABSTRACT FROM AUTHOR]

This paper proposes an online reinforcement learning(RL) for solving Stackelberg games with completely unknown dynamic systems. To deal with the hierarchical optimal control problem, the key is to find the solution of a two-level optimal control with the leader's optimal problem being constrained. Firstly, the leader–follower coupled Hamiltonian-Jacobi(HJ) equations subject to the follower's costate equation is derived. Secondly, an off-policy scheme is designed following the policy iteration(PI) algorithm based on model. The improved off-policy algorithm is given for obtaining the solution to the leader–follower coupled HJ equations. The algorithm is built without requiring any knowledge of the dynamic system, and the leader–follower optimal control policies are directly constructed by the extracted data. Meanwhile, the existence of the Stackelberg equilibrium is demonstrated. Lastly, NNs for each player are built, and NN learning is accomplished online with Kronecker product technique and vectorization, which simples the NN form and could decrease computation burden. Simulation examples are presented to demonstrate the proposed learning algorithm. [ABSTRACT FROM AUTHOR]

Communication in parallel systems imposes significant overhead which often turns out to be a bottleneck in parallel machine learning. To relieve some of this overhead, in this paper, we present EventGraD - an algorithm with event-triggered communication for stochastic gradient descent in parallel machine learning. The main idea of this algorithm is to modify the requirement of communication at every iteration in standard implementations of stochastic gradient descent in parallel machine learning to communicating only when necessary at certain iterations. We provide theoretical analysis of convergence of our proposed algorithm. We also implement the proposed algorithm for data-parallel training of a popular residual neural network used for training the CIFAR-10 dataset and show that EventGraD can reduce the communication load by up to 60% while retaining the same level of accuracy. In addition, EventGraD can be combined with other approaches such as Top-K sparsification to decrease communication further while maintaining accuracy. [ABSTRACT FROM AUTHOR]

Deep learning technologies have been rapidly developing recently. They have shown excellent performances in many fields. However, deep learning networks have weak adaptability to small sample sizes.Usually, deep learning networks require tens of thousands of samples to avoid overfitting. In contrast, the kernel method can deal with small sample sizes. In this paper, we use only dozens of samples to train a successful classifier. Deep multiple kernel learning (DMKL) has emerged in recent years. It combines the ideas of deep learning and multiple kernel learning. Unfortunately, the DMKL network with a fixed structure cannot adapt to data of different dimensions and sizes. Therefore, in this paper, we propose a novel depth-width-scaling multiple kernel learning (DWS-MKL) algorithm. It has the ability to adjust the architecture according to the input data. We optimize the estimation of leave-one-out error by using the span bound instead of the dual objective function. Finally, we choose a large number of data sets from the UCI benchmark data sets for classification tasks. We also evaluate DWS-MKL on the MNIST data set. The experimental results show that different frameworks have different performances on the same dataset. Our DWS-MKL algorithm obtains better classification results than the state-of-the-art kernel learning algorithms, as determined by a thorough comparison. The encouraging results demonstrate that our method has the potential to improve generalization performance of the model. [ABSTRACT FROM AUTHOR]

This paper investigates the fully complex mini-batch gradient algorithm for training complex-valued neural networks. Mini-batch gradient method has been widely used in neural network training, however, its convergence analysis is usually restricted to real-valued neural networks and of probability nature. By introducing a new Taylor mean value theorem for analytic functions, in this paper we establish deterministic convergence results for the fully complex mini-batch gradient algorithm under mild conditions. The deterministic convergence here means that the algorithm will deterministically converge, and both the weak convergence and strong convergence will be proved. Benefited from the newly introduced mean value theorem, our results are of global nature in that they are valid for arbitrarily given initial values of the weights. The theoretical findings are validated with a simulation example. [ABSTRACT FROM AUTHOR]

Recently, the Extreme Learning Machine (ELM) algorithm has been applied to various fields due to its rapidity and significant generalization performance. Traditionally, deep learning (DL) and wavelet neural networks (WNN) methods reach a high classification accuracy in machine learning applications. As a result, a new structure based on WNN, deep architecture and ELM is proposed in this paper. The proposed method is based on Extreme Learning Machine Auto-Encoder with DL structure and a composite wavelet activation function used in the hidden nodes. To evaluate the performance of our approach, we used standard benchmark data-sets, namely COIL-20, Pima Indian Diabetes (PID), MNIST and EMNIST. Experimental results show that our method offers satisfactory results and performance compared to other approaches. [ABSTRACT FROM AUTHOR]

Gradient-based algorithms are commonly used for training radial basis function neural network (RBFNN). However, it is still difficult to avoid vanishing gradient to improve the learning performance in the training process. For this reason, in this paper, an accelerated second-order learning (ASOL) algorithm is developed to train RBFNN. First, an adaptive expansion and pruning mechanism (AEPM) of gradient space, based on the integrity and orthogonality of hidden neurons, is designed. Then, the effective gradient information is constantly added to gradient space and the redundant gradient information is eliminated from gradient space. Second, with AEPM, the neurons are generated or pruned accordingly. In this way, a self-organizing RBFNN (SORBFNN) which reduces the structure complexity and improves the generalization ability is obtained. Then, the structure and parameters in the learning process can be optimized by the proposed ASOL-based SORBFNN (ASOL-SORBFNN). Third, some theoretical analyses including the efficiency of the proposed AEPM on avoiding the vanishing gradient and the stability of SORBFNN in the process of structural adjustment are given, then the successful application of the proposed ASOL-SORBFNN is guaranteed. Finally, to illustrate the advantages of the proposed ASOL-SORBFNN, several experimental studies are examined. By comparing with other existing approaches, the results show that ASOL-SORBFNN performs well in terms of both learning speed and prediction accuracy. [ABSTRACT FROM AUTHOR]

In the last decade it became increasingly apparent the inability of technical metrics such as accuracy, sustainability, and non-regressiveness to well characterize the behavior of intelligent systems. In fact, they are nowadays requested to meet also ethical requirements such as explainability, fairness, robustness, and privacy increasing our trust in their use in the wild. Of course often technical and ethical metrics are in tension between each other but the final goal is to be able to develop a new generation of more responsible and trustworthy machine learning. In this paper, we focus our attention on machine learning algorithms and associated predictive models, questioning for the first time, from a theoretical perspective, if it is possible to simultaneously guarantee their performance in terms of both technical and ethical metrics towards machine learning algorithms that we can trust. In particular, we will investigate for the first time both theory and practice of deterministic and randomized algorithms and associated predictive models showing the advantages and disadvantages of the different approaches. For this purpose we will leverage the most recent advances coming from the statistical learning theory: Complexity-Based Methods, Distribution Stability, PAC-Bayes, and Differential Privacy. Results will show that it is possible to develop consistent algorithms which generate predictive models with guarantees on multiple trustworthiness metrics. • AI is nowadays requested to optimize both technical and ethical metrics. • Focus on ML algorithms and associated models (both deterministic and randomized). • We prove that it is possible to develop consistent algorithms and models. • We bound generalization in terms of both technical and ethical metrics. • We leverage the most recent advances coming from the statistical learning theory. [ABSTRACT FROM AUTHOR]

The target net method has been the foundation of deep reinforcement learning since Deepmind first proposed it in 2015. Almost all the current popular reinforcement learning algorithms include target net. However, while the slowly updated target network improves the stability of the algorithm, it also reduces the performance of the algorithm. In this paper, the authors design a novel triple-network algorithm(TPN). TPN combines the temporal-difference(TD) algorithm and policy gradient(PG) theorem. Using three networks to estimate the state value(v), action value (q) , and policy(π). These networks have no primary or secondary distinction but are trained synchronously and influence each other. The author found that through this TPN architecture, the convergence and stability of the algorithm can be greatly improved without increasing the amount of calculation. Although it is only a basic framework at present. The calculation process of TPN is simple and easy to implement. Experiments prove that the convergence speed and stability of TPN in discrete cases are better than PPO. • Proposed a new deep reinforcement learning algorithm that does not rely on target networks. • Presented the complete derivation process of the TPN algorithm. • Verified the effectiveness of proposed algorithm through experiments. [ABSTRACT FROM AUTHOR]

Fuzzy rough set is a theoretical framework of fuzzy uncertainty management, and discernibility matrix offers a mathematical foundation for algorithm construction of feature learning. The approaches of fuzzy rough set and discernibility matrix have been successfully applied in single-label learning. However, few works have been done on investigating the foundation of fuzzy rough discernibility matrix on multi-label data. There will be two pivotal problems to be addressed when using fuzzy rough discernibility matrix for multi-label data analysis. One is how to extract sample-level and label-level correlations; and the other is how to utilize the discernibility matrix for algorithm construction. For this reason, in this paper the fuzzy rough discrimination matrix is introduced to deal with the problem of multi-label feature selection. First, the significance of labels in the label space is captured based on the label correlation. Labels with different significances contribute to different weights for measuring the similarity between samples. Hence, a sample similarity matrix in the label space is computed based on the label weighting strategy. Then, a framework of a fuzzy decision system is formalized, in which the discernibility matrix of fuzzy rough sets is introduced as a foundation to evaluate the sample discrimination ability of features. Under the discernibility matrix criterion, a multi-label learning algorithm is developed to select discriminative features from multi-label data. A series of experimental analysis verifies the effectiveness and efficiency of the proposed method. [ABSTRACT FROM AUTHOR]

Sparse Bayesian learning (SBL) provides state-of-the-art performance in accuracy, sparsity and probabilistic prediction for classification. In SBL, the regularization priors are automatically determined that avoids an exhaustive hyperparameter selection by cross-validation. However, scalability to large problems is a drawback of SBL because of the inversion of a potentially enormous covariance matrix for updating the regularization priors in every iteration. This paper develops an approximate SBL algorithm called ARP-SBL, where the regularization priors are approximated without inverting the covariance matrix. Therefore, our approach can easily scale up to problems with large data size or feature dimension. It alleviates the long training time and high memory complexity in SBL. Based on ARP-SBL, two scalable nonlinear SBL models: scalable relevance vector machine (ARP-RVM) and scalable sparse Bayesian extreme learning machine (ARP-SBELM) are developed for problems of large data size and large feature size respectively. Experiments on a variety of benchmarks have shown that the proposed models are with competitive accuracy compared to existing methods while i) converging faster; ii) requiring thousands of times less memory; iii) without exhaustive regularized hyperparameter selection; and iv) easily scaling up to large data size and high dimensional features. [ABSTRACT FROM AUTHOR]

A novel sparse tensor dictionary learning algorithm and a convolutional neural network (CNN) classification method based on this algorithm are proposed for hyperspectral image (HSI) in this paper. The HSI is intuitively represented as a three-dimensional (3-D) cube, and utilizing the joint spatial-spectral information can significantly improve the accuracy of HSI classification. Therefore, the proposed atom-substituted tensor dictionary learning (ASTDL) algorithm utilizes tensor techniques to extract 3-D joint spatial-spectral features from HSI cubes directly. Sparsity constraint is enforced on the coefficient tensors, which obeys the sparsity attribute of HSI. The proposed ASTDL enhanced CNN (ASTDL-CNN) classification method utilizes a two-dimensional (2-D) CNN to extract deep features from the feature tensors obtained by the ASTDL, and to perform the pixel-wise classification. The use of ASTDL, which extracts intrinsic tensor features before CNN, alleviates within-class spatial-spectral variation and reduces the requirement of CNN for the labeled data. Whereas, the CNN works as a 3-D classifier, and provides the final classification results. Besides, we perform the majority vote on the classification map obtained by CNN to refine the classification. The performance of the proposed method is evaluated on three real HSI data sets. The competitive results to the compared state-of-the-art methods demonstrate that the ASTDL-CNN can provide accurate and robust classification results. [ABSTRACT FROM AUTHOR]

This paper investigates a new bacterial colony-based feature selection algorithm to improve the classification accuracy of customers for personalized products recommendation. An attribute learning strategy is developed in this study to update the feature related population. Specifically, the features can be weighted according to their historic contributions to both the individual- and group-based subsets. Additionally, the frequency of appearance is also recorded for the feature candidates to improve the diversity of feature distribution and avoid the over-fitting. Based on the weight-based feature indexes and frequency of appearance records, the performance of feature subsets are enhanced by replacing the features being repeatedly appeared in a same vector. To explore the feasibility of the proposed method for the missing feature problems, the objective of the optimization is to minimize the classification error using the acceptable number of features. K-Nearest Neighbor is employed as the learning technique to cooperate with the proposed feature selection method. The effectiveness of the proposed feature selection method is demonstrated by performing test on the datasets from UCI machine learning repository and real-world data from Amazon customer reviews of products. Compared with other seven feature selections methods, the proposed feature selection algorithm outperforms the other algorithms by achieving higher classification accuracy rate using smaller features. [ABSTRACT FROM AUTHOR]

Abstract Dealing with high-dimensional dataset has always been an important problem and feature selection is one of useful tools. In this paper, we develop a new filter based supervised feature selection method by combining Trace Ratio Criterion of Linear Discriminant Analysis (TRC-LDA) and group sparsity regularization. The filter based supervised feature selection method is a classifier-independent method while the TRC-LDA criterion is a recently developed criterion for dimensionality reduction that can well preserve discriminative information of dataset. However, there are seldom methods by utilizing TRC-LDA criterion for feature selection. On the other hand, imposing the l2,0-norm to the projection matrix of TRC-LDA will force some rows in it to be zero while keep other rows nonzero making the index of nonzero rows to be the selected features, however, l 2,0 -nom minimizing problem is NP-hard and intractable. To solve the above problem, in this paper, we develop a new method, namely, Trace Ratio Criterion Discriminative Feature Selection (TRC-DFS), for feature selection. The proposed TRC-DFS has imposed l 2,1 -norm, i.e. an approximation of l 2,0 -norm, to the projection matrix W of TRC-LDA to achieve feature selection. As a result, the proposed TRC-DFS can both achieve feature selection as well as capture the discriminative structure of data. We also extend the proposed method with l 2, p -norm (0 < p < 1) regularization to grasp more sparse property and develop an iterative approach to calculate the optimal solution, which is rigorously proved to be converged. Extensive simulations based on several real-world datasets verify the effectiveness of the proposed methods. [ABSTRACT FROM AUTHOR]

Abstract Twin extreme learning machine (TELM) is an efficient and effective method for pattern classification, based on widely known extreme learning machine (ELM). However, TELM is mainly used to deal with supervised learning problems. In this paper, we extend TELM to handle semi-supervised learning problems and propose a novel Laplacian twin extreme learning machine (LapTELM), which simultaneously trains two related and paired semi-supervised ELMs with two nonparallel separating planes for the final classification. The proposed method exploits the geometry structure property of the unlabeled samples and incorporates it as a manifold regularization term. This allows LapTELM to reap the benefits of fully exploring the plentiful unlabeled samples while retaining the learning ability and efficiency of TELM. Moreover, the paper shows that semi-supervised and supervised TELM can form an unified learning framework. Compared with several mainstream semi-supervised learning methods, the experimental results on the synthetic and several real-world data sets verify the effectiveness and efficiency of LapTELM. [ABSTRACT FROM AUTHOR]

Distance metric learning is a branch of machine learning that aims to learn distances from the data, which enhances the performance of similarity-based algorithms. This tutorial provides a theoretical background and foundations on this topic and a comprehensive experimental analysis of the most-known algorithms. We start by describing the distance metric learning problem and its main mathematical foundations, divided into three main blocks: convex analysis, matrix analysis and information theory. Then, we will describe a representative set of the most popular distance metric learning methods used in classification. All the algorithms studied in this paper will be evaluated with exhaustive testing in order to analyze their capabilities in standard classification problems, particularly considering dimensionality reduction and kernelization. The results, verified by Bayesian statistical tests, highlight a set of outstanding algorithms. Finally, we will discuss several potential future prospects and challenges in this field. This tutorial will serve as a starting point in the domain of distance metric learning from both a theoretical and practical perspective. [ABSTRACT FROM AUTHOR]

• A novel cooperative hybrid evolutionary algorithm is designed for large scale MMSP. • Hybrid assignment and selection strategies are proposed based on the CC framework. • New evolutionary operations are adopted for the unit assignment and order sequence. As an important part of batch chemical industry scheduling problems, the multi-stage multi-product batch plant scheduling problem (MMSP) has been widely studied for decades. This problem is characterized by multiple stages with non-identical parallel units and operate based on customer orders. In this paper, we focus on the large scale MMSP and treat the minimization of make-span as the objective function. An efficient cooperative hybrid evolutionary algorithm is proposed based on the framework of cooperative co-evolution. First, a novel two-line encoding scheme is developed to represent the unit assignment and sequencing for orders respectively. Second, modified estimation of distribution algorithm (EDA) and differential evolutionary (DE) operations are proposed according to the feature of MMSP. EDA operation with a novel population-based incremental learning strategy is applied to handle the unit assignment variables. And novel DE operation based on a novel encoding method is adopted to deal with sequence variables. Then, two selection strategies are applied to preserve optimal and sub-optimal solutions for the proposed algorithm. The critical path based local search algorithm is adopted to further improve the efficiency of local optimization. The proposed algorithm has been tested by several instances with different sizes and characteristics. The numerical results and comparisons show that the proposed work is very competitive in solving large scale MMSP. [ABSTRACT FROM AUTHOR]

Breast ultrasound (BUS) is a powerful screening tool for examination of breast lesions. Recently, research attention has been paid to combining doctor's opinions and machine learning technology to build up a better computer-aided diagnosis (CAD) system. In this paper, we propose an improved approach that uses human-rating BI-RADS scores to classify the BUS samples. A BI-RADS feature scoring scheme is firstly adopted to standardize the descriptions on breast lesions, and then the diagnostic patterns are mined by a biclustering algorithm in the collected BI-RADS feature score dataset. With an input sample, the diagnostic patterns could be activated to different degrees which represent the high-level features by calculating the distance between input sample and patterns. The high-level features of the sample are input into a multi-layer perception neural network (MLPNN) and we use a cost matrix to convert the output from probabilities to classification cost. The structure of the MLPNN and the values of elements in cost matrix are optimized by Particle Swarm Optimization, and it finally classifies the input of BUS sample. According to the comparative experiments with other CAD approaches and the experienced sonographers, the proposed approach achieved the best sensitivity, indicating that it can serve as an assistant diagnostic system in clinical practices. [ABSTRACT FROM AUTHOR]

This paper investigates a online learning optimization problem in a distributed manner, where a set of agents aim to cooperatively minimize the sum of local time-varying cost functions while communications among agents depend on a sequence of time-varying directed graphs. For such a problem, we first propose a online distributed stochastic (sub)gradient push-sum algorithm by utilizing distributed optimization methods and push-sum protocols. Then we analyze the regret bounds for the proposed algorithm on the cases when the cost functions are convex and strongly convex, respectively. The bound on the expected regret for convex functions grows sub-linearly with order of O (T) , where T is the time horizon. When the cost functions are strongly convex with Lipschitz gradients, the regret bound has an improved rate with order of O (ln T). Numerical simulations on the localization in wireless sensor networks are used to show the effectiveness of the proposed algorithm. [ABSTRACT FROM AUTHOR]

Deep learning is a branch of neural network which has been intensively developed in the last decade. Due to the high-accuracy classification ability, the deep learning algorithms have been widely used in many fields, such as speech recognition, image recognition, and natural speech processing. However, they also show some shortcomings especially on the selection of some parameters in the network, including hyper-parameters, which is still treated as a time consuming task. In this paper, a modified ABC (ABC-ISB) optimization algorithm is proposed to automatically train the parameters of Feed-Forward Artificial Neural Networks, which is a typical a neural network. In the proposed ABC algorithm, we utilize the information of neighbors with better performance to accelerate the convergence of employed and onlooker bees respectively. In addition, a new selection strategy and a gbest-guided strategy are introduced to enhance the global search capability and balance the exploration and exploitation of the algorithm separately. The experimental results show our ABC-ISB is generally leading and competitive. [ABSTRACT FROM AUTHOR]

Hyperparameter optimization is a challenging process that has the potential to improve machine learning algorithms. Since it creates a remarkable computational burden for machine learning tasks, there have been few works coping with tuning strategies of a specific algorithm. In this paper, an improved Stochastic Gradient Descent (SGD) based on Fisher Maximization is developed for tuning hyperparameters of an Echo State Network (ESN) which has a wide range of applications. The results of the method are then compared with those of traditional Gradient Descent and Grid Search. According to the obtained results; 1) The scale of the data sets greatly affects the reliability of hyperparameter optimization results; 2) Feature selection is critical in terms of mean error of training when hyperparameter optimization is applied on some methods such as ESN; 3) SGD falls in a good local minima if Fisher Maximization is performed to find a good starting point. [ABSTRACT FROM AUTHOR]

• BNGBS is proposed to pursue an efficient learning process for classification task. • Triple incremental abilities are developed to learn more nodes, samples, and classes. • Extensive experiments are conducted on eight datasets to validate the effectiveness. As an ensemble algorithm, network boosting enjoys a powerful classification ability but suffers from the tedious and time-consuming training process. To tackle the problem, in this paper, a broad network gradient boosting system (BNGBS) is developed by integrating gradient boosting machine with broad networks, in which the classification loss caused by a base broad network is learned and eliminated by followed networks in a cascade manner. The proposed system is constructed as an additive model and can be easily optimized by a greedy strategy instead of the tedious back-propagation algorithm, resulting in a more efficient learning process. Meanwhile, triple incremental learning capabilities including the increment of feature nodes, increment of input samples, and increment of target classes are designed. The proposed system can be efficiently updated and expanded based on the current status instead of being entirely retrained when the demands for more feature nodes, input samples, and target classes are proposed. The node-increment ability allows to add more feature nodes into the built system if the current structures are not effective for learning. The sample-increment ability is developed to allow the model to keep learning from the coming batch data. The class-increment ability is used to tackle the issue that the coming batch data may contain unseen categories. In comparison with existing popular machine learning methods, comprehensive results based on eight benchmark datasets illustrate the effectiveness of the proposed broad network gradient boosting system for the classification task. [ABSTRACT FROM AUTHOR]

We explored the problem about function approximation error and complex mission adaptability in multi-agent deep reinforcement learning. This paper proposes a new multi-agent deep reinforcement learning algorithm framework named multi-agent time delayed deep deterministic policy gradient. Our work reduces the overestimation error of neural network approximation and variance of estimation result using dual-centered critic, group target network smoothing and delayed policy updating. According to experiment results, it improves the ability to adapt complex missions eventually. Then, we discuss that there is an inevitable overestimation issue about existing multi-agent algorithms about approximating real action-value equations with neural network. We also explain the approximate error of equations in the multi-agent deep deterministic policy gradient algorithm mathematically and experimentally. Finally, the application of our algorithm in the mixed cooperative competition experimental environment further demonstrates the effectiveness and generalization of our algorithm, especially improving the group's ability of adapting complex missions and completing more difficult missions. [ABSTRACT FROM AUTHOR]

Facial landmark tracking often adopts static models to generically fit per frame of video. This is considered inappropriate since such models ignore the informative correlation between previous and current frames. Moreover, most of these methods fail to balance the speed and accuracy for video-based facial landmark tracking simultaneously. In this paper, we propose an efficient online framework for video-based face alignment, named Incrementally Cascaded Broad Learning framework (ICBL). ICBL aims to continuously enhance the prediction capability of tracking model for sequential data. It is capable of learning the spatial appearance on specific-person statistics from continuous facial frames and using such knowledge to incrementally tune a cascade of regressors in parallel. To achieve this goal, we approximate the facial shape space by sampling from a dynamic distribution which is continuously updated by person-specific statistics from the tracked facial frames. This dramatically facilitates cascade regression to incrementally update all cascade-regressors in parallel, thus allowing a fast update of the whole model. Furthermore, we successfully incorporate both the linear and non-linear mappings into our parallel cascade framework and introduce Broad Learning (BL) algorithm as a solution for them simultaneously. Experimental results on the most popular and large-scale benchmark for facial landmark tracking show highly competitive performance of proposed ICBL in comparisons with the state-of-the-arts. The code of our ICBL framework has been available from https://github.com/CaifengLiu/Facial-landmark-tracking-by-ICBL. [ABSTRACT FROM AUTHOR]

Hyperparameter tuning is critical for the performance of machine learning algorithms. However, a noticeable limitation is the high computational cost of algorithm evaluation for complex models or for large datasets, which makes the tuning process highly inefficient. In this paper, we propose a novel model-based method for efficient hyperparameter optimization. Firstly, we frame this optimization process as a reinforcement learning problem and then employ an agent to tune hyperparameters sequentially. In addition, a model that learns how to evaluate an algorithm is used to speed up the training. However, model inaccuracy is further exacerbated by long-term use, resulting in collapse performance. We propose a novel method for controlling the model use by measuring the impact of the model on the policy and limiting it to a proper range. Thus, the horizon of the model use can be dynamically adjusted. We apply the proposed method to tune the hyperparameters of the extreme gradient boosting and convolutional neural networks on 101 tasks. The experimental results verify that the proposed method achieves the highest accuracy on 86.1% of the tasks, compared with other state-of-the-art methods and the average ranking of runtime is significant lower than all methods by using the predictive model. [ABSTRACT FROM AUTHOR]

Sparse representation and dictionary learning have been successfully applied in hyperspectral image classification. Generally, it is more effective to learn the sub-dictionary for each class and utilize multiple scale strategy. However, the sub-dictionary may only consider the within-class information and ignore the discriminative information. For multiscale sparse representation, the shape of the regions would not adaptively change according to context structure. This paper proposes a discriminative sub-dictionary learning algorithm and an adaptive multiscale superpixel classification strategy under sparse representation framework for hyperspectral image classification. In dictionary learning stage, the global constraint term can ensure the learnt sub-dictionaries contain more discriminative information. In sparse representation stage, the adaptive multiscale superpixel strategy can exploit the spatial context according to the structural information. The combination of the two strategies can further explore the spatial and spectral information in hyperspectral image. Experiments on four hyperspectral image datasets prove that the proposed method has a significant improvement in both quantitative and qualitative analysis comparing with state-of-the-art algorithms. Moreover, the proposed algorithm can produce satisfactory results for the imbalanced data problem involved in hyperspectral image. [ABSTRACT FROM AUTHOR]

An algorithm that generates an exponential number of stable states for the very well-known Hopfield Neural Network (HNN) is introduced in this paper. We show that the quantity of stable states depends on the dimension and number of components of the input pattern supporting noise. Extensive tests verify that the states generated by our algorithm are stable states and show the exponential storage capacity of a HNN. This paper opens the possibility of designing improved HNNs able to achieve exponential storage, and thus find their applicability in complex real-world problems. [ABSTRACT FROM AUTHOR]

This paper presents efficient algorithms for learning low-coherence dictionaries. First, a new algorithm based on proximal methods is proposed to solve the dictionary learning (DL) problem regularized with the mutual coherence of dictionary. This is unlike the previous approaches that solve a regularized problem where an approximate incoherence promoting term, instead of the mutual coherence, is used to encourage low-coherency. Then, a new solver is proposed for constrained low-coherence DL problem, i.e., a DL problem with an explicit constraint on the mutual coherence of the dictionary. As opposed to current methods, which follow a suboptimal two-step approach, the new algorithm directly solves the associated DL problem. Using previous studies, convergence of the new schemes to critical points of the associated cost functions is also provided. Furthermore, it is shown that the proposed algorithms have lower iteration complexity than existing algorithms. Our simulation results on learning low-coherence dictionaries for natural image patches as well as image classification based on discriminative over-complete dictionary learning demonstrate the superiority of the proposed algorithms compared with the state-of-the-art method. [ABSTRACT FROM AUTHOR]

Because of the sparse reward and the sequence of the complex multi-step task, there is a big challenge in reinforcement learning, that is, the agent needs to implement several consecutive sequential steps to complete the whole task without the intermediate reward. Reward Shaping and Curriculum Learning algorithms are always used to solve this challenge, but Reward Shaping is prone to sub-optimal policies and Curriculum Learning easily suffers from the catastrophic forgetting problem. In this paper, we propose a novel algorithm called Parallel Multi-Environment Shaping (PMES), where several sub-environments are built based on human knowledge to make the agent aware of the importance of intermediate steps, each of which corresponds to a key intermediate step. Specifically, the learning agent is trained under these parallel multiple environments including the original environment and several sub-environments by synchronous advantage actor-critic algorithm. And the PMES algorithm has the mechanism of adaptive reward shaping to adjust the reward function. In this way, PMES algorithm effectively incorporates human experience by multiple different environments rather than only shaping the reward function, which combines the benefits of Reward Shaping and Curriculum Learning algorithms while avoiding their drawbacks. Extensive experiments on the mini-game 'Build Marines' of StarCraft II environment show that our proposed algorithm is more effective than Reward Shaping, Curriculum Learning, and PLAID algorithms, which is almost close to the level of human Grandmaster. And compared with the existing work, it takes less time and computing resources to reach a good result. [ABSTRACT FROM AUTHOR]

This paper presents an adaptive neuro-fuzzy PID controller based on twin delayed deep deterministic policy gradient (TD3) algorithm for nonlinear systems. In this approach, the observation of the environment is embedded with information of a multiple input single output (MISO) fuzzy inference system (FIS) and have a specially defined fuzzy PID controller in neural network (NN) formation acting as the actor in the TD3 algorithm, which achieves automatic tuning of gains of fuzzy PID controller. From the control perspective, the controller combines the merits of both FIS and PID controller and utilizes reinforcement learning algorithm for optimizing parameters. From the reinforcement learning point of view, embedding the prior knowledge into the fuzzy PID controller incorporated in the actor network helps reduce the learning difficulty in the training process. The proposed method was tested on the cart-pole system in simulation environment with comparison of a linear PID controller, which demonstrates the robustness and generalization of the proposed approach. [ABSTRACT FROM AUTHOR]

One-shot learning has recently attracted growing attention to produce models which can classify significant events from a few or even no labeled examples. In this paper, we introduce a deep Q-network strategy into one-shot learning (OL-DQN) to design a more intelligent learner to infer whether to label a sample automatically or request the true label for the active-learning set-up. Then we conducted experiments in the ALOI dataset for the classification of objects recorded under various imaging circumstances and a dataset for handwriting recognition composed of both characters and digits to have a performance evaluation and application analysis of the proposed model respectively, and the obtained results demonstrate that our model can achieve a better trade-off between prediction accuracy and the need of label requests compared with a purely supervised task, a prior work AOL, and a conventional active learning algorithm QBC. [ABSTRACT FROM AUTHOR]

Fuzzy inference systems with reinforcement learning are currently being used in differential games to train agents with no prior experience. However, the reinforcement learning algorithms based on actor-critic structure have a drawback that the policy is depended on a probability distribution. In this paper, a novel fuzzy deterministic policy gradient algorithm is introduced and applied to classical 1-vs-1 constant-velocity pursuit-evasion differential games. The key goal is to self-learn the optimal strategy in the continuous action domain and obtain a specific physical meaning of the fuzzy rules. The novel proposed algorithm is based on the deterministic policy gradient theorem and the agent learns the near-optimal strategy under the actor-critic structure. The fuzzy inference system is applied as approximators so that the specific physical meaning can be obtained by the linguistic fuzzy rules. Furthermore, the proposed algorithm is applied to solve the decision-making problem of pursuit-evasion differential games. The result is compared with other existing algorithms and it elucidates that the proposed algorithm outperforms the precision and convergence efficiency. [ABSTRACT FROM AUTHOR]

In this paper we propose and discuss different Deep Learning-based ensemble algorithms for a problem of low-visibility events prediction due to fog. Specifically, seven different Deep Learning (DL) architectures have been considered, from which multiple individual learners are generated. Hyperparameters of the models, including parameters concerning data preprocessing, models architecture and training procedure, are randomly selected for each model within a pre-defined discrete range. Also, every model is trained with slightly different data sampled randomly, assuring that every models introduce variety in the ensemble. Then, three different information fusion techniques are employed to build the ensemble models. The influence of the filtering process and the elitism level (the percentage of the individual models entering the ensemble) is also assessed. The performance of the proposed methodology have been tested in two real problems of low-visibility events prediction due to orographical and radiation fog, at the north of Spain. Comparison with different Machine Learning, alternative DL algorithms and meteorological-based methods show the good performance of the proposed deep learning ensembles in this problem. [ABSTRACT FROM AUTHOR]

The person re-identification is an active research branch of the computer vision and attracts many researchers study on it. However, because of the variance in viewpoints, illumination, pedestrians' pose and some of other factors, building a robust person re-identification algorithm in open-world is rather challenging. In this paper, we propose two algorithms, indiscriminative patches trim strategy and multi-instance multi-label learning method, to fix the person re-identification in closed short-term surveillance network. There are two main contributions of this paper: first, we define the discriminative region in a person's image. Correspondingly, an adapted canopy-kmeans algorithm is proposed to evaluate the discriminability of image patches which form the discriminative regions. Besides, two strategies, in local image and in global gallery dataset, are proposed to filter out distractions. Finally, the true discriminative patches are employed to compute the similarity between two images, and then used to re-identify pedestrians. Second, we first introduce the multi-instance multi-label learning methods into re-identification, and hence we propose a framework of solving person re-identification by applying MIMLL. We employ two MIMLL methods, the MIMLBOOST and the MIMLSVM, to detect attributes in each image, and those detected attributes contribute to recognize different pedestrians. Experiments on two benchmark datasets show the competitive performance of the two proposed algorithms. [ABSTRACT FROM AUTHOR]

Abstract This paper aims to propose a distributed cooperative learning (DCL) algorithm for Radial Basis Function Network (RBFN), where training samples are often extremely large-scale, high-dimensional and located on distributed nodes over undirected and connected networks. The algorithm is investigated for RBFN with different basic functions, such as Gaussian function, multi-quadric function, thin plate spline function and so on. The proposed DCL–RBFN algorithm is based on Zero-Gradient-Sum (ZGS) distributed optimization strategy and works in a fully distributed fashion, which has no requirement on central node and communication of raw data from each node during the training process. Moreover, in order to avoid the waste of unnecessary network resources, an event-triggered communication mechanism is applied to our algorithm. Each node exchanges learned information with its neighboring nodes using the ZGS strategy under event-triggered mechanism. The convergence of the proposed DCL–RBFN and event-triggered DCL–RBFN algorithms are presented by using the Lyapunov method. It is proved that the proposed algorithms have exponential convergence speed and require less communication data over networks. Finally, a series of numerical simulations are given to verify the advantages of the proposed algorithms. [ABSTRACT FROM AUTHOR]

Abstract Feature selection is used in many applications in machine learning and bioinformatics. As a popular approach, feature selection can be implemented in the filter-manner based on the sparse solution of the l 1 regularization. Most study of the l 1 regularization concentrates on investigating the iteration solution of the problem or focuses on adapting sparsity to different applications. It is necessary to explore more deeply about how the sparsity learned with the l 1 regularization contributes to feature selection. In this paper, we make an effort to analyze the role of the l 1 regularization in feature selection from the perspective of information theory. We discover that the l 1 regularization contributes to minimizing the redundancy in feature selection. To avoid the complex computation of the l 1 optimization, we propose a novel feature selection algorithm, i.e. the Laplacian regularization based on mutual information (LRMI), which realizes the minimization of the redundancy in a new way, and incorporates the l 2 norm to achieve automatic grouping. Extensive experimental results demonstrate the superiority of LRMI over several traditional l 1 regularization based feature selection algorithms with less time consumption. [ABSTRACT FROM AUTHOR]

Abstract Multi-Instance learning (MIL) aims to predict labels of unlabeled bags by training a model with labeled bags. The usual assumption of existing MIL methods is that the underlying distribution of training data is the same as that of the testing data. However, this assumption may not be valid in practice, especially when training data from a source domain and testing data from a target domain are drawn from different distributions. In this paper, we put forward a novel algorithm Multi-Instance Transfer Metric Learning (MITML). Specially, MITML first attempts to bridge the distributions of different domains by using the bag weighting method. Then a consistent maximum likelihood estimation method is learned to construct an optimal distance metric and exploited to classify testing bags. Comprehensive experimental results on benchmark datasets have demonstrated that the learning performance of the proposed MITML algorithm is better than those of other state-of-the-art MIL algorithms. [ABSTRACT FROM AUTHOR]

Dimensionality reduction as one of most attractive topics in machine learning research area has aroused extensive attentions in recent years. In order to preserve the local structure of data, most of dimensionality reduction methods consider constructing the relationships among each sample and its k nearest neighbors, and they find the neighbors in original space by using Euclidean distance. Since the data in original space contain some noises and redundant features, finding the neighbors in original space is incorrect and may degrade the subsequent performance. Therefore, how to find the optimal k nearest neighbors for each sample is the key point to improve the robustness of model. In this paper, we propose a novel dimensionality reduction method, named Adaptive Neighborhood MinMax Projections (ANMMP) which finds the neighbors in the optimal subspace by solving Trace Ratio problem in which the noises and redundant features have been removed already. Meanwhile, the samples within same class are pulled together while the samples between different classes are pushed far away in such learned subspace. Besides, proposed model is a general approach which can be implemented easily and applied on other methods to improve the robustness. Extensive experiments conducted on several synthetic data and real-world data sets and achieve some encouraging performance with comparison to metric learning and feature extraction methods, which demonstrates the efficiency of our method. [ABSTRACT FROM AUTHOR]

Correntropy is a local similarity measure between two arbitrary variables, and it has been applied in a variety of learning algorithms to improve noise insensitivity. In this paper, based on the correntropy, a non-convex and bounded loss function is obtained which contains second and higher order moments of the classification margin. And the novel loss function is robust to noises and close to the 0–1 loss function. Then we introduce it into extreme learning machine (ELM), and propose a correntropy-based robust ELM framework for classification, trained by half quadratic optimization to cope with non-convexity of the algorithm. To evaluate robustness, feature noise and label noise are simulated to provide noisy environments. Experimental results on benchmark datasets demonstrate that the proposed algorithm is better than original algorithms and robust algorithms. Moreover, the superiority of proposed algorithm in noisy environment is more evident, which further proves its robustness to noises. [ABSTRACT FROM AUTHOR]

Twin support vector machines (TWSVM) is a new machine learning method based on the theory of Support Vector Machine (SVM). Unlike SVM, TWSVM would generate two non-parallel planes, such that each plane is closer to one of the two classes and is as far as possible from the other. In TWSVM, a pair of smaller sized quadratic programming problems (QPPs) is solved, instead of solving a single large one in SVM, making the computational speed of TWSVM approximately 4 times faster than the standard SVM. At present, TWSVM has become one of the popular methods because of its excellent learning performance. In this paper, the research progress of TWSVM is reviewed. Firstly, it analyzes the basic theory of TWSVM, then tracking describes the research progress of TWSVM including the learning model and specific applications in recent years, finally points out the research and development prospects. This helps researchers to effectively use TWSVM as an emerging research approach, encouraging them to work further on performance improvement. [ABSTRACT FROM AUTHOR]

Large numbers of data streams are today generated in many fields. A key challenge when learning from such streams is the problem of concept drift. Many methods, including many prototype methods, have been proposed in recent years to address this problem. This paper presents a refined taxonomy of instance selection and generation methods for the classification of data streams subject to concept drift. The taxonomy allows discrimination among a large number of methods which pre-existing taxonomies for offline instance selection methods did not distinguish. This makes possible a valuable new perspective on experimental results, and provides a framework for discussion of the concepts behind different algorithm-design approaches. We review a selection of modern algorithms for the purpose of illustrating the distinctions made by the taxonomy. We present the results of a numerical experiment which examined the performance of a number of representative methods on both synthetic and real-world data sets with and without concept drift, and discuss the implications for the directions of future research in light of the taxonomy. On the basis of the experimental results, we are able to give recommendations for the experimental evaluation of algorithms which may be proposed in the future. [ABSTRACT FROM AUTHOR]

Nonnegative matrix factorization (NMF) is a very popular unsupervised or semi-supervised learning method useful in various applications including data clustering, image processing, and semantic analysis of documents. This study focuses on Symmetric NMF (SNMF), which is a special case of NMF and can be useful in network analysis. Although there exist several algorithms for SNMF in literature, their convergence and initialization have not been well addressed. In this paper, we first discuss the convergence and initialization of existing algorithms for SNMF. We then propose a Converged Algorithm for SNMF (called CASNMF) which minimizes the Euclidean distance between a symmetrical matrix and its approximation of SNMF. Based on the optimization principle and the local auxiliary function method, we prove that our presented CASNMF does not only converge to a stationary point, but also could be applied to the wider range of SNMF problems. In addition, CASNMF does not require that the initial values are nonzero. To verify our theoretical results, experiments on three data sets are conducted by comparing our proposed CASNMF with other existing methods. [ABSTRACT FROM AUTHOR]

Word embeddings and CNN (convolutional neural networks) architecture are crucial ingredients of sentiment analysis. However, sentiment and lexicon embeddings are rarely used and CNN is incompetent to capture global features of sentence. To this end, semantic embeddings, sentiment embeddings and lexicon embeddings are applied for texts encoding, and three different attentions including attention vector, LSTM (long short term memory) attention and attentive pooling are integrated with CNN model in this paper. Additionally, a word and its context are explored to disambiguate the meaning of the word for rich input representation. To improve the performance of three different attention CNN models, CCR (cross-modality consistent regression) and transfer learning are presented. It is worth noticing that CCR and transfer learning are used in textual sentiment analysis for the first time. Finally, some experiments on two different datasets demonstrate that the proposed attention CNN models achieve the best or the next-best results against the existing state-of-the-art models. [ABSTRACT FROM AUTHOR]

There are two categories of partial shape matching problem: whole-to-part matching and part-to-part matching. The first establishes the best match between an open curve and a part of a closed curve, while the second is to find a part of the first input closed curve that can be well aligned with a part of the second input closed curve. In this paper, we present a novel approach to solve these two categories of the partial shape matching problem. First, we propose a novel shape descriptor, triangular centroid distances (TCDs), for shape representation; the TCDs shape descriptor is invariant to translation, rotation, scaling, and considerable shape deformations. Then, using the TCDs shape descriptor, we present a method to deal with the whole-to-part partial shape matching problem. Finally, we extend our work to part-to-part partial shape matching. Here, we propose a new approach, again using the TCDs shape descriptor, to solve the part-to-part partial shape matching problem. Experimental results demonstrate that our method outperforms existing methods in 2D nonrigid partial shape matching. [ABSTRACT FROM AUTHOR]

Freehand three-dimensional (3D) ultrasound imaging is an attractive research area because it is capable of providing large field of view and high in-plane resolution image to allow better illustration of complex anatomy structures. However, reconstructed image is corrupted with speckle noise and artifacts in the conventional reconstructed volume data. In this paper, we propose a simple but effective adaptive kernel regression method for volume reconstruction from freehand swept B-scan images. By creating a linear model for estimating the homogeneous region of the B-scan image and learning the parameters of the model with a supervised learning method, the statistical characteristic of speckle can be well recovered. With the learned linear model of speckle, we can easily estimate the homogenous region and reconstruct image with speckle reduction and edge preservation via the adaptive turning of the smoothing parameters of the kernel regression. Our algorithm lends itself to parallel processing, and yields a 288× speedup on a graphics processing unit (GPU). Experiments on the simulated data, ultrasonic abdominal phantom and in-vivo liver of human subject and comparisons with some classical and recent algorithms are used to demonstrate its improvements in both volume reconstruction accuracy and efficiency. [ABSTRACT FROM AUTHOR]

As having the closed form solution, the least squares support vector machine (LSSVM) has been widely used for classification and regression problems owing to its competitive performance compared with other types of SVMs. However, the LSSVM has two drawbacks: it is sensitive to outliers and its solution lacks sparseness. The robust LSSVM (R-LSSVM) partially overcomes the first drawback via its nonconvex truncated loss function, but it is unable to address the second drawback because its current algorithms produce dense solutions that are inefficient for training large-scale problems. In this paper, we interpret the robustness of the R-LSSVM from a re-weighted viewpoint and develop a primal R-LSSVM using the representer theorem. The new model may have a sparse solution. Then, we design a convergent sparse R-LSSVM (SR-LSSVM) algorithm to achieve a sparse solution of the primal R-LSSVM after obtaining a low-rank approximation of the kernel matrix. The new algorithm not only overcomes the two drawbacks of LSSVM simultaneously, it also has lower complexity than the existing algorithms. Therefore, it is very efficient at training large-scale problems. Numerous experimental results demonstrate that the SR-LSSVM can achieve better or comparable performance to other related algorithms in less training time, especially when used to train large-scale problems. [ABSTRACT FROM AUTHOR]