Search

Crowdsourcing services provide a fast, efficient, and cost-effective way to obtain large labeled data for supervised learning. Unfortunately, the quality of crowdsourced labels cannot satisfy the standards of practical applications. Ground-truth inference, simply called label integration, designs proper aggregation methods to infer the unknown true label of each instance (sample) from the multiple noisy label set provided by ordinary crowd labelers (workers). However, nearly all existing label integration methods focus solely on the multiple noisy label set per individual instance while totally ignoring the intercorrelation among multiple noisy label sets of different instances. To solve this problem, a multiple noisy label distribution propagation (MNLDP) method is proposed in this article. MNLDP at first estimates the multiple noisy label distribution of each instance from its multiple noisy label set and then propagates its multiple noisy label distribution to its nearest neighbors. Consequently, each instance absorbs a fraction of the multiple noisy label distributions from its nearest neighbors and yet simultaneously maintains a fraction of its own original multiple noisy label distribution. Empirical studies on a collection of an artificial dataset, six simulated UCI datasets, and three real-world crowdsourced datasets show that MNLDP outperforms all other existing state-of-the-art label integration methods in terms of the integration accuracy and classification accuracy. [ABSTRACT FROM AUTHOR]

This article investigates the stability and synchronization of nonautonomous reaction–diffusion neural networks with general time-varying delays. Compared with the existing works concerning reaction–diffusion neural networks, the main innovation of this article is that the network coefficients are time-varying, and the delays are general (which means that fewer constraints are posed on delays; for example, the commonly used conditions of differentiability and boundedness are no longer needed). By Green’s formula and some analytical techniques, some easily checkable criteria on stability and synchronization for the underlying neural networks are established. These obtained results not only improve some existing ones but also contain some novel results that have not yet been reported. The effectiveness and superiorities of the established criteria are verified by three numerical examples. [ABSTRACT FROM AUTHOR]

This article investigates the optimally distributed consensus control problem for discrete-time multiagent systems with completely unknown dynamics and computational ability differences. The problem can be viewed as solving nonzero-sum games with distributed reinforcement learning (RL), and each agent is a player in these games. First, to guarantee the real-time performance of learning algorithms, a data-based distributed control algorithm is proposed for multiagent systems using offline system interaction data sets. By utilizing the interactive data produced during the run of a real-time system, the proposed algorithm improves system performance based on distributed policy gradient RL. The convergence and stability are guaranteed based on functional analysis and the Lyapunov method. Second, to address asynchronous learning caused by computational ability differences in multiagent systems, the proposed algorithm is extended to an asynchronous version in which executing policy improvement or not of each agent is independent of its neighbors. Furthermore, an actor-critic structure, which contains two neural networks, is developed to implement the proposed algorithm in synchronous and asynchronous cases. Based on the method of weighted residuals, the convergence and optimality of the neural networks are guaranteed by proving the approximation errors converge to zero. Finally, simulations are conducted to show the effectiveness of the proposed algorithm. [ABSTRACT FROM AUTHOR]

Deep neural networks (DNNs) are shown to be excellent solutions to staggering and sophisticated problems in machine learning. A key reason for their success is due to the strong expressive power of function representation. For piecewise linear neural networks (PLNNs), the number of linear regions is a natural measure of their expressive power since it characterizes the number of linear pieces available to model complex patterns. In this article, we theoretically analyze the expressive power of PLNNs by counting and bounding the number of linear regions. We first refine the existing upper and lower bounds on the number of linear regions of PLNNs with rectified linear units (ReLU PLNNs). Next, we extend the analysis to PLNNs with general piecewise linear (PWL) activation functions and derive the exact maximum number of linear regions of single-layer PLNNs. Moreover, the upper and lower bounds on the number of linear regions of multilayer PLNNs are obtained, both of which scale polynomially with the number of neurons at each layer and pieces of PWL activation function but exponentially with the number of layers. This key property enables deep PLNNs with complex activation functions to outperform their shallow counterparts when computing highly complex and structured functions, which, to some extent, explains the performance improvement of deep PLNNs in classification and function fitting. [ABSTRACT FROM AUTHOR]

In this paper, we mainly investigate two issues for sequence labeling, namely, label imbalance and noisy data that are commonly seen in the scenario of named entity recognition (NER) and are largely ignored in the existing works. To address these two issues, a new method termed robust sequence labeling (RoSeq) is proposed. Specifically, to handle the label imbalance issue, we first incorporate label statistics in a novel conditional random field (CRF) loss. In addition, we design an additional loss to reduce the weights of overwhelming easy tokens for augmenting the CRF loss. To address the noisy training data, we adopt an adversarial training strategy to improve model generalization. In experiments, the proposed RoSeq achieves the state-of-the-art performances on CoNLL and English Twitter NER—88.07% on CoNLL-2002 Dutch, 87.33% on CoNLL-2002 Spanish, 52.94% on WNUT-2016 Twitter, and 43.03% on WNUT-2017 Twitter without using the additional data. [ABSTRACT FROM AUTHOR]

This paper considers the tracking synchronization problem for a class of coupled reaction–diffusion neural networks (CRDNNs) with undirected topology. For the case where the tracking trajectory has identical individual dynamic as that of the network nodes, the edge-based and vertex-based adaptive strategies on coupling strengths as well as adaptive controllers, which demand merely the local neighbor information, are proposed to synchronize the CRDNNs to the tracking trajectory. To reduce the control costs, an adaptive pinning control technique is employed. For the case where the tracking trajectory has different individual dynamic from that of the network nodes, the vertex-based adaptive strategy is proposed to drive the synchronization error to a relatively small area, which is adjustable according to the parameters of the adaptive strategy. This kind of adaptive design can enhance the robustness of the network against the external disturbance posed on the tracking trajectory. The obtained theoretical results are verified by two representative examples. [ABSTRACT FROM AUTHOR]

This paper investigates the cooperative output regulation problem for heterogeneous multiagent systems (MASs) under switching topology. Two novel distributed event-triggered adaptive control strategies based on state feedback and output feedback are developed, which can avoid using the minimal nonzero eigenvalue of Laplacian matrix associated with global system topologies. It is shown that under the proposed control protocols, MASs could achieve asymptotic tracking and disturbance rejection, and meanwhile, the amount of transmission data and communication cost among agents can be reduced. Then, the leader-following consensus problem of MASs is given as an application of our main results. Finally, an example is presented to verify the effectiveness of the proposed control schemes. [ABSTRACT FROM AUTHOR]

This paper is concerned with the guaranteed cost control problem for a class of Markov jump discrete-time neural networks (NNs) with event-triggered mechanism, asynchronous jumping, and fading channels. The Markov jump NNs are introduced to be close to reality, where the modes of the NNs and guaranteed cost controller are determined by two mutually independent Markov chains. The asynchronous phenomenon is considered, which increases the difficulty of designing required mode-dependent controller. The event-triggered mechanism is designed by comparing the relative measurement error with the last triggered state at the process of data transmission, which is used to eliminate dispensable transmission and reduce the networked energy consumption. In addition, the signal fading is considered for the effect of signal reflection and shadow in wireless networks, which is modeled by the novel Rice fading models. Some novel sufficient conditions are obtained to guarantee that the closed-loop system reaches a specified cost value under the designed jumping state feedback control law in terms of linear matrix inequalities. Finally, some simulation results are provided to illustrate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

This paper investigates the synchronization issue of coupled reaction–diffusion neural networks with directed topology via an adaptive approach. Due to the complexity of the network structure and the presence of space variables, it is difficult to design proper adaptive strategies on coupling weights to accomplish the synchronous goal. Under the assumptions of two kinds of special network structures, that is, directed spanning path and directed spanning tree, some novel edge-based adaptive laws, which utilized the local information of node dynamics fully are designed on the coupling weights for reaching synchronization. By constructing appropriate energy function, and utilizing some analytical techniques, several sufficient conditions are given. Finally, some simulation examples are given to verify the effectiveness of the obtained theoretical results. [ABSTRACT FROM PUBLISHER]

Due to hardware limitations, it is challenging for sensors to acquire images of high resolution in both spatial and spectral domains, which arouses a trend that utilizing a low-resolution hyperspectral image (LR-HSI) and a high-resolution multispectral image (HR-MSI) to fuse an HR-HSI in an unsupervised manner. Considering the fact that most existing methods are restricted by using linear spectral unmixing, we propose a nonlinear variational probabilistic generative model (NVPGM) for the unsupervised fusion task based on nonlinear unmixing. We model the joint full likelihood of the observed pixels in an LR-HSI and an HR-MSI, both of which are assumed to be generated from the corresponding latent representations, i.e., the abundance vectors. The sufficient statistics of the generative conditional distributions are nonlinear functions with respect to the latent variable, realized by neural networks, which results in a nonlinear spectral mixture model. For scalability and efficiency, we construct two recognition models to infer the latent representations, which are parameterized by neural networks as well. Simultaneously inferring the latent representations and optimizing the parameters are achieved using stochastic gradient variational inference, after which the target HR-HSI is retrieved via feedforward mapping. Though without supervised information about the HR-HSI, NVPGM still can be trained based on extra LR-HSI and HR-MSI data sets in advance unsupervisedly and processes the images at the test phase in real time. Three commonly used data sets are used to evaluate the effectiveness and efficiency of NVPGM, illustrating the outperformance of NVPGM in the unsupervised LR-HSI and HR-MSI fusion task. [ABSTRACT FROM AUTHOR]

This article aims to point out an error in the concerned paper and a possible correction to address the error. The correction is achieved by several revisions made on the corresponding assumption, event-triggering function, adaptive laws, and event-triggered control law. [ABSTRACT FROM AUTHOR]

In this paper, we study a new problem in the scenario of sequences labeling. To be exact, we consider that the training data are with annotation of various degrees, namely, fully labeled, unlabeled, and partially labeled sequences. The learning with fully un/labeled sequence refers to the standard setting in traditional un/supervised learning, and the proposed partially labeling specifies the subject that the element does not belong to. The partially labeled data are cheaper to obtain compared with the fully labeled data though it is less informative, especially when the tasks require a lot of domain knowledge. To solve such a practical challenge, we propose a novel deep conditional random field (CRF) model which utilizes an end-to-end learning manner to smoothly handle fully/un/partially labeled sequences within a unified framework. To the best of our knowledge, this could be one of the first works to utilize the partially labeled instance for sequence labeling, and the proposed algorithm unifies the deep learning and CRF in an end-to-end framework. Extensive experiments show that our method achieves state-of-the-art performance in two sequence labeling tasks on some popular data sets. [ABSTRACT FROM AUTHOR]

This brief is concerned with stability analysis for delayed neural networks (DNNs). By establishing polynomials and introducing slack variables reasonably, some improved delay-product type of auxiliary polynomial-based functions (APFs) is developed to exploit additional degrees of freedom and more information on extra states. Then, by constructing Lyapunov–Krasovskii functional using APFs and integrals of quadratic forms with high order scalar functions, a novel stability criterion is derived for DNNs, in which the benefits of the improved inequalities are fully integrated and the information on delay and its derivative is well reflected. By virtue of the advantages of APFs, more desirable performance is achieved through the proposed approach, which is demonstrated by the numerical examples. [ABSTRACT FROM AUTHOR]

Clustering algorithms based on deep neural networks have been widely studied for image analysis. Most existing methods require partial knowledge of the true labels, namely, the number of clusters, which is usually not available in practice. In this article, we propose a Bayesian nonparametric framework, deep nonparametric Bayes (DNB), for jointly learning image clusters and deep representations in a doubly unsupervised manner. In doubly unsupervised learning, we are dealing with the problem of “unknown unknowns,” where we estimate not only the unknown image labels but also the unknown number of labels as well. The proposed algorithm alternates between generating a potentially unbounded number of clusters in the forward pass and learning the deep networks in the backward pass. With the help of the Dirichlet process mixtures, the proposed method is able to partition the latent representations space without specifying the number of clusters a priori. An important feature of this work is that all the estimation is realized with an end-to-end solution, which is very different from the methods that rely on post hoc analysis to select the number of clusters. Another key idea in this article is to provide a principled solution to the problem of “trivial solution” for deep clustering, which has not been much studied in the current literature. With extensive experiments on benchmark datasets, we show that our doubly unsupervised method achieves good clustering performance and outperforms many other unsupervised image clustering methods. [ABSTRACT FROM AUTHOR]