Search

Deep neural networks (DNNs) are shown to be excellent solutions to staggering and sophisticated problems in machine learning. A key reason for their success is due to the strong expressive power of function representation. For piecewise linear neural networks (PLNNs), the number of linear regions is a natural measure of their expressive power since it characterizes the number of linear pieces available to model complex patterns. In this article, we theoretically analyze the expressive power of PLNNs by counting and bounding the number of linear regions. We first refine the existing upper and lower bounds on the number of linear regions of PLNNs with rectified linear units (ReLU PLNNs). Next, we extend the analysis to PLNNs with general piecewise linear (PWL) activation functions and derive the exact maximum number of linear regions of single-layer PLNNs. Moreover, the upper and lower bounds on the number of linear regions of multilayer PLNNs are obtained, both of which scale polynomially with the number of neurons at each layer and pieces of PWL activation function but exponentially with the number of layers. This key property enables deep PLNNs with complex activation functions to outperform their shallow counterparts when computing highly complex and structured functions, which, to some extent, explains the performance improvement of deep PLNNs in classification and function fitting. [ABSTRACT FROM AUTHOR]

This paper considers the tracking synchronization problem for a class of coupled reaction–diffusion neural networks (CRDNNs) with undirected topology. For the case where the tracking trajectory has identical individual dynamic as that of the network nodes, the edge-based and vertex-based adaptive strategies on coupling strengths as well as adaptive controllers, which demand merely the local neighbor information, are proposed to synchronize the CRDNNs to the tracking trajectory. To reduce the control costs, an adaptive pinning control technique is employed. For the case where the tracking trajectory has different individual dynamic from that of the network nodes, the vertex-based adaptive strategy is proposed to drive the synchronization error to a relatively small area, which is adjustable according to the parameters of the adaptive strategy. This kind of adaptive design can enhance the robustness of the network against the external disturbance posed on the tracking trajectory. The obtained theoretical results are verified by two representative examples. [ABSTRACT FROM AUTHOR]

This paper is concerned with the guaranteed cost control problem for a class of Markov jump discrete-time neural networks (NNs) with event-triggered mechanism, asynchronous jumping, and fading channels. The Markov jump NNs are introduced to be close to reality, where the modes of the NNs and guaranteed cost controller are determined by two mutually independent Markov chains. The asynchronous phenomenon is considered, which increases the difficulty of designing required mode-dependent controller. The event-triggered mechanism is designed by comparing the relative measurement error with the last triggered state at the process of data transmission, which is used to eliminate dispensable transmission and reduce the networked energy consumption. In addition, the signal fading is considered for the effect of signal reflection and shadow in wireless networks, which is modeled by the novel Rice fading models. Some novel sufficient conditions are obtained to guarantee that the closed-loop system reaches a specified cost value under the designed jumping state feedback control law in terms of linear matrix inequalities. Finally, some simulation results are provided to illustrate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

This paper investigates the synchronization issue of coupled reaction–diffusion neural networks with directed topology via an adaptive approach. Due to the complexity of the network structure and the presence of space variables, it is difficult to design proper adaptive strategies on coupling weights to accomplish the synchronous goal. Under the assumptions of two kinds of special network structures, that is, directed spanning path and directed spanning tree, some novel edge-based adaptive laws, which utilized the local information of node dynamics fully are designed on the coupling weights for reaching synchronization. By constructing appropriate energy function, and utilizing some analytical techniques, several sufficient conditions are given. Finally, some simulation examples are given to verify the effectiveness of the obtained theoretical results. [ABSTRACT FROM PUBLISHER]