Search

As a powerful data modeling tool, Heterogeneous Information Network (HIN) has been successfully used in auxiliary information exploitation to boost recommendation performance. For HIN based recommendation, it is challenging to extract and fuse useful features of user preferences and item attributes under different semantic paths in HINs. Existing methods leverage a pre-defined fusion function to integrate different semantics for recommendation, which cannot characterize the complex nonlinear interactions between users and items. In this paper, we present a general framework named MNRec, short for Meta-path fusion based Neural Recommendation, to extract and fuse user and item embeddings under different meta-paths for recommendation. Under the framework, we propose an instantiation of MNRec with Multi-Layer Perceptron (MLP) structure. It consists of two major steps, i.e., meta-path based heterogeneous network embedding and deep learning based rating prediction. Concretely, appropriate meta-paths are first designed according to domain knowledge. Then the embeddings of users and items are obtained through a meta-path and commuting matrix based heterogeneous network embedding method. Finally, in light of the powerful nonlinear modeling capabilities of deep neural networks, the learned embeddings under different meta-paths are integrated into a two-pathway MLP structure for rating prediction. Experimental results on three real-world datasets demonstrate the superiority and effectiveness of MNRec compared with state-of-the-art baselines in rating prediction. [ABSTRACT FROM AUTHOR]

Since input constraints and external disturbances are unavoidable in tracking control problems, how to obtain a controller in this case to save communication and data resources at the same time is very challenging. Aiming at these challenges, this paper develops a novel neural network (NN)-based event-triggered integral reinforcement learning (IRL) algorithm for constrained H ∞ tracking control problems. First, the constrained H ∞ tracking control problem is transformed into a regulation problem. Second, an event-triggered optimal controller is designed to reduce network transmission burden and improve resource utilization, where a novel threshold is proposed and its non-negativity can be guaranteed. Third, for implementation purpose, a novel NN-based event-triggered IRL algorithm is developed. In order to improve data utilization, the experience replay technique with an easy-to-verify condition is employed in the learning process. Theoretical analysis proves that the tracking error and weight estimation error are uniformly ultimately bounded. Finally, simulation verification shows the effectiveness of the present method. [ABSTRACT FROM AUTHOR]

This paper is concerned with the set-membership filtering issue for a class of artificial neural networks subject to mixed attacks. The encoding–decoding communication mechanism is adopted in the processing of data sharing between neurons. During the information exchanges among neurons, both injection and DoS attacks are concurrently considered to reflect the practical operating conditions of the investigated neural networks. The purpose of the addressed problem is to present an algorithm to estimate the neurons' states in the presence of mixed attacks, while guaranteeing the estimation errors at each neuron are confined within certain prescribed ellipsoidal region. Sufficient conditions are derived, in terms of convex optimization approach, to ensure the existence of desired filter, and the explicit filtering parameters are obtained via solving the provided set of matrix inequalities. Finally, a numerical simulation example is proposed to show the validity of the obtained theoretical results. [ABSTRACT FROM AUTHOR]

Neural networks have shown tremendous growth in recent years to solve numerous problems. Various types of neural networks have been introduced to deal with different types of problems. However, the main goal of any neural network is to transform the non-linearly separable input data into more linearly separable abstract features using a hierarchy of layers. These layers are combinations of linear and nonlinear functions. The most popular and common non-linearity layers are activation functions (AFs), such as Logistic Sigmoid, Tanh, ReLU, ELU, Swish and Mish. In this paper, a comprehensive overview and survey is presented for AFs in neural networks for deep learning. Different classes of AFs such as Logistic Sigmoid and Tanh based, ReLU based, ELU based, and Learning based are covered. Several characteristics of AFs such as output range, monotonicity, and smoothness are also pointed out. A performance comparison is also performed among 18 state-of-the-art AFs with different networks on different types of data. The insights of AFs are presented to benefit the researchers for doing further research and practitioners to select among different choices. The code used for experimental comparison is released at: https://github.com/shivram1987/ActivationFunctions. [ABSTRACT FROM AUTHOR]

This paper focuses on an adaptive learning and control problem for a class of discrete-time nonlinear uncertain systems operating under multiple environments. A novel intelligent learning control framework is proposed by using a combination of offline and online learning methods. Specifically, in the offline learning mode, a deterministic learning (DL) based adaptive dynamics learning approach is first proposed to achieve locally-accurate identification of associated nonlinear uncertain system dynamics under each anticipated individual environment, and the learned knowledge is obtained and stored in a set of constant radial basis function neural network models. Then, with the learned knowledge, an online adaptive learning control scheme is further developed, which consists of: (i) an online adaptive learning control mechanism composed of multiple experience-based controllers and a DL-based adaptive learning controller, aiming to provide desired control performance for the plant operating under each individual environment; and (ii) a learning-based recognition mechanism composed of multiple recognition estimators and a DL-based identifier, aiming to recognize the active environment and schedule appropriate control strategies in real time. To guarantee the system stability during environment transition, a robust quasi-sliding-mode controller is further developed and embedded in the overall controller architecture. With this new intelligent adaptive learning control framework, the overall system is capable of adapting not only to any anticipated (pre-defined) environment by re-utilizing the knowledge obtained from both offline and online learning, but also to unanticipated (new) environments by actively acquiring new knowledge online. Simulation studies are conducted to verify the effectiveness and advantages of this new framework. [ABSTRACT FROM AUTHOR]

Neural learning plays an important role in many applications. In this paper, we derive a new learning paradigm for neural networks. Most existing neural models train network parameters including connecting weights and biases via optimizing a loss or energy function. Inspired from the associative learning in brain, we propose to associate different patterns via modeling joint distribution of them based on a hierarchical architecture. We first define an energy function based on the distance between hierarchical features of different patterns. Then a Gibbs typological distribution is constructed according to the energy field. To optimize the model, it is necessary to estimate the gradient expectation via sampling. Different from the simple architecture of existing probabilistic neural models such as restricted Boltzmann machine, the difficulty of optimizing this model lies in uncomputable probability for sampling from the distribution. Then we propose an optimization based sampling method. After learning, conditional probability can be derived and the unknown pattern can be generated via sampling as well. Compared with existing neural learning models, the proposed deep associative learning can associate different patterns and can be directly applied to many learning problems. Experiments on problems of classification, image transformation, and image change detection verify the effectiveness of the proposed learning paradigm. [ABSTRACT FROM AUTHOR]

This paper proposes a computationally efficient and effective data-driven modeling framework for dynamical systems. The proposed modeling framework employs a collection of shallow neural networks known as Extreme Learning Machines (ELMs) to model local system behaviors along with data-driven inferred transitions among local models to establish a neural hybrid automaton model. First, the sampled system inputs are mapped to the corresponding feature spaces to obtain data-driven partitions, which subsequently define the transitions and invariants of the neural hybrid automaton model through a novel data-driven mode clustering process. Then, a collection of ELMs are trained to approximate the local dynamics. The learning processes integrate a segmented data merging procedure for location identification and a local dynamics modeling process. The proposed neural hybrid automaton models can capture behaviors of complex dynamical systems with high modeling precision but significantly lower computational complexities in computationally expensive tasks such as training and verification, which are traditionally considered to be computationally expensive tasks for neural network models. A computationally efficient set-valued reachability analysis method which is commonly used in safety verification is then developed based on interval analysis and a novel Split and Combine process. Finally, applications to modeling the limit cycle and human handwritten motions are presented to show the effectiveness and efficiency of our approach. [ABSTRACT FROM AUTHOR]