Search

• ANN, CART, and RF regression trees have shown the potential in modeling energy cost. • Three different strategies regarding variable selection were tested and compared. • Machine learning and Boruta method have produced the highest accuracy of prediction. • The model has extracted heating and occupational data as the most important. • The created model could be used to assess the concept of smart buildings and cities. The paper deals with modeling the cost of energy consumed in public buildings by leveraging three machine learning methods: artificial neural networks, CART, and random forest regression trees. Energy consumption is one of the major issues in global and national policies, therefore scientific efforts in creating prediction models of energy consumption and cost are highly important. One of the largest energy consumers in every state is its public sector, consisting of educational, health, public administration, military, and other types of public buildings. Recent technologies based on sensor networks and Big data platforms enable collection of large amounts of data that could be used to analyze energy consumption and cost. A real data from Croatian public sector is used in this paper including a large number of constructional, energetic, occupational, climate and other attributes. The algorithms for data pre-processing and modeling by optimizing parameters are suggested. Three strategies were tested: (1) with all available variables, (2) with a filter-based variable selection, and (3) with a wrapper-based variable selection which integrates Boruta algorithm and random forest. Prediction models of energy cost are created using two approaches: (a) comparative usage of artificial neural networks and two types of regression trees, CART and random forest, and (b) integration of RF-Boruta variable selection and machine learning methods for prediction. A cross-validation procedure was used to optimize the artificial neural network and regression tree topology, as well to select the most appropriate activation function. Along with creating a prediction model, the aim of the paper was also to extract the relevant predictors of energy cost in public buildings which are important in planning the construction or renovation of buildings. The results have shown that the second approach which integrates machine learning with Boruta method, where the random forest algorithm is used for both variable reduction and prediction modeling, has produced a higher accuracy of prediction than the individual usage of three machine learning methods. Such findings confirm the potential of hybrid machine learning methods which are suggested in previous research, but in favor of random forest method over CART and artificial neural networks. Regarding the variable selection, the model has extracted heating and occupational data as the most important, followed by constructional, cooling, electricity, and lighting attributes. The model could be implemented in public buildings information systems and their IoT networks within the concept of smart buildings and smart cities. [ABSTRACT FROM AUTHOR]

As a powerful data modeling tool, Heterogeneous Information Network (HIN) has been successfully used in auxiliary information exploitation to boost recommendation performance. For HIN based recommendation, it is challenging to extract and fuse useful features of user preferences and item attributes under different semantic paths in HINs. Existing methods leverage a pre-defined fusion function to integrate different semantics for recommendation, which cannot characterize the complex nonlinear interactions between users and items. In this paper, we present a general framework named MNRec, short for Meta-path fusion based Neural Recommendation, to extract and fuse user and item embeddings under different meta-paths for recommendation. Under the framework, we propose an instantiation of MNRec with Multi-Layer Perceptron (MLP) structure. It consists of two major steps, i.e., meta-path based heterogeneous network embedding and deep learning based rating prediction. Concretely, appropriate meta-paths are first designed according to domain knowledge. Then the embeddings of users and items are obtained through a meta-path and commuting matrix based heterogeneous network embedding method. Finally, in light of the powerful nonlinear modeling capabilities of deep neural networks, the learned embeddings under different meta-paths are integrated into a two-pathway MLP structure for rating prediction. Experimental results on three real-world datasets demonstrate the superiority and effectiveness of MNRec compared with state-of-the-art baselines in rating prediction. [ABSTRACT FROM AUTHOR]

Since input constraints and external disturbances are unavoidable in tracking control problems, how to obtain a controller in this case to save communication and data resources at the same time is very challenging. Aiming at these challenges, this paper develops a novel neural network (NN)-based event-triggered integral reinforcement learning (IRL) algorithm for constrained H ∞ tracking control problems. First, the constrained H ∞ tracking control problem is transformed into a regulation problem. Second, an event-triggered optimal controller is designed to reduce network transmission burden and improve resource utilization, where a novel threshold is proposed and its non-negativity can be guaranteed. Third, for implementation purpose, a novel NN-based event-triggered IRL algorithm is developed. In order to improve data utilization, the experience replay technique with an easy-to-verify condition is employed in the learning process. Theoretical analysis proves that the tracking error and weight estimation error are uniformly ultimately bounded. Finally, simulation verification shows the effectiveness of the present method. [ABSTRACT FROM AUTHOR]

This paper proposes finite-horizon optimal tracking control approach based on data for completely unknown discrete-time nonlinear affine systems. First, the identifier is designed by input and output data, which is used to identify system function and system model. And based on tracking error, the system function is transformed to the augmentation system with finite-time optimal performance. In finite time, by minimizing the performance index function, the iterative approximate dynamic programming (ADP) is utilized to solve Hamilton–Jacobi–Bellman (HJB) equation. The idea is carried by the policy iterative (PI) based on the model neural network, which makes the iterative control of the augmentation system available at the each step. At the same time, the action neural network is utilized to acquire the approximate optimal tracking control law and the critic neural network is used for approximating the optimal performance index function for the augmentation system. Afterwards, the paper show the analysis process that the convergence and stability for the iterative ADP algorithm and the weight estimation errors based on the PI, respectively. The end of the paper, a simulation example is applied to show the theoretical results and proposed approach. [ABSTRACT FROM AUTHOR]

This paper is concerned with the set-membership filtering issue for a class of artificial neural networks subject to mixed attacks. The encoding–decoding communication mechanism is adopted in the processing of data sharing between neurons. During the information exchanges among neurons, both injection and DoS attacks are concurrently considered to reflect the practical operating conditions of the investigated neural networks. The purpose of the addressed problem is to present an algorithm to estimate the neurons' states in the presence of mixed attacks, while guaranteeing the estimation errors at each neuron are confined within certain prescribed ellipsoidal region. Sufficient conditions are derived, in terms of convex optimization approach, to ensure the existence of desired filter, and the explicit filtering parameters are obtained via solving the provided set of matrix inequalities. Finally, a numerical simulation example is proposed to show the validity of the obtained theoretical results. [ABSTRACT FROM AUTHOR]

Neural networks have shown tremendous growth in recent years to solve numerous problems. Various types of neural networks have been introduced to deal with different types of problems. However, the main goal of any neural network is to transform the non-linearly separable input data into more linearly separable abstract features using a hierarchy of layers. These layers are combinations of linear and nonlinear functions. The most popular and common non-linearity layers are activation functions (AFs), such as Logistic Sigmoid, Tanh, ReLU, ELU, Swish and Mish. In this paper, a comprehensive overview and survey is presented for AFs in neural networks for deep learning. Different classes of AFs such as Logistic Sigmoid and Tanh based, ReLU based, ELU based, and Learning based are covered. Several characteristics of AFs such as output range, monotonicity, and smoothness are also pointed out. A performance comparison is also performed among 18 state-of-the-art AFs with different networks on different types of data. The insights of AFs are presented to benefit the researchers for doing further research and practitioners to select among different choices. The code used for experimental comparison is released at: https://github.com/shivram1987/ActivationFunctions. [ABSTRACT FROM AUTHOR]

Abstract In this paper, neural networks (NNs)-based adaptive dynamic surface control is investigated for a class of pure-feedback nonlinear systems with unknown time delay and input hysteresis nonlinearities. The design difficulty appeared in this paper due to time delay is handled by combining neural networks with finite covering lemma instead of using Krasovskii functionals, and the assumptions on the time-delay functions are removed based the finite covering lemma and NNs. Meanwhile, completely unknown backlash-like hysteresis control input that frequently exists in practice is also considered. By combining the adaptive backstepping recursive design technique with the universal approximation ability of neural networks, an adaptive neural control algorithm is systemically designed for the system under consideration, and the explosion of complexity exists in traditional backstepping design is avoided by using dynamic surface technique. It is shown that the designed adaptive controller can guarantee that all the signals are ultimately uniformly bounded and the desired signal can be tracked with a small domain of the origin. A numerical example and an example of a real plant for adjustable metal cutting system are provided to show the feasibility of the proposed method. [ABSTRACT FROM AUTHOR]

Neural Networks and Support Vector Machines (SVMs) are two of the most popular and efficient supervised classification models. However, in the context of large datasets many complexity issues arise due to high memory requirements and high computational cost. In the context of the application of Data Mining algorithms, data reduction techniques attempt to reduce the size of training datasets in terms of the number of instances by selecting some of the existing instances or by generating new training instances. The idea is to speed up the application of the data mining algorithm with minimum or no sacrifice in performance. Data reduction techniques have been extensively used in the context of k -Nearest Neighbor classification, a lazy classifier that works by directly using a training dataset rather than building a model. This paper explores the application of data reduction techniques as a preprocessing step before the training step of Neural Networks and SVMs. Furthermore, the paper proposes a new data reduction technique that is based on k-median clustering algorithm. Our experimental results illustrate that, in the case of SVMs, data reduction techniques can effectively reduce the dataset size incurring small performance degradation. In the case of Neural Networks, the performance loss is somewhat greater, for the same data reduction rate, but both SVM and Neural Network models outperform the k -NN approach that is typically used in Data Mining applications. [ABSTRACT FROM AUTHOR]

This paper proposes the design and implementation of a model-free tire slip control for a fast and highly nonlinear Anti-lock Braking System (ABS). A reinforcement Q-learning optimal control approach is inserted in a batch neural fitted scheme using two neural networks to approximate the value function and the controller, respectively. The transition samples required for learning high performance control can be collected by interacting with the process either by online exploiting the current iteration controller (or policy) under an ε-greedy exploration strategy, or by using data collected under any other controller that is capable of ensuring efficient exploration of the action-state space. Both approaches are highlighted in the paper. Fortunately, the ABS process fits this type of learning-by-interaction because it does not need an initial stabilizing controller. The validation case studies conducted on a real laboratory setup reveal that high control system performance can be achieved using the proposed approaches. Insightful comments on the observed control behavior are offered along with performance comparisons with several types of model-based and model-free controllers including relay, model-based optimal PI, an original model-free neural network state-feedback VRFT controller and a model-free neural network adaptive actor-critic one. With the ability to improve control performance starting from different supervisory controllers or to learn high performance controllers from scratch, the proposed Q-learning optimal control approach proves its performance in a wide operating range and is therefore recommended to its industrial application on ABS. [ABSTRACT FROM AUTHOR]

This paper investigates the L 2 – L ∞ state estimation problem for a class of delayed neural networks. Attention is focused on the design of a full-order state estimator such that the prescribed L 2 – L ∞ performance constraint can be ensured. By utilizing the time-delay information sufficiently, a novel L 2 – L ∞ performance analysis approach is proposed in this paper for the first time. Based on such an approach, the less conservative sufficient conditions are established in terms of linear matrix inequalities under which the L 2 – L ∞ performance level can be achieved for the estimation error dynamics. Several numerical examples show that the proposed approach in this paper is explicitly effective in reducing the possible conservatism. [ABSTRACT FROM AUTHOR]

This paper focuses on an adaptive learning and control problem for a class of discrete-time nonlinear uncertain systems operating under multiple environments. A novel intelligent learning control framework is proposed by using a combination of offline and online learning methods. Specifically, in the offline learning mode, a deterministic learning (DL) based adaptive dynamics learning approach is first proposed to achieve locally-accurate identification of associated nonlinear uncertain system dynamics under each anticipated individual environment, and the learned knowledge is obtained and stored in a set of constant radial basis function neural network models. Then, with the learned knowledge, an online adaptive learning control scheme is further developed, which consists of: (i) an online adaptive learning control mechanism composed of multiple experience-based controllers and a DL-based adaptive learning controller, aiming to provide desired control performance for the plant operating under each individual environment; and (ii) a learning-based recognition mechanism composed of multiple recognition estimators and a DL-based identifier, aiming to recognize the active environment and schedule appropriate control strategies in real time. To guarantee the system stability during environment transition, a robust quasi-sliding-mode controller is further developed and embedded in the overall controller architecture. With this new intelligent adaptive learning control framework, the overall system is capable of adapting not only to any anticipated (pre-defined) environment by re-utilizing the knowledge obtained from both offline and online learning, but also to unanticipated (new) environments by actively acquiring new knowledge online. Simulation studies are conducted to verify the effectiveness and advantages of this new framework. [ABSTRACT FROM AUTHOR]

This paper investigates the adaptive neural network optimal output feedback control design problem for nonlinear continuous-time systems with actuator saturation. The system dynamics and states of the controlled system are unknown. A neural network state observer is constructed to estimate the system states. This paper uses two neural networks, one is used to construct the neural network state observer, the other (critic neural network) is used to approximate the cost functions, which comprise an observer-critic architecture. In this architecture, the critic neural network weights are tuned based on both the current data and the previous data, thus the conditions of the persistent excitation in the previous literatures are relaxed. By utilizing adaptive dynamic programming approach, a new observer-based optimal control scheme is developed. It is proved that the proposed adaptive neural network output feedback optimal control scheme can ensure that the whole closed-loop system is stable. Moreover, the estimate errors of the critic neural network weights are asymptotically stable. A simulation example is given to validate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

This paper presents an artificial neural network to solve the quadratic zero-one programming problems under linear constraints. In this paper, by using the connection between integer and nonlinear programming, the quadratic zero-one programming problem is transformed into the quadratic programming problem with nonlinear constraints. Then, by using the nonlinear complementarity problem (NCP) function and penalty method this problem is transformed into an unconstrained optimization problem. It is shown that the Hessian matrix of the associated function in the unconstrained optimization problem is positive definite in the optimal point. To solve the unconstrained optimization problem an artificial neural network is used. The proposed neural network has a simple structure and a low complexity of implementation. It is shown here that the proposed artificial neural network is stable in the sense of Lyapunov. Finally, some numerical examples are given to show that the proposed model finds the optimal solution of this problem in the low convergence time. [ABSTRACT FROM AUTHOR]

In this paper, the exponential stability for a class of neural networks with time-varying delay is concerned. An improved integral inequality is derived which extends the auxiliary function-based integral inequality. A novel Lyapounov-Krasovskii functional (LKF) with some new integral terms is constructed. Based on the improved integral inequality and reciprocally convex combination approach, a less conservative exponential stability criterion for the neural networks with time-varying delay is obtained. The effectiveness of the proposed method in this paper is illustrated via numerical examples. [ABSTRACT FROM AUTHOR]

This paper presents an approach of a social emotional model, which allows to extract the social emotion of a group of intelligent entities. The emotional model PAD allows to represent the emotion of an intelligent entity in 3-D space, allowing the representation of different emotional states. The social emotional model presented in this paper uses individual emotions of each one of the entities, which are represented in the emotional space PAD . Using a social emotional model within intelligent entities allows the creation of more real simulations, in which emotional states can influence decision-making. The result of this social emotional mode is represented by a series of examples, which are intended to represent a number of situations in which the emotions of each individual modify the emotion of the group. Moreover, the paper introduces an example which employs the proposed model in order to learn and predict future actions trying to influence in the social emotion of a group of people. [ABSTRACT FROM AUTHOR]

This paper is concerned with the passivity problem for neural networks with leakage delay and time-varying transmission delay. Using some novel Lyapunov-Krasovskii (L-K) functionals, and combining with the Wirtinger integral inequality, some less conservative delay-dependent passivity conditions are proposed in terms of linear matrix inequalities (LMIs). Different from the existing approaches, the model transformation is abandoned in this paper, and the proposed L-K functionals incorporate the interconnected terms related to both the leakage delay and the time-varying delay. Numerical examples show that the obtained conditions are less conservative. [ABSTRACT FROM AUTHOR]

This paper presents a comprehensive overview of modelling, simulation and implementation of neural networks, taking into account that two aims have emerged in this area: the improvement of our understanding of the behaviour of the nervous system and the need to find inspiration from it to build systems with the advantages provided by nature to perform certain relevant tasks. The development and evolution of different topics related to neural networks is described (simulators, implementations, and real-world applications) showing that the field has acquired maturity and consolidation, proven by its competitiveness in solving real-world problems. The paper also shows how, over time, artificial neural networks have contributed to fundamental concepts at the birth and development of other disciplines such as Computational Neuroscience, Neuro-engineering, Computational Intelligence and Machine Learning. A better understanding of the human brain is considered one of the challenges of this century, and to achieve it, as this paper goes on to describe, several important national and multinational projects and initiatives are marking the way to follow in neural-network research. [ABSTRACT FROM AUTHOR]

Neural learning plays an important role in many applications. In this paper, we derive a new learning paradigm for neural networks. Most existing neural models train network parameters including connecting weights and biases via optimizing a loss or energy function. Inspired from the associative learning in brain, we propose to associate different patterns via modeling joint distribution of them based on a hierarchical architecture. We first define an energy function based on the distance between hierarchical features of different patterns. Then a Gibbs typological distribution is constructed according to the energy field. To optimize the model, it is necessary to estimate the gradient expectation via sampling. Different from the simple architecture of existing probabilistic neural models such as restricted Boltzmann machine, the difficulty of optimizing this model lies in uncomputable probability for sampling from the distribution. Then we propose an optimization based sampling method. After learning, conditional probability can be derived and the unknown pattern can be generated via sampling as well. Compared with existing neural learning models, the proposed deep associative learning can associate different patterns and can be directly applied to many learning problems. Experiments on problems of classification, image transformation, and image change detection verify the effectiveness of the proposed learning paradigm. [ABSTRACT FROM AUTHOR]

This paper proposes a computationally efficient and effective data-driven modeling framework for dynamical systems. The proposed modeling framework employs a collection of shallow neural networks known as Extreme Learning Machines (ELMs) to model local system behaviors along with data-driven inferred transitions among local models to establish a neural hybrid automaton model. First, the sampled system inputs are mapped to the corresponding feature spaces to obtain data-driven partitions, which subsequently define the transitions and invariants of the neural hybrid automaton model through a novel data-driven mode clustering process. Then, a collection of ELMs are trained to approximate the local dynamics. The learning processes integrate a segmented data merging procedure for location identification and a local dynamics modeling process. The proposed neural hybrid automaton models can capture behaviors of complex dynamical systems with high modeling precision but significantly lower computational complexities in computationally expensive tasks such as training and verification, which are traditionally considered to be computationally expensive tasks for neural network models. A computationally efficient set-valued reachability analysis method which is commonly used in safety verification is then developed based on interval analysis and a novel Split and Combine process. Finally, applications to modeling the limit cycle and human handwritten motions are presented to show the effectiveness and efficiency of our approach. [ABSTRACT FROM AUTHOR]

This paper is concerned with the problem of passivity analysis of neural networks with an interval time-varying delay. Unlike existing results in the literature, the time-delay considered in this paper is subjected to interval time-varying without any restriction on the rate of change. Based on novel refined Jensen inequalities and by constructing an improved Lyapunov–Krasovskii functional (LKF), which fully utilizes information of the neuron activation functions, new delay-dependent conditions that ensure the passivity of the network are derived in terms of tractable linear matrix inequalities (LMIs) which can be effectively solved by various computational tools. The effectiveness and improvement over existing results of the proposed method in this paper are illustrated through numerical examples. [ABSTRACT FROM AUTHOR]

The problem of synchronization via sampled-data and saturating controller is considered for stochastic time-delay neural networks with Lévy noise and Markovian switching parameters in this paper. By using of the generalized Itô׳s formula and the Lyapunov functional method, an LMI-based sufficient condition is established to ensure the mean square synchronization of the master system and the slave system. Meanwhile, the gain of the sample data and saturating controller is determined. The sufficient condition depends on not only the switching mode and time-delay, but also the upper and the lower bound of sampling intervals. A numerical example is provided to verify the usefulness of the criterion proposed in this paper. [ABSTRACT FROM AUTHOR]

In this paper, the problem of passivity analysis for memristive neural networks with interval multiple time-varying delays is studied. More precisely, the multiple time-varying delays include not only the time-varying delay in the discrete term but also the time-varying delay in the leakage term. Moreover, this paper provides an improved passivity criteria for neural networks with the above two delays varying in their respective intervals under the joint action of differential inclusions, set-valued maps and Lyapunov theory. By constructing a novel Lyapunov–Krasovskii functional together with triple integral terms and employing first-order reciprocally convex method, second-order reciprocally convex method, free-weighting matrices technique and zero equalities, the improved passivity criteria are derived to guarantee that the input and output of the considered memristive neural networks satisfy a prescribed passivity-inequality constraint. Also it is assumed that the lower bounds of the activation functions can be positive, negative or zero. Meanwhile, it is worth pointing out that all of these criteria can be reduced to be applied not only to the memristive neural networks with only interval time-varying delay in the discrete term but also to the memristive neural networks with both multiple time-varying delays not containing interval terms. Further, the obtained conditions are formulated in terms of linear matrix inequalities which can be easily solved by using some standard numerical packages. Finally, two numerical examples are given to show the effectiveness and less conservatism of the proposed criteria. [ABSTRACT FROM AUTHOR]

This paper concerns with the passivity and mean square exponential stability problems for discrete-time stochastic fuzzy neural network with parameter uncertainties based on an interval type-2 (IT2) fuzzy model. For the nonlinear stochastic fuzzy neural network, novel sufficient conditions are presented by linear matrix inequalities (LMIs) to guarantee the passivity and mean square exponential stability of the resulting system, and the parameter uncertainties are handled via the IT2 fuzzy model approach. The main contribution of this paper is that we first propose the IT2 T–S discrete-time stochastic fuzzy neural network. Finally, a numerical example is provided to testify the effectiveness of the proposed scheme. [ABSTRACT FROM AUTHOR]

Neural network learning is usually time-consuming since backpropagation needs to compute full gradients and backpropagate them across multiple layers. Despite its success of existing works in accelerating propagation through sparseness, the relevant theoretical characteristics remain under-researched and empirical studies found that they suffer from the loss of information contained in unpropagated gradients. To tackle these problems, this paper presents a unified sparse backpropagation framework and provides a detailed analysis of its theoretical characteristics. Analysis reveals that when applied to a multilayer perceptron, our framework essentially performs gradient descent using an estimated gradient similar enough to the true gradient, resulting in convergence in probability under certain conditions. Furthermore, a simple yet effective algorithm named m emorized s parse b ack p ropagation (MSBP) is proposed to remedy the problem of information loss by storing unpropagated gradients in memory for learning in the next steps. Experimental results demonstrate that the proposed MSBP is effective to alleviate the information loss in traditional sparse backpropagation while achieving comparable acceleration. [ABSTRACT FROM AUTHOR]

Neural networks are the main strength of modern artificial intelligence; they have demonstrated revolutionary performance in a wide range of applications. In practice, the weights of neural networks are generally obtained indirectly using iterative training methods. Such methods are inefficient and problematic in many respects. Besides, neural networks trained end-to-end by such methods are typical black box models that are hard to interpret. Thus, it would be significantly better if the weights of a neural network could be calculated directly. In this paper, we located the key for calculating the weights of a neural network directly: assigning proper targets to the hidden units. Furthermore, if such targets are assigned, the neural network becomes a white box model that is easy to interpret. Thus, we propose a framework for solving the weights of a neural network and provide a sample implementation of the framework. The implementation was tested in various classification and regression experiments. Compared with neural networks trained using traditional methods, the constructed ones using solved weights had similar or better performance on many tasks, while remaining interpretable. Given the early stage of the proposed approach, many improvements are expectable in future developments. [ABSTRACT FROM AUTHOR]

This paper presents a novel constrained integration (CINT) method for solving initial boundary value partial differential equations (PDEs). The CINT method combines classical Galerkin methods with a constrained backpropogation training approach to obtain an artificial neural network representation of the PDE solution that approximately satisfies the boundary conditions at every integration step. The advantage of CINT over existing methods is that it is readily applicable to solving PDEs on irregular domains, and requires no special modification for domains with complex geometries. Furthermore, the CINT method provides a semi-analytical solution that is infinitely differentiable. In this paper the CINT method is demonstrated on two hyperbolic and one parabolic initial boundary value problems with a known analytical solutions that can be used for performance comparison. The numerical results show that, when compared to the most efficient finite element methods, the CINT method achieves significant improvements both in terms of computational time and accuracy. [ABSTRACT FROM AUTHOR]

A vision-based anticipatory controller for the autonomous indoor navigation of an unmanned aerial vehicle (UAV) is the topic of this paper. A dual Feedforward/Feedback architecture has been used as the UAV׳s controller and the K-NN classifier using the gray level image histogram as discriminant variables has been applied for landmarks recognition. After a brief description of the UAV, we first identify the two main components of its autonomous navigation, namely, the landmark recognition and the dual controller based on cerebellar system of living beings, then we focus on the anticipatory module that has been implemented by an artificial neural network. Afterwards, the paper describes the experimental setup and discusses the experimental results centered mainly on the basic UAV׳s behavior of landmark approximation maneuver, which in topological navigation is known as the beaconing or homing problem. [ABSTRACT FROM AUTHOR]

The purpose of this paper is to exhibit the internal process of the Growing Functional Modules (GFM) controllers. These learning based controllers are graphically designed by the user, interconnecting four kinds of components: Global Goals, Acting Modules, Sensing Modules and Sensations. Global Goals specify the intrinsic motivations, Acting and Sensing Modules develop their respective functionalities while Sensations provide the controlled system׳s feedback. GFM Controllers engage in a learning process to satisfy the specified motivations while interacting with the environment and thus can be considered as artificial brains. The traditional systems programming task is replaced by a design phase that consists in graphically configuring and interconnecting the GFM components. This paper gives a description of the aforementioned components and of their specific roles in the learning based control process. The behavior of the controller and its components is exemplified with the characterization of a simple control application. [ABSTRACT FROM AUTHOR]

Artificial Neural Networks have been found to be very efficient universal approximators. Single Layer Feedforward Networks (SLFN) are the most popular and easy to train. The neurons in these networks can use both sigmoidal functions and radial basis functions (RBF) as activation functions. Both functions have been shown to work very efficiently. Sigmoidal networks are already very well described in the literature. This paper will focus on the construction of a SLFN architecture using RBF neurons. There are many algorithms that are used to construct or train networks to solve function approximation problems. In this paper, an algorithm which is a modification of the Incremental Extreme Learning Machine (I-ELM) family of algorithms is proposed. The proposed algorithm eliminates randomness in the learning process with respect to center positions and widths of the RBF neurons. To do this, the input with the highest error magnitude is saved during error calculation and then used as the center for the next incrementally added neuron. Then the radius of the new neuron is iteratively chosen using Nelder–Mead׳s Simplex method. This allows the universal approximation properties of I-ELM to be preserved while greatly reducing the sizes of the trained RBF networks. [ABSTRACT FROM AUTHOR]

This paper focuses on a new kind of artificial neural networks – the Laplace transform artificial neural networks (LTANN). It is proposed to use the Laplace transform instead of ordinary weights and a linear activation function of an artificial neuron. This extension allows to use artificial neural networks in new areas. The ordinary description of artificial neural networks is a special case of the description proposed in this paper. Three models of different geological structures based on the LTANN are discussed in this paper. Using these models, it is possible to classify an unknown geological structure as the structure without anomaly, the structure with a sinkhole or the structure with a loose zone. [ABSTRACT FROM AUTHOR]

This paper describes the algorithms used to estimate the state of consumption of a pump based on dynamic pressure or vibrations. To create algorithms, the author used computational intelligence methods in the form of neural networks. In order to perform the analysis, data analysis systems were designed based on three neural networks: multilayer perceptron neural network (MLP), generalized regression neural network (GRNN) and probabilistic neural network (PNN). Processing of the input signal in the final result of the analysis consisted of several steps. First, the measurement data were preprocessed (delete constant component, normalization, standardization, reduction, fast Fourier transform (FFT), etc.), and training and test sets were prepared using the matrices with the expected system answers. The last step was the analysis, consisting of design data analysis systems based on artificial neural networks and their learning and testing. On the basis of the obtained results the effectiveness of neural networks and the methods of pre-processing of the signals applied to approximate the state of consumption of the displacement pump were evaluated. Design systems were evaluated based on accuracy (generated error) and complexity (number of parameters and training time) criteria. The main contribution of the paper is to design and compare methods for pre-processing the signal, and to design and compare the effectiveness of the three neural networks in the diagnosis consumption of a positive displacement pump. [ABSTRACT FROM AUTHOR]

Finite-time synchronization means the optimality in convergence time, thus many contributions have been made to it in the literature. However, to the best of our knowledge, most of the existing results on finite-time synchronization do not include time-delay. Considering the fact that time-delays especially infinite-time distributed delays are inevitably existing in neural networks, this paper aims to study global synchronization in finite time of neural networks with both time-varying discrete delay and infinite-time distributed delay (mixed delays). The techniques that we apply in this paper are not only different from the techniques employed in existing papers, but also applicable to differential systems with or without delay. Based on new Lyapunov-Krasovskii functional candidate and the new analysis techniques, sufficient conditions guaranteeing the finite-time synchronization of the addressed neural networks are derived by using a class of simple discontinuous state feedback controller. Conditions for realizing finite-time synchronization of neural networks with finite-time distributed delay and without delay are also given. Moreover, estimation of the upper bound of synchronization-time is also provided for neural networks with finite-time distributed delay and without delay. It is shown that the synchronization-time depends on both the initial values and the time-delays of the drive-response systems. Numerical examples demonstrate the effectiveness of the theoretical results. [ABSTRACT FROM AUTHOR]

This paper revisits the problem of passivity analysis for neural networks with time-varying delays. A new delay-dependent criterion is obtained in terms of linear matrix inequalities, guaranteeing that the input and output of the considered neural network satisfy a prescribed passivity-inequality constraint. This newly presented criterion does not require all the symmetric matrices involved in the employed quadratic Lyapunov-Krasovskii functional to be positive definite. This feature is remarkable since it sheds new light on the traditional ideas for constructing Lyapunov-Krasovskii functionals. More importantly, the conservatism of delay-dependent passivity conditions can be reduced due to the relaxation on the positive-definiteness of every Lyapunov matrix. It is shown both theoretically and numerically that the passivity criterion proposed in this paper is truly less conservative than some of the latest results in the literature. [ABSTRACT FROM AUTHOR]

In this paper existence and asymptotic stability of asymptotically almost automorphic solution of impulsive neural networks with delay is discussed. The results are established by using various fixed point theorems and Lyapunov-like functional. As far as we know, this is the first paper to discuss such kind of solutions for impulsive neural networks. At the end, we give few examples to illustrate our theoretical findings. One can see that the numerical simulation results show asymptotically almost automorphic behaviour of the solution. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, a finite-time lag synchronization of coupled neural networks with time delay is investigated. By means of the Lyapunov stability theory, a feedback controller is designed for achieving lag synchronization between two delayed neural networks systems in finite time. Paper extends some basic results from the area of finite time to time-delay systems. Numerical simulations on coupled Lu neural systems illustrate the effectiveness of the results. [Copyright &y& Elsevier]

This paper addresses the problem of composite adaptive learning and tracking control for discrete-time nonlinear uncertain systems in general normal form. To deal with the system's unstructured nonlinear uncertain dynamics, novel adaptive neural network (NN) learning control strategies are proposed by extending methodologies from the continuous-time deterministic learning theory. Both state-feedback and output-feedback cases are considered. The proposed control schemes are compelling in the sense that (i) they are capable of rendering not only stable tracking control, but also locally-accurate learning/identification of unknown system dynamics during stable closed-loop control; (ii) the learned knowledge can be effectively represented and stored as constant NN models, whose weights are guaranteed to partially converge to ideal/optimal values. Based on this, experience-based controllers are also developed to pursue improved tracking control performance without online adaptation. In particular for the output-feedback control case, a new observer-less adaptive NN learning control scheme is proposed without resorting to high-gain observers, followed by an observer-less output-feedback experience-based controller. Numerical simulations have been conducted to demonstrate the effectiveness of the proposed approaches. [ABSTRACT FROM AUTHOR]

Classifying visual information is an apparently simple and effortless task in our everyday routine, but can we automatically predict what we see from signals emitted by the brain? While other researchers have already attempted to answer this question, we are the first to show that a commercially available BCI could be effectively used for visual image classification in real-world scenarios – when testing takes place at a completely different time than training data collection. The task is difficult, as it requires relating the noisy and low-level EEG signals to complex and highly semantic visual categories. In this paper, we propose different learning approaches and show that simpler classifiers such as Ridge Regression with Gabor filtering of the input EEG signal could be more effective than the powerful Long Short Term Memory Networks and Convolutional Neural Networks in this case of limited and noisy training data. We analyzed the importance of each electrode for the visual classification task and noticed that the sensors with the highest accuracy were the ones that recorded brain activity from regions known to be correlated more with higher level recognition and cognitive processes and less to lower-level visual signal processing. The result is also in accordance with research in computer vision with deep neural networks, which shows that semantic visual features are learned only at higher levels of neural depth. While EEG signals are weaker by themselves for the task of visual classification, we demonstrate that they could be powerful when combined with deep visual features extracted from the image, improving performance from 91% to over 97% in a multi-class recognition setting. Our tests show that EEG input brings additional information that is not learned by artificial deep networks on the given image training set. Thus, a commercially available BCI could be effectively used in conjunction with a deep learning based vision system to form together a stronger visual recognition system that is suitable for real-world applications. [ABSTRACT FROM AUTHOR]

Activity-dependent plasticity plays an important role in biological neural network learning. Unlike the backpropagation-based learning in artificial neural networks that depends on supervised signals, biological neurons adjust themselves using their own historical behaviors as a clue to benefit information processing. Inspired by this biological neural mechanism, this paper proposes a novel neuron model, named Activity-Dependent Neuron (ADN). The basic idea of ADN is to add a gate in the neuron model to control its signal conduction capability, that is, the gate will facilitate transmission of important signals while suppress trivial signals. This idea can be achieved by self-tuning of activation behaviors. We also develop an iterative training algorithm, so that the ADNs can be smoothly incorporated into deep neural networks to jointly learn with the network weights. Experimental results found that the ADNs can efficiently improve the noise-resistant capability. Compared with the state-of-the-art, it is more robust to unforeseen noises. It does not need noise data for training. [ABSTRACT FROM AUTHOR]

Abstract: This paper investigates the problem of exponential stability for a class of discrete-time stochastic neural networks with randomly time-varying delays. In the concerned model, stochastic disturbance is described by a Brownian motion, and the time-varying delays satisfy , moreover, the effects of both variation range and probability distribution of time-varying delays are taken into consideration in the proposed approach. By constructing a new Lyapunov–Krasovskii functional, combining with the stochastic stability theory and the free-weighting matrix method, a delay-distribution-dependent exponential stability criterion is obtained in terms of LMIs. Compared with some previous results, the new conditions obtained in this paper are less conservative. Finally, two numerical examples are provided to show the effectiveness of the proposed theoretical results. [Copyright &y& Elsevier]

Abstract: This paper investigates the problem of the existence, uniqueness and global asymptotic stability of the equilibrium point for the class of neural networks with multiple time delays and parameter uncertainties. Under the assumption that the activation functions are globally Lipschitz continuous, we derive a new criterion for the robust stability of a class of delayed neural networks by utilizing the Lyapunov stability theorems and the Homomorphic mapping theorem. Different from those previously published conditions in the recent literature, the robust stability result presented in this paper not only establishes a time-independent relationship between the network parameters of the neural network, but also takes into account the number the neurons of the designed neural system. Some illustrative numerical examples are also given to make a detailed comparison between our result and the previously published corresponding results. This comparison proves that our result is new and can be considered an alternative condition to those of the previously reported robust stability results. [Copyright &y& Elsevier]

Inspired by the prescribed performance control (PPC), a new performance function, called finite-time performance function (FTPF), is first defined in this paper, by which a novel finite-time control design process is introduced. This work is also the first to solve the finite-time control issue for a class of transmission control protocol/active queue management (TCP/AQM) networks. Meanwhile, with the aid of FTPF, PPC and neural networks, a new adaptive practically pre-assigned finite-time controller is designed, which guarantees that the queue length q (t) tracks the desired queue q ref in finite-time and all the signals of the closed-loop system are semi-globally practical finite-time stable (SGPFS). Finally, simulation results are shown to illustrate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

This paper studies the controller design for an autonomous underwater vehicle (AUV) with the target tracking task. Considering the uncertainty the nonlinear longitudinal model, a sliding mode controller is designed. Meanwhile the neural networks (NNs) are used to approximate the unknown nonlinear function in the model. To improve the NNs learning rapidity, the prediction error which reflect the learning performance is constructed, further the updating law is designed utilizing the composite learning technique. The system stability is guaranteed through the Lyapunov approach. The simulation results verify that the designed method could force the AUV to track the target until rendezvous, and the model uncertainty is addressed better via the composite learning algorithm. [ABSTRACT FROM AUTHOR]

This paper investigates the impulsive effects on the finite time boundedness (FTB) and the FTB-stabilization for a high-order neutral-type neural networks (HONNNs) with leakage delay and mixed time-varying delay including both discrete and infinite distributed delays. The average impulsive interval technique and Lyapunov–Krasovskii functional (LKF) method are employed to derive sufficient conditions for the effect of the time delay in the leakage term on FTB in presence of destabilizing impulses. These conditions are in the form of linear matrix inequalities (LMIs) which can be numerically checked. Moreover, a state delay-free feedback control is designed to ensure the FTB-stabilization for impulsive HONNNs. Our theoretical results are further substantiated by numerical examples with graphical illustrations to confirm them and show their effectiveness. [ABSTRACT FROM AUTHOR]

State estimation and synchronization of neural networks (NNs) have recently received ever-increasing research interests due to a large number of successful applications in various fields such as repetitive learning, classification of patterns, nonlinear control, adaptive control, image processing, and so forth. Owing to limited communication bandwidth, the network-induced phenomena and the adopted communication scheduling may cause inevitable negative effects or the degradation of dynamical performance of NNs. This paper, from the perspective of dynamical behavior, renders a comprehensive summary on the recent advances of communication-embedded NNs and their application in nonlinear control. First, some common network-induced phenomena and communication protocols are roughly introduced from both models and mechanisms. Then, the state estimation and synchronization analysis of various NNs with network-induced phenomena are systematically reviewed and some interesting results are separately presented. Furthermore, the latest advances for the cases with communication protocols are profoundly surveyed from three aspects: state estimation, synchronization analysis and applications in control realm. Finally, some challenged problems are listed and several potential future research directions are highlighted. [ABSTRACT FROM AUTHOR]

Most users typically interact with products only through implicit feedback such as clicks or purchases rather than explicit user-provided information like product ratings. Learning to rank products according to individual preferences using only implicit feedback can be helpful to make useful recommendations. In this paper, a neural network architecture to solve collaborative filtering problems for personalized rankings on implicit feedback datasets is presented. It is shown how a layer of constant weights forces the network to learn pairwise item rankings. Additionally, similarities between the proposed neural network and a matrix factorization model trained with the Bayesian Personalized Ranking optimization criterion are proven. The experiments indicate state-of-the-art performance for the task of personalized ranking. [ABSTRACT FROM AUTHOR]

Abstract In this paper, an online integral reinforcement learning strategy is proposed to deal with robust constrained control problems using event-triggered mechanism for nonlinear Continuous-Time (C-T) systems with external disturbances. The novel design of constrained control law is addressed together with the adaptive event-triggered condition by guaranteeing the optimal performance and system stability. An adaptive online actor-critic Neural Network (NN) reinforcement learning scheme is developed to approximate the optimal solution of the complicated Hamilton–Jacobi–Isaacs equation. Meanwhile, the convergence of NN weight errors and the event-triggered closed-loop system stability are demonstrated to be uniform ultimate bounded by Lyapunov analysis under the proposed triggering condition. Moreover, event-triggered H ∞ tracking control with input constrains and limited network communication is also presented by establishing an augmented system. Finally, simulation results are provided to show the algorithm validity. [ABSTRACT FROM AUTHOR]

Abstract This paper deals with the global exponential stability problem of neural networks with time-varying delay. A novel Lyapounov–Krasovskii functional (LKF), which contains a common double integral term, an augmented double integral term and two delay-product-type terms, is constructed to analyze the exponential stability. An auxiliary function-based integral inequality (AFBII) and two special forms of it are applied to estimate the upper bounds of single integral terms produced in the time derivation of the LKF candidate. By using the novel LKF and AFBII, some new cross terms of matrix variables are included in linear matrix inequalities (LMIs). As a result, a less conservative delay-dependent global exponential stability criterion is proposed. Finally, two numerical examples are provided to demonstrate the effectiveness of the proposed criterion. [ABSTRACT FROM AUTHOR]

Abstract Functional networks (FNs) are extensions of neural networks (NNs). Unlike NNs, FNs considers general functional models instead of sigmoid-like models. Additionally, in FNs, there are no weights associated with the links that connect neurons. In this paper, we review the research progress and applications of FNs models in recent years. First, we introduce FNs architecture, three typical functional models and the learning process, and we explain the differences between NNs and FNs. Second, we discuss recent applications of FNs that have been introduced in many fields, such as time series prediction, differential and functional equations, pattern classification, detection and prediction, approximation computation, complex system modeling, computer-aided design (CAD), and linear and nonlinear regression. Finally, we present some remarks on future research directions for FNs. [ABSTRACT FROM AUTHOR]

Abstract This paper examines the problem of asymptotic stability of continuous neural networks with time-varying delay via a new Lyapunov–Krasovskii functional (LKF). First, a suitable quadratic functional is constructed, which coordinates with the use of the orthogonal-polynomials-based integral inequality. Second, the novel proposed LKF contains more state vectors of neural networks, so that more state information can be exploited adequately. By combining the new proposed LKF and orthogonal-polynomials-based integral inequality, novel delay-dependent stability criteria with less conservatism are established in the form of linear matrix inequalities (LMIs). Finally, two commonly-used numerical examples are provided to show the effectiveness and improvement of the proposed criteria. [ABSTRACT FROM AUTHOR]

Abstract This paper proposed an online image-based visual servoing (IBVS) controller for manipulator systems with dead-zone input. The adaptive neural networks (NNs) are used to approximate the unknown nonlinear dynamics. The Barrier Lyapunov Function (BLF) is constructed to overcome the visibility constraint problem, in which both the constant symmetric barriers and time-varying asymmetric barriers are considered. With the proposed control method, it is proved that all the signals in the closed-loop system are semi-globally uniformly bounded and the image error is remained in a bounded compact set. Finally, simulation examples are given to illustrate the effectiveness of the proposed control method. [ABSTRACT FROM AUTHOR]