Search

This paper mainly focuses on how to construct concept lattice effectively and efficiently based on improved variable precision rough set. On the basis of preprocessing formal concept, one algorithm that can determine the value range of variable precision parameter β according to the approximate classification quality is proposed. An improved β -upper and lower distribution attribute reduction algorithm is also proposed based on the improved variable precision rough set, the algorithm can be used for attribute reduction on the original data of the concept lattice, and to eliminate the redundant knowledge or noises of the formal context. For the reduced formal context, the paper combines the concept construction algorithm with an improved rule acquisition algorithm seamlessly, and proposes a novel approach of concept lattice construction based on improved variable precision rough set. Finally, a concept lattice generation prototype system is developed, this paper also performs comprehensive experiments, and the effectiveness of the improved algorithm is proved through the experimental results. [ABSTRACT FROM AUTHOR]

Deep learning algorithms are a subset of the machine learning algorithms, which aim at discovering multiple levels of distributed representations. Recently, numerous deep learning algorithms have been proposed to solve traditional artificial intelligence problems. This work aims to review the state-of-the-art in deep learning algorithms in computer vision by highlighting the contributions and challenges from over 210 recent research papers. It first gives an overview of various deep learning approaches and their recent developments, and then briefly describes their applications in diverse vision tasks, such as image classification, object detection, image retrieval, semantic segmentation and human pose estimation. Finally, the paper summarizes the future trends and challenges in designing and training deep neural networks. [ABSTRACT FROM AUTHOR]

A huge volume of high spatial resolution hyperspectral imagery (HSI) data sets can currently be acquired. However, making full use of the information within the HSI is still a huge problem. The exploitation of spatial information is playing a more and more important role in the classification of remote sensing data. How to efficiently extract the spatial feature for HSI has become a critical task. In this paper, we propose a dropout k -means based framework to extract an effective hierarchical spatial feature for HSI. This paper focuses on unsupervised hierarchical feature learning representation. The proposed framework was tested on two HSIs. The extensive experimental results clearly show that the proposed dropout k -means based framework achieves a superior classification performance. [ABSTRACT FROM AUTHOR]

Visual recognition of crowd dynamics has had a huge impact on several applications including surveillance, situation awareness, homeland security and intelligent environments. However, the state-of-the-art in crowd analysis has become diverse due to factors such as: (a) the underlying definition of a crowd, (b) the constituent elements of the crowd processing model, (c) its application, hence (d) the dataset and (e) the evaluation criteria available for benchmarking. Although such diversity is healthy, the techniques for crowd modelling thus developed have failed to establish credibility therefore becoming unreliable and of questionable validity across different research disciplines. This paper aims to give an account of such issues by deducing key statistical evidence from the existing literature and providing recommendations towards focusing on the general aspects of techniques rather than any specific algorithm. The advances and trends in crowd analysis are drawn in the context of crowd modelling studies published in leading journals and conferences over the past 7 years. Finally, this paper shall also provide a qualitative and quantitative comparison of some existing methods using various publicly available crowd datasets to substantiate some of the theoretical claims. [ABSTRACT FROM AUTHOR]

The purpose of this paper is to investigate the problem of passivity analysis for delayed uncertain stochastic neural networks (DUSNNs) with discrete and distributed time-varying delays. The novelty of this paper lies in the consideration of a new integral inequality proved to be less conservatism than celebrated Jensen׳s inequality and takes fully the relationship between the terms in the Leibniz–Newton formula within the framework of linear matrix inequalities (LMIs). By constructing a suitable Lyapunov–Krasovskii functional with triple integral terms and using Jensen׳s inequality, integral inequality technique and linear matrix inequality frame work, which guarantees stability for the passivity of addressed neural networks. This LMI can be easily solved via convex optimization techniques. Using several examples from the literature, it is shown that the proposed stabilization theorem is less conservative than previous results. Finally, the technique is applied to benchmark problem, showing how to derive efficient stability criteria for realistic problems, using the proposed technique. [ABSTRACT FROM AUTHOR]

Auto-encoder—a tricky three-layered neural network, known as auto-association before, constructs the “building block” of deep learning, which has been demonstrated to achieve good performance in various domains. In this paper, we try to investigate the dimensionality reduction ability of auto-encoder, and see if it has some kind of good property that might accumulate when being stacked and thus contribute to the success of deep learning. Based on the above idea, this paper starts from auto-encoder and focuses on its ability to reduce the dimensionality, trying to understand the difference between auto-encoder and state-of-the-art dimensionality reduction methods. Experiments are conducted both on the synthesized data for an intuitive understanding of the method, mainly on two and three-dimensional spaces for better visualization, and on some real datasets, including MNIST and Olivetti face datasets. The results show that auto-encoder can indeed learn something different from other methods. Besides, we preliminarily investigate the influence of the number of hidden layer nodes on the performance of auto-encoder and its possible relation with the intrinsic dimensionality of input data. [ABSTRACT FROM AUTHOR]

In this paper, we discuss the long-term time series forecasting using a Multilayer Neural Network with Multi-Valued Neurons (MLMVN). This is a complex-valued neural network with a derivative-free backpropagation learning algorithm. We evaluate the proposed approach using a real-world data set describing the dynamic behavior of an oilfield asset located in the coastal swamps of the Gulf of Mexico. We show that MLMVN can be efficiently applied to univariate and multivariate one-step- and multi-step ahead prediction of reservoir dynamics. This paper is not only intended for proposing to use a complex-valued neural network for forecasting, but to deeper study some important aspects of the application of ANN models to time series forecasting that could be of the particular interest for pattern recognition community. [ABSTRACT FROM AUTHOR]

In this paper, an adaptive predictive control algorithm is employed to controlling a class of continuous stirred tank reactor (CSTR) system. The main contribute of this paper is that the CSTR system are in discrete-time form and non-symmetric dead-zone inputs are considered here. The design parameters of control algorithm for the CSTR systems are not so much than before, such that the calculated amount of the control algorithm is less than before. By considering the Radial basis function neural networks (RBFNN), the unknown functions are approximated, the mean value theorem is utilized in the algorithm design process. Based on the Lyapunov analysis method, and choosing the design parameters appropriately, all the signals in the closed-loop system are proved to be semi-global uniformly ultimately bounded (SGUUB) and the tracking error is converged to a small compact set. A simulation example for CSTR systems is studied to demonstrate the effectiveness of the proposed approach. [ABSTRACT FROM AUTHOR]

The problem of synchronization via sampled-data and saturating controller is considered for stochastic time-delay neural networks with Lévy noise and Markovian switching parameters in this paper. By using of the generalized Itô׳s formula and the Lyapunov functional method, an LMI-based sufficient condition is established to ensure the mean square synchronization of the master system and the slave system. Meanwhile, the gain of the sample data and saturating controller is determined. The sufficient condition depends on not only the switching mode and time-delay, but also the upper and the lower bound of sampling intervals. A numerical example is provided to verify the usefulness of the criterion proposed in this paper. [ABSTRACT FROM AUTHOR]

We consider shunting inhibitory cellular neural networks with inputs and outputs that are chaotic in a modified Li-Yorke sense. The original Li-Yorke definition of chaos has been modified such that infinitely many periodic motions separated from the motions of the scrambled set are now replaced with almost periodic ones. Another principal novelty of the paper is that chaos is obtained as solutions of differential equations (neural networks) which are perturbed chaotically . To construct the original chaos, the special set of piecewise continuous postsynaptic currents is applied. It is shown that a control can be realized for the extended chaos in an effective way. This is the first time in the theory of neural networks that the Ott–Grebogi–Yorke control method is used to stabilize almost periodic motions. Our techniques can be performed to investigate chaotic dynamics in human brain activities, communication security, combinatorial optimization problems and control of legged robots. The results of this paper were announced in the 5th International Conference on Nonlinear Science and Complexity (August 4–9, 2014, Xi׳an, China) and in an International Conference on Nonlinear Dynamics and Complexity (May 11–15, 2015, La Manga, Spain). [ABSTRACT FROM AUTHOR]

This paper contributes to the understanding and implementation of different types of locomotion in a snake-like robot by using a proposed unified neural oscillator model as central pattern generator (CPG). A neural oscillator network is suggested so that several rhythmic movements can be generated in order to mimic the actual snake movements on land. The unified coupled neural oscillators in the network provide the desired orientation and displacement for the snake-like robot and they are applied to the different types of snake locomotion by just altering some of the parameters proposed in the paper. The design of snake-like robot, consisting of a series of wheeled links (links with active wheels) connected with each other using the passive prismatic and revolute joints, is also suggested in the paper. The passive prismatic joints facilitate the change of lengths of wheel-less links and in this way the robot can be elongated as well as shortened. In this paper the serpentine, rectilinear, and concertina types of actual snake locomotion are realized, through simulation studies, by using the unified neural oscillator model proposed for the snake-like robots. [ABSTRACT FROM AUTHOR]

A teaching-learning-based optimization algorithm (TLBO) which uses a variable population size in the form of a triangle form (VTTLBO) is proposed in the paper. The main goal of the proposed method is to decrease the computing cost of original TLBO and extend it for optimizing the parameters of artificial neural network (ANN). In the proposed algorithm, the evolutionary process is divided into some equal periods according to the maximal generation. The population size in each period is changed in form of triangle. In the linear increasing phase of population’s number, some new individuals are generated with gauss distribution by using the adaptive mean and variance of the population. In the linear decreasing phase of population׳s number, some highly similar individuals are deleted. To compare the performance of the proposed method with some other methods, Saw-tooth teaching-learning-based optimization algorithm is also designed with simulating the basic principle of Saw-tooth genetic algorithm (STGA), and some other EAs with the fixed population size are also simulated. A variety of benchmark problems and system modeling and prediction problems with ANN are tested in this paper, the results indicate that the computation cost of the given method is small and the convergence accuracy and speed of it are high. [ABSTRACT FROM AUTHOR]

This paper investigates the design of auxiliary error compensation for adaptive neural control of the longitudinal dynamics of a flexible air-breathing hypersonic vehicle (FAHV) with magnitude constraints on actuators. The control objective pursued is to steer velocity and altitude to follow their respective reference trajectories in the presence of actuator saturation and system uncertainties. To guarantee the exploited controller’s robustness with respect to parametric uncertainties, neural network (NN) is applied to approximate the lumped uncertainty of each subsystem of FAHV model. Different from the traditional parameter updating technique, in this paper, the minimal-learning-parameter (MLP) scheme is introduced to estimate the norm rather than the elements of NN’s weight vector while the computational load is reduced. The special contribution is that novel auxiliary systems are developed to compensate both the tracking errors and desired control laws, based on which the explored controller can still provide effective tracking of velocity and altitude commands when the actuators are saturated. Finally, numerical simulations are performed to illustrate the command tracking performance of the proposed strategy. [ABSTRACT FROM AUTHOR]

In this paper, we examine the problem of learning sparse representations of visual patterns in the context of artificial and biological vision systems. There are a myriad of strategies for sparse coding that often result in similar feature properties for the learned feature set. Typically this results in a bank of Gabor-like or edge filters that are sensitive to a range of distinct angular and radial frequencies. The theory and experimentation that is presented in this paper serves to provide a better understanding of a number of specific properties related to low-level feature learning. This includes close examination of the role of phase pairing in complex cells, the role of depth information and its relationship to variation of intensity and chroma, and deriving hybrid features that borrow from both analytic forms and statistical methods. Together, these specific examples provide context for more general discussion of effective strategies for feature learning. In particular, we make the case that imposing additional constraints on mechanisms for feature learning inspired by biological vision systems can be useful in guiding constrained optimization towards convergence, or specific desirable computational properties for representation of visual input in artificial vision systems. [ABSTRACT FROM AUTHOR]

In this paper, the problem of passivity analysis for memristive neural networks with interval multiple time-varying delays is studied. More precisely, the multiple time-varying delays include not only the time-varying delay in the discrete term but also the time-varying delay in the leakage term. Moreover, this paper provides an improved passivity criteria for neural networks with the above two delays varying in their respective intervals under the joint action of differential inclusions, set-valued maps and Lyapunov theory. By constructing a novel Lyapunov–Krasovskii functional together with triple integral terms and employing first-order reciprocally convex method, second-order reciprocally convex method, free-weighting matrices technique and zero equalities, the improved passivity criteria are derived to guarantee that the input and output of the considered memristive neural networks satisfy a prescribed passivity-inequality constraint. Also it is assumed that the lower bounds of the activation functions can be positive, negative or zero. Meanwhile, it is worth pointing out that all of these criteria can be reduced to be applied not only to the memristive neural networks with only interval time-varying delay in the discrete term but also to the memristive neural networks with both multiple time-varying delays not containing interval terms. Further, the obtained conditions are formulated in terms of linear matrix inequalities which can be easily solved by using some standard numerical packages. Finally, two numerical examples are given to show the effectiveness and less conservatism of the proposed criteria. [ABSTRACT FROM AUTHOR]

This paper addresses region aggregation issues of multi-agent networks (MANs) that consist of finite number of isolated sub-multi-agent systems (sub-MASs) with multiple equilibria (ME) in multi-dimensional coordinate systems (MDCSs). To investigate aggregation of this kind of MANs via switching strategies, switched systems with ME are used to describe such networks under the case that each agent has a unique equilibrium point and all the equilibria are different from each other. Based on the novel stability concepts of region stability ( RS ) and exponential region stability ( ERS ) introduced, this paper proposes several RS and ERS results for sub-MASs with ME via the maximum energy function method, and then presents several region aggregation results for such MANs under three suitable assumptions. Also, a numerical example illustrates the effectiveness and practicality of our new results. [ABSTRACT FROM AUTHOR]

This paper concerns with the passivity and mean square exponential stability problems for discrete-time stochastic fuzzy neural network with parameter uncertainties based on an interval type-2 (IT2) fuzzy model. For the nonlinear stochastic fuzzy neural network, novel sufficient conditions are presented by linear matrix inequalities (LMIs) to guarantee the passivity and mean square exponential stability of the resulting system, and the parameter uncertainties are handled via the IT2 fuzzy model approach. The main contribution of this paper is that we first propose the IT2 T–S discrete-time stochastic fuzzy neural network. Finally, a numerical example is provided to testify the effectiveness of the proposed scheme. [ABSTRACT FROM AUTHOR]

In this paper, we are concerned with the existence, uniqueness and global asymptotic stability of equilibrium point for a class of complex-valued Cohen–Grossberg neural networks with time delays. By using homeomorphism theory, new matrix inequality techniques and inequality techniques, several LMI-based sufficient conditions on the existence, uniqueness and global asymptotic stability of equilibrium point for above complex-valued Cohen–Grossberg neural networks with two classes of complex-valued activation functions, behaved functions and amplification functions are obtained. So far, only the stabilities of complex-valued recurrent neural networks, Hopfield neural networks and Cellular neural networks have been studied. Hence, it is for the first place that the stability of complex-valued Cohen–Grossberg neural networks is discussed in this paper. [ABSTRACT FROM AUTHOR]

This paper focuses on a new kind of artificial neural networks – the Z-transform artificial neural networks (ZTANNs). It is proposed to use the Z-transform instead of ordinary weights and a linear activation function of an artificial neuron. This extension allows to use artificial neural networks in new areas. The ordinary description of artificial neural networks is a special case of the description proposed in this paper. It also contains a description of the use of the ZTANN for automatic identification of objects in digital control system. [ABSTRACT FROM AUTHOR]

In this paper, an adaptive neural network tracking control is studied for a class of uncertain nonlinear systems. The studied systems are in discrete-time form and unknown time-delay is considered here. Up to now, the research works on nonlinear discrete-time main focus on systems without time-delay, so the problem of the unknown time-delay will be solved in this paper. Based on the Lipschitz or norm-boundedness assumption of the unknown nonlinearities, the mean-value theorem is utilized to solve the unknown time-delay problem. In order to overcome the noncausal problem, the strict-feedback systems will be transformed into a special form. The radial basis functions neural networks (RBFNN) are utilized to approximate the unknown functions of the systems, the adaptation laws and the controllers are designed based on the transformed systems. By using the Lyapunov analysis, it is proven that the closed-loop system is stable in the sense that semi-globally uniformly ultimately bounded (SGUUB) and the output tracking errors converge to a bounded compact set. A simulation example is used to illustrate the effectiveness of the proposed algorithm. [ABSTRACT FROM AUTHOR]

This paper firstly treats of the anti-synchronization of a class of memristor-based neural networks with mixed time delays and reaction–diffusion terms. An adaptive controller is designed such that the response system can be synchronized with the drive system by utilizing Lyapunov stability theory. Furthermore, the synchronization criteria derived in this paper are associated with the time switching. In addition, the H ∞ anti-synchronization of memristive neural networks has been studied as well. Therefore, the results obtained in this paper are more general and extend the previous work. Finally, two numerical examples and their simulations are given to show the effectiveness of the obtained results. [ABSTRACT FROM AUTHOR]

In the paper, an adaptive neural controller for the tracking problem of a direct-current (DC) motor is investigated. Because the unknown functions are included in the systems, the neural networks are used to estimate the unknown functions. In this study, the state variables of DC motor are required to be constrained in the compact set. The main contribution of this paper is that the proposed scheme is successfully to integrate barrier Lyapunov function to avoid the violation of the constraints. Based on Lyapunov analysis, it is proved that the output of the DC motor follows a desired trajectory and all the signals of the systems are guaranteed to be bounded. A simulation result is shown to confirm the effectiveness of the proposed scheme. [ABSTRACT FROM AUTHOR]

The ability to precisely predict behavior of a ship can be useful for different ship systems, for example, dynamic positioning, video or artillery systems. To automatically maintain a ship position and heading, the dynamic positioning system has to know future behavior of the ship as exactly as possible. The same applies to the ship video and artillery systems which to continuously track a target have to predict both the location of the target and orientation of the ship deck in relation to the target in successive points in time. In this paper, evolutionary neural networks are proposed as ship behavior predictors. To perform the task, they are supplied with the information about ship spatial orientation (Euler angles) acquired from inertial navigational systems. In experiments reported in the paper, both monolithic and modular recurrent neural networks were tested. To build them, a neuro-evolutionary method called Assembler Encoding with Evolvable Operations was applied. As the point of reference for the networks two other prediction methods were used: the first is Linear Regression with Correction, i.e. the most effective method in preliminary experiments, whereas the second is Autoregressive Integrated Moving Average, which seems to be currently the most general tool for forecasting a time series. [ABSTRACT FROM AUTHOR]

This paper deals with the robust H ∞ control problem for a class of uncertain neural networks with discrete interval and distributed time-varying delays. The main purpose of this paper is to estimate robust asymptotic stability of the given neural network with H ∞ performance analysis γ . By constructing novel Lyapunov–Krasovskii functionals with triple integral terms, several new less conservative delay-dependent stability conditions for H ∞ control are obtained in terms of linear matrix inequalities. Numerical examples are given to illustrate the effectiveness of the proposed theoretical results. The method given in this paper shows less conservative results when comparing with some existing methods. [ABSTRACT FROM AUTHOR]

This paper proposes using the Parallel Layer Perceptron (PLP) network, instead of the Single Layer Feedforward neural network (SLFN) in the Extreme Learning Machine (ELM) framework. Differently from the SLFNs which consider cascade layers, the PLP is designed to accomplish also parallel layers, being the SLFN its particular case. This paper explores a particular PLP configuration which considers a nonlinear layer in parallel with a linear layer. For n inputs and m nonlinear neurons, it provides ( n + 1 ) m linear parameters, while the SLFN would have only m linear parameters (one for each hidden neuron). Since the ELM is based on adjusting only the linear parameters using the least squares estimate (LSE), the PLP network provides more freedom for the proper adjustment. Results from 12 regression and 6 classification problems are presented considering the training and test errors, the linear vector norm and the system condition number. They point out that the PLP-ELM framework is more efficient than the SLFN-ELM approach. [ABSTRACT FROM AUTHOR]

This paper can be regarded as the continuation of the work of the authors contained in papers (2015). At the same time, it represents the extension of the papers Lou and Cui (2007, [24] ), Sannay (2007, [34] ) and Acka et al. (2004, [1] ). This work discusses a generalized model of high-order Hopfield-type neural networks with time-varying delays. By utilizing Lyapunov functional method and the linear inequality approach, some new stability criteria for such system are derived. The results are related to the size of delays and impulses. The exponential convergence rate of the equilibrium point is also estimated. Finally, we analyze and interpret four numerical examples proving the efficiency of our theoretical results and showing that impulse can be used to stabilize and exponentially stabilize some high-order Hopfield-type neural networks. [ABSTRACT FROM AUTHOR]

Integrating simplified or partial physics models with data-driven machine learning models is an emerging concept targeted at facilitating generalizability and extrapolability of complex system behavior predictions. In this paper, we introduce a novel machine learning based fusion model MIDPhyNet that decomposes, memorizes, and integrates first principle physics-based information with data-driven models. In MIDPhyNet the output of partial physics is decomposed into Intrinsic Mode Functions (IMFs), which are then infused to a Memorization Unit to generate embedded vectors. A Prediction Unit synthesizes all of the data to generate prediction results. We test the performance of MIDPhyNet on modeling the behavior of dynamic systems such as an inverted pendulum under wind drag. The results clearly demonstrate the performance benefits of our hybrid architecture over both purely data-driven models and state-of-art hybrid models in terms of generalizability and extrapolability. The MIDPhyNet architecture's superiority is most significant when the models are trained over sparse data sets and in general, MIDPhyNet provides a generic way to explore how physical information can be infused with data-driven models. [ABSTRACT FROM AUTHOR]

Recently, image-text matching based on local region-word semantic alignment has attracted considerable research attention. The fine-grained interplay can be achieved by aggregating the similarities of the region-word pairs. However, the similarities of aligned region-word pairs are treated equally in most cross-modal matching literatures, without considering their respective importance. Moreover, the local alignment methods are prone to bring about a global semantic drift due to the ignorance of thematic considerations for the image-text pairs. In this paper, a novel Dual-View Semantic Inference (DVSI) network is proposed to leverage both local and global semantic matching in a holistic deep framework. For the local view, a region enhancement module is proposed to mine the priorities for different regions in the image, which provides differentiate abilities to discover the latent region-word relationships. For the global view, the overall semantics of image is summarized for global semantic matching to avoid global semantic drift. The two views are unified together for final image-text matching. Extensive experiments conducted on MSCOCO and Flicr30K demonstrate the effectiveness of the proposed DVSI. [ABSTRACT FROM AUTHOR]

6DOF camera relocalization is an important component of autonomous driving and navigation. Deep learning has recently emerged as a promising technique to tackle this problem. In this paper, we present a novel relative geometry-aware Siamese neural network to enhance the performance of deep learning-based methods through explicitly exploiting the relative geometry constraints between images. We perform multi-task learning and predict the absolute and relative poses simultaneously. We regularize the shared-weight twin networks in both the pose and feature domains to ensure that the estimated poses are globally as well as locally correct. We employ metric learning and design a novel adaptive metric distance loss to learn a feature that is capable of distinguishing poses of visually similar images from different locations.We evaluate the proposed method on public indoor and outdoor benchmarks and the experimental results demonstrate that our method can significantly improve localization performance. Furthermore, extensive ablation evaluations are conducted to demonstrate the effectiveness of different terms of the loss function. [ABSTRACT FROM AUTHOR]

• The MBAN is proposed to learn invariant semantic representation of text image through mutual guidance mechanism. • The proposed method can deal with multiple types of irregularity, particularly blurring and arbitrary shape in this paper. • The proposed model helps alleviae the attention drift problem. • The proposed method achieves comparable or superior performance on standard benchmarks. Reading irregular text of arbitrary shape or low quality in natural scene images is a challenging task. Existing irregular scene text recognition methods mainly focus on irregular text with arbitrary shape, but rarely focus on irregular text of low quality. In this work, we propose a simple but effective method for recognizing irregular texts with arbitrary shape and low quality simultaneously. The proposed Multi-Branch guided Attention Network (MBAN) makes mutual guidance among multi-branch data in training, so as to learn invariant semantic representation between regular text images and the corresponding irregular images. Compared with the standard attention framework for text recognition, MBAN can significantly improve the performance of irregular text recognition while preserving similar performance for regular text recognition. In addition, regarding the attention drift problem encountered in standard attention network, MBAN can significantly improve the accuracy of alignment factors at each time step. We verify the effectiveness of MBAN in irregular text recognition and attention drift problem through extensive experiments. The performance of MBAN is shown to be comparable on regular datasets and superior on some irregular datasets with state-of-the-art methods. [ABSTRACT FROM AUTHOR]

• The NTZNNM is developed for the solution of time-invariant/varying Yang-Baxter-type matrix equation problem in the presence of noises from a control-theoretic viewpoint. • The time-varying matrix-formed state variable of NTZNNM is proven to globally/exponentially converge to the matrix-formed theoretical solution. • The robustness and superiority of GNTZNNM are guaranteed with the aid of different activation functions for handling time-invariant/varying Yang-Baxter-type matrix equation problems. Time-invariant/varying Yang-Baxter-type matrix equation problems in the presence of noises often arise in the fields of scientific computation and engineering implementation. Noises are ubiquitous and unavoidable in real systems but most existing models carry out the time-invariant/varying Yang-Baxter-type matrix equation problem with an indispensable precondition that the solving process is free of noises. In this paper, a noise-tolerant zeroing neural network model (NTZNNM) is first proposed, analyzed and verified by feat of a classical zeroing neural network model (ZNNM) from a control-theoretic viewpoint, and note that NTZNNM behaves efficiently in online solving the time-invariant/varying Yang-Baxter-type matrix equation problem with different measurement noises. Moreover, a general noise-tolerant zeroing neural network model (GNTZNNM) derived from a general noise-tolerant zeroing neural dynamic model (GNTZNDM) is developed and utilized to accelerating the convergent rate and enhancing the robustness. Then, theoretical results further demonstrate that the presented NTZNNM owns the ability to globally/exponentially converge with different measurement noises. Furthermore, the global convergence of GNTZNDM with different monotonically-increasing odd activation functions is also investigated and analyzed in detail. Besides, numerical results are provided to substantiate the efficiency, availability and superiority of the developed NTZNNM and GNTZNNM for time-invariant/varying Yang-Baxter-type matrix equation problems with inherent tolerance to noises. [ABSTRACT FROM AUTHOR]

This paper presents a method to identify an unknown discrete-time nonlinear system, using high-order neural networks and high-order sliding mode algorithms, which may be subject to internal and external disturbances. Based on the information obtained from available system states, a high-order neural network model is proposed to approximate the system dynamics. Neural network weights are trained by means of the unscented Kalman filter and high-order sliding mode observer. A simulation example is included to illustrate effectiveness of the proposed scheme. [ABSTRACT FROM AUTHOR]

A novel memristive synapse model based on the HP memristor is proposed in this paper, which can address the problem of synaptic weight infinite modulations. The sliding threshold mechanism of the Bienenstock-Cooper-Munro rule (BCM) is used to redefine the memristance (i.e. synaptic weight) adjustment process of the memristive synapse model. Based on the proposed memristor-based synapse and Leaky Integrate-and-Fire neurons, a spiking neural network (SNN) hardware fragment is constructed, where spike trains with different frequencies are used to evaluate the stability performance of the proposed SNN hardware. Results show that compared to other approaches, the network is stable under different stimuli due to the characteristics of the memristor-based synapse model, and prove that the proposed synapse model is able to mimic biological synaptic behaviour and the problem of synaptic weight infinite modulations is addressed. [ABSTRACT FROM AUTHOR]

In current efficient convolutional neural networks, 1 × 1 convolution is widely used. However, the amount of computation and the number of parameters of 1 × 1 convolution layers account for a large part of these neural network models. In this paper, we propose to use linear-phase pointwise convolution kernels (LPPC kernels) to reduce the computational complexities and storage costs of these neural networks. We design four types of LPPC kernels based on the parity of the number of input channels and symmetry of the weights of the pointwise convolution kernel. Experimental results show that Type-I LPPC kernels can compress some popular networks better with a small reduction in accuracy than the other types of LPPC kernels. The LPPC kernels can be used as new 1 × 1 convolution kernels to design efficient neural network architectures in the future. Moreover, the LPPC kernels are friendly to low-power hardware accelerator design to achieve lower memory access cost and smaller model size. [ABSTRACT FROM AUTHOR]

Educational institutions continually strive to improve the services they offer, their aim is to have the best possible teaching staff, increase the quality of teaching and the academic performance of their students. A knowledge of the factors that affect student learning could help universities and study centres adjust their curricula and teaching methods to the needs of their students. One of the first measures employed by teaching institutions was to create Virtual Learning Environments (VLEs). This type of environment makes it possible to attract a larger number of students because it enables them to study from wherever they are in the world, meaning that the student's location is no longer a constraint. Moreover, VLEs facilitate access to teaching resources, they make it easier to monitor the activity of the teaching staff and of the interactions between students and teachers. Thus, online environments make it possible to assess the factors that cause the students' academic performance to increase or decrease. To understand the factors that influence the university learning process, this paper applies a series of automatic learning techniques to a public dataset, including tree-based models and different types of Artificial Neural Networks (ANNs). Having applied these techniques to the dataset, the number of times students have accessed the resources made available on VLE platforms has been identified as a key factor affecting student performance. This factor has been analysed by conducting a real case study which has involved 120 students doing a masters degree over a VLE platform. Concretely, the case study participants were masters degree students in areas related to computer engineering at the University of Salamanca. [ABSTRACT FROM AUTHOR]

This paper is concerned with the synchronization of bi-directional associative memory (BAM) neural networks with diffusion and time-varying delays, in which both synchronizing and desynchronizing impulses are considered. Due to the existence of the diffusion effect in artificial neural networks, synchronization control involves the dynamical behaviors of master–slave systems both in space and time. An impulsive pinning controller is designed for realizing the spatio-temporal synchronization of neural networks. By the Lyapunov functional theory and the comparison principle, exponential synchronization criteria are established, which consist of algebraic inequalities and are easy to implement. Two numerical examples show that controlling only a portion of neurons in the slave neural network can achieve the synchronization of master–slave systems. [ABSTRACT FROM AUTHOR]

In this paper, we propose a new neural network-based iterative learning leader-less consensus control algorithm for the second-order uncertain time-varying nonlinear multi-agent systems, where the position of each agent is constrained in an open interval which can be pre-selected and the control direction can be non-identical and partially unknown. With the help of a novel barrier composite energy function and multiple piece-wise Nussbaum functions, the designed completely distributed learning protocols can make the positions and velocities of the agents achieve the exact consensus on a finite time interval as the iteration number increases to infinity. Meanwhile, the position trajectories of the agents at each iteration do not exceed the constraints bounds. Besides, to testify the feasibility and utility of the presented algorithm, two simulation examples are given in the end. [ABSTRACT FROM AUTHOR]

In this paper, we construct forecasting models (multivariate long short-term memory recurrent neural networks and multiple linear regression) for the resource usage prediction of four MapReduce applications and applications executed within the Spark framework. We have evaluated the impact of a sample size to prediction accuracy. Also, we propose a phase modelling approach for read/write-intensive applications. Our results show that models based on long short-term memory recurrent neural networks exhibit a higher accuracy than multiple linear regression models and the intensive characteristics of a resource are closely related to the prediction accuracy of forecasting models. We investigated the hyperparameter tuning of such models and showed that a randomly initialised, shallow, well-tuned network may outperform deeper models that use stacked autoencoder initialisation. Furthermore, multivariate long short-term memory recurrent neural network models are more sensitive to sample size than multiple linear regression models. We show that an LSTM model trained in a specific machine may be used to predict the resource usage in another machine. [ABSTRACT FROM AUTHOR]

This paper concerns a model of hierarchical neural network with D n × D n -symmetry and time delays. The linear stability of the steady state is obtained via an artful factorization technique of the characteristic equation corresponding to the linearized system. And we also perform a delay-induced Hopf bifurcation analysis involving spatio-temporal patterns of bifurcated periodic solutions relying on the equivariant Hopf bifurcation theory for retarded functional differential equations and the representation method of Lie groups. Some numerical simulations for a simple example are carried out to demonstrate the application of our theoretical conclusion that multiple periodic orbits could occur simultaneously from Hopf bifurcation in the model. [ABSTRACT FROM AUTHOR]

This survey presents a review of state-of-the-art deep neural network architectures, algorithms, and systems in speech and vision applications. Recent advances in deep artificial neural network algorithms and architectures have spurred rapid innovation and development of intelligent speech and vision systems. With availability of vast amounts of sensor data and cloud computing for processing and training of deep neural networks, and with increased sophistication in mobile and embedded technology, the next-generation intelligent systems are poised to revolutionize personal and commercial computing. This survey begins by providing background and evolution of some of the most successful deep learning models for intelligent speech and vision systems to date. An overview of large-scale industrial research and development efforts is provided to emphasize future trends and prospects of intelligent speech and vision systems. Robust and efficient intelligent systems demand low-latency and high fidelity in resource-constrained hardware platforms such as mobile devices, robots, and automobiles. Therefore, this survey also provides a summary of key challenges and recent successes in running deep neural networks on hardware-restricted platforms, i.e. within limited memory, battery life, and processing capabilities. Finally, emerging applications of speech and vision across disciplines such as affective computing, intelligent transportation, and precision medicine are discussed. To our knowledge, this paper provides one of the most comprehensive surveys on the latest developments in intelligent speech and vision applications from the perspectives of both software and hardware systems. Many of these emerging technologies using deep neural networks show tremendous promise to revolutionize research and development for future speech and vision systems. [ABSTRACT FROM AUTHOR]

We propose a class of Clifford-valued neutral-type fuzzy cellular neural network models with leakage delays on time scales, give the definition and basic properties of Clifford-valued almost automorphic functions on time scales, and prove a composition theorem of Clifford-valued almost automorphic functions on time scales, and on this basis, we study the existence and global exponential stability of almost automorphic solutions of this class of neural networks. The research in this paper can unify the research of real-valued neural networks, complex-valued neural networks and quaternion-valued neural networks, as well as the research of continuous-time neural networks and discrete-time neural networks. Finally, an example illustrates the feasibility of the results obtained. [ABSTRACT FROM AUTHOR]

Using neural network modeling, we address the problem of system identification for continuous multivariate systems, whose structures vary around an operating point. Structural changes in the system are of combinatorial nature, and some of them may be very rare; they may be actionable for the purpose of controlling the system. Although our ultimate goal is both system identification and control , we only address the problem of identification in this paper. We propose and study a novel neural network architecture called LEAP net, for Latent Encoding of Atypical Perturbation. Our method maps system structure changes to neural net structure changes, using structural actionable variables. We demonstrate empirically that LEAP nets can be trained with a natural observational distribution, very concentrated around a "reference" operating point of the system, and yet generalize to rare (or unseen) structural changes. We validate the generalization properties of LEAP nets theoretically in particular cases. We apply our technique to power transmission grids, in which high voltage lines are disconnected and re-connected with one-another from time to time, either accidentally or willfully. We discuss extensions of our approach to actionable variables, which are continuous (instead of discrete, in the case of our application) and make connections between our problem setting, transfer learning, causal inference, and reinforcement learning. [ABSTRACT FROM AUTHOR]

Weightless Neural Networks (WNNs) are Artificial Neural Networks based on RAM memory broadly explored as solution for pattern recognition applications. Memory-oriented solutions for pattern recognition are typically very simple, and can be easily implemented in hardware and software. Nonetheless, the straightforward implementation of a WNN requires a large amount of memory resources making its adoption impracticable on memory constrained systems. In this paper, we establish a foundational relationship between WNN and Bloom filters, presenting a novel unified framework which encompasses the two. In particular, we indicate that a WNN can be framed as a memory segmented Bloom filter. Leveraging such finding, we propose a new model of WNNs which utilizes Bloom filters to implement RAM nodes. Bloom filters reduce memory requirements, and allow false positives when determining if a given pattern was already seen in data. We experimentally found that for pattern recognition purposes such false positives can build robustness into the system. The experimental results show that our model using Bloom filters achieves competitive accuracy, training time and testing time, consuming up to 6 orders of magnitude less memory resources when compared against the standard Weightless Neural Network model. [ABSTRACT FROM AUTHOR]

• Weightless Artificial Neural Networks (WANNs) are known to be more cheaper and faster than traditional neural networks. • WiSARD is the most adopted WANN model, having already been used in many different tasks and domains. • ClusWiSARD is an extension that improves accuracy in tasks where patterns of the same class are very heterogeneous. • The n-Tuple Regression Network (nTRN) was the only WANN model capable of performing regression. • This work proposes new models: Regression WiSARD and ClusRegression WiSARD, which are based in nTRN, WiSARD and ClusWiSARD. • In this work the novel proposed models are analyzed, including in the exploration of three different ensemble architectures. This paper explores two new weightless neural network models, Regression WiSARD and ClusRegression WiSARD, in the challenging task of predicting the total palm oil production of a set of 28 (twenty eight) differently located sites under different climate and soil profiles. Both models were derived from Kolcz and Allinson's n -Tuple Regression weightless neural model and obtained mean absolute error (MAE) rates of 0.09097 and 0.09173, respectively. Such results are very competitive with the state-of-the-art (0.07983), whilst being four orders of magnitude faster during the training phase. Additionally the models have been tested on three classic regression datasets, also presenting competitive performance with respect to other models often used in this type of task. [ABSTRACT FROM AUTHOR]

Deep learning is a branch of neural network which has been intensively developed in the last decade. Due to the high-accuracy classification ability, the deep learning algorithms have been widely used in many fields, such as speech recognition, image recognition, and natural speech processing. However, they also show some shortcomings especially on the selection of some parameters in the network, including hyper-parameters, which is still treated as a time consuming task. In this paper, a modified ABC (ABC-ISB) optimization algorithm is proposed to automatically train the parameters of Feed-Forward Artificial Neural Networks, which is a typical a neural network. In the proposed ABC algorithm, we utilize the information of neighbors with better performance to accelerate the convergence of employed and onlooker bees respectively. In addition, a new selection strategy and a gbest-guided strategy are introduced to enhance the global search capability and balance the exploration and exploitation of the algorithm separately. The experimental results show our ABC-ISB is generally leading and competitive. [ABSTRACT FROM AUTHOR]

Neural network learning is usually time-consuming since backpropagation needs to compute full gradients and backpropagate them across multiple layers. Despite its success of existing works in accelerating propagation through sparseness, the relevant theoretical characteristics remain under-researched and empirical studies found that they suffer from the loss of information contained in unpropagated gradients. To tackle these problems, this paper presents a unified sparse backpropagation framework and provides a detailed analysis of its theoretical characteristics. Analysis reveals that when applied to a multilayer perceptron, our framework essentially performs gradient descent using an estimated gradient similar enough to the true gradient, resulting in convergence in probability under certain conditions. Furthermore, a simple yet effective algorithm named m emorized s parse b ack p ropagation (MSBP) is proposed to remedy the problem of information loss by storing unpropagated gradients in memory for learning in the next steps. Experimental results demonstrate that the proposed MSBP is effective to alleviate the information loss in traditional sparse backpropagation while achieving comparable acceleration. [ABSTRACT FROM AUTHOR]

Electroencephalography (EEG) based Brain-Computer Interface (BCI) enables subjects to communicate with the outside world or control equipment using brain signals without passing through muscles and nerves. Many researchers in recent years have studied the non-invasive BCI systems. However, the efficiency of the intention decoding algorithm is affected by the random non-stationary and low signal-to-noise ratio characteristics of the EEG signal. Furthermore, channel selection is another important issue in BCI systems intention recognition. During intention recognition in BCI systems, the unnecessary information produced by redundant electrodes affects the decoding rate and deplete system resources. In this paper, we introduce a recurrent-convolution neural network model for intention recognition by learning decomposed spatio-temporal representations. We apply the novel Gradient-Class Activation Mapping (Grad-CAM) visualization technology to the channel selection. Grad-CAM uses the gradient of any classification, flowing into the last convolutional layer to produce a coarse localization map. Since the pixels of the localization map correspond to the spatial regions where the electrodes are placed, we select the channels that are more important for decision-making. We conduct an experiment using the public motor imagery EEG dataset EEGMMIDB. The experimental results demonstrate that our method achieves an accuracy of 97.36% at the full channel, outperforming many state-of-the-art models and baseline models. Although the decoding rate of our model is the same as the best model compared, our model has fewer parameters with faster training time. After the channel selection, our model maintains the intention decoding performance of 92.31% while reducing the number of channels by nearly half and saving system resources. Our method achieves an optimal trade-off between performance and the number of electrode channels for EEG intention decoding. [ABSTRACT FROM AUTHOR]

Image-based food pattern classification poses new challenges for mainstream computer vision algorithms. Recent works on feature fusion technique have significantly boosted the generalization performances of food categorization tasks. However, the use of representation learning in the training process of feature fusion has rarely been explored. This study addresses the issue through a new supervised subnetwork-based feature encoding and pattern classification model, termed a wide hierarchical subnetwork-based neural network (Wi-HSNN). In particular, Wi-HSNN is a subnet-based iterative training process in which one pair of subnets is added to the framework in each iteration. Furthermore, instead of learning the optimal representations with the whole dataset, this paper introduces a batch-by-batch parallel scheme of Wi-HSNN to process large-scale datasets, such as Place365 set with more than 1.8 million samples. Extensive evaluations on eight benchmark datasets from food classification to scene image recognition demonstrated that the proposed solution has better representation learning capacity compared to existing encoding methods, and achieves stronger performance than existing approaches for food image classification tasks. [ABSTRACT FROM AUTHOR]

Pooling layers help reduce redundancy and the number of parameters in deep neural networks without the need of performing additional learning processes. Although these operators are able to deal with both single-label and multi-label problems they are specifically aimed at reducing feature space. However, in the case of multi-label data, this should also be done in the label space. On the other hand, in spite of their success, existing pooling operators are not ideal when handling (multi-label) datasets that do not have an explicit topological organization. In this paper, we present a deep neural architecture using bidirectional association-based pooling layers to extract high-level features and labels in multi-label classification problems. Our approach uses an association function to detect distinct pairs of neurons that will be aggregated into pooled neurons. In the first pooling layer, our proposal computes the Pearson correlation among the variables as the basis to quantify the association values. In addition, we propose an iterative procedure that allows estimating the association degree among pooled neurons in deeper layers without the need of recomputing the correlation matrix. The main advantage of this deep neural architecture is that it allows extracting high-level features and labels on datasets with no specific topological organization. The numerical results show that our bidirectional neural network helps reduce the number of problem features and labels while preserving network's discriminatory power. [ABSTRACT FROM AUTHOR]

In this paper, zeroing neural network models are redesigned and analyzed from a control-theoretical framework for online solving time-varying full-rank Moore–Penrose inversions. To solve time-varying full-rank Moore–Penrose inverse problems with different noises in real time, some modified zeroing neural network models are developed, analyzed and investigated from the perspective of control. Furthermore, the proposed zeroing neural network models globally converge to the theoretical solution of the full-rank Moore–Penrose inverse problem without noises, and exponentially converge to the exact solution in the presence of noises, which are demonstrated theoretically. Moreover, in comparison with existing models, numerical simulations are provided to substantiate the feasibility and superiority of the proposed modified neural network for online solving time-varying full-rank Moore–Penrose problems with inherent tolerance to noises. In addition, the numerical results infer that different activation functions can be applied to accelerate the convergence speed of the zeroing neural network model. Finally, the proposed zeroing neural network models are applied to the motion generation of redundant robot manipulators, which illustrates its high efficiency and robustness. [ABSTRACT FROM AUTHOR]