Search

More recently, deep neural networks received much attention from the recommender systems community where CNNs and RNNs were applied in different recommendation contexts and achieved state-of-the-art results on many publicly available benchmarks. While the success of the neural models came from their high expressiveness, the uninterpretability problem still one of their main drawbacks, which can have a negative side effect on the whole learning process. A good candidate to reduce the uninterpretability is using Probabilistic Soft Logic (PSL), which showed a strong performance in dealing with this problem. Moreover, sparsity is another major problem of the most previous recommendation systems. In this paper, we introduce a new recommender system framework based on the generalized distillation principle that combines two modeling approaches: PSL for the knowledge-driven modeling approach and deep neural networks for the data-driven approach. Experimental results on publicly available datasets show that our method significantly outperforms the previous state-of-the-art based on deep neural networks emphasizing the utility of PSL rules in handling inconsistencies, reducing the uninterpretability of the neural models and shows promising results in dealing with sparsity. [ABSTRACT FROM AUTHOR]

Content-based image retrieval (CBIR) aims to retrieve images from a given image collection according to similarities of image contents. In this paper, we focus on retrieval of scene images. We propose a siamese spatial attentive model which bases on siamese architecture and incorporates attention mechanism to generate compatible image embeddings. It extracts local features with a convolutional neural network (CNN), which starts with pre-trained parameters and is well fine-tuned for retrieval. Spatial attention pooling is proposed to take feature maps as input and generate weights for local features, which are then used to refine local features via weighted sum-pooling. Such pooling alleviates impacts from disturbance and concentrates on meaningful parts of images. Therefore, the model is able to output robust representations for noisy images. We also propose a multi-stage training scheme for the model, which leads to better performance than normal one-pass training scheme. Extensive experimental results on benchmark image retrieval datasets show that our model is competitive in retrieval performance. [ABSTRACT FROM AUTHOR]

Impurity detection involves detecting small impurities in the liquid inside an opaque glass bottle with complex textures by looking through the bottleneck. Sometimes experts have to observe continuous frames to determine the existence of an impurity. In recent years, region-based convolutional neural networks have gained incremental successes in common object detection tasks. However, sequential impurity detections present more challenging issues than detecting targets in a single frame, because consecutive motions and appearance changes of impurities cannot be captured using those common object detectors. In this paper, we propose a simple and controllable ensemble architecture to alleviate this problem. Specifically, a siamese fusion network is used to generate impurity proposals, then an attention model based on visual features and trajectories is proposed to localize a unique region proposal in each frame, finally, a sequential region proposal classifier using a long-term recurrent convolutional network is applied to refine impurity detection performances. The proposed method achieves 79.81 % mAP on IML-DET datasets, outperforming a comparable state-of-the-art Mask R-CNN model. [ABSTRACT FROM AUTHOR]

Spiking neurons encode information through their spiking temporal patterns. Although the precise spike-timing based encoding scheme has long been recognised, the exact mechanism that underlies the learning of such precise spike-timing in the brain remains an open question. Most of the existing learning methods for spiking neurons are based on synaptic weight adjustment. However, biological evidences suggest that synaptic delays can also be modulated to play an important role in the learning process. This paper investigates the viability of integrating synaptic delay plasticity into supervised learning and proposes a novel learning method that adjusts both the synaptic delays and weights of the learning neurons to make them fire precisely timed spikes, that is referred to as synaptic delay-weight plasticity. Remote Supervised Method (ReSuMe) and Perceptron Based Spiking Neuron Learning Rule (PBSNLR), two representative supervised learning methods, are studied to illustrate how the synaptic delay-weight plasticity works. The performance of the proposed learning method is thoroughly evaluated on synthetic data and is further demonstrated on real-world classification tasks. The experiments show that the synaptic delay-weight learning method outperforms the traditional synaptic weight learning methods in many ways. [ABSTRACT FROM AUTHOR]

This paper investigates a new hybrid projective synchronization scheme between two coupled fractional-order complex networks with different sizes. The hybrid projective synchronization studied in this paper includes complete synchronization of the states of the nodes in each network and projective synchronization of the states of a pair of nodes from both networks. Based on the stability theorem of fractional-order differential system and adaptive control technique, some sufficient conditions for guaranteeing the existence of the hybrid projective synchronization are derived. Two examples are given to show the effectiveness of the proposed methods. [ABSTRACT FROM AUTHOR]

In this paper, least trimmed squares (LTS) estimators, frequently used in robust (or resistant) linear parametric regression problems, will be generalized to nonparametric LTS neural networks for nonlinear regression problems. Emphasis is put particularly on the robustness against outliers. This provides alternative learning machines when faced with general nonlinear learning problems. Simple weight updating rules based on gradient descent and iteratively reweighted least squares (IRLS) algorithms will be provided. The important parameter of trimming percentage for the data at hand can be determined by cross validation. Some simulated and real-world data will be used to illustrate the use of LTS neural networks. We will compare the robustness against outliers for usual neural networks with least squares criterion and the proposed LTS neural networks. Simulation results show that the LTS neural networks proposed in this paper have good robustness against outliers. [ABSTRACT FROM AUTHOR]

The problem of stability via adaptive controller is considered for time-delay neutral-type neural networks with stochastic noise and Markovian switching in this paper. A new criterion of almost sure (a.s.) asymptotic stability for a general neutral-type stochastic differential equation is proposed. Based on this criterion, and by using of the generalized Itô׳s formula and the M-matrix method, a delay dependent sufficient condition is established to ensure the almost sure asymptotic stability for neutral-type neural networks with stochastic perturbation and Markovian switching. Meanwhile, the update law of the feedback control is determined. A numerical example is provided to verify the usefulness of the criterion proposed in this paper. [ABSTRACT FROM AUTHOR]

This paper presents a novel constrained integration (CINT) method for solving initial boundary value partial differential equations (PDEs). The CINT method combines classical Galerkin methods with a constrained backpropogation training approach to obtain an artificial neural network representation of the PDE solution that approximately satisfies the boundary conditions at every integration step. The advantage of CINT over existing methods is that it is readily applicable to solving PDEs on irregular domains, and requires no special modification for domains with complex geometries. Furthermore, the CINT method provides a semi-analytical solution that is infinitely differentiable. In this paper the CINT method is demonstrated on two hyperbolic and one parabolic initial boundary value problems with a known analytical solutions that can be used for performance comparison. The numerical results show that, when compared to the most efficient finite element methods, the CINT method achieves significant improvements both in terms of computational time and accuracy. [ABSTRACT FROM AUTHOR]

This paper presents a recurrent neural network to support vector machine (SVM) learning in pattern classification arising widespread applications in a variety of setting. The SVM learning problem in classification is first converted into an equivalent quadratic programming (QP) formulation, and then a recurrent neural network for SVM learning is proposed. The proposed neural network is guaranteed to obtain the optimal solution of support vector classification. It is also shown that the proposed neural network model is stable in the sense of Lyapunov and it is globally convergent to an exact optimal solution of the QP problem. Several illustrative examples are provided to show the feasibility and the efficiency of the proposed method in this paper. [ABSTRACT FROM AUTHOR]

The cooperative exponential synchronization of multi-agent chaotic delayed neural networks (DNNs) with switching topology, stochastic disturbance and impulsive disturbance is investigated in this paper. Based on the Lyapunov stability theory, algebraic graph theory, matrix theory and the helpful stochastic Halanay inequality technique, some sufficient conditions are presented to guarantee the cooperative exponential synchronization for multi-agent chaotic DNNs with switching topology. Compared with the existing works about synchronization (or consensus) of multi-agent systems, the proposed method in this paper can provide a more general framework for the cooperative synchronization of nonlinear multi-agent systems with or without time delays, stochastic and impulsive disturbances. The famous master–slave (drive-response) synchronization of chaotic DNNs is a special case of this paper, and therefore the derived results can also be favorable for practical application in secure communication. Simulation results finally verify the effectiveness of the proposed synchronization control algorithm. [ABSTRACT FROM AUTHOR]

A vision-based anticipatory controller for the autonomous indoor navigation of an unmanned aerial vehicle (UAV) is the topic of this paper. A dual Feedforward/Feedback architecture has been used as the UAV׳s controller and the K-NN classifier using the gray level image histogram as discriminant variables has been applied for landmarks recognition. After a brief description of the UAV, we first identify the two main components of its autonomous navigation, namely, the landmark recognition and the dual controller based on cerebellar system of living beings, then we focus on the anticipatory module that has been implemented by an artificial neural network. Afterwards, the paper describes the experimental setup and discusses the experimental results centered mainly on the basic UAV׳s behavior of landmark approximation maneuver, which in topological navigation is known as the beaconing or homing problem. [ABSTRACT FROM AUTHOR]

The purpose of this paper is to exhibit the internal process of the Growing Functional Modules (GFM) controllers. These learning based controllers are graphically designed by the user, interconnecting four kinds of components: Global Goals, Acting Modules, Sensing Modules and Sensations. Global Goals specify the intrinsic motivations, Acting and Sensing Modules develop their respective functionalities while Sensations provide the controlled system׳s feedback. GFM Controllers engage in a learning process to satisfy the specified motivations while interacting with the environment and thus can be considered as artificial brains. The traditional systems programming task is replaced by a design phase that consists in graphically configuring and interconnecting the GFM components. This paper gives a description of the aforementioned components and of their specific roles in the learning based control process. The behavior of the controller and its components is exemplified with the characterization of a simple control application. [ABSTRACT FROM AUTHOR]

Artificial Neural Networks have been found to be very efficient universal approximators. Single Layer Feedforward Networks (SLFN) are the most popular and easy to train. The neurons in these networks can use both sigmoidal functions and radial basis functions (RBF) as activation functions. Both functions have been shown to work very efficiently. Sigmoidal networks are already very well described in the literature. This paper will focus on the construction of a SLFN architecture using RBF neurons. There are many algorithms that are used to construct or train networks to solve function approximation problems. In this paper, an algorithm which is a modification of the Incremental Extreme Learning Machine (I-ELM) family of algorithms is proposed. The proposed algorithm eliminates randomness in the learning process with respect to center positions and widths of the RBF neurons. To do this, the input with the highest error magnitude is saved during error calculation and then used as the center for the next incrementally added neuron. Then the radius of the new neuron is iteratively chosen using Nelder–Mead׳s Simplex method. This allows the universal approximation properties of I-ELM to be preserved while greatly reducing the sizes of the trained RBF networks. [ABSTRACT FROM AUTHOR]

In this paper we propose a novel Computational Attention Models (CAM) that fuses bottom-up, top-down and salient motion visual cues to compute visual salience in surveillance videos. When dealing with a number of visual features/cues in a system, it is always challenging to combine or fuse them. As there is no commonly agreed natural way of combining different conspicuity maps obtained from different features: face and motion for example, the challenge is thus to find the right mix of visual cues to get a salience map that is the closest to a corresponding gaze map? In the literature many CAMs have used fixed weights for combining different visual cues. This is computationally attractive but is a very crude way of combining the different cues. Furthermore, the weights are typically set in an ad hoc fashion. Therefore in this paper, we propose a machine learning approach, using an Artificial Neural Network (ANN) to estimate these weights. The ANN is trained using gaze maps, obtained by eye tracking in psycho-physical experiments. These weights are then used to combine the conspicuities of the different visual cues in our CAM, which is later applied to surveillance videos. The proposed model is designed in a way to consider important visual cues typically present in surveillance videos, and to combine their conspicuities via ANN. The obtained results are encouraging and show a clear improvement over state-of-the-art CAMs. [ABSTRACT FROM AUTHOR]

This paper investigates the problem of robust stabilization for a class of discrete-time stochastic neural networks with randomly occurring discrete and distributed time-varying delays. More precisely, the neuron activation functions are assumed to be more general and satisfy sector-like nonlinearities. Moreover, the effects of both variation range and probability distribution of mixed time-delays are taken into consideration in the proposed problem. The main objective of this paper is to design a state feedback reliable H ∞ controller such that for all admissible uncertainties as well as actuator failure cases, the resulting closed-loop form of considered neural network is robustly asymptotically stable while satisfying a prescribed H ∞ performance constraint. Linear matrix inequality approach together with proper construction of Lyapunov–Krasovskii functional is employed for obtaining delay dependent sufficient conditions for the existence of robust reliable H ∞ controller. The obtained results are formulated in terms of linear matrix inequalities (LMIs) which can be easily solved by using the MATLAB LMI toolbox. Finally, a numerical example with simulation results is provided to illustrate the effectiveness of the obtained control law and less conservativeness of the proposed results. [ABSTRACT FROM AUTHOR]

The cellular neural network (CNN) is one of the classic artificial neural networks. During the past decades, the one-dimensional CNN models and their applications have not yet been paid enough enthusiasm too. For this reason, this paper proposes a simplified one-dimensional CNN model and then designs a pairwise network using this model to demonstrate its applicability. This pairwise CNN consists of two parallel one-dimensional CNNs, a fixed master and a movable slave. Using this pairwise CNN, an algorithm is developed to perform the global alignment of two DNA sequences. In this algorithm, the slave moves forward step by step, and the cell states of the master are computed in the meanwhile. Based on all the states obtained in all time steps, a state selection array is generated then a global alignment path is determined from this array. Under the guidance of the alignment path, two DNA sequences are globally aligned by inserting blank spaces in the appropriate positions of these two sequences. Experiments on aligning the DNA sequences from the publicly available databases of the NCBI with this method are carried out in this paper and compared with the other two methods. Through evaluating computation time and similarity, these experiments prove that the proposed one-dimensional CNN model is effective, and the alignment algorithm based on a pairwise CNN of the model is efficient, obtaining higher similarity with less computation time than the other two. [ABSTRACT FROM AUTHOR]

Creation of a computationally efficient and biologically plausible neuron and neural network models is an exceptionally important occupation, the results of which are much needed for the neuroscience in general or for the neurophysiology in partial. Besides, the applying neural network technological areas are always looking for novel improvements of the methodological approaches, algorithms, computational facilities. In this paper, a new (in some aspects) hybrid integrate-and-fire (HIF) neuron model, based on N -shaped Na + and K + ionic current–voltage characteristics and a double discrete reset mechanism, is proposed. In the results of computational experiments, the spike series from a single spiking to the burst and chaotic burst trains are demonstrated. In the other part of the paper, analogous FitzHugh–Nagumo neuron model as a generalized neural networks system with an expansion to an inclusion of a coupling kernel function has been considered. The mean field approximation of neuronal potentials and recovery currents inside a neuron cluster was used. The biologically more realistic nonlinear N-shaped Na + characteristic and kernel functions (one - or two-dimensional) were applied. A possibility to present the nonlinear integral differential equations with kernel functions under the Fourier transformation by partial differential equations was used that allowed us to overcome the analytical and numerical modeling difficulties. The equivalence of two kinds of solutions was additionally confirmed based on the error evaluation. The approach of the equivalent partial differential equations was successfully employed to solve the system with the heterogeneous synaptic functions. The analytical studies are corroborated by many numerical modeling experiments. [ABSTRACT FROM AUTHOR]

This paper focuses on a new kind of artificial neural networks – the Laplace transform artificial neural networks (LTANN). It is proposed to use the Laplace transform instead of ordinary weights and a linear activation function of an artificial neuron. This extension allows to use artificial neural networks in new areas. The ordinary description of artificial neural networks is a special case of the description proposed in this paper. Three models of different geological structures based on the LTANN are discussed in this paper. Using these models, it is possible to classify an unknown geological structure as the structure without anomaly, the structure with a sinkhole or the structure with a loose zone. [ABSTRACT FROM AUTHOR]

In this paper, an effective hybrid optimization strategy by incorporating the adaptive optimization of particle swarm optimization (PSO) into genetic algorithm (GA), namely HPSOGA, is used for determining the parameters of radial basis function neural networks (number of neurons, their respective centers and radii) automatically. While this task depends upon operator׳s experience with trial and error due to lack of prior knowledge, or based on gradient algorithms which are highly dependent on initial values. In this paper, hybrid evolutionary algorithms are used to automatically build a radial basis function neural networks (RBF-NN) that solves a specified problem, related to rainfall forecasting in this case. In HPSOGA, individuals in a new generation are created through three approaches to improve the global optimization performance, which are elitist strategy, PSO strategy and GA strategy. The upper-half of the best-performing individuals in a population are regarded as elites, whereas the half of the worst-performing individuals are regarded as a swarm. The group constituted by the elites are enhanced by selection, crossover and mutation operation on these enhanced elites. HPSOGA is applied to RBF-NN design for rainfall prediction. The performance of HPSOGA is compared to pure GA in these basis function neural networks design problems, showing that the hybrid strategy is of more effective global exploration ability and to avoid premature convergence. Our findings reveal that the hybrid optimization strategy proposed here may be used as a promising alternative forecasting tool for higher forecasting accuracy and better generalization ability. [ABSTRACT FROM AUTHOR]

This paper is concerned with the issue of pinning synchronization of a general model of complex dynamical networks, which can well describe practical architectures of many realistic complex systems. Compared with some existing works, the distinctive feature of the considered model in this paper includes (i) the network topologies with variable structures corresponding directed graph; (ii) the multi-linked configuration with nonlinear coupling and time-varying delays. Some generic criteria for pinning global synchronization of such dynamical network are presented based on Lyapunov stability theory on delayed dynamical systems. It is shown that these criteria can provide a novel and effective adaptive pinning strategy, which is very convenient to implement in practice since the design of the adaptive control law is independent of time-varying delays. Furthermore, it is interesting to find that when the nodes with low in-degrees are pinned firstly, the pinning control scheme is more efficient. Subsequently, the theoretic results are applied to a general two-linked network consisting of Hopfield neuron oscillator. Finally, numerical simulations demonstrate that the proposed pinning adaptive synchronization criteria are practical and effective to pin two-linked Hopfield neural networks to an equilibrium, periodic orbit and chaotic attractor. [ABSTRACT FROM AUTHOR]

This paper proposes a novel optimization method that utilizes a multi-population parallel genetic algorithm (GA) to simultaneously optimize the structure and system latency of evolvable block-based neural networks (BbNNs). In the past, BbNNs have been successfully deployed for various kinds of classification problems with promising results. However, existing BbNN architectures do not explicitly model or optimize the latency of the system. Optimization of system latency is important, because it improves performance, reduces power consumption, and allows a more deterministic behavior that is needed for recurrent modes. The BbNN hardware model proposed in this paper is implemented as a synchronous architecture with registered outputs. This architecture will permit the system latency to be deterministic. To facilitate the automated layout of the neuron block interconnects within the BbNN structure, an array interconnect algorithm is presented. The proposed latency-optimized BbNN is implemented in hardware, and the architecture resembles mobile devices with integrated graphics processing units (GPUs). The hardware implementation is verified and analyzed using various case studies. The power consumption of our prototyping platform was measured, and it is shown that approximately 364 nJ of energy can be saved for every computational cycle (latency) reduced. All presented case studies show that the proposed BbNN model provides an optimal latency value while still matching the high accuracy reported in previous works. [ABSTRACT FROM AUTHOR]

Memristive neural networks are a novel topic in the design and construction of brain-like circuitry system. This paper addresses a challenging problem: How to derive some less conservative theoretical results on the stability and attractability for memristive neural networks? In this paper, a new comparison method and segmentation method of state space are developed. A succinct criterion is provided to ascertain the global exponential stability and the estimate for location of equilibrium point. The obtained criterion is the improvement and extension of the existing results in the literature. The applicability of the proposed framework can be extended to general memristive neurodynamic systems. [ABSTRACT FROM AUTHOR]

In this paper, the passivity problem of memristor-based neural networks (MNNs) with time-varying delays is investigated. New delay-dependent criteria are established for the passivity of MNNs. The time-varying delays of our paper are not necessary to be differentiable, so our results are less conservative, which enrich and improve the earlier publications. An example is given to demonstrate the effectiveness of the obtained results. [ABSTRACT FROM AUTHOR]

This paper explains and models selected associative processes that take place in biological associative neural systems. Such associative systems allow us to form, expand, and exploit knowledge in a human-like way. They trigger artificial associations for previously trained and even new contexts taking into account the previous states of neurons of such systems, which are necessary for associative knowledge formation. The associative systems can automatically generalize and be creative. This paper reveals and explains the important generalization mechanisms of biological associative systems that can be modelled in artificial neural associative systems and used for practical associative neurocomputations. The associative mechanisms enable even generalization of rules or algorithms. These systems can reproduce generalization not only to train and classify static objects but also to form new sequences, which enables creativity of these systems. Because neuron groups are activated in a specific order, the associative conclusions are reached very quickly. Knowledge and active associations can also substitute many laborious and time-consuming searching processes that are used in contemporary computer science. [ABSTRACT FROM AUTHOR]

This paper describes the algorithms used to estimate the state of consumption of a pump based on dynamic pressure or vibrations. To create algorithms, the author used computational intelligence methods in the form of neural networks. In order to perform the analysis, data analysis systems were designed based on three neural networks: multilayer perceptron neural network (MLP), generalized regression neural network (GRNN) and probabilistic neural network (PNN). Processing of the input signal in the final result of the analysis consisted of several steps. First, the measurement data were preprocessed (delete constant component, normalization, standardization, reduction, fast Fourier transform (FFT), etc.), and training and test sets were prepared using the matrices with the expected system answers. The last step was the analysis, consisting of design data analysis systems based on artificial neural networks and their learning and testing. On the basis of the obtained results the effectiveness of neural networks and the methods of pre-processing of the signals applied to approximate the state of consumption of the displacement pump were evaluated. Design systems were evaluated based on accuracy (generated error) and complexity (number of parameters and training time) criteria. The main contribution of the paper is to design and compare methods for pre-processing the signal, and to design and compare the effectiveness of the three neural networks in the diagnosis consumption of a positive displacement pump. [ABSTRACT FROM AUTHOR]

In this paper, finite time dual neural networks with a new activation function are presented to solve quadratic programming problems. The activation function has two tunable parameters, which give more flexibility to design the neural networks. By Lyapunov theorem, finite-time stability can be derived for the proposed neural networks, and the actual optimal solutions of the quadratic programming problems can be obtained in finite time interval. Different from the existing recurrent neural networks for solving the quadratic programming problems, the neural networks of this paper have a faster convergent speed, at the same time, they can reduce oscillation when delay appears, and have less sensitivity to additive noise with careful selection of the parameters. Simulations are presented to evaluate the performance of the neural networks with the tunable activation function. In addition, the proposed neural networks are applied to estimate parameters for an energy model of belt conveyors. The effectiveness of our methods are validated by theoretical analysis and numerical simulations. [ABSTRACT FROM AUTHOR]

This paper is concerned with finite-time synchronization for a class of delayed neural networks with Cohen-Grossberg type. Different from the existing related results, the time-delayed feedback strategy is utilized to investigate finite-time synchronization of delayed Cohen-Grossberg neural networks. By constructing Lyapunov functions and using differential inequalities, several new and effective criteria are derived to realize local and global synchronization in finite time of the addressed neural networks based on two different time-delayed feedback controllers. Besides, the upper bounds of the settling time of synchronization are estimated. Furthermore, as corollaries, some sufficient conditions are given to achieve finite-time synchronization of delayed cellular neural networks. Finally, some numerical examples are provided to verify the theoretical results established in this paper. [ABSTRACT FROM AUTHOR]

Finite-time synchronization means the optimality in convergence time, thus many contributions have been made to it in the literature. However, to the best of our knowledge, most of the existing results on finite-time synchronization do not include time-delay. Considering the fact that time-delays especially infinite-time distributed delays are inevitably existing in neural networks, this paper aims to study global synchronization in finite time of neural networks with both time-varying discrete delay and infinite-time distributed delay (mixed delays). The techniques that we apply in this paper are not only different from the techniques employed in existing papers, but also applicable to differential systems with or without delay. Based on new Lyapunov-Krasovskii functional candidate and the new analysis techniques, sufficient conditions guaranteeing the finite-time synchronization of the addressed neural networks are derived by using a class of simple discontinuous state feedback controller. Conditions for realizing finite-time synchronization of neural networks with finite-time distributed delay and without delay are also given. Moreover, estimation of the upper bound of synchronization-time is also provided for neural networks with finite-time distributed delay and without delay. It is shown that the synchronization-time depends on both the initial values and the time-delays of the drive-response systems. Numerical examples demonstrate the effectiveness of the theoretical results. [ABSTRACT FROM AUTHOR]

This paper revisits the problem of passivity analysis for neural networks with time-varying delays. A new delay-dependent criterion is obtained in terms of linear matrix inequalities, guaranteeing that the input and output of the considered neural network satisfy a prescribed passivity-inequality constraint. This newly presented criterion does not require all the symmetric matrices involved in the employed quadratic Lyapunov-Krasovskii functional to be positive definite. This feature is remarkable since it sheds new light on the traditional ideas for constructing Lyapunov-Krasovskii functionals. More importantly, the conservatism of delay-dependent passivity conditions can be reduced due to the relaxation on the positive-definiteness of every Lyapunov matrix. It is shown both theoretically and numerically that the passivity criterion proposed in this paper is truly less conservative than some of the latest results in the literature. [ABSTRACT FROM AUTHOR]

DNA microarray technology has enabled biologists to study all the genes within an entire organism to obtain a global view of genes? interaction and regulation. This technology has a great potential in obtaining a deep understanding of the functional organization of cells. This paper is concerned with improving the processes involved in the analysis of microarray image data. The main focus is to clarify an image?s feature space in an unsupervised manner. Rather than using the raw microarray image, it proposes to produce filtered versions of the image data by applying nonlinear anisotropic diffusion so that the dynamic range of the image could be increased and, therefore, a better ability of signal extraction could be achieved. In this paper, a novel segmentation algorithm is developed. This algorithm is based on the Cellular Neural Network computational paradigm integrated with median and anisotropic diffusion filters. The AnaLogic CNN Simulation Toolbox for MATLAB (InstantVision Toolboxes for MATLAB) is used during the segmentation process. Quantitative comparisons among the proposed methods and GenePix® are carried out from both the objective and subjective points of view. It is shown that analogic algorithm integrated with Complex Diffusion filter is the best one to be applied to accomplish the segmentation. [ABSTRACT FROM AUTHOR]

In this paper existence and asymptotic stability of asymptotically almost automorphic solution of impulsive neural networks with delay is discussed. The results are established by using various fixed point theorems and Lyapunov-like functional. As far as we know, this is the first paper to discuss such kind of solutions for impulsive neural networks. At the end, we give few examples to illustrate our theoretical findings. One can see that the numerical simulation results show asymptotically almost automorphic behaviour of the solution. [ABSTRACT FROM AUTHOR]

In this paper a polynomial fuzzy regression model with fuzzy independent variables and fuzzy parameters is discussed. Within this paper the fuzzy neural network model is used to obtain an estimate for the fuzzy parameters in a statistical sense. Based on the extension principle, a simple algorithm from the cost function of the fuzzy neural network is proposed, in order to find the approximate parameters. Finally, we illustrate our approach by some numerical examples. [ABSTRACT FROM AUTHOR]

In this paper, an intelligent swarm based-wavelet neural network for affective mobile designed is presented. The contribution on this paper is to develop a new intelligent particle swarm optimization (iPSO), where a fuzzy logic system developed based on human knowledge is proposed to determine the inertia weight for the swarm movement of the PSO and the control parameter of a newly introduced cross-mutated operation. The proposed iPSO is used to optimize the parameters of wavelet neural network. An affective design of mobile phones is used to evaluate the effectiveness of the proposed iPSO. It has been found that significantly better results in a statistical sense can be obtained by the iPSO comparing with the existing hybrid PSO methods. [ABSTRACT FROM AUTHOR]

Deep neural networks (DNN), such as convolutional neural networks (CNN) have been widely used for object recognition. However, they are usually unable to ensure the required intra-class compactness and inter-class separability in the kernel space. These are known to be important in pattern recognition for achieving both robustness and accuracy. In this paper, we propose to integrate a kernelized Min-Max objective in the DNN training in order to explicitly enforce both kernelized within-class compactness and between-class margin. The involved kernel space is implicitly mapped from the feature space associated with a certain upper layer of DNN by exploiting a kernel trick, while the Min-Max objective in this space is interpolated with the original DNN loss function and finally optimized in the training phase. With a very small additional computation cost, the proposed strategy can be easily integrated in different DNN models without changing any other part of the original model. The comparative recognition accuracy of the proposed method is evaluated with multiple DNN models (including shallow CNN, deep CNN and deep residual neural network models) on two benchmark datasets: CIFAR-10 and CIFAR-100. Extensive experimental results demonstrate that the integration of kernelized Min-Max objective in the training of DNN models can achieve better results compared to state-of-the-art models, without incurring additional model complexity. [ABSTRACT FROM AUTHOR]

Neural networks are the main strength of modern artificial intelligence; they have demonstrated revolutionary performance in a wide range of applications. In practice, the weights of neural networks are generally obtained indirectly using iterative training methods. Such methods are inefficient and problematic in many respects. Besides, neural networks trained end-to-end by such methods are typical black box models that are hard to interpret. Thus, it would be significantly better if the weights of a neural network could be calculated directly. In this paper, we located the key for calculating the weights of a neural network directly: assigning proper targets to the hidden units. Furthermore, if such targets are assigned, the neural network becomes a white box model that is easy to interpret. Thus, we propose a framework for solving the weights of a neural network and provide a sample implementation of the framework. The implementation was tested in various classification and regression experiments. Compared with neural networks trained using traditional methods, the constructed ones using solved weights had similar or better performance on many tasks, while remaining interpretable. Given the early stage of the proposed approach, many improvements are expectable in future developments. [ABSTRACT FROM AUTHOR]

• A framework based on Bidirectional Gated Neural Networks is proposed for fault diagnosis with uncertainty in dynamic environments. • A Sample Sensitive Bidirectional Gated Neural Networks model is developed to tackle imbalanced fault diagnosis challenges. • Cost sensitive active learning is used to explore unlabeled data. • Effective methods are developed to address both binary Fault Diagnosis and multi-class Fault Diagnosis. Most existing fault diagnosis methods may fail in the following three scenarios: (1) serial correlations exist in the process data; (2) fault data are much less than normal data; and (3) it is impractical to obtain enough labeled data. In this paper, a novel form of the bidirectional gated recurrent unit (BGRU) is developed to underpin effective and efficient fault diagnosis using cost sensitive active learning. Specifically, BGRU is devised to consider the dynamic behavior of a complex process. In the training phase of BGRU, the idea of weighting each training example is proposed to reduce the effect of class imbalance. Besides, in order to explore the unlabeled data, cost sensitive active learning is utilized to select the candidate instances. The effectiveness of the proposed method is evaluated on the Tennessee Eastman (TE) dataset and a real plasma etching process dataset. The experiment results show that the proposed cost senstive active learning bidirectional gated recurrent unit (CSALBGRU) method achieves better performance in both binary fault diagnosis and multi-class fault diagnosis. [ABSTRACT FROM AUTHOR]

Discovering the relationship between two entities in a sentence is one of the most important tasks in natural language processing. Due to the superior feature learning capacity, deep learning-based relationship extraction methods have become prevailing. However, most of them rely on the features provided by tools such as dependency parser, part-of-speech tagger, or named entity recognizers. Moreover, although models such as convolutional neural networks or recurrent neural networks have their merits in text feature extraction, the semantic representation of entities' relationships are still far from optimal. To address this issue, this paper proposes a novel end-to-end multi-level semantic representation enhancement network (MLSREN). MLSREN can enhance the semantic representation of entities from word, phrase, and context level. Furthermore, to extract deeper semantic information of phrases with fewer network parameters, a LightText-CNN is proposed for local feature extraction. Extensive experiments are conducted on the SemEval-2010 task 8 dataset, the results demonstrate that our model significantly improves state-of-the-art performance. To better understand performance achieved by MLSREN, we also analyze the contribution of each network component as well as the problems in the feature selection of the current relationship extraction models. [ABSTRACT FROM AUTHOR]

• Structured artificial neural networks. • Propagation matrix. • PID controller. • Computational intelligence. • Bulk resumption process. The tuning of proportional–integral–derivative (PID) controllers has been extensively used in the industry, this adjustment is performed by means of conventional methods, such as the Ziegler–Nichols and trial-and-error that often fails to address inherent complexity of industrial processes. To overcome these problems of PID tuning, a neurocomputing adaptive novel methodology is presented in this paper. The proposed methodology is based on a structured artificial neural network (S-ANN) and a propagation matrix of PID actions that was developed to support the tuning. The problem formulation and S-ANN-based solution are tightly connected, i.e., the nodes and layers of S-ANN topology are ruled by the order of the propagation matrix. For given operational conditions of the plant and disturbance in its parameters, the performance of the trained network is evaluated for tasks within the scope of learning on a bulk resumption process in ore mining system, focusing the control system output accuracy and convergence of the S-ANN algorithm. [ABSTRACT FROM AUTHOR]

In this paper, we consider a four-neuron bi-directional associative memory (BAM, for short) neural network with two delays. We choose connection weights and the sum of delays as bifurcation parameters and derive the critical values where a double Hopf bifurcation may occur by analyzing the associated characteristic equation which is a fourth-degree polynomial exponential equation. Meanwhile, we obtain some parameter conditions on the existence of invariant 2-tori of the truncated normal form near the bifurcation point by the center manifold theorem and normal form method. Despite the fact that the higher-degree terms may destroy the invariant 2-tori of the truncated normal form, we prove that the neural network model has quasi-periodic invariant 2-tori for most of the parameter set where the truncated normal form possesses invariant 2-tori in a sufficiently small neighborhood of the bifurcation point. Numerical examples and simulations are given to support the theoretical analysis. [ABSTRACT FROM AUTHOR]

This paper proposes a deep convolutional neural network model for ordinal regression by considering a family of probabilistic ordinal link functions in the output layer. The link functions are those used for cumulative link models, which are traditional statistical linear models based on projecting each pattern into a 1-dimensional space. A set of ordered thresholds splits this space into the different classes of the problem. In our case, the projections are estimated by a non-linear deep neural network. To further improve the results, we combine these ordinal models with a loss function that takes into account the distance between the categories, based on the weighted Kappa index. Three different link functions are studied in the experimental study, and the results are contrasted with a statistical analysis. The experiments run over two different ordinal classification problems and the statistical tests confirm that these models improve the results of a nominal model and outperform other robust proposals considered in the literature. [ABSTRACT FROM AUTHOR]

In recent years, SiamFC-based trackers have received much attention because of their great potentials in balancing tracking accuracy and speed. However, the robustness of most such trackers is greatly affected by the large deformations of targets. We argue that in the cross-correlation operation which is widely used by modern SiamFC-based trackers, the static correlation between the template kernel and the feature maps of test sample is difficult to adapt to the large deformation of the target object. In this paper, we propose a Siamese deformable cross-correlation network (SiamDCN), which introduces the deformable cross-correlation operation into SiamFC in an online self-adaptive way, for robust visual tracking. Compared to the previous SiamFC-based trackers, our SiamDCN is more robust to the large deformations of targets through dynamically and adaptively adjusting the location of correlation calculation for each element of the template kernel in the cross-correlation operation. Moreover, we build a twofold Siamese network, named SiamDCN+, which consists of a SiamDCN branch and a SiamFC branch, for accurate and real-time visual tracking after observing that the features learned in SiamFC are static and discriminative, whereas those in SiamDCN are dynamic and robust, and they complement each other. Extensive experiments on three public benchmarks, OTB2015, VOT2016, and VOT2017, show that the proposed SiamDCN achieves superior localization accuracy than its baseline tracker SiamFC and the proposed SiamDCN+ achieves competitive performance compared to state-of-the-art real-time trackers, while running beyond 40 FPS. [ABSTRACT FROM AUTHOR]

Artificial neural networks experience serious catastrophic forgetting (or interference) when information is learned sequentially. A significant effort in the machine learning community is devoted to the solution of this problem. Many approaches to overcome the catastrophic interference (CI) find parallels with an organization of the human memory system. In this paper, we provide a review of biologically inspired approaches for CI prevention. The main emphasis is made on the development of methods inspired by generative properties of the brain. We developed and tested several methods for preventing CI using an artificial dataset generated on the base of previous experience of neural network. The proposed methods include the activation maximization approach, the method based on Bayesian learning, and the method based on generative neural networks. The methods based on a combination of episodic memory (several stored samples) and semantic memory (sampling of posterior probability function) show superiority compared to other recent methods devoted to CI prevention. Based on generative approaches, the biologically plausible mechanisms of active forgetting and memory reconsolidation are also demonstrated. The proof of concept experiments were performed on several publicly available datasets. [ABSTRACT FROM AUTHOR]

With the prosperity of the location-based social networks, next point-of-interest (POI) recommendation has become an increasingly significant requirement since it can benefit both users and business. Obtaining insight into user mobility for the next POI recommendations is a vital yet challenging task. Existing approaches to understanding user mobility mainly gloss over the check-in sequence, making it fail to explicitly capture the subtle POI–POI interactions across the entire user check-in history and distinguish relevant check-ins from the irrelevant. In this paper, we proposed a novel recommendation approach, namely geographical-temporal awareness hierarchical attention network (GT-HAN) to resolve those issues. We first establish a geographical-temporal attention network to simultaneously uncover the overall sequence dependence and the subtle POI–POI relationships. Then, a context-specific co-attention network was designed to learn to change user preferences by adaptively selecting relevant check-in activities from check-in histories, which enabled GT-HAN to distinguish degrees of user preference for different check-ins. Finally, we make a POI recommendation using a conditional probability distribution function. Experimental results on real world datasets (obtained from Foursquare and Gowalla) show that the GT-HAN model significantly outperforms current state-of-the-art approaches, and demonstrating the benefits produced by new technologies incorporated into GT-HAN. [ABSTRACT FROM AUTHOR]

Reinforcement learning (RL) of optimal policies against an opponent agent also with learning capability is still challenging in Markov games. A variety of algorithms have been proposed for solving this problem such as the traditional Q-learning-based RL (QbRL) algorithms as well as the state-of-the-art neural-network-based RL (NNbRL) algorithms. However, the QbRL approaches have poor generalization capability for complex problems with non-stationary opponents, while the learned policies by NNbRL algorithms are lack of explainability and transparency. In this paper, we propose an algorithm X-OMQ(λ) that integrates eXtended Classifier System (XCS) with opponent modelling for concurrent reinforcement learners in zero-sum Markov Games. The algorithm can learn general, accurate, and interpretable action selection rules and allow policy optimization using the genetic algorithm (GA). Besides, the X-OMQ(λ) agent optimizes the established opponent's model while simultaneously learning to select actions in a goal-directed manner. In addition, we use the eligibility trace mechanism to further speed up the learning process. In the reinforcement component, not only the classifiers in the action set are updated, but other relevant classifiers are also updated in a certain proportion. We demonstrate the performance of the proposed algorithm in the hunter prey problem and two adversarial soccer scenarios where the opponent is allowed to learn with several benchmark QbRL and NNbRL algorithms. The results show that our method has similar learning performance with the NNbRL algorithms while our method requires no prior knowledge of the opponent or the environment. Moreover, the learned action selection rules are also interpretable while having generalization capability. [ABSTRACT FROM AUTHOR]

A delayed cellular neural networks with Caputo fractional-order derivative has been discussed in this paper. Firstly, the existence and uniqueness of S -asymptotically ω -periodic oscillation of the model are investigated by some important features of Mittag-Leffler functions and contraction mapping principle. Secondly, global asymptotical stability of the model is also studied by using Laplace transform, comparison principle and stability theorem of linear delayed Caputo fractional-order differential equations. Some better results are derived to improve and extend a few existing research findings. The research thoughts in this literature could be applied to research other fractional-order models in neural networks and physical areas. [ABSTRACT FROM AUTHOR]

Recently, many machine learning and statistical models such as non-linear regressions, the Single Index, Multi-index, Varying Coefficient Index Models and Two-layer Neural Networks can be reduced to or be seen as a special case of a new model which is called the Stochastic Linear Combination of Non-linear Regressions model. However, due to the high non-convexity of the problem, there is no previous work study how to estimate the model. In this paper, we provide the first study on how to estimate the model efficiently and scalably. Specifically, we first show that with some mild assumptions, if the variate vector x is multivariate Gaussian, then there is an algorithm whose output vectors have ℓ 2 -norm estimation errors of O (p n) with high probability, where p is the dimension of x and n is the number of samples. The key idea of the proof is based on an observation motived by the Stein's lemma. Then we extend our result to the case where x is bounded and sub-Gaussian using the zero-bias transformation, which could be seen as a generalization of the classic Stein's lemma. We also show that with some additional assumptions there is an algorithm whose output vectors have ℓ ∞ -norm estimation errors of O (1 p + p n) with high probability. We also provide a concrete example to show that there exists some link function which satisfies the previous assumptions. Finally, for both Gaussian and sub-Gaussian cases we propose a faster sub-sampling based algorithm and show that when the sub-sample sizes are large enough then the estimation errors will not be sacrificed by too much. Experiments for both cases support our theoretical results. To the best of our knowledge, this is the first work that studies and provides theoretical guarantees for the stochastic linear combination of non-linear regressions model. [ABSTRACT FROM AUTHOR]

Functional link artificial neural network (FLANN) has received much attention due to its wide applicability. The Van der Pol-Duffing oscillator (VdPDO)-based nonlinear systems, which own complex dynamical behaviors, identification of such nonlinear model is vital. This paper exploits the nonlocality of fractional calculus, aiming to enhance the identification accuracy of the VdPDO-based nonlinear systems. The proposed combined FEM-LMS (CFEM-LMS) algorithm, which is based on the FLANN structure, convexly combines the least mean square (LMS) algorithm and the newly proposed fractional-order error modified LMS (FEM-LMS) algorithm. The CFEM-LMS algorithm has improved performance and can dynamically adapt to the nonlinearity of the system. As an added contribution, a novel mixing parameter adaptation criterion is proposed for performance improvement. Extensive simulation results in the context of VdPDO-based nonlinear system identification demonstrate the superiority of the proposed algorithm as compare to state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Knowledge Graphs (KGs) have been applied to various application scenarios including Web searching, Q&A, recommendation system, natural language processing and so on. However, the vast majority of Knowledge Bases (KBs) are incomplete, necessitating a demand for KB completion (KBC). Methods of KBC used in the mainstream current knowledge base include the latent factor model, the random walk model and recent popular methods based on reinforcement learning, which performs well in their respective areas of expertise. Recurrent neural network (RNN) and its variants model temporal data by remembering information for long periods, however, whether they also have the ability to use the information they have already remembered to achieve complex reasoning in the knowledge graph. In this paper, we produce a novel framework (ALSTM) based on the Attention mechanism and Long Short-Term Memory (LSTM), which associates structure learning with parameter learning of first-order logical rules in an end-to-end differentiable neural networks model. In this framework, we designed a memory system and employed a multi-head dot product attention (MHDPA) to interact and update the memories embedded in the memory system for reasoning purposes. This is also consistent with the process of human cognition and reasoning, looking for enlightenment for the future in historical memory. In addition, we explored the use of inductive bias in deep learning to facilitate learning of entities, relations, and rules. Experiments establish the efficiency and effectiveness of our model and show that our method achieves better performance in tasks which include fact prediction and link prediction than baseline models on several benchmark datasets such as WN18RR, FB15K-237 and NELL-995. [ABSTRACT FROM AUTHOR]

This work presents a novel paradigmatic revision of traditional neural networks, using network theoretic methods and Conformal Geometric Algebra. A unique theoretical framework called the 'hyperfield cognition framework' expands upon the mathematical foundations of neural networks in five-dimensional Conformal Geometric Algebraic space. This framework allows one to construct a novel theoretical computational engine, which is similar to a standard artificial neural network, but admits numerous added benefits: permits multiple training programs simultaneously, affords computational multiplicity in a single engine, reduces sensitivity to adversarial perturbations in training sets, affords broader capabilities and plasticity in the training of networks and produces robust and streamlined 'neural networks'. We call this novel five-dimensional neural network a 'hyperfield cognition network' (HCN). This paper demonstrates the utility and merit of the proposed Hyperfield Cognition Network by presenting two case studies, the first of which investigates the fuel efficiency of various automobiles and the second of which models the residuary resistance per unit weight of displacement of ships. [ABSTRACT FROM AUTHOR]

In this paper, we present a new granular classifier in two versions (iterative and non–iterative), by adopting some ideas originating from a kind of Functional Link Artificial Neural Network and the Functional Network schemes. These two architectures are substantially the same: they both use a function basis instead of the usual activation function, but they are different for the learning algorithm. We augment them from the perspective of Granular Computing and information granules, designing a new kind of classifier and two learning algorithms, by taking into account granularity of information. The proposed classifier exhibits the advantages of the granular architectures, that is higher accuracy and transparency. We formally discuss the convergence of the iterative learning scheme. We carry out some numerical experiments using publicly available data, by comparing the results against those results produced by the state-of-the-art methods. In particular, we achieved sound results by invoking the iterative learning scheme. [ABSTRACT FROM AUTHOR]