Search

Abstract: Language understanding is a long-standing problem in computer science. However, the human brain is capable of processing complex languages with seemingly no difficulties. This paper shows a model for language understanding using biologically plausible neural networks composed of associative memories. The model is able to deal with ambiguities on the single word and grammatical level. The language system is embedded into a robot in order to demonstrate the correct semantical understanding of the input sentences by letting the robot perform corresponding actions. For that purpose, a simple neural action planning system has been combined with neural networks for visual object recognition and visual attention control mechanisms. [Copyright &y& Elsevier]

Abstract: In this paper, we propose the concept of a manifold of color perception through empirical observation that the center-surround properties of images in a perceptually similar environment define a manifold in the high dimensional space. Such a manifold representation can be learned using a novel recurrent neural network based learning algorithm. Unlike the conventional recurrent neural network model in which the memory is stored in an attractive fixed point at discrete locations in the state space, the dynamics of the proposed learning algorithm represent memory as a nonlinear line of attraction. The region of convergence around the nonlinear line is defined by the statistical characteristics of the training data. This learned manifold can then be used as a basis for color correction of the images having different color perception to the learned color perception. Experimental results show that the proposed recurrent neural network learning algorithm is capable of color balance the lighting variations in images captured in different environments successfully. [Copyright &y& Elsevier]

Abstract: This study aims to develop a generalized framework of an SOM called a modular network SOM (mnSOM). The mnSOM has an array structure consisting of functional modules that are trainable neural networks, e.g., multi-layer perceptrons (MLPs), instead of the vector units of the conventional SOM. In the case of MLP-modules, an mnSOM learns a group of systems or functions in terms of the input–output relationships in parallel with generating a feature map of them. Thus an mnSOM with MLP modules is an SOM in function space rather than in vector space. In this paper, first, as an example, we focus on a class of mnSOM that consists of MLP modules and introduce the architecture and algorithm. Then, a more generalized framework is described. Finally, some simulation results of an MLP-module-mnSOM are presented. [Copyright &y& Elsevier]

Abstract: This paper is concerned with the state estimation problem for a class of Markovian neural networks with discrete and distributed time-delays. The neural networks have a finite number of modes, and the modes may jump from one to another according to a Markov chain. The main purpose is to estimate the neuron states, through available output measurements, such that for all admissible time-delays, the dynamics of the estimation error is globally asymptotically stable in the mean square. An effective linear matrix inequality approach is developed to solve the neuron state estimation problem. Both the existence conditions and the explicit characterization of the desired estimator are derived. Furthermore, it is shown that the traditional stability analysis issue for delayed neural networks with Markovian jumping parameters can be included as a special case of our main results. Finally, numerical examples are given to illustrate the applicability of the proposed design method. [Copyright &y& Elsevier]

Abstract: This paper is concerned with the stability analysis problem for a new class of discrete-time recurrent neural networks with mixed time-delays. The mixed time-delays that consist of both the discrete and distributed time-delays are addressed, for the first time, when analyzing the asymptotic stability for discrete-time neural networks. The activation functions are not required to be differentiable or strictly monotonic. The existence of the equilibrium point is first proved under mild conditions. By constructing a new Lyapnuov–Krasovskii functional, a linear matrix inequality (LMI) approach is developed to establish sufficient conditions for the discrete-time neural networks to be globally asymptotically stable. As an extension, we further consider the stability analysis problem for the same class of neural networks but with state-dependent stochastic disturbances. All the conditions obtained are expressed in terms of LMIs whose feasibility can be easily checked by using the numerically efficient Matlab LMI Toolbox. A simulation example is presented to show the usefulness of the derived LMI-based stability condition. [Copyright &y& Elsevier]

Abstract: In this paper, a novel approach termed Enhanced Dynamic Self-Generated Fuzzy Q-Learning (EDSGFQL) for automatically generating Fuzzy Inference Systems (FISs) is presented. In the EDSGFQL approach, structure identification and parameter estimations of FISs are achieved via Unsupervised Learning (UL) (including Reinforcement Learning (RL)). Instead of using Supervised Learning (SL), UL clustering methods are adopted for input space clustering when generating FISs. At the same time, structure and preconditioning parts of a FIS are generated in a RL manner in that fuzzy rules are adjusted and deleted according to reinforcement signals. The proposed EDSGFQL methodologies can automatically create, delete and adjust fuzzy rules dynamically. Simulation studies on wall-following and obstacle avoidance tasks by a mobile robot show that the proposed approach is superior in generating efficient FISs. [Copyright &y& Elsevier]

Abstract: Understanding the design principle of reward functions is a substantial challenge both in artificial intelligence and neuroscience. Successful acquisition of a task usually requires not only rewards for goals, but also for intermediate states to promote effective exploration. This paper proposes a method for designing ‘intrinsic’ rewards of autonomous agents by combining constrained policy gradient reinforcement learning and embodied evolution. To validate the method, we use Cyber Rodent robots, in which collision avoidance, recharging from battery packs, and ‘mating’ by software reproduction are three major ‘extrinsic’ rewards. We show in hardware experiments that the robots can find appropriate ‘intrinsic’ rewards for the vision of battery packs and other robots to promote approach behaviors. [Copyright &y& Elsevier]

Abstract: This paper proposes a novel learning method for a mixture of recurrent neural network (RNN) experts model, which can acquire the ability to generate desired sequences by dynamically switching between experts. Our method is based on maximum likelihood estimation, using a gradient descent algorithm. This approach is similar to that used in conventional methods; however, we modify the likelihood function by adding a mechanism to alter the variance for each expert. The proposed method is demonstrated to successfully learn Markov chain switching among a set of 9 Lissajous curves, for which the conventional method fails. The learning performance, analyzed in terms of the generalization capability, of the proposed method is also shown to be superior to that of the conventional method. With the addition of a gating network, the proposed method is successfully applied to the learning of sensory–motor flows for a small humanoid robot as a realistic problem of time series prediction and generation. [Copyright &y& Elsevier]

Abstract: In this paper, the asymptotic stability is investigated for a class of cellular neural networks with interval time-varying delay (that is, ). By introducing a novel Lyapunov functional with the idea of partitioning the lower bound of the time-varying delay, a new criterion of asymptotic stability is derived in terms of a linear matrix inequality (LMI), which can be efficiently solved via standard numerical software. The criterion proves to be less conservative than most of the existing results, and the conservatism could be notably reduced by thinning the delay partitioning. Two examples are provided to demonstrate the less conservatism and effectiveness of the proposed stability conditions. [Copyright &y& Elsevier]

Abstract: This paper introduces an approach called Clustering and Co-evolution to Construct Neural Network Ensembles (CONE). This approach creates neural network ensembles in an innovative way, by explicitly partitioning the input space through a clustering method. The clustering method allows a reduction in the number of nodes of the neural networks that compose the ensemble, thus reducing the execution time of the learning process. This is an important characteristic especially when evolutionary algorithms are used. The clustering method also ensures that different neural networks specialize in different regions of the input space, working in a divide-and-conquer way, to maintain and improve the accuracy. Besides, the clustering method facilitates the understanding of the system and makes a straightforward distributed implementation possible. The experiments performed with seven classification databases and three different co-evolutionary algorithms show that CONE considerably reduces the execution time without prejudicing (and even improving) the accuracy, even when a distributed implementation is not used. [Copyright &y& Elsevier]

Abstract: This paper presents a new online clustering algorithm called SAKM (Self-Adaptive Kernel Machine) which is developed to learn continuously evolving clusters from non-stationary data. Based on SVM and kernel methods, the SAKM algorithm uses a fast adaptive learning procedure to take into account variations over time. Dedicated to online clustering in a multi-class environment, the algorithm designs an unsupervised neural architecture with self-adaptive abilities. Based on a specific kernel-induced similarity measure, the SAKM learning procedures consist of four main stages: Creation, Adaptation, Fusion and Elimination. In addition to these properties, the SAKM algorithm is attractive to be computationally efficient in online learning of real-drifting targets. After a theoretical study of the error convergence bound of the SAKM local learning, a comparison with NORMA and ALMA algorithms is made. In the end, some experiments conducted on simulation data, UCI benchmarks and real data are given to illustrate the capacities of the SAKM algorithm for online clustering in non-stationary and multi-class environment. [Copyright &y& Elsevier]

Abstract: The purpose of this paper is to present some relevant theoretical results on the asymptotic behaviour of finite neural networks (on lattices) when they are subjected to fixed boundary conditions. This work focuses on two different topics of interest from the biological point of view. First, it exhibits a link between the possible updating iteration modes in these networks, whatever the number of dimensions is. It proves that the effects of boundary conditions on neural networks do not depend on the updating iteration mode under the hypothesis of synaptic weight symmetry. Thus, if the asymptotic behaviour admits phase transitions, these phase transitions are observable for many updating iteration modes (from synchrony to asynchrony). Then, it shows that boundaries have no significant impact on one-dimensional neural networks. In order to prove this property, we present a new general mathematical approach based on the use of a projectivity matrix in order to simplify the problem. This approach allows the theoretical study of the asymptotic dynamics and of the boundary influence in neural networks. We will also introduce the numerical tools generalising the method in order to study phase transitions in more complex cases. [Copyright &y& Elsevier]

Abstract: This paper gives new simulation results on the asymptotic behaviour of theoretical neural networks on and following an extended Hopfield law. It specifically focuses on the influence of fixed boundary conditions on such networks. First, we will generalise the theoretical results already obtained for attractive networks in one dimension to more complicated neural networks. Then, we will focus on two-dimensional neural networks. Theoretical results have already been found for the nearest neighbours Ising model in 2D with translation-invariant local isotropic interactions. We will detail what happens for this kind of interaction in neural networks and we will also focus on more complicated interactions, i.e., interactions that are not local, neither isotropic, nor translation-invariant. For all these kinds of interactions, we will show that fixed boundary conditions have significant impacts on the asymptotic behaviour of such networks. These impacts result in the emergence of phase transitions whose geometric shape will be numerically characterised. [Copyright &y& Elsevier]

Abstract: The radial basis function (RBF) networks are one of the most widely used models for function approximation in the regression problem. In the learning paradigm, the best approximation is recursively or iteratively searched for based on observed data (teacher signals). One encounters difficulties in such a process when two component basis functions become identical, or when the magnitude of one component becomes null. In this case, the number of the components reduces by one, and then the reduced component recovers as the learning process proceeds further, provided such a component is necessary for the best approximation. Strange behaviors, especially the plateau phenomena, have been observed in dynamics of learning when such reduction occurs. There exist singularities in the space of parameters, and the above reduction takes place at the singular regions. This paper focuses on a detailed analysis of the dynamical behaviors of learning near the overlap and elimination singularities in RBF networks, based on the averaged learning equation that is applicable to both on-line and batch mode learning. We analyze the stability on the overlap singularity by solving the eigenvalues of the Hessian explicitly. Based on the stability analysis, we plot the analytical dynamic vector fields near the singularity, which are then compared to those real trajectories obtained by a numeric method. We also confirm the existence of the plateaus in both batch and on-line learning by simulation. [Copyright &y& Elsevier]

Abstract: This paper proposes a GRG (Greedy Rule Generation) algorithm, a new method for generating classification rules from a data set with discrete attributes. The algorithm is “greedy” in the sense that at every iteration, it searches for the best rule to generate. The criteria for the best rule include the number of samples and the size of subspaces that it covers, as well as the number of attributes in the rule. This method is employed for extracting rules from neural networks that have been trained and pruned for solving classification problems. The classification rules are extracted from the neural networks using the standard decompositional approach. Neural networks with one hidden layer are trained and the proposed GRG algorithm is applied to their discretized hidden unit activation values. Our experimental results show that neural network rule extraction with the GRG method produces rule sets that are accurate and concise. Application of GRG directly on three medical data sets with discrete attributes also demonstrates its effectiveness for rule generation. [Copyright &y& Elsevier]

Abstract: We study the framework of Clifford algebra for the design of neural architectures capable of processing different geometric entities. The benefits of this model-based computation over standard real-valued networks are demonstrated. One particular example thereof is the new class of so-called Spinor Clifford neurons. The paper provides a sound theoretical basis to Clifford neural computation. For that purpose the new concepts of isomorphic neurons and isomorphic representations are introduced. A unified training rule for Clifford MLPs is also provided. The topic of activation functions for Clifford MLPs is discussed in detail for all two-dimensional Clifford algebras for the first time. [Copyright &y& Elsevier]

Abstract: Most of the existing research on multivariate time series concerns supervised forecasting problems. In comparison, little research has been devoted to their exploration through unsupervised clustering and visualization. In this paper, the capabilities of Generative Topographic Mapping Through Time, a model with foundations in probability theory, that performs simultaneous time series clustering and visualization, are assessed in detail. Focus is placed on the visualization of the evolution of signal regimes and the exploration of sudden transitions, for which a novel identification index is defined. The interpretability of time series clustering results may become extremely difficult, even in exploratory visualization, for high dimensional datasets. Here, we define and test an unsupervised time series relevance determination method, fully integrated in the Generative Topographic Mapping Through Time model, that can be used as a basis for time series selection. This method should ease the interpretation of time series clustering results. [Copyright &y& Elsevier]

Abstract: Over successive stages, the visual system develops neurons that respond with view, size and position invariance to objects or faces. A number of computational models have been developed to explain how transform-invariant cells could develop in the visual system. However, a major limitation of computer modelling studies to date has been that the visual stimuli are typically presented one at a time to the network during training. In this paper, we investigate how vision models may self-organize when multiple stimuli are presented together within each visual image during training. We show that as the number of independent stimuli grows large enough, standard competitive neural networks can suddenly switch from learning representations of the multi-stimulus input patterns to representing the individual stimuli. Furthermore, the competitive networks can learn transform (e.g. position or view) invariant representations of the individual stimuli if the network is presented with input patterns containing multiple transforming stimuli during training. Finally, we extend these results to a multi-layer hierarchical network model (VisNet) of the ventral visual system. The network is trained on input images containing multiple rotating 3D objects. We show that the network is able to develop view-invariant representations of the individual objects. [Copyright &y& Elsevier]

Abstract: Many real life problems require the classification of items into naturally ordered classes. These problems are traditionally handled by conventional methods intended for the classification of nominal classes where the order relation is ignored. This paper introduces a new machine learning paradigm intended for multi-class classification problems where the classes are ordered. The theoretical development of this paradigm is carried out under the key idea that the random variable class associated with a given query should follow a unimodal distribution. In this context, two approaches are considered: a parametric, where the random variable class is assumed to follow a specific discrete distribution; a nonparametric, where the random variable class is assumed to be distribution-free. In either case, the unimodal model can be implemented in practice by means of feedforward neural networks and support vector machines, for instance. Nevertheless, our main focus is on feedforward neural networks. We also introduce a new coefficient, , to measure the performance of ordinal data classifiers. An experimental study with artificial and real datasets is presented in order to illustrate the performances of both parametric and nonparametric approaches and compare them with the performances of other methods. The superiority of the parametric approach is suggested, namely when flexible discrete distributions, a new concept introduced here, are considered. [Copyright &y& Elsevier]

Abstract: In the application of variable structure control to engine air–fuel ratio, the ratio is subjected to chattering due to system uncertainty, such as unknown parameters or time varying dynamics. This paper proposes an adaptive neural network method to estimate two immeasurable physical parameters on-line and to compensate for the model uncertainty and engine time varying dynamics, so that the chattering is substantially reduced and the air–fuel ratio is regulated within the desired range of the stoichiometric value. The adaptive law of the neural network is derived using the Lyapunov method, so that the stability of the whole system and the convergence of the networks are guaranteed. Computer simulations based on a mean value engine model demonstrate the effectiveness of the technique. [Copyright &y& Elsevier]

Abstract: This paper investigates the relation between over-fitting and weight size in neural network regression. The over-fitting of a network to Gaussian noise is discussed. Using re-parametrization, a network function is represented as a bounded function multiplied by a coefficient . This is considered to bound the squared sum of the outputs of at given inputs away from a positive constant , which restricts the weight size of a network and enables the probabilistic upper bound of the degree of over-fitting to be derived. This reveals that the order of the probabilistic upper bound can change depending on . By applying the bound to analyze the over-fitting behavior of one Gaussian unit, it is shown that the probability of obtaining an extremely small value for the width parameter in training is close to one when the sample size is large. [Copyright &y& Elsevier]

Abstract: The paper presents theoretical results on the global exponential periodicity and global exponential stability of a class of recurrent neural networks with various general activation functions and time-varying delays. The general activation functions include monotone nondecreasing functions, globally Lipschitz continuous and monotone nondecreasing functions, semi-Lipschitz continuous mixed monotone functions, and Lipschitz continuous functions. For each class of activation functions, testable algebraic criteria for ascertaining global exponential periodicity and global exponential stability of a class of recurrent neural networks are derived by using the comparison principle and the theory of monotone operator. Furthermore, the rate of exponential convergence and bounds of attractive domain of periodic oscillations or equilibrium points are also estimated. The convergence analysis based on the generalization of activation functions widens the application scope for the model design of neural networks. In addition, the new effective analytical method enriches the toolbox for the qualitative analysis of neural networks. [Copyright &y& Elsevier]

Abstract: This paper focuses on the evolution of Fuzzy ARTMAP neural network classifiers, using genetic algorithms, with the objective of improving generalization performance (classification accuracy of the ART network on unseen test data) and alleviating the ART category proliferation problem (the problem of creating more than necessary ART network categories to solve a classification problem). We refer to the resulting architecture as GFAM. We demonstrate through extensive experimentation that GFAM exhibits good generalization and is of small size (creates few ART categories), while consuming reasonable computational effort. In a number of classification problems, GFAM produces the optimal classifier. Furthermore, we compare the performance of GFAM with other competitive ARTMAP classifiers that have appeared in the literature and addressed the category proliferation problem in ART. We illustrate that GFAM produces improved results over these architectures, as well as other competitive classifiers. [Copyright &y& Elsevier]

Abstract: In this paper, we investigate the existence and stability of the equilibrium point of Cohen–Grossberg neural networks with time-varying delays. Under easily verified conditions, exponential stability is obtained when the delay is finite, while asymptotic stability is obtained when the delay is infinite. Moreover, the stability obtained is robust. The only condition for the delay term is continuity. The results obtained here improve and extend to those in the literature. [Copyright &y& Elsevier]

Abstract: This paper is concerned with the problem of robust stability for stochastic interval delayed additive neural networks (SIDANN) with Markovian switching. The time delay is assumed to be time-varying. In such neural networks, the features of stochastic systems, interval systems, time-varying delay systems and Markovian switching are taken into account. The mathematical model of this kind of neural networks is first proposed. Secondly, the global exponential stability in the mean square is studied for the SIDANN with Markovian switching. Based on the Lyapunov method, several stability conditions are presented, which can be expressed in terms of linear matrix inequalities. As a subsequent result, the stochastic interval additive neural networks with time-varying delay are also discussed. A sufficient condition is given to determine its stability. Finally, two simulation examples are provided to illustrate the effectiveness of the results developed. [Copyright &y& Elsevier]

Abstract: Competitive learning neural networks are regarded as a powerful tool for online data clustering to represent a non-stationary probability distribution with a fixed number of weight vectors. One difficulty in practical applications of competitive learning neural networks to online data clustering is that most of them require heuristically-predetermined threshold parameters for balancing a trade-off between convergence accuracy, i.e. error minimization performance, and speed of adaptation to the changes in source statistics. Although adaptation acceleration is achievable by relocating a “useless” node so that it becomes useful, excessive relocation often disturbs error minimization. Hence, both of the adaptation speed and the error minimization performance sensitively depend on threshold parameters to determine whether a node should be relocated or not. In general, it is difficult to know adequate threshold parameters a priori. This paper proposes a novel criterion for decision making of node relocation without heuristically predetermined thresholds. According to the proposed criterion, a node is relocated only if the relocation task improves partial distortion entropy, which is an online optimality metric reliable from the viewpoint of error minimization. Hence, node relocation is carried out without disturbing error minimization. As a result, both quick adaptation and error minimization are simultaneously accomplished without any carefully predefined parameters. Experimental results clarify the validity of the proposed criterion. Competitive learning with the criterion is clearly superior to other representative algorithms in terms of both quick adaptation and error minimization performance. [Copyright &y& Elsevier]

Abstract: A novel significant vector (SV) regression algorithm is proposed in this paper based on an analysis of Chen’s orthogonal least squares (OLS) regression algorithm. The proposed regularized SV algorithm finds the significant vectors in a successive greedy process in which, compared to the classical OLS algorithm, the orthogonalization has been removed from the algorithm. The performance of the proposed algorithm is comparable to the OLS algorithm while it saves a lot of time complexities in implementing the orthogonalization needed in the OLS algorithm. [Copyright &y& Elsevier]

Abstract: In this paper, a linear matrix inequality (LMI) to global asymptotic stability of the delayed Cohen–Grossberg neural network is investigated by means of nonsmooth analysis. Several new sufficient conditions are presented to ascertain the uniqueness of the equilibrium point and the global asymptotic stability of the neural network. It is noted that the results herein require neither the smoothness of the behaved function, or the activation function nor the boundedness of the activation function. In addition, from theoretical analysis, it is found that the condition for ensuring the global asymptotic stability of the neural network also implies the uniqueness of equilibrium. The obtained results improve many earlier ones and are easy to apply. Some simulation results are shown to substantiate the theoretical results. [Copyright &y& Elsevier]

Abstract: In this paper, without assuming strict positivity of amplifier functions, boundedness of activation functions, or symmetry of the connection matrix, dynamical behaviors of delayed Cohen–Grossberg neural networks with nonnegative equilibria are studied. Based on the theory of the nonlinear complementary problem (NCP), a sufficient condition is derived guaranteeing the existence and uniqueness of the nonnegative equilibrium in the NCP sense. Moreover, this condition also guarantees the -global asymptotic stability of the nonnegative equilibrium in the first orthant. The result is compared with some previous results and numerical examples are presented to indicate the viability of our theoretical analysis. [Copyright &y& Elsevier]

Abstract: Natural phenomena are multistationary and are composed of a number of interacting processes, so one single model handling all processes often suffers from inaccuracies. A solution is to partition data in relation to such processes using the available domain knowledge or expert judgment, to train separate models for each of the processes, and to merge them in a modular model (committee). In this paper a problem of water flow forecast in watershed hydrology is considered where the flow process can be presented as consisting of two subprocesses — base flow and excess flow, so that these two processes can be separated. Several approaches to data separation techniques are studied. Two case studies with different forecast horizons are considered. Parameters of the algorithms responsible for data partitioning are optimized using genetic algorithms and global pattern search. It was found that modularization of ANN models using domain knowledge makes models more accurate, if compared with a global model trained on the whole data set, especially when forecast horizon (and hence the complexity of the modelled processes) is increased. [Copyright &y& Elsevier]

Abstract: An environmentally adaptive system for prediction of acoustic transmission loss (TL) in the atmosphere is developed in this paper. This system uses several back propagation neural network predictors, each corresponding to a specific environmental condition. The outputs of the expert predictors are combined using a fuzzy confidence measure and a nonlinear fusion system. Using this prediction methodology the computational intractability of traditional acoustic model-based approaches is eliminated. The proposed TL prediction system is tested on two synthetic acoustic data sets for a wide range of geometrical, source and environmental conditions including both nonturbulent and turbulent atmospheres. Test results of the system showed root mean square (RMS) errors of 1.84 dB for the nonturbulent and 1.36 dB for the turbulent conditions, respectively, which are acceptable levels for near real-time performance. Additionally, the environmentally adaptive system demonstrated improved TL prediction accuracy at high frequencies and large values of horizontal separation between source and receiver. [Copyright &y& Elsevier]

Abstract: Synoptic downscaling models are used in climatology to predict values of weather elements at one or more stations based on values of synoptic-scale atmospheric circulation variables. This paper presents a hybrid method for climate prediction and downscaling that couples an analog, i.e., -nearest neighbor, model to an artificial neural network (ANN) model. In the proposed method, which is based on nonlinear principal predictor analysis (NLPPA), the analog model is embedded inside an ANN, forming its output layer. Nonlinear analog predictor analysis (NLAPA) is a flexible model that maintains the ability of the analog model to preserve inter-variable relationships and model non-normal and conditional variables (such as precipitation), while taking advantage of NLPPA’s ability to define an optimal set of analog predictors that maximize predictive performance. Performance on both synthetic and real-world hydroclimatological benchmark tasks indicates that the NLAPA model is capable of outperforming other forms of analog models commonly used in synoptic downscaling. [Copyright &y& Elsevier]

Abstract: With deregulation and growth of the power industry, many power system elements such as generators, transmission lines, are driven to operate near their maximum capacity, especially those serving heavy load centres. Wide Area Controllers (WACs) using wide area or global signals can provide remote auxiliary control signals to local controllers such as automatic voltage regulators, power system stabilizers, etc. to damp out system oscillations. However, since the power system is highly nonlinear with fast changing dynamics, it is a challenging problem to design an online system monitor/estimator, which can provide dynamic intra-area and inter-area information such speed deviations of generators to an adaptive WAC continuously. This paper presents a new kind of recurrent neural networks, called the Echo State Network (ESN), for the online design of a Wide Area Monitor (WAM) for a multimachine power system. A single ESN is used to predict the speed deviations of four generators in two different areas. The performance of this ESN WAM is evaluated for small and large disturbances on the power system. Results for an ESN based WAM and a Time-Delayed Neural Network (TDNN)-based WAM are presented and compared. The advantages of the ESN WAM are that it learns the dynamics of the power system in a shorter training time with a higher accuracy and with considerably fewer weights to be adapted compared to the design-based on a TDNN. [Copyright &y& Elsevier]

Abstract: Fuzzy ARTMAP neural networks have been proven to be good classifiers on a variety of classification problems. However, the time that Fuzzy ARTMAP takes to converge to a solution increases rapidly as the number of patterns used for training is increased. In this paper we examine the time Fuzzy ARTMAP takes to converge to a solution and we propose a coarse grain parallelization technique, based on a pipeline approach, to speed-up the training process. In particular, we have parallelized Fuzzy ARTMAP without the match-tracking mechanism. We provide a series of theorems and associated proofs that show the characteristics of Fuzzy ARTMAP’s, without matchtracking, parallel implementation. Results run on a BEOWULF cluster with three large databases show linear speedup as a function of the number of processors used in the pipeline. The databases used for our experiments are the Forrest CoverType database from the UCI Machine Learning repository and two artificial databases, where the data generated were 16-dimensional Gaussian distributed data belonging to two distinct classes, with different amounts of overlap (5% and 15%). [Copyright &y& Elsevier]

Abstract: This paper presents new necessary and sufficient conditions for absolute stability of asymmetric neural networks. The main result is based on a solvable Lie algebra condition, which generalizes existing results for symmetric and normal neural networks. An exponential convergence estimate of the neural networks is also obtained. Further, it is demonstrated how to generate larger sets of weight matrices for absolute stability of the neural networks from known normal weight matrices through simple procedures. The approach is nontrivial in the sense that non-normal matrices can possibly be contained in the resulting weight matrix set. And the results also provide finite checking for robust stability of neural networks in the presence of parameter uncertainties. [Copyright &y& Elsevier]

Abstract: An important drawback of many artificial neural networks (ANN) is their lack of explanation capability [Andrews, R., Diederich, J., & Tickle, A. B. (1996). A survey and critique of techniques for extracting rules from trained artificial neural networks. Knowledge-Based Systems, 8, 373–389]. This paper starts with a survey of algorithms which attempt to explain the ANN output. We then present HYPINV, 1 [1] HYPINV stands for an algorithm which extracts HYPerplanes using INVersion. a new explanation algorithm which relies on network inversion; i.e. calculating the ANN input which produces a desired output. HYPINV is a pedagogical algorithm, that extracts rules, in the form of hyperplanes. It is able to generate rules with arbitrarily desired fidelity, maintaining a fidelity–complexity tradeoff. To our knowledge, HYPINV is the only pedagogical rule extraction method, which extracts hyperplane rules from continuous or binary attribute neural networks. Different network inversion techniques, involving gradient descent as well as an evolutionary algorithm, are presented. An information theoretic treatment of rule extraction is presented. HYPINV is applied to example synthetic problems, to a real aerospace problem, and compared with similar algorithms using benchmark problems. [Copyright &y& Elsevier]

Abstract: This paper is concerned with analysis problem for the global exponential stability of a class of recurrent neural networks (RNNs) with mixed discrete and distributed delays. We first prove the existence and uniqueness of the equilibrium point under mild conditions, assuming neither differentiability nor strict monotonicity for the activation function. Then, by employing a new Lyapunov–Krasovskii functional, a linear matrix inequality (LMI) approach is developed to establish sufficient conditions for the RNNs to be globally exponentially stable. Therefore, the global exponential stability of the delayed RNNs can be easily checked by utilizing the numerically efficient Matlab LMI toolbox, and no tuning of parameters is required. A simulation example is exploited to show the usefulness of the derived LMI-based stability conditions. [Copyright &y& Elsevier]

Abstract: In the past few decades, neural networks have been extensively adopted in various applications ranging from simple synaptic memory coding to sophisticated pattern recognition problems such as scene analysis and robot vision. Moreover, current studies on neuroscience and physiology have reported that in a typical scene segmentation problem our major senses of perception (e.g. vision, olfaction, etc.) are highly chaotic and involved non-linear neural dynamics and oscillations. In this paper, the author proposes an innovative chaotic neural oscillator—namely the Lee-oscillator (Lee''s Chaotic Neural Oscillator) to provide a chaotic neural coding and information processing scheme. To illustrate the capability of Lee-oscillators upon pattern association, a chaotic auto-associative network, namely Lee-Associator (Lee''s Chaotic Auto-associator) is constructed. Different from classical auto-associators such as the celebrated Hopfield network, which provides time-independent and static pattern association scheme, the Lee-Associator provides a remarkable progressive memory association scheme (what the author called ‘Progressive Memory Recalling Scheme, PMRS’) during the chaotic memory association. This is exactly consistent with the latest research in psychiatry and perception psychology on dynamic memory recalling schemes, as well as the implications and analogues to human perception as illustrated by the remarkable Rubin-vase experiment on visual psychology. [Copyright &y& Elsevier]

Abstract: In this paper a neural network model for spatio-temporal coordination of hand gesture during prehension is proposed. The model includes a simplified control strategy for whole hand shaping during grasping tasks, that provides a realistic coordination among fingers. This strategy uses the increasing evidence that supports the view of a synergistic control of whole fingers during prehension. In this control scheme, only two parameters are needed to define the evolution of hand shape during the task performance. The proposal involves the design and development of a Library of Hand Gestures consisting of motor primitives for finger pre-shaping of an anthropomorphic dextrous hand. Through computer simulations, we show how neural dynamics of the model leads to simulated grasping movements with human-like kinematic features. The model can provide clear-cut predictions for experimental evaluation at both the behavioural and neural levels as well as a neural control system for a dextrous robotic hand. [Copyright &y& Elsevier]

Abstract: This paper considers the problem of exponential stability analysis of neural networks with time-varying delays. The activation functions are assumed to be globally Lipschitz continuous. A linear matrix inequality (LMI) approach is developed to derive sufficient conditions ensuring the delayed neural network to have a unique equilibrium point, which is globally exponentially stable. The proposed LMI conditions can be checked easily by recently developed algorithms solving LMIs. Examples are provided to demonstrate the reduced conservativeness of the proposed results. [Copyright &y& Elsevier]

Abstract: This paper presents a spiking neuron circuit with a triangular base signal. The circuit can output rich pulse-trains and the dynamics can be analyzed using a piecewise linear one-dimensional pulse-position map. Applying cross-switching to two neurons we construct a pulse-coupled system whose dynamics can be integrated into a composite map of the pulse position maps of two neurons. The composite map is piecewise linear and precise analysis is possible. We can clarify various interesting phenomena caused by the pulse-coupling. For example, periodic behavior of each neuron is changed into chaotic behavior and chaotic behavior of each neuron is changed into periodic behavior. These results provide basic information to construct flexible pulse-coupled neural networks. [Copyright &y& Elsevier]

Abstract: In this paper, we outline the approach we have developed to construct an emotion-recognising system. It is based on guidance from psychological studies of emotion, as well as from the nature of emotion in its interaction with attention. A neural network architecture is constructed to be able to handle the fusion of different modalities (facial features, prosody and lexical content in speech). Results from the network are given and their implications discussed, as are implications for future direction for the research. [Copyright &y& Elsevier]

Abstract: A competitive neural network model was recently proposed to describe the dynamics of cortical maps, where there are two types of memories: long-term and short-term memories. Such a network is characterized by a system of differential equations with two types of variables, one models the fast neural activity and the other models the slow modification of synaptic strength. In this paper, we introduce a time delay parameter into the neural network model to characterize the signal transmission delays in real neural systems and the finite switch speed in the circuit implementations of neural networks. Then, we analyze the global exponential stability of the delayed competitive neural networks with different time scales. We allow the model has non-differentiable and unbounded functions, and use the nonsmooth analysis techniques to prove the existence and uniqueness of the equilibrium, and derive a new sufficient condition ensuring global exponential stability of the networks. [Copyright &y& Elsevier]

Abstract: In this paper, a general class of recurrent neural networks with time-varying delays is studied. Some novel and sufficient conditions are given to guarantee the global exponential stability of the equilibrium point and the existence of periodic solutions for such delayed neural networks. Comparing with some previous literature, in which the time-varying delays were assumed to be differentiable and their derivatives were simultaneously required to be not greater than 1, the restrictions on the time-varying delays are removed. Therefore, our results obtained here improve and extend some previously related results. Finally, two numerical examples are provided to illustrate our theorems. [Copyright &y& Elsevier]

Abstract: This paper studies the global output convergence of a class of recurrent neural networks with globally Lipschitz continuous and monotone nondecreasing activation functions and locally Lipschitz continuous time-varying inputs. We establish two sufficient conditions for global output convergence of this class of neural networks. Symmetry in the connection weight matrix is not required in the present results which extend the existing ones. [Copyright &y& Elsevier]

Abstract: This paper presents a family of layered feed-forward networks that is able to uniformly approximate functions on any metric space, and also on a wide variety of non-metric spaces. Non-Euclidean input spaces are frequently encountered in practice, while usual approximation schemes are guaranteed to work only on Euclidean metric spaces. Theoretical foundations are provided, as well as practical algorithms and illustrative examples. This tool potentially constitutes a significant extension of the common notion of ‘universal approximation capability’. [Copyright &y& Elsevier]

Abstract: Principal component analysis (PCA) has been generalized to complex principal component analysis (CPCA), which has been widely applied to complex-valued data, two-dimensional vector fields, and complexified real data through the Hilbert transform. Nonlinear PCA (NLPCA) can also be performed using auto-associative feed-forward neural network (NN) models, which allows the extraction of nonlinear features in the data set. This paper introduces a nonlinear complex PCA (NLCPCA) method, which allows nonlinear feature extraction and dimension reduction in complex-valued data sets. The NLCPCA uses the architecture of the NLPCA network, but with complex variables (including complex weight and bias parameters). The application of NLCPCA on test problems confirms its ability to extract nonlinear features missed by the CPCA. For similar number of model parameters, the NLCPCA captures more variance of a data set than the alternative real approach (i.e. replacing each complex variable by two real variables and applying NLPCA). The NLCPCA is also used to perform nonlinear Hilbert PCA (NLHPCA) on complexified real data. The NLHPCA applied to the tropical Pacific sea surface temperatures extracts the El Niño-Southern Oscillation signal better than the linear Hilbert PCA. [Copyright &y& Elsevier]

This paper proposes a neural network model that has an ability to restore missing portions of partly occluded patterns. It is a multi-layered hierarchical neural network, in which visual information is processed by interaction of bottom-up and top-down signals. Memories of learned patterns are stored in the connections between cells. Occluded parts of a pattern are reconstructed mainly by top-down signals from higher stages of the network, while the unoccluded parts are reproduced mainly by signals from lower stages. The restoration progresses successfully, even if the occluded pattern is a deformed version of a learned pattern. The model tries to complete even an unlearned pattern by interpolating and extrapolating visible edges. Resemblance of local features to other learned patterns are also utilized for the restoration. [Copyright &y& Elsevier]

Abstract: In this paper we present a self-organizing neural network model of early lexical development called DevLex. The network consists of two self-organizing maps (a growing semantic map and a growing phonological map) that are connected via associative links trained by Hebbian learning. The model captures a number of important phenomena that occur in early lexical acquisition by children, as it allows for the representation of a dynamically changing linguistic environment in language learning. In our simulations, DevLex develops topographically organized representations for linguistic categories over time, models lexical confusion as a function of word density and semantic similarity, and shows age-of-acquisition effects in the course of learning a growing lexicon. These results match up with patterns from empirical research on lexical development, and have significant implications for models of language acquisition based on self-organizing neural networks. [Copyright &y& Elsevier]

Abstract: The current paper reviews a connectionist model, the recurrent neural network with parametric biases (RNNPB), in which multiple behavior schemata can be learned by the network in a distributed manner. The parametric biases in the network play an essential role in both generating and recognizing behavior patterns. They act as a mirror system by means of self-organizing adequate memory structures. Three different robot experiments are reviewed: robot and user interactions; learning and generating different types of dynamic patterns; and linguistic-behavior binding. The hallmark of this study is explaining how self-organizing internal structures can contribute to generalization in learning, and diversity in behavior generation, in the proposed distributed representation scheme. [Copyright &y& Elsevier]