Search

Abstract Biological neural networks are systems of extraordinary computational capabilities shaped by evolution, development, and lifelong learning. The interplay of these elements leads to the emergence of biological intelligence. Inspired by such intricate natural phenomena, Evolved Plastic Artificial Neural Networks (EPANNs) employ simulated evolution in-silico to breed plastic neural networks with the aim to autonomously design and create learning systems. EPANN experiments evolve networks that include both innate properties and the ability to change and learn in response to experiences in different environments and problem domains. EPANNs' aims include autonomously creating learning systems, bootstrapping learning from scratch, recovering performance in unseen conditions, testing the computational advantages of particular neural components, and deriving hypotheses on the emergence of biological learning. Thus, EPANNs may include a large variety of different neuron types and dynamics, network architectures, plasticity rules, and other factors. While EPANNs have seen considerable progress over the last two decades, current scientific and technological advances in artificial neural networks are setting the conditions for radically new approaches and results. Exploiting the increased availability of computational resources and of simulation environments, the often challenging task of hand-designing learning neural networks could be replaced by more autonomous and creative processes. This paper brings together a variety of inspiring ideas that define the field of EPANNs. The main methods and results are reviewed. Finally, new opportunities and possible developments are presented. [ABSTRACT FROM AUTHOR]

Abstract: This paper presents new experimental results on the variadic neural network (VNN) [McGregor, S. (2007). Neural network processing for multiset data. In Proceedings: Vol. 4668. Artificial neural networks — ICANN 2007, 17th international conference (pp. 460–470). Springer]. The inputs to a variadic network are an arbitrary-length list of -tuples of real numbers, where is fixed, and the function computed by the network is unaffected by permutation of the inputs. This paper describes improvements in the training algorithm for the variadic perceptron, based on a constructive cascade topology, and performance of the improved networks on geometric problems inspired by vector graphics. Further development may allow practical application of these or similar networks to vector graphics processing and statistical analysis. [Copyright &y& Elsevier]

Abstract: In an earlier paper [Fukushima, K., & Tohyama, K. (2005). Analysis of optic flow: a neural network model. In ICONIP 2005, International Conference on Neural Information Processing (pp. 312–317); Tohyama, K., & Fukushima, K. (2005). Neural network model for extracting 98 optic flow. Neural Networks, 18(5–6), 549–556], we proposed a neural network extracting optic flow based on vector field hypothesis. We started discussion there, however, from the stage where local velocities of visual objects have already been extracted. This paper proposes a new mechanism of extracting local velocity from retinal images, and adds it to the previous network. The network has a hierarchical multilayered architecture. X- and Y-cells of the retina extract spatial and temporal contrast of brightness. The network contains several types of V1 cells, namely, S- C- and V-cells. S- and C-cells extract orientated edges. V-cells extract local velocities, based on the signals from Y- and S-cells. MT cells extract relative velocities between adjoining small visual fields. MST cells add the responses of MT cells to extract a specific optic flow, such as rotation and expansion/contraction. The difference in type of optic flow extracted by MST cells can be created simply by the difference in relative locations of inhibitory to excitatory areas in the receptive fields of MT cells. [Copyright &y& Elsevier]

Abstract: In this paper, we propose a novel framework based on a collective network of evolutionary binary classifiers (CNBC) to address the problems of feature and class scalability. The main goal of the proposed framework is to achieve a high classification performance over dynamic audio and video repositories. The proposed framework adopts a “Divide and Conquer” approach in which an individual network of binary classifiers (NBC) is allocated to discriminate each audio class. An evolutionary search is applied to find the best binary classifier in each NBC with respect to a given criterion. Through the incremental evolution sessions, the CNBC framework can dynamically adapt to each new incoming class or feature set without resorting to a full-scale re-training or re-configuration. Therefore, the CNBC framework is particularly designed for dynamically varying databases where no conventional static classifiers can adapt to such changes. In short, it is entirely a novel topology, an unprecedented approach for dynamic, content/data adaptive and scalable audio classification. A large set of audio features can be effectively used in the framework, where the CNBCs make appropriate selections and combinations so as to achieve the highest discrimination among individual audio classes. Experiments demonstrate a high classification accuracy (above 90%) and efficiency of the proposed framework over large and dynamic audio databases. [Copyright &y& Elsevier]

Abstract: Language understanding is a long-standing problem in computer science. However, the human brain is capable of processing complex languages with seemingly no difficulties. This paper shows a model for language understanding using biologically plausible neural networks composed of associative memories. The model is able to deal with ambiguities on the single word and grammatical level. The language system is embedded into a robot in order to demonstrate the correct semantical understanding of the input sentences by letting the robot perform corresponding actions. For that purpose, a simple neural action planning system has been combined with neural networks for visual object recognition and visual attention control mechanisms. [Copyright &y& Elsevier]

Abstract: In this paper, we propose the concept of a manifold of color perception through empirical observation that the center-surround properties of images in a perceptually similar environment define a manifold in the high dimensional space. Such a manifold representation can be learned using a novel recurrent neural network based learning algorithm. Unlike the conventional recurrent neural network model in which the memory is stored in an attractive fixed point at discrete locations in the state space, the dynamics of the proposed learning algorithm represent memory as a nonlinear line of attraction. The region of convergence around the nonlinear line is defined by the statistical characteristics of the training data. This learned manifold can then be used as a basis for color correction of the images having different color perception to the learned color perception. Experimental results show that the proposed recurrent neural network learning algorithm is capable of color balance the lighting variations in images captured in different environments successfully. [Copyright &y& Elsevier]

Abstract: This paper is concerned with the stability analysis problem for a new class of discrete-time recurrent neural networks with mixed time-delays. The mixed time-delays that consist of both the discrete and distributed time-delays are addressed, for the first time, when analyzing the asymptotic stability for discrete-time neural networks. The activation functions are not required to be differentiable or strictly monotonic. The existence of the equilibrium point is first proved under mild conditions. By constructing a new Lyapnuov–Krasovskii functional, a linear matrix inequality (LMI) approach is developed to establish sufficient conditions for the discrete-time neural networks to be globally asymptotically stable. As an extension, we further consider the stability analysis problem for the same class of neural networks but with state-dependent stochastic disturbances. All the conditions obtained are expressed in terms of LMIs whose feasibility can be easily checked by using the numerically efficient Matlab LMI Toolbox. A simulation example is presented to show the usefulness of the derived LMI-based stability condition. [Copyright &y& Elsevier]

Abstract: This paper introduces an approach called Clustering and Co-evolution to Construct Neural Network Ensembles (CONE). This approach creates neural network ensembles in an innovative way, by explicitly partitioning the input space through a clustering method. The clustering method allows a reduction in the number of nodes of the neural networks that compose the ensemble, thus reducing the execution time of the learning process. This is an important characteristic especially when evolutionary algorithms are used. The clustering method also ensures that different neural networks specialize in different regions of the input space, working in a divide-and-conquer way, to maintain and improve the accuracy. Besides, the clustering method facilitates the understanding of the system and makes a straightforward distributed implementation possible. The experiments performed with seven classification databases and three different co-evolutionary algorithms show that CONE considerably reduces the execution time without prejudicing (and even improving) the accuracy, even when a distributed implementation is not used. [Copyright &y& Elsevier]

Abstract: The purpose of this paper is to present some relevant theoretical results on the asymptotic behaviour of finite neural networks (on lattices) when they are subjected to fixed boundary conditions. This work focuses on two different topics of interest from the biological point of view. First, it exhibits a link between the possible updating iteration modes in these networks, whatever the number of dimensions is. It proves that the effects of boundary conditions on neural networks do not depend on the updating iteration mode under the hypothesis of synaptic weight symmetry. Thus, if the asymptotic behaviour admits phase transitions, these phase transitions are observable for many updating iteration modes (from synchrony to asynchrony). Then, it shows that boundaries have no significant impact on one-dimensional neural networks. In order to prove this property, we present a new general mathematical approach based on the use of a projectivity matrix in order to simplify the problem. This approach allows the theoretical study of the asymptotic dynamics and of the boundary influence in neural networks. We will also introduce the numerical tools generalising the method in order to study phase transitions in more complex cases. [Copyright &y& Elsevier]

Abstract: We study the framework of Clifford algebra for the design of neural architectures capable of processing different geometric entities. The benefits of this model-based computation over standard real-valued networks are demonstrated. One particular example thereof is the new class of so-called Spinor Clifford neurons. The paper provides a sound theoretical basis to Clifford neural computation. For that purpose the new concepts of isomorphic neurons and isomorphic representations are introduced. A unified training rule for Clifford MLPs is also provided. The topic of activation functions for Clifford MLPs is discussed in detail for all two-dimensional Clifford algebras for the first time. [Copyright &y& Elsevier]

Abstract: An environmentally adaptive system for prediction of acoustic transmission loss (TL) in the atmosphere is developed in this paper. This system uses several back propagation neural network predictors, each corresponding to a specific environmental condition. The outputs of the expert predictors are combined using a fuzzy confidence measure and a nonlinear fusion system. Using this prediction methodology the computational intractability of traditional acoustic model-based approaches is eliminated. The proposed TL prediction system is tested on two synthetic acoustic data sets for a wide range of geometrical, source and environmental conditions including both nonturbulent and turbulent atmospheres. Test results of the system showed root mean square (RMS) errors of 1.84 dB for the nonturbulent and 1.36 dB for the turbulent conditions, respectively, which are acceptable levels for near real-time performance. Additionally, the environmentally adaptive system demonstrated improved TL prediction accuracy at high frequencies and large values of horizontal separation between source and receiver. [Copyright &y& Elsevier]

Abstract: Synoptic downscaling models are used in climatology to predict values of weather elements at one or more stations based on values of synoptic-scale atmospheric circulation variables. This paper presents a hybrid method for climate prediction and downscaling that couples an analog, i.e., -nearest neighbor, model to an artificial neural network (ANN) model. In the proposed method, which is based on nonlinear principal predictor analysis (NLPPA), the analog model is embedded inside an ANN, forming its output layer. Nonlinear analog predictor analysis (NLAPA) is a flexible model that maintains the ability of the analog model to preserve inter-variable relationships and model non-normal and conditional variables (such as precipitation), while taking advantage of NLPPA’s ability to define an optimal set of analog predictors that maximize predictive performance. Performance on both synthetic and real-world hydroclimatological benchmark tasks indicates that the NLAPA model is capable of outperforming other forms of analog models commonly used in synoptic downscaling. [Copyright &y& Elsevier]

Abstract: In this paper, a general class of recurrent neural networks with time-varying delays is studied. Some novel and sufficient conditions are given to guarantee the global exponential stability of the equilibrium point and the existence of periodic solutions for such delayed neural networks. Comparing with some previous literature, in which the time-varying delays were assumed to be differentiable and their derivatives were simultaneously required to be not greater than 1, the restrictions on the time-varying delays are removed. Therefore, our results obtained here improve and extend some previously related results. Finally, two numerical examples are provided to illustrate our theorems. [Copyright &y& Elsevier]

Abstract: This paper studies the global output convergence of a class of recurrent neural networks with globally Lipschitz continuous and monotone nondecreasing activation functions and locally Lipschitz continuous time-varying inputs. We establish two sufficient conditions for global output convergence of this class of neural networks. Symmetry in the connection weight matrix is not required in the present results which extend the existing ones. [Copyright &y& Elsevier]

This paper derives a new sufficient condition for the exponential stability of the equilibrium point for delayed neural networks with time varying delays by employing a Lyapunov-Krasovskii functional and using Linear Matrix Inequality (LMI) approach. This result establishes a relation between the delay time and the parameters of the network. The result is also compared with the most recent result derived in the literature. [Copyright &y& Elsevier]

In this paper, a class of nonautonomous cellular neural networks is studied. By constructing a suitable Liapunov functional, applying the boundedness theorem for general functional–differential equations and the Banach fixed point theorem, a series of new criteria are obtained on the boundedness, global exponential stability, and existence of periodic solutions. [Copyright &y& Elsevier]

This paper presents a recurrent neural network for solving strict convex quadratic programming problems and related linear piecewise equations. Compared with the existing neural networks for quadratic program, the proposed neural network has a one-layer structure with a low model complexity. Moreover, the proposed neural network is shown to have a finite-time convergence and exponential convergence. Illustrative examples further show the good performance of the proposed neural network in real-time applications. [Copyright &y& Elsevier]

In this paper, we propose a new network growth model and its learning algorithm to more precisely model such a real-world growing network as the Web. Unlike the conventional models, we have incorporated directional attachment and community structure for this purpose. We show that the proposed model exhibits a degree distribution with a power-law tail, which is an important characteristic of many large-scale real-world networks including the Web. Using real Web data, we experimentally show that predictive ability can be improved by incorporating directional attachment and community structure. Also, using synthetic data, we experimentally show that predictive ability can definitely be improved by incorporating community structure. [Copyright &y& Elsevier]