Search

We describe two simple simulations in which artificial organisms evolve an ability to respond to inputs from within their own body and these inputs themselves can evolve. In the first simulation the organisms develop an ability to respond to a pain signal caused by body damage by stopping looking for food when they feel pain since resting while the body is damaged accelerates healing of the body and increases the individual's survival chances. In the second simulation the pain signal itself evolves, that is, the body develops a tendency to send pain signals to the nervous system when the body is damaged. The results are discussed in terms of an internal robotics in which the robot's body has an internal structure and not only an external morphology and the neural network that controls the robot's behavior responds to inputs both from the external environment and from within the body. [ABSTRACT FROM AUTHOR]

This paper address the kinematic variables control problem for the low-speed manoeuvring of a low cost and underactuated underwater vehicle. Control of underwater vehicles is not simple, mainly due to the non-linear and coupled character of plant equations, the lack of a precise model of vehicle dynamics and parameters, as well as the appearance of internal and external perturbations. The proposed methodology is an approach that makes use of a pioneering algorithm in underwater vehicles, based on the fusion of a robust or sliding mode controller and an adaptive fuzzy system, including the advantages of both systems. The main property of this methodology is that it relaxes the required knowledge of vehicle model, reducing the cost of its design. [ABSTRACT FROM AUTHOR]

Simbad is an open source Java 3d robot simulator for scientific and educational purposes. It is mainly dedicated to researchers and programmers who want a simple basis for studying Situated Artificial Intelligence, Machine Learning, and more generally AI algorithms, in the context of Autonomous Robotics and Autonomous Agents. It is is kept voluntarily readable and simple for fast implementation in the field of Research and/or Education. Moreover, Simbad embeds two stand-alone additional packages : a Neural Network library (feed-forward NN, recurrent NN, etc.) and an Artificial Evolution Framework for Genetic Algorithm, Evolutionary Strategies and Genetic Programming. These packages are targeted towards Evolutionary Robotics. The Simbad Package is available from http://simbad.sourceforge.net/ under the conditions of the GPL (GNU General Public Licence). [ABSTRACT FROM AUTHOR]

In recent years, the advent of robust tracking systems has enabled behavioral analysis of individuals based on their trajectories. An analysis method based on a Point Distribution Model (PDM) is presented here. It is an unsupervised modeling of the trajectories in order to extract behavioral features. The applicability of this method has been demonstrated on trajectories of a realistically simulated mobile robot endowed with various controllers that lead to different patterns of motion. Results show that this analysis method is able to clearly classify controllers in the PDM-transformed space, an operation extremely difficult in the original space. The analysis also provides a link between the behaviors and trajectory differences. [ABSTRACT FROM AUTHOR]

In mobile robotics, it is common to find different control programs designed to achieve a particular robot task. It is often necessary to compare the performance of such controllers. So far this is usually done qualitatively, because of a lack of quantitative behaviour analysis methods. In this paper we present a novel approach to compare robot control codes quantitatively, based on system identification. Using the NARMAX system identification process, we "translate" the original behaviour into a transparent, analysable mathematical model of the original behaviour. We then use statistical methods and sensitivity analysis to compare models quantitatively. We demonstrate our approach by comparing two different robot control programs, which were designed to drive a Magellan Pro robot through door-like openings. [ABSTRACT FROM AUTHOR]

In this paper we describe how a population of simulated robots evolved for the ability to solve a collective navigation problem develop individual and social/communication skills. In particular, we analyze the evolutionary origins of motor and signaling behaviors. Obtained results indicate that signals and the meaning of the signals produced by evolved robots are grounded not only on the robots sensory-motor system but also on robots' behavioral capabilities previously acquired. Moreover, the analysis of the co-evolution of robots individual and communicative abilities indicate how innovation in the former might create the adaptive basis for further innovations in the latter and vice versa. [ABSTRACT FROM AUTHOR]

In mixed societies of robots and cockroaches, several insect-like-robot (Insbot) and animals interact in order to perform collective decision-making. Many gregarious species are able to collectively select a resting site without any leadership. The key process is based on the modulation of the probability of leaving the shelter according to the total population under this shelter and its light intensity. It is important that cockroaches perceive the robot as a "congener". This recognition is mainly based on a chemical blend. The aim of this study is to validate experimentally (1) the behavioral patterns expressed by the cockroaches in presence of shelters and of an Insbot, and (2) the important role played by the chemical blend on collective decision-makings. [ABSTRACT FROM AUTHOR]

An important problem for societies of natural and artificial animals is to converge upon a similar language in order to communicate. We call this the language convergence problem. In this paper we study the complexity of finding the optimal (in terms of time to convergence) algorithm for language convergence. We map the language convergence problem to instances of a Decentralized Partially Observable Markov Decision Process to show that the complexity can vary from P-complete to NEXP-complete based on the scenario being studied. [ABSTRACT FROM AUTHOR]

In this paper, research work on Arisco is described. Arisco is a social robot built around a robotic head with gesture ability, visual and auditive perception and learning. It is intended for interacting with people. The general architecture is first described in the paper. Then, the learning capacity of Arisco is addressed. It learns and performs associations between different stimulus responses through several dynamic neural networks, guided by motivational drives. Main contribution of this paper is the integration in a real robot of conditioning learning models based on a neural competitive network. A number of experiments are discussed, covering stimulus competition, habituation and first and second order conditioning. [ABSTRACT FROM AUTHOR]

Strategies of incremental evolution of artificial neural systems have been suggested over the last decade to overcome the scalability problem of evolutionary robotics. In this article two methods are introduced that support the evolution of neural couplings and extensions of recurrent neural networks of general type. These two methods are applied to combine and extend already evolved behavioral functionality of an autonomous robot in order to compare the structure-function relations of the resulting networks with those of the initial structures. The results of these investigations indicate that the emergent dynamics of the resulting networks turn these control structures into irreducible systems. We will argue that this leads to several consequences. One is, that the scalability problem of evolutionary robotics remains unsolved, no matter which type of incremental evolution is applied. [ABSTRACT FROM AUTHOR]

This paper deals with a new kind of robotic control, based on Chua's nonlinear circuit called Cellular Neural Network (CNN). A CNN is a net of coupled circuits, connected in a grid structure, which inherits its features and properties from the well known Artificial Neural Network and Cellular Automata. It has been demonstrated that CNNs are able of universal computation, many cognitive processes such as pattern recognition, features extraction, image processing, and mathematical simulations of nonlinear equations such as Navier-Stokes equations, reaction-diffusion equations, and so on. Using an approach like Evolutionary Robotics, we evolved, instead of Neural Networks, CNNs by using Genetic Algorithms (GAs), for controlling the behavior of an hexapod robot in a simulated environment. We developed a Java3D software in which physical simulations are carried on by using different kind of robots. In this program, a module for evolving the robot's behavior by GAs has been implemented. Furthermore, many advanced sensors and actuators complete the evolution of the robot's behavior. The evolved behavior of our robots is very similar to that of real insects, and we analyzed the pathways these agents perform in the simulated environment. [ABSTRACT FROM AUTHOR]

In the context of minimally cognitive behavior, we used multi-robotic systems to investigate the emergence of communication and cooperation during the evolution of recurrent neural networks. The networks are systematically analyzed to identify their relevant dynamical properties. Evolution efficiently adapts these properties through small structural changes within the networks when specific environmental conditions are altered, such as the number of interacting robots. The findings signify the importance of reducing the predefined knowledge about resulting behaviors, dynamical properties of control, and the topology of neural networks in order to utilize the strength of the Evolutionary Robotics approach to Artificial Life. [ABSTRACT FROM AUTHOR]

We study the role of the agent interaction topology in distributed language learning. In particular, we utilize the replicator- mutator framework of language evolution for the creation of an emergent agent interaction topology that leads to quick convergence. In our system, it is the links between agents that are treated as the units of selection and replication, rather than the languages themselves. We use the Noisy Preferential Attachment algorithm, which is a special case of the replicator-mutator process, for generating the topology. The advantage of the NPA algorithm is that, in the short-term, it produces a scale-free interaction network, which is helpful for rapid exploration of the space of languages present in the population. A change of parameter settings then ensures convergence because it guarantees the emergence of a single dominant node which is chosen as teacher almost always. [ABSTRACT FROM AUTHOR]

The question "What drove foragers to farm?" has drawn answers from many different disciplines, often in the form of verbal models. Here, we take one such model, that of the ideal free distribution, and implement it as an agent-based computer simulation. Populations distribute themselves according to the marginal quality of different habitats, predicting settlement patterns and subsistence methods over both time and space. Our experiments and our analyses thereof show that central conclusions of the ideal free distribution model are reproduced by our agent-based simulation, while at the same time offering new insights into the theory's underlying assumptions. Generally, we demonstrate how agent-based models can make use of empirical data to reconstruct realistic environmental and cultural contexts, enabling concrete tests of the explanatory power of anthropological models put forward to explain historical developments, such as agricultural transitions, in specific times and places. [ABSTRACT FROM AUTHOR]

This paper investigates the dynamics of cumulative cultural evolution in a simulation concerning the evolution of language. This simulation integrates the iterated learning model with the Talking Heads experiment in which a population of agents evolves a language to communicate geometrical coloured objects by playing guessing games and transmitting the language from one generation to the next. The results show that cumulative cultural evolution is possible if the language becomes highly regular, which only happens if the language is transmitted from generation to generation. [ABSTRACT FROM AUTHOR]

A honeybee colony has to work highly efficient to survive. Most of a honeybee's energetic demands are satisfied by consuming carbohydrates which are collected by forager bees in form of nectar from flowering plants. The storage of this nectar is performed by another specialised group of bees. The size of the two workgroups (foragers and receivers) are precisely regulated by dances performed by forager bees, a process that represents adaptive behaviour of a superorganism. We implemented these mechanisms in a simulation of a honeybee colony to investigate the possible advantages of bigger colonies in nectar foraging. [ABSTRACT FROM AUTHOR]

Gregarious insects, like cockroaches, aggregate in shelters during their resting period. How do individuals reach a collegial decision? What is the relation between the distributions of the individuals and the parameter values characterizing the population and the environment? With a model based on experimental data, we demonstrated that the collegial decision is based on the relation between the individual resting time in a shelter and the population in this shelter. We extended this model to the case where different sub-groups may interact and where the crowding effect under the shelters influences the aggregation. This second model shows that depending on the interaction between the sub-groups and the crowding effect, different patterns are observed such as segregation of the different sub-group or the aggregation of the whole population. [ABSTRACT FROM AUTHOR]

This paper tackles the issue of designing homogeneous neuro-controllers with artificial evolution in order to control groups of robots that differ in terms of sensory capabilites. In order to accomplish a common goal, the agents have to complement the partial "view" they have of the environment. The results obtained prove that the agents are capable of cooperating and coordinating their actions in order to carry out a navigation task. A preliminary analysis of the mechanisms underlying the group behaviour is provided. [ABSTRACT FROM AUTHOR]

"Learning" has been well studied in several research areas such as psychology, brain science, computer science and robotics. In these studies, many experiments using animals have been performed. On the other hand, several researchers have been studying adaptive interactions and task learning through interactions between humans and robots. We then focus on adaptive interactions between animals and robots. The purpose of our research is to develop a framework of adaptive interactions between animals and robots through interaction experiments between rats and a robotic rat. We propose a novel behavior generation algorithm for the robot to enable it to autonomously teach a simple behavior task to rats as an example of adaptive interaction. This algorithm was implemented in the robot and the experimental setup, and then verified through the experiment. [ABSTRACT FROM AUTHOR]

In imitation learning, agents are trained to carry out certain actions by examining a demonstration of the task at hand. Though common in robotics, little work has been done in translating these concepts to computer games. Given that present-day games generally use antiquated AI techniques which can often lead to stilted, mechanical and conspicuously artificial behaviour, it seems likely that approaches based on the imitation of human players may produce agents which convey a more humanlike impression than their traditional counterparts. At the same time, there exists no formal method of quantifying what constitutes a ‘humanlike' impression; an equivalent of the Turing test is needed, with the requirement that an agent's appearance and behaviour be capable of deceiving an observer into misidentifying it as human. The aims of this paper are thus threefold; we describe an approach to the imitation of strategic behaviour and motion, propose a formal method of quantifying the degree to which different agents are perceived as ‘humanlike', and present the results of a series of experiments using these two systems. [ABSTRACT FROM AUTHOR]

In this article we study the convergence of the positions of a multi-agent system in a cyclic pursuit under asynchronism and time delays. Each agent is assumed to operate on an infinite sequence of behaviors modeled by a finite state machine, which is represented by a discrete asynchronous mathematical model on a higher-level. The results on the convergence of the synchronous model are used in the proof of convergence of the asynchronous system. Numerical simulations are also performed to verify the theoretical results. [ABSTRACT FROM AUTHOR]

This paper investigates the relationship between spatially embedded neural network models and modularity. It is hypothesised that spatial constraints lead to a greater chance of evolving modular structures. Firstly, this is tested in a minimally modular task/controller scenario. Spatial networks were shown to possess the ability to generate modular controllers which were not found in standard, non-spatial forms of network connectivity. We then apply this insight to examine the effect of varying degrees of spatial constraint on the modularity of a controller operating in a more complex, situated and embodied simulated environment. We conclude that a bias towards modularity is perhaps not always a desirable property for a control system paradigm to possess. [ABSTRACT FROM AUTHOR]

This paper reports an experiment in which artificial foraging agents with dynamic, recurrent neural network architectures, are "evolved" within a simulated ecosystem. The resultant agents can compare different food values to "go for more," and display similar comparison performance to that found in biological subjects. We propose and apply a novel methodology for analysing these networks, seeking to recover their quantity representations within an Approximationist framework. We focus on Localist representation, seeking to interpret single units as conveying representative information through their average activities. One unit is identified that passes our "representation test", representing quantity by inverse accumulation. [ABSTRACT FROM AUTHOR]

We introduce a simulation model of coordinated adaptive behavior of agents with homogeneous elements connected by springs. The agents accomplish a hill climbing task by coordinating the dynamics of active elements and responding to the spatial pattern of surface convexity. Transition between discrete movement patterns is observed while moving in a rugged environment. The difference of dynamics between approaching hills and avoiding hollows is distinct and can be interpreted as a dynamic categorization of bumps on the surface. [ABSTRACT FROM AUTHOR]

We present a robot motivational system design framework. The framework represents the underlying (possibly conflicting) goals of the robot as a set of drives, while ensuring comparable drive levels and providing a mechanism for drive priority adaptation during the robot's lifetime. The resulting drive reward signals are compatible with existing reinforcement learning methods for balancing multiple reward functions. We illustrate the framework with an experiment that demonstrates some of its benefits. [ABSTRACT FROM AUTHOR]

We present an approach for evolutionary design of the driving style of an agent, remotely operating a scale model of a car running in a fastest possible way. The agent perceives the environment from a video camera and conveys its actions to the car via standard radio control transmitter. In order to cope with the video feed latency we propose an anticipatory modeling in which the agent considers its current actions based on the anticipated intrinsic (rather than currently available, outdated) state of the car and its surrounding. The driving style is first evolved offline on a software model of the car and then adapted online to the real world. An online evolutionary adaptation of the offline-obtained best styles to the needs to avoid a small obstacle results in lap times that are virtually the same as the best lap times achieved on the same track without obstacles. Presented work is a step towards the automated design of the control software of remotely operated vehicles capable to find an optimal solution to various tasks in different environmental situations. The results, also, can be seen as an attempt to explore the feasibility of developing a framework of adaptive racing games in which the human competes against a computer with matching capabilities, both operating physical, scale models of cars. [ABSTRACT FROM AUTHOR]

Using an incremental multi-objective evolutionary algorithm and the ModNet encoding, we generated working neuro-controllers for target-following behavior in a simulated flapping-wing animat. To this end, we evolved tail controllers that were combined with two closed-loop wing-beat controllers previously generated, and able to secure straight flight at constant altitude and speed. The corresponding results demonstrate that a wing-beat strategy that consists in continuously adapting the twist of the external wing panel leads to better manoeuvring capabilities than another strategy that adapts the beating amplitude. Such differences suggest that further improvements in flying control should better rely on some sort of automatic incremental evolution procedure than on any hand-designed decomposition of the problem. [ABSTRACT FROM AUTHOR]

This article presents a multi-agent simulation of an abstract ecosystem which is inhabited by two species: a predator species and a prey species. Both species show the typical behaviors found in such an ecological relationship that are: hunting behavior and escaping behavior. In the simulation, the actors make behavioral decisions according to "genetically fixed" weighting parameters. These parameters determine which prey item is selected by the predator and which predators are avoided the most by prey. Thus these parameters shape the decisions performed by both species. We incorporated artificial evolution by allowing successful animals to pass their features to their offspring, a process that includes mutation and recombination of these "genes". The simulation shows that different kinds of optimal behavioral choices emerge out of artificial evolution, when the simulation is run with different physiological and morphological parameters of the actors. [ABSTRACT FROM AUTHOR]

The current work addresses the problem of redesigning brain-inspired artificial cognitive systems in order to gradually enrich them with advanced cognitive skills. In the proposed approach, properly formulated neural agents are employed to represent brain areas. A cooperative coevolutionary method, with the inherent ability to co-adapt substructures, supports the design of agents. Interestingly enough, the same method provides a consistent mechanism to reconfigure (if necessary) the structure of agents, facilitating follow-up modelling efforts. In the present work we demonstrate partial redesign of a brain-inspired cognitive system, in order to furnish it with learning abilities. The implemented model is successfully embedded in a simulated robotic platform which supports environmental interaction, exhibiting the ability of the improved cognitive system to adopt, in real-time, two different operating strategies. [ABSTRACT FROM AUTHOR]

We evolve spiking neural networks that implement a seek-push-release drive for a simple simulated agent interacting with objects. The evolved agents display minimally-cognitive behavior, by switching as a function of context between the three sub-behaviors and by being able to discriminate relative object size. The neural controllers have either static synapses or synapses featuring spike-timing-dependent plasticity (STDP). Both types of networks are able to solve the task with similar efficacy, but networks with plastic synapses evolved faster. In the evolved networks, plasticity plays a minor role during the interaction with the environment and is used mostly to tune synapses when networks start to function. [ABSTRACT FROM AUTHOR]

In this paper we present a novel information-theoretic measure of spatiotemporal coordination in a modular robotic system, and use it as a fitness function in evolving the system. This approach exemplifies a new methodology formalizing co-evolution in multi-agent adaptive systems: information-driven evolutionary design. The methodology attempts to link together different aspects of information transfer involved in adaptive systems, and suggests to approximate direct task-specific fitness functions with intrinsic selection pressures. In particular, the information-theoretic measure of coordination employed in this work estimates the generalized correlation entropy K2 and the generalized excess entropy E2 computed over a multivariate time series of actuators' states. The simulated modular robotic system evolved according to the new measure exhibits regular locomotion and performs well in challenging terrains. [ABSTRACT FROM AUTHOR]

This work is concerned with the study of the application of the MDB (Multilevel Darwinist Brain) evolution based Cognitive Architecture in real robots performing adaptive learning tasks. The experiments described here display the capabilities of this architecture when dealing with tasks that involve real time learning from a teacher and real time adaptation to changes in the goals provided or the communication pattern used by the teacher. One of the consequences of the interaction of the robot with the environment through the MDB is the generation of induced behaviors that allow the robot to continue its operation when no teacher is present. The experiments were carried out using a Sony AIBO robot and a Pioneer 2 robot with the same mechanism running on both just to demonstrate the robustness of the approach. [ABSTRACT FROM AUTHOR]

In this paper we apply incremental evolution for automatic synthesis of neural network controllers for a group of physically connected mobile robots called s-bots. The robots should be able to safely and cooperatively perform phototaxis in an arena containing holes. We experiment with two approaches to incremental evolution, namely behavioral decomposition and environmental complexity increase. Our results are compared with results obtained in a previous study where several non-incremental evolutionary algorithms were tested and in which the evolved controllers were shown to transfer successfully to real robots. Surprisingly, none of the incremental evolutionary strategies performs any better than the non-incremental approach. We discuss the main reasons for this and why it can be difficult to apply incremental evolution successfully in highly integrated tasks. [ABSTRACT FROM AUTHOR]

In the last few years a lot of work has been done to discover why GasNets outperform other network types in terms of evolvability. In this work GasNets are again compared to CTRNNs on a shape discrimination task. This task is used as to solve it, or gain an advantage, a controller does not need timers or pattern generators. We show that GasNets are outperformed by CTRNNs in terms of evolvability on this task and possible reasons for the disadvantages of GasNets are investigated. It is shown that, on a simple task where there is no necessity for a timer or pattern generator, there may be other issues which are better tackled by CTRNNs. [ABSTRACT FROM AUTHOR]

It has been suggested that evolving developmental programs instead of direct genotype-phenotype mappings may increase the scalability of Genetic Algorithms. Many of these Artificial Embryogeny (AE) models have been proposed and their evolutionary properties are being investigated. One of these properties concerns the fault-tolerance of at least a particular class of AE, which models the development of artificial multicellular organisms. It has been shown that such AE evolves designs capable of recovering phenotypic faults during development, even if fault-tolerance is not selected for during evolution. This type of adaptivity is clearly very interesting both for theoretical reasons and possible robotic applications. In this paper we provide empirical evidence collected from a multicellular AE model showing a subtle relationship between evolution and development. These results explain why developmental fault-tolerance necessarily emerges during evolution. [ABSTRACT FROM AUTHOR]

This paper describes work carried out to investigate whether a classic reaction-diffusion (RD) system could be used to control a ‘minimally cognitive' animat. The reaction-diffusion system chosen was that first described by Gray and Scott (Gray-Scott) and the minimally cognitive behaviors those used by Beer et. al involving the fixation and discrimination of diamond and circle shapes by a whiskered animat. The parameters of this RD-controller were evolved using an evolutionary, or genetic, algorithm (GA). [ABSTRACT FROM AUTHOR]

Magicicada is the genus of periodical cicadas which display a unique combination of long life cycles, periodicity, and mass emergences. Their nymphs live underground and stay immobile before constructing an exit tunnel in the spring of their 13th or 17th year, depending on the species. Once out, the adult insects live only for a few weeks with one sole purpose: reproduction. Both 13 and 17 are prime numbers; why did the cicadas "choose" these lengths for their life cycles? Two are the most interesting hypotheses (limited resources and hybridization avoidance) drawn by biologists, both bringing to the conclusion that the prime number cycles were selected because they were least likely to emerge with other cycles. If that's the case, then these lengths would have been selected via a sort of "tacit coordination by evolution". In the agent based model presented here, it is shown how the two major hypotheses must be both present in order to cause the emergency of the prime numbers based life cycles. A very important point here is that the agents in the model are not endowed with a "calculating" ability, in particular they lack the capacity of determining divisors. In the model no form of learning is present; the emergency of prime numbers is then a fact of evolutionary biology, a natural selection by adaptation, a la Darwin's theory known as "survival of the fittest". Collective behaviour thus emerges from simple atomic reactions at agents' level and from their reactions to the constraints imposed by the environment. In order to explore the space of parameters and to understand their role in the evolutionary selection of the life cycles, the multi-run technique is used (i.e.: changing a value at a time, the others being the same). [ABSTRACT FROM AUTHOR]

Self-organization and the phenomenon of emergence play an essential role in living systems and form a challenge to artificial life systems. This is not only because systems become more lifelike, but also since self-organization may help in reducing the design efforts in creating complex behavior systems. The present paper studies self-exploration based on a general approach to the self-organization of behavior, which has been developed and tested in various examples in recent years. This is a step towards autonomous early robot development. We consider agents under the close sensorimotor coupling paradigm with a certain cognitive ability realized by an internal forward model. Starting from tabula rasa initial conditions we overcome the bootstrapping problem and show emerging self-exploration. Apart from that, we analyze the effect of limited actions, which lead to deprivation of the world model. We show that our paradigm explicitly avoids this by producing purposive actions in a natural way. Examples are given using a simulated simple wheeled robot and a spherical robot driven by shifting internal masses. [ABSTRACT FROM AUTHOR]

When monkeys tackle novel complex behavioral tasks by trial-and-error they select actions from repertoires of sensorimotor primitives that allow them to search solutions in a space which is coarser than the space of fine movements. Neuroscientific findings suggested that upper-limb sensorimotor primitives might be encoded, in terms of the final goal-postures they pursue, in premotor cortex. A previous work by the authors reproduced these results in a model based on the idea that cortical pathways learn sensorimotor primitives while basal ganglia learn to assemble and trigger them to pursue complex reward-based goals. This paper extends that model in several directions: a) it uses a Kohonen network to create a neural map with population encoding of postural primitives; b) it proposes an actor-critic reinforcement learning algorithm capable of learning to select those primitives in a biologically plausible fashion (i.e., through a dynamic competition between postures); c) it proposes a procedure to pre-train the actor to select promising primitives when tackling novel reinforcement learning tasks. Some tests (obtained with a task used for studying monkeys engaged in learning reaching-action sequences) show that the model is computationally sound and capable of learning to select sensorimotor primitives from the postures' continuous space on the basis of their population encoding. [ABSTRACT FROM AUTHOR]

The moth Macroglossum stellatarum can learn the colour and sometimes the odour of a rewarding food source. We present data from 20 different experiments with different combinations of blue and yellow artificial flowers and the two odours honeysuckle and lavender. The experiments show that learning about the odours depends on the colour used. By training on different colour-odour combinations and testing on others, it becomes possible to investigate the exact relation between the two modalities during learning. Three computational models were tested in the same experimental situations as the real moths and their predictions were compared to the experimental data. The average error over all experiments as well as the largest deviation from the experimental data were calculated. Neither the Rescorla-Wagner model or a learning model with independent learning for each stimulus component were able to explain the experimental data. We present the new categorisation model, which assumes that the moth learns a template for the sensory attributes of the rewarding stimulus. This model produces behaviour that closely matches that of the real moth in all 20 experiments. [ABSTRACT FROM AUTHOR]

Jean is a model of early cognitive development based loosely on Piaget's theory of sensori-motor and pre-operational thought [1]. Like an infant, Jean repeatedly executes schemas, gradually extending its schemas to accommodate new experiences. Jean's environment is a simulated "playpen" in which Jean and other objects move about and interact. Jean's cognitive development depends on several integrated functions: a simple perceptual system, an action-selection system, a motivational system, a long-term memory, and learning methods. This paper provides an overview of Jean's architecture and schemas, and it focuses on how Jean learns schemas and transfers them to new situations. [ABSTRACT FROM AUTHOR]

A central role in the development process of children is played by self-exploratory activities. Through a playful interaction with the surrounding environment, they test their own capabilities, explore novel situations, and understand how their actions affect the world. During this kind of exploration, interesting situations may be discovered. By learning to reach these situations, a child incrementally develops more and more complex skills. Inspired by studies from psychology, neuroscience, and machine learning, we designed SMILe (Self-Motivated Incremental Learning), a learning framework that allows artificial agents to autonomously identify and learn a set of abilities useful to face several different tasks, through an iterated three phase process: by means of a random exploration of the environment (babbling phase), the agent identifies interesting situations and generates an intrinsic motivation (motivating phase) aimed at learning how to get into these situations (skill acquisition phase). This process incrementally increases the skills of the agent, so that new interesting configurations can be experienced. We present results on two gridworld environments to show how SMILe makes it possible to learn skills that enable the agent to perform well and robustly in many different tasks. [ABSTRACT FROM AUTHOR]

In this paper, we train a robot to learn online a task of obstacles avoidance. The robot has at its disposal only its visual input from a linear camera in an arena whose walls are composed of random black and white stripes. The robot is controlled by a recurrent spiking neural network (integrate and fire). The learning rule is the spike-time dependent plasticity (STDP) and its counterpart - the so-called anti-STDP. Since the task itself requires some temporal integration, the neural substrate is the network's own dynamics. The behaviors of avoidance we obtain are homogenous and elegant. In addition, we observe the emergence of a neural selectivity to the distance after the learning process. [ABSTRACT FROM AUTHOR]

In a reward-seeking task performed in a continuous environment, our previous work compared several Actor-Critic (AC) architectures implementing dopamine-like reinforcement learning mechanisms in the rat's basal ganglia. The task complexity imposes the coordination of several AC submodules, each module being an expert trained in a particular subset of the task. We showed that the classical method where the choice of the expert to train at a given time depends on each expert's performance suffered from strong limitations. We rather proposed to cluster the continuous state space by an ad hoc method that lacked autonomy and generalization abilities. In the present work we have combined the mixture of experts with self-organizing maps in order to cluster autonomously the experts' responsibility space. On the one hand, we find that classical Kohonen maps give very variable results: some task decompositions provide very good and stable reinforcement learning performances, whereas some others are unadapted to the task. Moreover, they require the number of experts to be set a priori. On the other hand, algorithms like Growing Neural Gas or Growing When Required have the property to choose autonomously and incrementally the number of experts to train. They lead to good performances, even if they are still weaker than our hand-tuned task decomposition and than the best Kohonen maps that we got. We finally discuss on propositions about what information to add to these algorithms, such as knowledge of current behavior, in order to make the task decomposition appropriate to the reinforcement learning process. [ABSTRACT FROM AUTHOR]

The idea that synaptic plasticity holds the key to the neural basis of learning and memory is now widely accepted in neuroscience. The precise mechanism of changes in synaptic strength has, however, remained elusive. Neurobiological research has led to the postulation of many models of plasticity, and among the most contemporary are spike-timing dependent plasticity (STDP) and long-term potentiation (LTP). The STDP model is based on the observation of single, distinct pairs of pre- and post- synaptic spikes, but it is less clear how it evolves dynamically under the input of long trains of spikes, which characterise normal brain activity. This research explores the emergent properties of a spiking artificial neural network which incorporates both STDP and LTP. Previous findings are replicated in most instances, and some interesting additional observations are made. These highlight the profound influence which initial conditions and synaptic input have on the evolution of synaptic weights. [ABSTRACT FROM AUTHOR]

When neurons fire together they wire together. This is Donald Hebb's famous postulate. However, Hebbian learning is inherently unstable because synaptic weights will self amplify themselves: the more a synapse is able to drive a postsynaptic cell the more the synaptic weight will grow. We present a new biologically realistic way how to stabilise synaptic weights by introducing a third factor which switches on or off learning so that self amplification is minimised. The third factor can be identified by the activity of dopaminergic neurons in VTA which fire when a reward has been encountered. This leads to a new interpretation of the dopamine signal which goes beyond the classical prediction error hypothesis. The model is tested by a real world task where a robot has to find "food disks" in an environment. [ABSTRACT FROM AUTHOR]

We present a practical application of sensorimotor self- simulation for a mobile robot. Using its self-simulation, the robot can reason about its ability to perform tasks, despite having no model of many of its internal processes and thus no way to create an a priori configuration space in which to search. We suggest that this in-the-head rehearsal of tasks is particularly useful when the tasks carry a high risk of robot "death", as it provides a source of negative feedback in perfect safety. This approach is a useful complement to existing work using forward models for anticipatory behaviour. A minimal system is shown to be effective in simulation and real-world experiments. The virtues and limitations of the approach are discussed and future work suggested. [ABSTRACT FROM AUTHOR]

We present a navigation and planning system using vision for extracting non predefined landmarks, a dead-reckoning system generating the integrated movement and a topological map. Localisation and planning remain possible even if the map is partially unknown. An omnidirectional camera gives a panoramic images from which unpredefined landmarks are extracted. The set of landmarks and their azimuths relative to a fixed orientation defines a particular location without any need of an external environment map. Transitions between two locations recognized at time t and t-1 are explicitly coded, and define spatio-temporal transitions. These transitions are the sensory-motor unit chosen to support planning. During exploration, a topological map (our cognitive map) is learned on-line from these transitions without any cartesian coordinates nor occupancy grids. The edges of this map may be modified in order to take into account dynamical changes of the environment. The transitions are linked with the integrated movement used for moving from one place to the others. When planning is required, the activities of transitions coding for the required goal in the cognitive map are enough to bias predicted transitions and to obtain the required movement. [ABSTRACT FROM AUTHOR]

Several models have been proposed for visual homing in insects. These work well in small-scale environments but performance usually degrades significantly when the scale of the environment is increased. We address this problem by extending one such algorithm, the average landmark vector (ALV) model, by using a novel approach to waypoint selection during the construction of multi-leg routes for visual homing. The algorithm, guided by observations of insect behaviour, identifies locations on the boundaries between visual locales and uses them as waypoints. Using this approach, a simulated agent is shown to be capable of significantly better autonomous exploration and navigation through large-scale environments than the standard ALV homing algorithm. [ABSTRACT FROM AUTHOR]

Active vision may be useful to perform landmark-based navigation where landmark relationship requires active scanning of the environment. In this article we explore this hypothesis by evolving the neural system controlling vision and behavior of a mobile robot equipped with a pan/tilt camera so that it can discriminate visual patterns and arrive at the goal zone. The experimental setup employed in this article requires the robot to actively move its gaze direction and integrate information over time in order to accomplish the task. We show that the evolved robot can detect separate features in a sequential manner and discriminate the spatial relationships. An intriguing hypothesis on landmark-based navigation in insects derives from the present results. [ABSTRACT FROM AUTHOR]