Search

In order to solve the problem of stability control for biped robots, the concept of stability training is proposed by using a training platform to exert random disturbance with amplitude limitation on robots that are to be trained. In this work, an approach to achieve a posture stabilizing capability based on stability training and reinforcement learning is explored and verified by simulations. An automatic abstraction method for state space is proposed by using the Gauss basis function and inner evaluation indexes to speed up the learning process. Hierarchical structure stabilizer using the Monte Carlo method is designed according to the concept of variable ZMP. Training samples are extracted from the state transition of the stability training process using balance controllers based on the robot dynamic model. The stabilizers are trained with and without applying the automatic abstraction of state space. Then simulation tests of them are conducted under conditions where the training platform exerts amplitude-limited random disturbances on the robot. Also, the influence of the model errors is studied by introducing deviations of the CoM position during the simulation tests. By comparing the simulation results of two learning stabilizers and the model-based balance controller, it is demonstrated that the designed stabilizer can achieve approximate success rate of the ideal model-based balance controller and exert all the driving ability of the robot under the large disturbance condition of 30 inclination of the platform. Also, the effects of the model error can be overcome by retraining using state transition data with the model error. The active training concept is proposed by applying a training platform.The training platform disturbs robots on it with amplitude-limited random motions.An automatic abstraction method is proposed for the high-dimensional state space.A learning posture stabilizer with hierarchical structure is designed.

Embodied and situated view of cognition is a transdisciplinary framework which stresses the importance of real time and dynamical interaction of an agent with the surrounding environment. This article presents a series of evolutionary robotics experiments that operationalize such concept, training miniature two-wheeled mobile robots to autonomously solve a temporal task. In order to provide a numerical description of the robots’ behavior, chaotic measures are estimated on the attractor reconstructed from the recorded positions of the agent. Chaos theory provides a rigorous mathematical framework consistent with an antireductionist approach, useful for understanding embodied and situated systems while avoiding a decomposition of the integrated system brain–body–environment. Time series are analyzed in detail using nonlinear mathematical tools in order to verify the presence of low-dimensional deterministic dynamical systems, a fundamental prerequisite for chaos theory. In particular, the recorded time series are evaluated with nonlinear prediction error to unveil deterministic dynamics, cross-prediction error to determine the stationarity of the signal, and surrogate data testing to verify the existence of nonlinear components in the underlying system. Estimators for quantifying level of chaos and fractal dimension are applied to suitable datasets. Results show that robots governed by a chaotic dynamic are more efficient at adapting to environments never experience during evolution, demonstrating robustness towards novel and unpredictable situations. Furthermore, chaotic measures, in particular fractal dimension, are correlated with the performance if robots exhibit a similar behavioral strategy.

Evolutionary Robotics (ER) is a field of study that has shown much promise in automating the development of robotic controllers and morphologies. The use of simulators as an alternative to real-world robots is often employed to reduce the time required to develop effective controllers in the ER process. However, the development of adequate simulators is often time-consuming and complex. Simulators are usually constructed from physics models and/or are based on empirically collected data. The vast majority of simulation approaches are based on physics models which can become complex and require specialised knowledge. Alternative simulation approaches that simplify and automate the modelling of real-world phenomena can provide certain advantages over traditional approaches. An alternative simulation approach, such as Artificial Neural Networks (ANNs) that model the real-world phenomena based on empirical data are relatively simple to construct and requires little specialised knowledge. ANN-based simulators are traditionally constructed before the ER process can begin and require the sampling of real-world experimental data. Disadvantages to the traditional approach to ANN-based simulator construction are that the simulator must be created before the ER process can be initiated and a large amount of behavioural data must be collected in order to accurately predict future behaviours. Previous research has successfully demonstrated that ANN-based simulators and controllers can be developed concurrently during the ER process for a simple differentially-steered wheeled robot, while the current research has demonstrated the concurrent approach using a complex snake-like robot. The viability of the concurrent approach was demonstrated on a real-world snake-like robot by performing trajectory planning tasks. Influential factors related to the success of the concurrent approach were also studied by investigating the effects various parameter settings had on success.

Robotic simulators are often used to speed up the Evolutionary Robotics (ER) process. Most simulation approaches are based on physics modelling. However, physics-based simulators can become complex to develop and require prior knowledge of the robotic system. Robotics simulators can be constructed using Machine Learning techniques, such as Artificial Neural Networks (ANNs). ANN-based simulator development usually requires a lengthy behavioural data collection period before the simulator can be trained and used to evaluate controllers during the ER process. The Bootstrapped Neuro-Simulation (BNS) approach can be used to simultaneously collect behavioural data, train an ANN-based simulator and evolve controllers for a particular robotic problem. This paper investigates proposed improvements to the BNS approach and demonstrates the viability of the approach by optimising gait controllers for a Hexapod and Snake robot platform.

Evolutionary Robotics (ER) strives for the automatic creation of robotic controllers and morphologies. The ER process is normally performed in simulation in order to reduce the time required and robot wear. Simulator development is a time consuming process which requires expert knowledge and must traditionally be completed before the ER process can commence. Traditional simulators have limited accuracy, can be computationally expensive and typically do not account for minor operational differences between physical robots.This research proposes the automatic creation of simulators concurrently with the normal ER process. The simulator is derived from an Artificial Neural Network (ANN) to remove the need for formulating an analytical model for the robot. The ANN simulator is improved concurrently with the ER process through real-world controller evaluations which continuously generate behavioural data. Simultaneously, the ER process is informed by the improving simulator to evolve better controllers which are periodically evaluated in the real-world. Hence, the concurrent processes provide further targeted behavioural data for simulator improvement.The concurrent and real-time creation of both controllers and ANN-based simulators is successfully demonstrated for a differentially-steered mobile robot. Various parameter settings in the proposed algorithm are investigated to determine factors pertinent to the success of the proposed approach. Controllers are developed for trajectory planning using Evolutionary Robotics and a differentially-steered mobile robot.A novel approach is proposed for the concurrent development of controllers and a simulator.Robot behaviours are simulated using Artificial Neural Networks.An extensive parameter comparison study of the proposed approach is conducted.

This paper proposes the use of a modular robotic architecture in order to produce feasible robots through evolution. To this end, the main requirements the architecture must fulfill are analyzed and a top-down methodology is employed to obtain the different types of modules that make it up. Specifically, the problem of how to increase the evolvability or evolution friendliness of the system is addressed by considering a heterogeneous modular architecture with a large number of connection faces per module. Afterwards, a prototypical implementation of these modules with the required features is described and different experiments provide an indication of how versatile the architecture is for evolving robot morphologies and control for specific tasks and how easy it is to build them. We present a modular architecture to produce feasible robots through evolution.The architecture is based on a set of a heterogeneous modules.The modules contain a large number of connection faces per module.The design and the implementation of prototype modules is described in detail.Different experiments show its potential for evolving robot morphologies and control.

In this paper it is presented a CPG approach based on phase oscillators to bipedal locomotion where the designer with little a priori knowledge is able to incrementally add basic motion primitives, reaching bipedal walking and other locomotor behaviors as final result. The proposed CPG aims to be a model free solution for the generation of bipedal walking, not requiring the use of inverse kinematical models and previously defined joint trajectories. The proposed incremental construction of bipedal walking allows an easier parametrization and performance evaluation throughout the design process. Furthermore, the approach provides for a developmental mechanism, which enables progressively building a motor repertoire. It would easily benefit from evolutionary robotics and machine learning to explore this aspect. The proposed CPG system also offers a good substrate for the inclusion of feedback mechanisms for modulation and adaptation. It is explored a phase regulation mechanism using load sensory information, observable in vertebrate legged animals. Results from simulations, on HOAP and DARwIn-OP in Webots software show the adequacy of the locomotor system to generate bipedal walk on different robots. Experiments on a DARwIn-OP demonstrates how it can accomplish locomotion and how the proposed work can generalize, achieving several distinct locomotor behaviors.

Reconfigurable robots are set to become a vital factor in the theoretical development and practical utilization of robotics. The core problem in this scientific area is steady information transfer between a swarm and its organisms and vice versa. To this end, we present a basic theoretical framework that stipulates the interoperation between the two modes. We evaluate our proposed framework by constructing 100 mobile microrobots of three different types that initiate the processes of self-reconfigurability and self-repair. The autonomous decision to self-aggregate to an organism mainly derives from the necessity to overcome existing obstructive environmental conditions, e.g. ramps or clefts. The methodological dichotomy that we have chosen to evaluate our concept was to pursue in parallel an approach based on embodied distributed cognition and an evolutionary path mainly based on artificial genomes and reproduction. In this paper, we evaluate these two different approaches in two distinct grand challenges and present the main results.

The modular walking machine Octavio is a bio-inspired robot designed to serve as a testbed for modular neural locomotion control. It consists of up to eight control- and energy-autonomous leg modules, each equipped with 3 active and 2 passive compliant joints and various proprioceptive sensors. Legs may be either used in single leg (treadmill) experiments or can be quickly attached to and detached from bodies with different morphologies. Body morphologies include 4-, 6- and 8-legged machines. Neurocybernetic control is developed and optimized using evolutionary techniques together with a physical simulation of the machine and its environment. This article gives an overview of the machines mechanics, electronics, firmware, configuration and control software. Simple examples demonstrate how the behavior of the simulated and the physical machines are controlled by e.g. neurobiologically motivated modular neural networks.

This paper deals with the study of scaling up behaviors in evolutive robotics (ER). Complex behaviors were obtained from simple ones. Each behavior is supported by an artificial neural network (ANN)-based controller or neurocontroller. Hence, a method for the generation of a hierarchy of neurocontrollers, resorting to the paradigm of Layered Evolution (LE), is developed and verified experimentally through computer simulations and tests in a Khepera^^^(R) micro-robot. Several behavioral modules are initially evolved using specialized neurocontrollers based on different ANN paradigms. The results show that simple behaviors coordination through LE is a feasible strategy that gives rise to emergent complex behaviors. These complex behaviors can then solve real-world problems efficiently. From a pure evolutionary perspective, however, the methodology presented is too much dependent on user's prior knowledge about the problem to solve and also that evolution take place in a rigid, prescribed framework. Mobile robot's navigation in an unknown environment is used as a test bed for the proposed scaling strategies.

This paper surveys fitness functions used in the field of evolutionary robotics (ER). Evolutionary robotics is a field of research that applies artificial evolution to generate control systems for autonomous robots. During evolution, robots attempt to perform a given task in a given environment. The controllers in the better performing robots are selected, altered and propagated to perform the task again in an iterative process that mimics some aspects of natural evolution. A key component of this process-one might argue, the key component-is the measurement of fitness in the evolving controllers. ER is one of a host of machine learning methods that rely on interaction with, and feedback from, a complex dynamic environment to drive synthesis of controllers for autonomous agents. These methods have the potential to lead to the development of robots that can adapt to uncharacterized environments and which may be able to perform tasks that human designers do not completely understand. In order to achieve this, issues regarding fitness evaluation must be addressed. In this paper we survey current ER research and focus on work that involved real robots. The surveyed research is organized according to the degree of a priori knowledge used to formulate the various fitness functions employed during evolution. The underlying motivation for this is to identify methods that allow the development of the greatest degree of novel control, while requiring the minimum amount of a priori task knowledge from the designer.

Developmental robotics is concerned with the design of algorithms that promote robot adaptation and learning through qualitative growth of behaviour and increasing levels of competence. This paper uses ideas and inspiration from early infant psychology (up to three months of age) to examine how robot systems could discover the structure of their local sensory-motor spaces and learn how to coordinate these for the control of action. An experimental learning model is described and results from robotic experiments using the model are presented and discussed.

Evolutionary robotics (ER) is a field of research that applies artificial evolution toward the automatic design and synthesis of intelligent robot controllers. The preceding decade saw numerous advances in evolutionary robotics hardware and software systems. However, the sophistication of resulting robot controllers has remained nearly static over this period of time. Here, we make the case that current methods of controller fitness evaluation are primary factors limiting the further development of ER. To address this, we define a form of fitness evaluation that relies on intra-population competition. In this research, complex neural networks were trained to control robots playing a competitive team game. To limit the amount of human bias or know-how injected into the evolving controllers, selection was based on whether controllers won or lost games. The robots relied on video sensing of their environment, and the neural networks required on the order of 150 inputs. This represents an order of magnitude increase in sensor complexity compared to other research in this field. Evolved controllers were tested extensively in real fully-autonomous robots and in simulation. Results and experiments are presented to characterize the training process and the acquisition of controller competency under different evolutionary conditions.

In this paper we investigated the morphology and controller of biped robots. We viewed them as design components that together can induce dynamically stable bipedal locomotion. We conducted coupled evolution of the morphology and controller of a biped robot, consisting of nine links and eight joints, actuated by oscillators without sensor feedback in three-dimensional simulation. As a result, both pseudo-passive dynamic walkers and active-control walkers emerged, but the pseudo-passive dynamic walkers showed more dynamic stability than the active-control walkers. This is because compliant components in morphology function as noise filters and passive oscillators. Analysis on this latter class of walkers revealed that this was achieved by two novel functions: self-stabilization and self-regulation. Because these functions were handled by the passive dynamics induced in the robot morphology, due to its compliance, we concluded that a computational trade-off between the controller and morphology occurs in these devices. Finally, we have concluded that appropriate compliance is a key to achieving dynamical stability during locomotion.

This paper proposes a mobile printer system (MPS) based on multi-robot cooperation. The system consists of multiple mobile robots, a wireless LAN system, a graphic user interface (GUI), and a host computer. The GUI comprises a user input section, a task allocation optimization section, and a control and communication section. Its operation is as follows: a user draws a picture on an input window of the GUI, and then the host computer commands client printer-robots to reproduce the same on a paper in a finite time. To control multiple robots during this process, two kinds of multi-robot control architectures along with a collision-free arbitration configuration are proposed. One is a decentralized control architecture, which employs subsumption architecture based on behavior-based robotics. The robots continue to seek the nearest line and to draw it repeatedly until all lines are drawn. This architecture needs no pre-planning and is fault-tolerant. Another is a centralized control architecture, which employs an evolutionary algorithm (EA). The host computer optimizes the task (finding time-optimal path) allocation for each robot by using an evolutionary algorithm and sends the optimized job sequence to the robots. To minimize the elapsed time in drawing all the lines, an evolutionary algorithm with a representation of an individual suitable for the MPS is employed for the task optimization. This architecture can minimize the elapsed time effectively and offers the option of distance-optimality in addition to time-optimality. The proposed architectures are simulated and real experiments with three omni-directional robots are carried out to demonstrate the effectiveness and the applicability of the proposed mobile printer system.

This article explores the assumption that a deeper (quantitative) understanding of the information-theoretic implications of sensory-motor coordination can help endow robots not only with better sensory morphologies, but also with better exploration strategies. Speciflcally, we investigate by means of statistical and informationtheoretic measures, to what extent sensory-motor coordinated activity can generate and structure information in the sensory channels of a simulated agent interacting with its surrounding environment. The results show how the usage of correlation, entropy, and mutual information can be employed (a) to segment an observed behavior into distinct behavioral states, (b) to analyze the informational relationship between the difierent components of the sensory-motor apparatus, and (c) to identify patterns (or flngerprints) in the sensory-motor interaction between the agent and its local environment.

In this paper, we Study coordinated motion in a swarm robotic system, called a swarm-bot. A swann-bot is a self-assembling and self-organising artifact, composed of a swarm of s-bots, mobile robots with the ability to connect to and disconnect from each other. The swann-bot concept is particularly suited for tasks that require all-terrain navigation abilities, such as space exploration or rescue in collapsed buildings. As a first step toward the development of more complex control strategies, we investigate the case in which a swarm-bot has to explore an arena while avoiding falling into holes. In such a scenario, individual s-bots have sensory-motor limitations that prevent them navigating efficiently. These limitations can be overcome if the s-bots are made to cooperate. In particular, we exploit the s-bots' ability to physically connect to each other. In order to synthesise the s-bots' controller, we rely on artificial evolution, which we show to be a powerful tool for the production of simple and effective solutions to the hole avoidance task.

Autonomous intelligent agents often must complete non-Markovian sequential tasks, which require complex recurrent neural controllers. In order to improve the convergence of evolution and reduce the computation time, this paper proposes application of an extended evolutionary algorithm. We implemented an extended multi-population genetic algorithm (EMPGA), where subpopulations apply different evolutionary strategies. In addition, subpopulations compete and cooperate among each other. Results show that EMPGA outperformed single population genetic algorithm (SPGA) by efficiently distributing the number of individuals among subpopulations as different strategies became successful during the course of evolution. In addition, the comparison with other multi-population GA shows that competition between subpopulations improved the quality of solution. The evolved neural controllers were also tested in the real hardware of Cyber Rodent robot.

This paper presents results generated with a new evolutionary robotics (ER) simulation environment and its complementary real mobile robot colony research test-bed. Neural controllers producing mobile robot maze searching and exploration behaviors using binary tactile sensors as inputs were evolved in a simulated environment and subsequently transferred to and tested on real robots in a physical environment. There has been a considerable amount of proof-of-concept and demonstration research done in the field of ER control in recent years, most of which has focused on elementary behaviors such as object avoidance and homing. Artificial neural networks (ANN) are the most commonly used evolvable controller paradigm found in current ER literature. Much of the research reported to date has been restricted to the implementation of very simple behaviors using small ANN controllers. In order to move beyond the proof-of-concept stage our ER research was designed to train larger more complicated ANN controllers, and to implement those controllers on real robots quickly and efficiently. To achieve this a physical robot test-bed that includes a colony of eight real robots with advanced computing and communication abilities was designed and built. The real robot platform has been coupled to a simulation environment that facilitates the direct wireless transfer of evolved neural controllers from simulation to real robots (and vice versa). We believe that it is the simultaneous development of ER computing systems in both the simulated and the physical worlds that will produce advances in mobile robot colony research. Our simulation and training environment development focuses on the definition and training of our new class of ANNs, networks that include multiple hidden layers, and time-delayed and recurrent connections. Our physical mobile robot design focuses on maximizing computing and communications power while minimizing robot size, weight, and energy usage. The simulation and ANN-evolution environment was developed using MATLAB. To allow for efficient control software portability our physical evolutionary robots (EvBots) are equipped with a PC-104-based computer running a custom distribution of Linux and connected to the Internet via a wireless network connection. In addition to other high-level computing applications, the mobile robots run a condensed version of MATLAB, enabling ANN controllers evolved in simulation to be transferred directly onto physical robots without any alteration to the code. This is the first paper in a series to be published cataloging our results in this field. © 2004 Elsevier B.V. All rights reserved.

In this work, we describe the evolutionary training of artificial neural network controllers for competitive team game playing behaviors by teams of real mobile robots. This research emphasized the development of methods to automate the production of behavioral robot controllers. We seek methods that do not require a human designer to define specific intermediate behaviors for a complex robot task. The work made use of a real mobile robot colony (EVolutionary roBOTs) and a closely coupled computer-based simulated training environment. The acquisition of behavior in an evolutionary robotics system was demonstrated using a robotic version of the game Capture the Flag . In this game, played by two teams of competing robots, each team tries to defend its own goal while trying to ‘attack’ another goal defended by the other team. Robot neural controllers relied entirely on processed video data for sensing of their environment. Robot controllers were evolved in a simulated environment using evolutionary training algorithms. In the evolutionary process, each generation consisted of a competitive tournament of games played between the controllers in an evolving population. Robot controllers were selected based on whether they won or lost games in the course of a tournament. Following a tournament, the neural controllers were ranked competitively according to how many games they won and the population was propagated using a mutation and replacement strategy. After several hundred generations, the best performing controllers were transferred to teams of real mobile robots, where they exhibited behaviors similar to those seen in simulation including basic navigation, the ability to distinguish between different types of objects, and goal tending behaviors.

We have to prepare the evaluation (fitness) function to evaluate the performance of the robot when we apply the machine learning techniques to the robot application. In many cases, the fitness function is composed of several aspects. Simple implementation to cope with the multiple fitness functions is a weighted summation. This paper presents an adaptive fitness function for the evolutionary computation to obtain the purposive behaviors through changing the weights for the fitness function. As an example task, a basic behavior in a simplified soccer game (shooting a ball into the opponent goal) is selected to show the validity of the adaptive fitness function. Simulation results and real experiments are shown, and a discussion is given. © 2002 Elsevier Science B.V. All rights reserved.

We examine the eventual role of surprise in two domains of human endeavor: classical engineering and what we call “emergent engineering”, with examples relevant to the field of robotics. Placing ourselves within the formal framework of the recently proposed “emergence test”, we argue that the element of surprise, central in the test, serves to illuminate a fundamental difference between industrial and autonomous robots: unsurprise is demanded of classically engineered automation, while a mild form of surprise — unsurprising surprise — must of necessity be tolerated in biologically inspired systems, including behavior-based robotics.

Evolutionary robotics is a research area that makes use of evolutionary computation (EC) to provide a means of learning in robots. In this paper, we discuss a new way of integrating the actual robot and its model during EC. This method, which involves the co-evolution of model parameters, is applied to the problem of learning gaits for hexapod robots. The form of EC used is the cyclic genetic algorithm (CGA). Tests done in simulation and on the robot show that the CGA operating on the co-evolving model of the robot can adapt to changes in the robot’s capabilities to provide a system of anytime learning.

We have used an automatic programming method called genetic programming (GP) for control of a miniature robot. Our earlier work on real-time learning suffered from the drawback of the learning time being limited by the response dynamics of the robot's environment. In order to overcome this problem we have devised a new technique which allows learning from past experiences that are stored in memory. The new method shows its advantage when perfect behavior emerges in experiments quickly and reliably. It is tested on two control tasks, obstacle avoiding and wall following behavior, both in simulation and on the real robot platform Khepera.

The new wave in robotics research began to blossom in the mid-1980s with the convergence of a number of seemingly diverse research trends. All of these were inspired, to different degrees, by biological systems and include neuro-computing, behaviorbased robotics, artificial life, evolutionary methods and behavioral modeling. The new wave is, in a sense, situated between the high-level planning robotics of pre-1980s deliberative artificial intelligence (AI) and low-level control theory. It grew out of a drive for simplicity, adaptivity, and an attitude that "nature knows best"; natural stupiddity rather than Artificial Intelligence. It should be noted, however, there are many different levels of "biological plausibility" in the robotics literature. They range from general

Recently, a new approach involving a form of simulated evolution has been proposed to build autonomous robots. However, it is still not clear if this approach is adequate for real life problems. In this paper we show how control systems that perform a non-trivial sequence of behaviors can be obtained with this methodology by “canalizing” the evolutionary process in the right direction. In the experiment described in the paper, a mobile robot was successfully trained to keep clear an arena surrounded by walls by locating, recognizing, and grasping “garbage” objects and by taking collected objects outside the arena. The controller of the robot was evolved in simulation and then downloaded and tested on the real robot. We also show that while a given amount of supervision may canalize the evolutionary process in the right direction the addition of unnecessary constraints can delay the evolution of the desired behavior.

We give an overview of evolutionary robotics research at Sussex over the last five years. We explain and justify our distinctive approaches to (artificial) evolution, and to the nature of robot control systems that are evolved. Results are presented from research with evolved controllers for autonomous mobile robots, simulated robots, co-evolved animats, real robots with software controllers, and a real robot with a controller directly evolved in hardware.