Search

The ability of robots to write Chinese strokes, which is recognized as a sophisticated task, involves complicated kinematic control algorithms. The conventional approaches for robotic writing of Chinese strokes often suffer from limited font generation methods, which limits the ability of robots to perform high-quality writing. This paper instead proposes a developmental evolutionary learning framework that enables a robot to learn to write fundamental Chinese strokes. The framework first considers the learning process of robotic writing as an evolutionary easy-to-difficult procedure. Then, a developmental learning mechanism called “Lift-constraint, act and saturate” that stems from developmental robotics is used to determine how the robot learns tasks ranging from simple to difficult by building on the learning results from the easy tasks. The developmental constraints, which include altitude adjustments, number of mutation points, and stroke trajectory points, determine the learning complexity of robot writing. The developmental algorithm divides the evolutionary procedure into three developmental learning stages. In each stage, the stroke trajectory points gradually increase, while the number of mutation points and adjustment altitudes gradually decrease, allowing the learning difficulties involved in these three stages to be categorized as easy, medium, and difficult. Our robot starts with an easy learning task and then gradually progresses to the medium and difficult tasks. Under various developmental constraint setups in each stage, the robot applies an evolutionary algorithm to handle the basic shapes of the Chinese strokes and eventually acquires the ability to write with good quality. The experimental results demonstrate that the proposed framework allows a calligraphic robot to gradually learn to write five fundamental Chinese strokes and also reveal a developmental pattern similar to that of humans. Compared to an evolutionary algorithm without the developmental mechanism, the proposed framework achieves good writing quality more rapidly.

Robotics currently lacks fully autonomous capabilities, especially where task knowledge is incomplete and optimal robotic solutions cannot be pre-engineered. The intersection of evolutionary robotics, artificial life and embodied artificial intelligence presents a promising paradigm for generating multitask problem-solvers suitable for adapting over extended periods in unexplored, remote and hazardous environments. To address the automation of evolving robotic systems, we propose fully autonomous, embodied artificial-life factories and laboratories, situated in various environments as multi-task problem-solvers. Such integrated factories and laboratories would be adaptive solution designers, producing fit-for-purpose physical robots with accelerated artificial evolution that experiment to continually discover new tasks. Such tasks would be stepping-stones towards accomplishing given mission objectives over extended periods (days to decades). Rather than being purely speculative, prerequisite technologies to realize such factories have been experimentally demonstrated. Currently, vast scientific and enterprise opportunities await in applications such as asteroid mining, terraforming, space and deep sea exploration, though no suitable solution exists. The proposed embodied artificial-life factories and laboratories, termed: AutoFac, use robot production equipment run by artificial evolution controllers to collect and synthesize environmental information (from robotic sensory systems). Such information is merged with current needs and mission objectives to create new robot embodiment and task definitions that are environmentally adapted and balance task-oriented behavior with exploration. AutoFac is thus generalist (deployable in many environments) but continually produces specialist solutions within such environments a perpetual robot machine.

Displaced communication, whereby individuals communicate regarding a subject that is not immediately present (spatially or temporally), is one of the key innovations of human language. It also occurs in a few animal species, most notably the honeybee, where the waggle dance is used to communicate the location and quality of a patch of flowers. However, it is difficult to study how it emerged given the paucity of species displaying this capacity and the fact that it often occurs via complex multimodal signals. To address this issue, we developed a novel paradigm in which we conducted experimental evolution with foraging agents endowed with neural networks that regulate their movement and the production of signals. Displaced communication readily evolved but, surprisingly, agents did not use signal amplitude to convey information on food location. Instead, they used signal onset-delay and duration-based mode of communication, which depends on the motion of the agent within a communication area. When agents were experimentally prevented from using these modes of communication, they evolved to use signal amplitude instead. Interestingly, this mode of communication was more efficient and led to higher performance. Subsequent controlled experiments suggested that this more efficient mode of communication failed to evolve because it took more generations to emerge than communication grounded on the onset-delay and length of signaling. These results reveal that displaced communication is likely to initially evolve from non-communicative behavioral cues providing incidental information with evolution later leading to more efficient communication systems through a ritualization process., Laurent Keller received Swiss NSF (https://www.snf.ch/en) grant nr. R32I3_141063. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

We introduce a method that permits to co-evolve the body and the control properties of robots. It can be used to adapt the morphological traits of robots with a hand-designed morphological bauplan or to evolve the morphological bauplan as well. Our results indicate that robots with co-adapted body and control traits outperform robots with fixed hand-designed morphologies. Interestingly, the advantage is not due to the selection of better morphologies but rather to the mutual scaffolding process that results from the possibility to co-adapt the morphological traits to the control traits and vice versa. Our results also demonstrate that morphological variations do not necessarily have destructive effects on robots’ skills.

In evolutionary robotics, robot controllers are often evolved in simulation, as using the physical robot for fitness evaluation can take a prohibitively long time. Simulators provide a quick way to evaluate controller fitness. A simulator is tasked with providing appropriate sensor information to the controller. If the robot has an on-board camera, an entire virtual visual environment is needed to simulate the camera’s signal. In the past, these visual environments have been constructed by hand, requiring the use of hand-crafted models, textures and lighting, which is a tedious and time-consuming process. This paper proposes a deep neural network-based architecture for simulating visual environments. The neural networks are trained exclusively from images captured from the robot, creating a 3-dimensional visual environment without using hand-crafted models, textures or lighting. It does not rely on any external domain specific datasets, as all training data is captured in the physical environment. Robot controllers were evolved in simulation to discern between objects with different colours and shapes, and they successfully completed the same task in the real world.

This paper focuses on the collective cognition by robotic swarms. The robotic swarms expected to perform tasks that are beyond the capability of a single robot by collective behavior that emerge from local interactions, similar to biological swarms. However, the robotic swarms have to rely on the collective cognition more than biological swarms when considering the limitation in sensory capabilities and the cost of each robot. In this paper, we develop controllers for a robotic swarm to accomplish a foraging task that requires collective cognition. In this task, robots have to both collectively distinguish two objects, namely food and poison, and cooperatively transport food objects to the nest. We applied an evolutionary robotics approach with the covariance matrix adaptation evolution strategy to develop controllers for robotic swarms. The results of computer simulations show that collective cognition was successfully developed, which allows the robots to transport only food objects. In addition, we also perform experiments to examine the scalability and the flexibility of the developed controllers.

Theory of mind (ToM) is the ability to understand others’ mental states (e.g., intentions). Studies on human ToM show that the way we understand others’ mental states is very efficient, in the sense that observing only some portion of others’ behaviors can lead to successful performance. Recently, ToM has gained interest in robotics to build robots that can engage in complex social interactions. Although it has been shown that robots can infer others’ internal states, there has been limited focus on the data utilization of ToM mechanisms in robots. Here we show that robots can infer others’ intentions based on limited information by selectively and flexibly using behavioral cues similar to humans. To test such data utilization, we impaired certain parts of an actor robot’s behavioral information given to the observer, and compared the observer’s performance under each impairment condition. We found that although the observer’s performance was not perfect compared to when all information was available, it could infer the actor’s mind to a degree if the goal-relevant information was intact. These results demonstrate that, similar to humans, robots can learn to infer others’ mental states with limited information.

The concept of an artificial corporeal machine that can reproduce has attracted the attention of researchers from various fields over the past century. Some have approached the topic with a desire to understand biological life and develop artificial versions; others have examined it as a potentially practical way to use material resources from the moon and Mars to bootstrap the exploration and colonization of the solar system. This review considers both bodies of literature, with an emphasis on the underlying principles required to make self-replicating robotic systems from raw materials a reality. We then illustrate these principles with machines from our laboratory and others and discuss how advances in new manufacturing processes such as 3-D printing can have a synergistic effect in advancing the development of such systems.

The framework of Simulation-to-real learning, i.e, learning policies in simulation and transferring those policies to the real world is one of the most promising approaches towards data-efficient learning in robotics. However, due to the inevitable reality gap between the simulation and the real world, a policy learned in the simulation may not always generate a safe behaviour on the real robot. As a result, during adaptation of the policy in the real world, the robot may damage itself or cause harm to its surroundings. In this work, we introduce a novel learning algorithm called SafeAPT that leverages a diverse repertoire of policies evolved in the simulation and transfers the most promising safe policy to the real robot through episodic interaction. To achieve this, SafeAPT iteratively learns a probabilistic reward model as well as a safety model using real-world observations combined with simulated experiences as priors. Then, it performs Bayesian optimization on the repertoire with the reward model while maintaining the specified safety constraint using the safety model. SafeAPT allows a robot to adapt to a wide range of goals safely with the same repertoire of policies evolved in the simulation. We compare SafeAPT with several baselines, both in simulated and real robotic experiments and show that SafeAPT finds high-performance policies within a few minutes in the real world while minimizing safety violations during the interactions., Comment: Under review. For video of the paper http://tiny.cc/safeAPT

The aim of this paper is to present the genetic algorithm used in programming a solar tracker robot. The solar tracker robot can be used to increase the efficiency of solar panels by rotating them toward the Sun. When the efficiency of solar panels is increased, they can reduce the space occupied by them. Efficiency can also be increased by genetic algorithms, where a solar panel can evolve and thus track the Sun more precisely. The genetic algorithm implemented in the solar tracker is the main focus of this paper. The genetic algorithm is presented with a flow chart and formulas. After this, the system is implemented on a solar tracker robot and validated with real experiments. The solar tracker robot algorithm is implemented in the C language on a microcontroller built on an FPGA platform.

In this paper, we present an implementation of social learning for swarm robotics. We consider social learning as a distributed online reinforcement learning method applied to a collective of robots where sensing, acting and coordination are performed on a local basis. While some issues are specific to artificial systems, such as the general objective of learning efficient (and ideally, optimal) behavioural strategies to fulfill a task defined by a supervisor, some other issues are shared with social learning in natural systems. We discuss some of these issues, paving the way towards cumulative cultural evolution in robot swarms, which could enable complex social organization necessary to achieve challenging robotic tasks.This article is part of a discussion meeting issue ‘The emergence of collective knowledge and cumulative culture in animals, humans and machines’.

Simultaneously evolving morphologies (bodies) and controllers (brains) of robots can cause a mismatch between the inherited body and brain in the offspring. To mitigate this problem, the addition of an infant learning period by the so-called Triangle of Life framework has been proposed relatively long ago. However, an empirical assessment is still lacking to-date. In this paper we investigate the effects of such a learning mechanism from different perspectives. Using extensive simulations we show that learning can greatly increase task performance and reduce the number of generations required to reach a certain fitness level compared to the purely evolutionary approach. Furthermore, although learning only directly affects the controllers, we demonstrate that the evolved morphologies will be also different. This provides a quantitative demonstration that changes in the brain can induce changes in the body. Finally, we examine the concept of morphological intelligence quantified by the ability of a given body to learn. We observe that the learning delta, the performance difference between the inherited and the learned brain, is growing throughout the evolutionary process. This shows that evolution is producing robots with an increasing plasticity, that is, consecutive generations are becoming better and better learners which in turn makes them better and better at the given task. All in all, our results demonstrate that the Triangle of Life is not only a concept of theoretical interest, but a system architecture with practical benefits., Comment: Frontiers in Robotics and AI. arXiv admin note: text overlap with arXiv:2107.08249

© Copyright © 2021 Eiben.This paper takes a critical look at the concept of real-world robot evolution discussing specific challenges for making it practicable. After a brief review of the state of the art several enablers are discussed in detail. It is noted that sample efficient evolution is one of the key prerequisites and there are various promising directions towards this in different stages of maturity, including learning as part of the evolutionary system, genotype filtering, and hybridizing real-world evolution with simulations in a new way. Furthermore, it is emphasized that an evolutionary system that works in the real world needs robots that work in the real world. Obvious as it may seem, to achieve this significant complexification of the robots and their tasks is needed compared to the current practice. Finally, the importance of not only building but also understanding evolving robot systems is emphasised, stating that in order to have the technology work we also need the science behind it.

What makes a quality-diversity (QD) algorithm effective? This question is increasingly studied as such algorithms gain in popularity. Recent work on Novelty Search linked its efficiency to the evolution of a population under a dynamic selection pressure defined by the novelty metric, which allows the population to converge to highly evolvable individuals, on which spending the evolutionary budget allows to navigate the behavior space. MAP-Elites, another popular QD algorithm, does not use an explicit population, instead sampling the parent population directly from the behavioral archive at each generation. This sampling can be uniform, or biased according to criteria such as curiosity, which favor individuals that previously generated successful offsprings. In this article, we show that such improved selection schemes are efficient because they create a dynamic pseudo-population that mimics the desirable qualities of a Novelty Search population. We do this by proposing another simpler selection scheme, youth, that simply focuses the evolutionary budget on such a dynamic pseudo-population, and by showing it exhibits similar behavior and performance as curiosity without explicitly selecting for previously successful parents. This has important implications for the design of future QD algorithms taking the best of both archive-based and population-based methods.

Organisms with different body morphology and movement dynamics have distinct abilities to move through the environment. Despite such truism, there is a lack of general principles that predict which shapes and dynamics make the organisms more fit to move. Studying a minimal yet embodied soft robot model under the influence of gravity, we find three features that predict robot locomotion fitness: (1) A larger body is better. (2) Two-point contact with the ground is better than one-point contact. (3) Out-of-phase oscillating body parts increase locomotion fitness. These design principles can guide the selection rules for evolutionary algorithms to obtain robots with higher locomotion fitness.

Locating odour sources is a hard task that has been addressed with a large variety of AI methods to produce search strategies with different levels of efficiency and robustness. However, it is still not clear how to evaluate those strategies. Simply evaluating the robot's ability to reach the goal may produce deceptive fitness values, favouring poor strategies that do not generalise. Conversely, including prior knowledge may bias the learning process. This work studies the impact of evaluation functions with various degrees of prior knowledge, in evolving search strategies. The baseline is set by performing multiple evaluations of each strategy with a function that only evaluates the task efficiency. A function was found that is able to produce strategies with equivalent performance to those of the baseline, whilst performing a single evaluation.

Robots are still missing the ability to adapt to new environments. However, biological systems are able to adapt to new environments with ease; perhaps because they have the ability to react to environmental input during a growth phase with changes not only in behaviour, but also morphology. Yet within the field of robots, environmental based development of morphology is an under researched area. In this paper we use an evolutionary algorithm to evolve neural cellular automata capable of inducing environmental based developmental plasticity in robots. We use the kinetic energy of each cell and its neighbours as an input to our network, the output of which determines the position of new cell growth. We evolve our neural cellular automata first in three individual environments and then also for performance in multiple environments. We show that the networks that use environmental feedback outperform those that do not and that by introducing environmental feedback during development, more adaptive and better performing robots are potentially possible.

Engineers, control theorists, and neuroscientists often view the delay imposed by finite signal propagation velocities as a problem that needs to be compensated for or avoided. In this article, we consider the alternative possibility that in some cases, signal delay can be used functionally, that is, as an essential component of a cognitive system. To investigate this idea, we evolve a minimal robot controller to solve a basic stimulus-distinction task. The controller is constrained so that the solution must utilize a delayed recurrent signal. Different from previous evolutionary robotics studies, our controller is modeled using delay differential equations, which (unlike the ordinary differential equations of conventional continuous-time recurrent neural networks) can accurately capture delays in signal propagation. We analyze the evolved controller and its interaction with its environment using classical dynamical systems techniques. The analysis shows what kinds of invariant sets underlie the various successful and unsuccessful performances of the robot, and what kinds of bifurcations produce these invariant sets. In the second phase of our analysis, we turn our attention to the parameter θ, which describes the amount of signal delay included in the model. We show how the delay destabilizes certain attractors that would exist if there were no delay and creates other stable attractors, resulting in an agent that performs well at the target task.

This work presents a genetic programming control design methodology that extends the traditional behavior–based control strategy towards a synthetic-analytic perspective. The proposed approach considers the internal and external dynamics of the system, providing solutions to a general structure, and including analytic functions, which can be studied within the Control Theory framework. The method is illustrated for the tracking control problem under bounded velocity restrictions of a nonholonomic wheeled mobile robot . A classic Control Theory (CT) based controller that solves the tracking problem (but not the velocity constraint requirement) is chosen from the literature; based on its stability properties , a modified structure where the search of suitable analytic basis behaviors, fulfilling both control objectives simultaneously, can be introduced. The proposed framework takes the form of a learning process based on Genetic Programming (GP) which generates a set of nonlinear tracking controllers satisfying pre-specified velocity bounds. A collection of 9113 suitable nonlinear solutions were obtained to augment the ground controller. Simulations and real–time experiments are performed to illustrate the effectiveness of the methodology through the testing of the models with the best performance, as well as those with lower structural complexity.

I present criteria or benchmarks of personhood that robotic agents must pass for them to match us in social intelligence and indicate some current successes and difficulties for the future with res...

SUMMARYThis paper investigates artificial neural network (ANN)-based simulators as an alternative to physics-based approaches for evolving controllers in simulation for a complex snake-like robot. Prior research has been limited to robots or controllers that are relatively simple. Benchmarks are performed in order to identify effective simulator topologies. Additionally, various controller evolution strategies are proposed, investigated and compared. Using ANN-based simulators for controller fitness estimation during controller evolution is demonstrated to be a viable approach for the high-dimensional problem specified in this work.

It has often been found that students appreciate hands-on work, and find that they learn more with courses that include a project than those relying solely on conventional lectures and tests. This type of project driven learning is a key component of “Inquiry-based learning” (IBL), which aims at teaching methodology as well as content by incorporating the student as an actor rather than a spectator. Robotics applications are especially well-suited for IBL due to the value of trial and error experience, the multiple possibilities for students to implement their own ideas, and the importance of programming, problem-solving and electro-mechanical skills in real world engineering and science jobs. Furthermore, robotics platforms can be useful teaching media and learning tools for a variety of topics. Here, we present RoboGen: an open-source, web-based, software and hardware platform for Robotics and Artificial Intelligence with a particular focus on Evolutionary Robotics. We describe the platform in detail, compare it to existing alternatives, and present results of its use as a platform for Inquiry-based learning within a master's level course at the Ecole Polytechnique Fédérale de Lausanne (EPFL).

SummaryMission planning is a complex motion planning problem specified by using Temporal Logic constituting of Boolean and temporal operators, typically solved by model verification algorithms with an exponential complexity. The paper proposes co-evolutionary optimization thus building an iterative solution to the problem. The language for mission specification is generic enough to represent everyday missions, while specific enough to design heuristics. The mission is broken into components which cooperate with each other. The experiments confirm that the robot is able to outperform the search, evolutionary and model verification techniques. The results are demonstrated by using a Pioneer LX robot.

In swimming virtual creatures, there is often a disparity between the level of detail in simulating a swimmer’s body and that of the fluid it moves in. In order to address this disparity, we have developed a new approach to modelling swimming virtual creatures using pseudo-soft bodies and particle-based fluids, which has sufficient realism to investigate a larger range of body-environment interactions than are usually included. As this comes with increased computational costs, which may be severe, we have also developed a means of reducing the volume of fluid that must be simulated.

The Evolutionary Robotics (ER) process has been applied extensively to developing control programs to achieve locomotion in legged robots, as an automated alternative to the arduous task of manually creating control programs for such robots. The evolution of such controllers is typically performed in simulation by making use of a physics engine-based robotic simulator. Making use of such physics-based simulators does, however, have certain challenges associated with it, such as these simulators’ computational inefficiency, potential issues with lack of accuracy and the human effort required to construct such simulators. The current study therefore proposed and investigated an alternative method of simulation for a hexapod (six-legged) robot in the ER process, and directly compared this newly-proposed simulation method to traditional physics-based simulation. This alternative robotic simulator was built based solely on experimental data acquired directly from observing the behaviour of the robot. This data was used to construct a simulator for the robot based on Artificial Neural Networks (ANNs). To compare this novel simulation method to traditional physics simulation, the ANN-based simulators were used to evolve simple open-loop locomotion controllers for the robot in simulation. The real-world performance of these controllers was compared to that of controllers evolved in a more traditional physics-based simulator. The obtained results indicated that the use of ANN-based simulators produced controllers which could successfully perform the required locomotion task on the real-world robot. In addition, the controllers evolved using the ANN-based simulators allowed the real-world robot to move further than those evolved in the physics-based simulator and the ANN-based simulators were vastly more computationally efficient than the physics-based simulator. This study thus decisively indicated that ANN-based simulators offer a superior alternative to widely-used physics simulators in ER for the locomotion task considered.

Evolutionary robotics is concerned with optimizing autonomous robots for one or more specific tasks. Remarkably, the energy needed to operate autonomously is hardly ever considered. This is quite striking because energy consumption is a crucial factor in real-world applications and ignoring this aspect can increase the reality gap. In this paper, we aim to mitigate this problem by extending our robot simulator framework with a model of a battery module and studying its effect on robot evolution. The key idea is to include energy efficiency in the definition of fitness. The robots will need to evolve to achieve high gait speed and low energy consumption. Since our system evolves the robots’ morphologies as well as their controllers, we investigate the effect of the energy extension on the morphologies and on the behavior of the evolved robots. The results show that by including the energy consumption, the evolution is not only able to achieve higher task performance (robot speed), but it reaches good performance faster. Inspecting the evolved robots and their behaviors discloses that these improvements are not only caused by better morphologies, but also by better settings of the robots’ controller parameters.

This research proposed augmenting Evolutionary Robotics controllers with their own robotic simulator. The motivation behind this idea is that the controller could use the simulator to approximate the effects of its actions. This would allow the controller to maintain an approximation of the robot’s location and orientation. It was hypothesised that this information could enhance a controller’s navigation ability. Both regular and augmented controllers were tested and compared on a task that required them to employ adaptive navigation strategies to efficiently solve the task. Augmented controllers were shown to perform better and possess superior behaviours in comparison to regular controllers.

We demonstrate the training of Spiking Neural Networks (SNN) in a novel multi-agent Evolutionary Robotics (ER) framework inspired by competitive evolutionary environments in nature. The topology of a SNN along with morphological parameters of the bot it controls in the ER environment is together treated as a phenotype. Rules of the framework select certain bots and their SNNs for reproduction and others for elimination based on their efficacy in capturing food in a competitive environment. While the bots and their SNNs are not explicitly trained to survive or reproduce using loss functions, these drives emerge implicitly as they evolve to hunt food and survive. Their efficiency in capturing food exhibits the evolutionary signature of punctuated equilibrium. We use this signature to compare the performances of two evolutionary inheritance algorithms on the phenotypes, Mutation and Crossover with Mutation, using ensembles of 100 experiments for each algorithm. We find that Crossover with Mutation promotes 40% faster learning in the SNN than mere Mutation with a statistically significant margin.

An open question for both natural and artificial evolutionary systems is how, and under what environmental and evolutionary conditions complexity evolves. This study investigates the impact of increasingly complex task environments on the evolution of robot complexity. Specifically, the impact of evolving body-brain couplings on locomotive task performance, where robot evolution was directed by either body-brain exploration (novelty search) or objective-based (fitness function) evolutionary search. Results indicated that novelty search enabled the evolution of increased robot body-brain complexity and efficacy given specific environment conditions. The key contribution is thus the demonstration that body-brain exploration is suitable for evolving robot complexity that enables high fitness robots in specific environments.

Quality-Diversity (QD) optimisation is a new family of learning algorithms that aims at generating collections of diverse and high-performing solutions. Among those algorithms, the recently introduced Covariance Matrix Adaptation MAP-Elites (CMA-ME) algorithm proposes the concept of emitters, which uses a predefined heuristic to drive the algorithm's exploration. This algorithm was shown to outperform MAP-Elites, a popular QD algorithm that has demonstrated promising results in numerous applications. In this paper, we introduce Multi-Emitter MAP-Elites (ME-MAP-Elites), an algorithm that directly extends CMA-ME and improves its quality, diversity and data efficiency. It leverages the diversity of a heterogeneous set of emitters, in which each emitter type improves the optimisation process in different ways. A bandit algorithm dynamically finds the best selection of emitters depending on the current situation. We evaluate the performance of ME-MAP-Elites on six tasks, ranging from standard optimisation problems (in 100 dimensions) to complex locomotion tasks in robotics. Our comparisons against CMA-ME and MAP-Elites show that ME-MAP-Elites is faster at providing collections of solutions that are significantly more diverse and higher performing. Moreover, in cases where no fruitful synergy can be found between the different emitters, ME-MAP-Elites is equivalent to the best of the compared algorithms.

Autonomous robots are increasingly used in remote and hazardous environments, where damage to sensory-actuator systems cannot be easily repaired. Such robots must therefore have controllers that continue to function effectively given unexpected malfunctions and damage to robot morphology. This study applies the Intelligent Trial and Error (IT&E) algorithm to adapt hexapod robot control to various leg failures and demonstrates the IT&E map-size parameter as a critical parameter in influencing IT&E adaptive task performance. We evaluate robot adaptation for multiple leg failures on two different map-sizes in simulation and validate evolved controllers on a physical hexapod robot. Results demonstrate a trade-off between adapted gait speed and adaptation duration, dependent on adaptation task complexity (leg damage incurred), where map-size is crucial for generating behavioural diversity required for adaptation.

The social brain hypothesis proposes that enlarged brains have evolved in response to the increasing cognitive demands that complex social life in larger groups places on primates and other mammals. However, this reasoning can be challenged by evidence that brain size has decreased in the evolutionary transitions from solitary to social larger groups in the case of Neolithic humans and some eusocial insects. Different hypotheses can be identified in the literature to explain this reduction in brain size. We evaluate some of them from the perspective of recent approaches to cognitive science, which support the idea that the basis of cognition can span over brain, body, and environment. Here we show through a minimal cognitive model using an evolutionary robotics methodology that the neural complexity, in terms of neural entropy and degrees of freedom of neural activity, of smaller-brained agents evolved in social interaction is comparable to the neural complexity of larger-brained agents evolved in solitary conditions. The nonlinear time series analysis of agents' neural activity reveals that the decoupled smaller neural network is intrinsically lower dimensional than the decoupled larger neural network. However, when smaller-brained agents are interacting, their actual neural complexity goes beyond its intrinsic limits achieving results comparable to those obtained by larger-brained solitary agents. This suggests that the smaller-brained agents are able to enhance their neural complexity through social interaction, thereby offsetting the reduced brain size.

Multi-level evolution (MLE) is a novel robotic design paradigm which decomposes the design problem into layered sub-tasks that involve concurrent search for appropriate materials, component geometry and overall morphology. This has a number of advantages, mainly in terms of quality and scalability. In this paper, we present a hierarchical approach to robotic design based on the MLE architecture. The design problem involves finding a robotic design which can be used to perform a specific locomotion task. At the materials layer, we put together a simple collection of materials which are represented by combinations of mechanical properties such as friction and restitution. At the components layer we combine these materials with geometric design to form robot limbs. Finally, at the robot layer we introduce these evolved limbs into robotic body-plans and learn control policies to form complete robots. Quality-diversity algorithms at each level allow for the discovery of a wide variety of reusable elements. The results strongly support the initial claims for the benefits of MLE, allowing for the discovery of designs that would otherwise be difficult to achieve with conventional design paradigms.

Taking inspiration from the navigation ability of humans, this study investigated a method of providing robotic controllers with a basic sense of position. It incorporated robotic simulators into robotic controllers to provide them with a mechanism to approximate the effects their actions had on the robot. Controllers with and without internal simulators were tested and compared. The proposed controller architecture was shown to outperform the regular controller architecture. However, the longer an internal simulator was executed, the more inaccurate it became. Thus, the performance of controllers with internal simulators reduced over time unless their internal simulator was periodically corrected.

In nature, very few animals locomote on two legs. Static bipedalism can be found in four limbed and five limbed animals like dogs, cats, birds, monkeys and kangaroos, but it cannot be seen in hexapods or other multi-limbed animals. In this paper, we present a simulation with a novel perspective on the evolution of static bipedalism, with a virtual creature evolving its body and controllers, and we apply an evolutionary algorithm to explore the locomotion transition from octapods to bipods. We find that the presence of four limbs in the evolutionary trajectory of the creature scaffolds a parametric jump that enables bipedalism, and shows that hexapods, without undergoing such transformation, struggle to evolve into bipeds. An analysis of the transitional parameters points to the role of a shorter femur length in helping maintain the stability of the body, and the tibia length is responsible for improving the forward speed.

The ability of a robot to adapt in-mission to achieve an assigned goal is highly desirable. This thesis project places an emphasis on employing learning-based intelligent control methodologies to the development and implementation of an autonomous unmanned aerial vehicle (UAV). Flight control is carried out by evolving spiking neural networks (SNNs) with Hebbian plasticity. The proposed implementation is capable of learning and self-adaptation to model variations and uncertainties when the controller learned in simulation is deployed on a physical platform. Controller development for small multicopters often relies on simulations as an intermediate step, providing cheap, parallelisable, observable and reproducible optimisation with no risk of damage to hardware. Although model-based approaches have been widely utilised in the process of development, loss of performance can be observed on the target platform due to simplification of system dynamics in simulation (e.g., aerodynamics, servo dynamics, sensor uncertainties). Ignorance of these effects in simulation can significantly deteriorate performance when the controller is deployed. Previous approaches often require mathematical or simulation models with a high level of accuracy which can be difficult to obtain. This thesis, on the other hand, attempts to cross the reality gap between a low-fidelity simulation and the real platform. This is done using synaptic plasticity to adapt the SNN controller evolved in simulation to the actual UAV dynamics. The primary contribution of this work is the implementation of a procedural methodology for SNN control that integrates bioinspired learning mechanisms with artificial evolution, with an SNN library package (i.e. eSpinn) developed by the author. Distinct from existing SNN simulators that mainly focus on large-scale neuron interactions and learning mechanisms from a neuroscience perspective, the eSpinn library draws particular attention to embedded implementations on hardware that is applicable for problems in the robotic domain. This C++ software package is not only able to support simulations in the MATLAB and Python environment, allowing rapid prototyping and validation in simulation; but also capable of seamless transition between simulation and deployment on the embedded platforms. This work implements a modified version of the NEAT neuroevolution algorithm and leverages the power of evolutionary computation to discover functional controller compositions and optimise plasticity mechanisms for online adaptation. With the eSpinn software package the development of spiking neurocontrollers for all degrees of freedom of the UAV is demonstrated in simulation. Plastic height control is carried out on a physical hexacopter platform. Through a set of experiments it is shown that the evolved plastic controller can maintain its functionality by self-adapting to model changes and uncertainties that take place after evolutionary training, and consequently exhibit better performance than its non-plastic counterpart.

Evolutionary Algorithms have been applied in robotics over the last quarter of a century to simultaneously evolve robot body morphology and controller. However, the literature shows that in this area, one is still unable to generate robots that perform better than conventional manual designs, even for simple tasks. It is noted that the main hindrance to satisfactory evolution is poor controller generated as a result of the simultaneous variation of the morphology and controller. As the controller is a result of pure evolution, it is not given a chance to improve before fitness is calculated, which is the equivalent of reproducing immediately after birth in biological evolution. Therefore, to improve co-evolution and to bring artificial robot evolution a step closer to biological evolution, this paper introduces Reinforced Co-evolution Algorithm (ReCoAl), which is a hybrid of an Evolutionary and a Reinforcement Learning algorithm. It combines the evolutionary and learning processes found in nature to co-evolve robot morphology and controller. ReCoAl works by allowing a direct policy gradient based RL algorithm to improve the controller of an evolved robot to better utilise the available morphological resources before fitness evaluation. The ReCoAl is tested for evolving mobile robots to perform navigation and obstacle avoidance. The findings indicate that the controller learning process has both positive and negative effects on the progress of evolution, similar to observations in evolutionary biology. It is also shown how, depending on the effectiveness of the learning algorithm, the evolver generates robots with similar fitness in different generations.

Designing robots by hand can be costly and time consuming, especially if the robots have to be created with novel materials, or be robust to internal or external changes. In order to create robots automatically, without the need for human intervention, it is necessary to optimise both the behaviour and the body design of the robot. However, when co-optimising the morphology and controller of a locomoting agent the morphology tends to converge prematurely, reaching a local optimum. Approaches such as explicit protection of morphological innovation have been used to reduce this problem, but it might also be possible to increase exploration of morphologies using a more indirect approach. We explore how changing the environment, where the agent locomotes, affects the convergence of morphologies. The agents’ morphologies and controllers are co-optimised, while the environments the agents locomote in are evolved open-endedly with the Paired Open-Ended Trailblazer (POET). We compare the diversity, fitness and robustness of agents evolving in environments generated by POET to agents evolved in handcrafted curricula of environments. Our agents each contain of a population of individuals being evolved with a genetic algorithm. This population is called the agent-population. We show that agent-populations evolving in open-endedly evolving environments exhibit larger morphological diversity than agent-populations evolving in hand crafted curricula of environments. POET proved capable of creating a curriculum of environments which encouraged both diversity and quality in the populations. This suggests that POET may be capable of reducing premature convergence in co-optimisation of morphology and controllers.

Evolution has been a topic of interest that has been explored extensively throughout recent history. Ever since the proposal of the evolutionary theory by Darwin, there have been attempts made to validate, extend, explore and exploit the different aspects of this theory. This study sets forth methods to explore elements of what has been proposed by Darwinian evolution using Braitenberg vehicles, simple machines with sensory-motor couplings in an environmental context. In this study, a simulation environment is set up based upon the descriptions of neurophysiologist Valentino Braitenberg. This simulation environment is then utilized to carry out the thought experiments envisioned by Braitenberg for the simplest nontrivial kinds of vehicles. The methodology and results for the carried out experiments are detailed in this paper. Apart from an understanding of whether the environmental setup affects evolution, specifically the number of light sources (stimuli) and location of these light sources, the experiments show an interesting trend regarding dynamic equilibrium of the evolutionary process, the ramifications of which might not have been understood well enough previously. It is concluded that ecological setup, as well as the initial genetic makeup of the vehicles, play a crucial role in the evolution of vehicles in scenarios laid out in this study. Further, the placement of stimuli (location of lights) and the number of the stimuli have a visible effect on the survivability of vehicle types (species).

In evolutionary robotics, we evaluate individuals by placing them in an initial configuration in the environment, and then measure their fitness over a period of time. The choice of initial configuration has a direct impact on the fitness of an individual and thereby also the overarching evolutionary process. In this paper, we propose the concept of dynamic initial configurations, which is an initial configuration that is neither random nor fixed, but develops dynamically in response to the evolutionary process. As an example we have implemented a competitive co-evolutionary algorithm where initial configurations and controllers are evolved together to solve an obstacle avoidance task of a mobile robot. We show that, while a evolutionary approach taken from literature consistently fails, the co-evolutionary approach succeeds in 22 out of 25runs. This example demonstrates the benefit of dynamic initial configurations, but more work is needed to establish if the concept generalizes to more complex tasks, environments and morphologies.

Robots operating in the real world will experience a range of different environments and tasks. It is essential for the robot to have the ability to adapt to its surroundings to work efficiently in changing conditions. Evolutionary robotics aims to solve this by optimizing both the control and body (morphology) of a robot, allowing adaptation to internal, as well as external factors. Most work in this field has been done in physics simulators, which are relatively simple and not able to replicate the richness of interactions found in the real world. Solutions that rely on the complex interplay among control, body, and environment are therefore rarely found. In this article, we rely solely on real-world evaluations and apply evolutionary search to yield combinations of morphology and control for our mechanically self-reconfiguring quadruped robot. We evolve solutions on two distinct physical surfaces and analyze the results in terms of both control and morphology. We then transition to two previously unseen surfaces to demonstrate the generality of our method. We find that the evolutionary search finds high-performing and diverse morphology-controller configurations by adapting both control and body to the different properties of the physical environments. We additionally find that morphology and control vary with statistical significance between the environments. Moreover, we observe that our method allows for morphology and control parameters to transfer to previously unseen terrains, demonstrating the generality of our approach.

We outline a perspective on the future of evolutionary robotics and discuss a long-term vision regarding robots that evolve in the real world. We argue that such systems offer significant potential for advancing both science and engineering. For science, evolving robots can be used to investigate fundamental issues about evolution and the emergence of embodied intelligence. For engineering, artificial evolution can be used as a tool that produces good designs in difficult applications in complex unstructured environments with (partially) unknown and possibly changing conditions. This implies a new paradigm, second-order software engineering, where instead of directly developing a system for a given application, we develop an evolutionary system that will develop the target system for us. Importantly, this also holds for the hardware; with a complete evolutionary robot system, both the software and the hardware are evolved. In this chapter, we discuss the long-term vision, elaborate on the main challenges, and present the initial results of an ongoing research project concerned with the first tangible implementation of such a robot system.

Advances in rapid prototyping have opened up new avenues of research within Evolutionary Robotics in which not only controllers but also the body plans (morphologies) of robots can evolve in real-time and real-space. However, this also introduces new challenges, in that robot models that can be instantiated from an encoding in simulation might not be manufacturable in practice (due to constraints associated with the 3D printing and/or automated assembly processes). We introduce a representation for evolving (wheeled) robots with a printed plastic skeleton, and evaluate three variants of a novelty-search algorithm in terms of their ability to produce populations of manufacturable but diverse robots. While the set of manufacturable robots discovered represent only a small fraction of the overall search space of all robots, all methods are shown to be capable of generating a diverse population of manufacturable robots that we conjecture is large enough to seed an evolving robotic ecosystem.

Bipedal locomotion has several key challenges, such as balancing, foot placement, and gait optimization. We reach optimality from a very early age by using natural supports, such as our parent’s hands, chairs, and training wheels, and bootstrap a new knowledge from the recently acquired one. In this paper, we propose a scaffolded learning method from an evolutionary robotics perspective, where a biped creature achieves stable and independent bipedal walking while exploiting the natural scaffold of its changing morphology to create a third limb. Hence, we compare three conditions of scaffolded learning to reach bipedalism, and we prove that a performance-based scaffold is the most conducive to accelerate the learning of ontogenetic bipedal walking. Beyond a pedagogical experiment, this work presents a powerful tool to accelerate learning on robots.

In this review we introduce the framework of reality-assisted evolution to summarize a growing trend towards combining model-based and model-free approaches to improve the design of physically embodied soft robots. In silico, data-driven models build, adapt and improve representations of the target system using real-world experimental data. By simulating huge numbers of virtual robots using these data-driven models, optimization algorithms can illuminate multiple design candidates for transference to the real world. In reality, large-scale physical experimentation facilitates the fabrication, testing and analysis of multiple candidate designs. Automated assembly and reconfigurable modular systems enable significantly higher numbers of real-world design evaluations than previously possible. Large volumes of ground-truth data gathered via physical experimentation can be returned to the virtual environment to improve data-driven models and guide optimization. Grounding the design process in physical experimentation ensures the complexity of virtual robot designs does not outpace the model limitations or available fabrication technologies. We outline key developments in the design of physically embodied soft robots under the framework of reality-assisted evolution., Comment: Final manuscript, accepted for publication in Artificial Life

This article studies the performance of different evolutionary strategies for deep reinforcement learning policy optimization. The policy will be centred in an important swarm robotic task: the aggregation of simple robots in the environment. The main inspiration for robotic swarm comes from the observation of social animals. Ants, bees, birds, and fish are some examples of how simple individuals can succeed when they gather in groups. In addition, is important to highlight that aggregation may be considered as a previous requirement to tackle another tasks.Due the design of a swarm behaviour is a comprehensive process of experimentation, one of the current solutions is learn a policy able to control a robot. Gradient descent techniques have demonstrated their learning power in the field of neural networks and reinforcement learning, among others. But some of these techniques are difficult to be applied in the field of robotics because the requirements needed to calculate the gradient to be informative are not met. For that reason, in this article we are going to use and compare the evolutionary strategies CMA-ES, PEPG, SES, GA y OpenAI-ES. A fast simulator, based in the differential robot Mbot Ranger, will be used. Once the aggregation task is learned we will compare each strategy for different swarm sizes to analyse the convergence time, the quality of the policies, its scalability and its capacity to generalize.

Few evolved robots have been tested outside simulation due to real world experiments being resource and time expensive. In this paper, we discuss different approaches to physically implement evolved robots and propose a modular robot system with external automatic reconfiguration. While the morphological space is reduced, it provides us with a fast, reusable, fully autonomous system to evolve physical robots in reality.

Multi-level evolution is a bottom-up robotic design paradigm which decomposes the design problem into layered sub-tasks that involve concurrent search for appropriate materials, component geometry and overall morphology. Each of the three layers operate with the goal of building a library of diverse candidate solutions which will be used either as building blocks for the layer above or provided to the decision maker for final use. In this paper we provide a theoretical discussion on the concepts and technologies that could potentially be used as building blocks for this framework.

An unsolved problem in both natural and artificial evolutionary systems is determining the exact environmental and evolutionary conditions that enable complexity to evolve [16]. This is especially pertinent in evolutionary robotics [4] where possible problem-solving behavior is constrained by brain (controller) and body (morphology) complexity [12]. We evaluate the impact of environments and complexity costs on robotic controller and morphology evolution across various evolutionary robotics task scenarios. This study uses evolutionary robotics as an experimental platform to investigate the arrow of complexity hypothesis [3], previously demonstrated to hold in artificial evolutionary systems given an imposed complexity cost [2]. Specifically, we test whether energy costs imposed on evolving robot controller and morphology complexity enables the evolution of increasingly complex controller and morphological designs concomitant with increasing environment complexity. Morphological complexity was equated with possible sensor configurations for a physical counterpart Khepera III mobile robot [8], and neural complexity was equated to artificial neural network topological configurations that coupled with a robot's evolved morphology.

Evolutionary computation provides methods to automatically generate controllers for swarm robotics. Many approaches rely on optimization and the targeted behavior is quantified in form of a fitness function. Other methods, like novelty search, increase exploration by putting selective pressure on unexplored behavior space using a domain-specific behavioral distance function. In contrast, minimize surprise leads to the emergence of diverse behaviors by using an intrinsic motivation as fitness, that is, high prediction accuracy. We compare a standard genetic algorithm, novelty search and minimize surprise in a swarm robotics setting to evolve diverse behaviors and show that minimize surprise is competitive to novelty search.