Search

In Evolutionary Robotics where both body and brain are malleable, it is common practice to evaluate individuals in isolated environments. With the objective of implementing a more naturally plausible system, we designed a single interactive ecosystem for robots to be evaluated in. In this ecosystem robots are physically present and can interact each other and we implemented decentralized rules for mate selection and reproduction. To study the effects of evaluating robots in an interactive ecosystem has on evolution, we compare the evolutionary process with a more traditional, oracle–based approach. In our analysis, we observe how the different approach has a substantial impact on the final behaviour and morphology of the robots, while maintaining decent fitness performance.

Most work in evolutionary robotics centers on evolving a controller for a fixed body plan. However, previous studies suggest that simultaneously evolving both controller and body plan could open up many interesting possibilities. However, the joint optimization of body plan and control via evolutionary processes can be challenging in rich morphological spaces. This is because offspring can have body plans that are very different from either of their parents, leading to a potential mismatch between the structure of an inherited neural controller and the new body. To address this, we propose a framework that combines an evolutionary algorithm to generate body plans and a learning algorithm to optimize the parameters of a neural controller. The topology of this controller is created once the body plan of each offspring has been generated. The key novelty of the approach is to add an external archive for storing learned controllers that map to explicit 'types' of robots (where this is defined with respect to the features of the body plan). By initiating learning from a controller with an appropriate structure inherited from the archive, rather than from a randomly initialized one, we show that both the speed and magnitude of learning increase over time when compared to an approach that starts from scratch, using two tasks and three environments. The framework also provides new insights into the complex interactions between evolution and learning.

Several methods have already been proposed to automatically design control software for robot swarms by assembling predefined modules. Yet, so far, the modules on which these methods operate have always been defined manually in a process that is time consuming, requires domain knowledge, and must be performed by an expert. Motivated by the goal of automatizing the definition of these modules, we propose an approach in which repertoires of modules, in the form of neural networks, are automatically generated via a quality-diversity evolutionary algorithm. To illustrate the proposal, we introduce Nata, a novel approach belonging to the AutoMoDe family. Nata automatically generates probabilistic finite-state machines in which states are selected from a repertoire of neural networks, and transition conditions are selected from a set of rules based on the sensory capabilities of the robotic platform considered. Both the repertoire of neural networks and the set of transition rules are automatically generated a priori, once and for all, in a mission-agnostic way. We study Nata on three missions, both in simulation and with real robots. Nata is the first modular automatic design method that assembles modules that were themselves generated automatically., SCOPUS: ar.j, info:eu-repo/semantics/published

Not all generate-and-test search algorithms are created equal. Bayesian Optimization (BO) invests a lot of computation time to generate the candidate solution that best balances the predicted value and the uncertainty given all previous data, taking increasingly more time as the number of evaluations performed grows. Evolutionary Algorithms (EA) on the other hand rely on search heuristics that typically do not depend on all previous data and can be done in constant time. Both BO and EA community typically assess their performance as a function of the number of evaluations, i.e., data efficiency. However, this is unfair once we start to compare the efficiency of these classes of algorithms, as the overhead times to generate candidate solutions are significantly different. We suggest to measure the efficiency of generate-and-test search algorithms as the expected gain in the objective value per unit of computation time spent, i.e., time efficiency. To the time-efficient search algorithm, we therefore propose a new algorithm, a combination of BO and an EA, BEA for short, that starts with BO, then transfers knowledge to an EA, and subsequently runs the EA. We compare the BEA with BO, the EA, Differential Evolution (DE), and Particle Swarm Optimization (PSO). The results show that BEA outperforms BO, the EA, DE and PSO in terms of time efficiency, and ultimately leads to better performance on well-known benchmark objective functions with many local optima. Moreover, we test BEA, BO, and the EA on nine test cases of robot learning problems and here again we find that BEA outperforms the other algorithms.

Due to the lack of systematic empirical analyses and comparisons of ideas and methods, a clearly established state of the art is still missing in the optimization-based design of robot swarms. In this paper, we propose an experimental protocol for the comparison of fully-automatic design methods. This protocol is characterized by two notable elements: a way to define benchmarks for the evaluation and comparison of design methods, and a sampling strategy that minimizes the variance when estimating their expected performance. To define generally applicable benchmarks, we introduce the notion of mission generator: a tool to generate missions that mimic those a design method will eventually have to solve. To minimize the variance of the performance estimation, we show that, under some common assumptions, one should adopt the sampling strategy that maximizes the number of missions considered&, SCOPUS: ar.j, info:eu-repo/semantics/published

In this paper, we reintroduce evolutionary algorithms into Auto-MoDe, an automatic design approach which optimizes behavioural modules into a probabilistic finite automaton. We evaluate three approaches, with different encodings of the probabilistic finite automaton phenotype, and observe their performances. This work opens modular designs to more advanced evolutionary robotics methods, such as novelty search and embodied evolution.

The joint evolution of bodies and brains (morphologies and controllers) is one of the grand challenges of Evolutionary Robotics. Related work is conducted in various morphological spaces, including, modular robots and voxel-based artificial organisms, most commonly evolving robots for a good gait, i.e., for walking as far as possible without a target, using an open-loop controller. Here we investigate a practically more relevant task: directed locomotion, i.e., the ability to walk in a target direction. To this end, we use closed-loop controllers based on an extension of the popular CPG-networks and compare them to the traditional open-loop setup, where robots are walking 'blindly', but walking in the right direction is rewarded by higher fitness. Our results disclose that the new system does not only lead to better task performance (which was expected), but also to very different morphologies and gaits. In other words, we obtain surprising results showing that adding the ability to utilise sensory feedback does not only lead to better performance, but also to different 'life forms'.

Not all generate-and-test search algorithms are created equal. Bayesian Optimization (BO) invests a lot of computation time to generate the candidate solution that best balances the predicted value and the uncertainty given all previous data, taking increasingly more time as the number of evaluations performed grows. Evolutionary Algorithms (EA) on the other hand rely on search heuristics that typically do not depend on all previous data and can be done in constant time. Both BO and EA community typically assess their performance as a function of the number of evaluations, i.e., data efficiency. However, this is unfair once we start to compare the efficiency of these classes of algorithms, as the overhead times to generate candidate solutions are significantly different. We suggest to measure the efficiency of generate-and-test search algorithms as the expected gain in the objective value per unit of computation time spent, i.e., time efficiency. To the time-efficient search algorithm, we therefore propose a new algorithm, a combination of BO and an EA, BEA for short, that starts with BO, then transfers knowledge to an EA, and subsequently runs the EA. We compare the BEA with BO, the EA, Differential Evolution (DE), and Particle Swarm Optimization (PSO). The results show that BEA outperforms BO, the EA, DE and PSO in terms of time efficiency, and ultimately leads to better performance on well-known benchmark objective functions with many local optima. Moreover, we test BEA, BO, and the EA on nine test cases of robot learning problems and here again we find that BEA outperforms the other algorithms.

Rapid developments in evolutionary computation, robotics, 3D-printing, and material science are enabling advanced systems of robots that can autonomously reproduce and evolve. The emerging technology of robot evolution challenges existing AI ethics because the inherent adaptivity, stochasticity, and complexity of evolutionary systems severely weaken human control and induce new types of hazards. In this paper we address the question how robot evolution can be responsibly controlled to avoid safety risks. We discuss risks related to robot multiplication, maladaptation, and domination and suggest solutions for meaningful human control. Such concerns may seem far-fetched now, however, we posit that awareness must be created before the technology becomes mature.

The ability of a robot to adapt in-mission to achieve an assigned goal is highly desirable. This thesis project places an emphasis on employing learning-based intelligent control methodologies to the development and implementation of an autonomous unmanned aerial vehicle (UAV). Flight control is carried out by evolving spiking neural networks (SNNs) with Hebbian plasticity. The proposed implementation is capable of learning and self-adaptation to model variations and uncertainties when the controller learned in simulation is deployed on a physical platform.Controller development for small multicopters often relies on simulations as an intermediate step, providing cheap, parallelisable, observable and reproducible optimisation with no risk of damage to hardware. Although model-based approaches have been widely utilised in the process of development, loss of performance can be observed on the target platform due to simplification of system dynamics in simulation (e.g., aerodynamics, servo dynamics, sensor uncertainties). Ignorance of these effects in simulation can significantly deteriorate performance when the controller is deployed. Previous approaches often require mathematical or simulation models with a high level of accuracy which can be difficult to obtain. This thesis, on the other hand, attempts to cross the reality gap between a low-fidelity simulation and the real platform. This is done using synaptic plasticity to adapt the SNN controller evolved in simulation to the actual UAV dynamics.The primary contribution of this work is the implementation of a procedural methodology for SNN control that integrates bioinspired learning mechanisms with artificial evolution, with an SNN library package (i.e. eSpinn) developed by the author. Distinct from existing SNN simulators that mainly focus on large-scale neuron interactions and learning mechanisms from a neuroscience perspective, the eSpinn library draws particular attention to embedded implementations on hardwar

Jamming Grippers are a novel class of soft robotic actuators that can robustly grasp and manipulate objects of arbitrary shape. They are formed by placing a granular material within a flexible skin connected to a vacuum pump and function by pressing the un-jammed gripper against a target object and evacuating the air to transition the material to a jammed (solid) state, gripping the target object. However, due to the complex interactions between grain morphology and target object shape, much uncertainty still remains regarding optimal constituent grain shapes for specific gripping applications. We address this challenge by utilising a modern Evolutionary Algorithm, NSGA-III, combined with a Discrete Element Method soft robot model to perform a multi-objective optimisation of grain morphology for use in jamming grippers for a range of target object sizes. Our approach optimises the microscopic properties of the system to elicit bespoke functional granular material performance driven by the complex relationship between the individual particle morphologies and the related emergent behaviour of the bulk state. Results establish the important contribution of grain morphology to gripper performance and the critical role of local surface curvature and the length scale governed by the relative sizes of the grains and target object.

The joint evolution of morphologies and controllers of robots leads to a problem: Even if the parents have well-matching bodies and brains, the stochastic recombination can break this match and cause a body-brain mismatch in their offspring. This can be mitigated by having newborn robots perform a learning process that optimizes their inherited brain quickly after birth. An adequate learning method should work on all possible robot morphologies and be efficient. In this paper we apply Bayesian Optimization and Differential Evolution as learning algorithms and compare them on a test suite of different robot bodies.

Most evolutionary robotics studies focus on evolving some targeted behavior without considering energy usage. In this paper, we extend our simulator with a battery model to take energy consumption into account in a system where robot morphologies and controllers evolve simultaneously. The results show that including the energy consumption in the fitness in a multi-objective fashion reduces the average size of robot bodies while reducing their speed. However, robots generated without size reduction can achieve speeds comparable to robots from the baseline set.

Previous research has shown automated robotic mechanism design to be both deceptive (prone to local minima) and rife with linkage problems (having highly interdependent parameters). This results in a barrier to optimization that is unable to be breached by simply applying more iterations and computational power. The research also indicates that a graph structure model of the robot in combination with an evolutionary algorithm yields useful robotic mechanisms for a limited set of simple problems. This thesis expands on this pre-existing representation by introducing an indirect model that can be used to include both controllers, motors and other new elements in the representation. Besides this extension of the mechanism model, a framework for the automated design optimization task itself is introduced. This thesis shows an equivalence between an operator based representation of the mechanisms and the graph based representation. These operators represent modifications on the mechanism structure and/or parameters. By recognizing the operators as paths in this model a graph of the search space itself can be constructed. In this graph the vertices are mechanisms and the edges are operators. Using the the operator-mechanism equivalence it is shown that designing an optimization algorithm is equivalent to (1) choosing how vertices in the space are grouped together. (2) choosing how the vertices of this search space are connected beforehand by either implicitly or explicitly picking operators and projecting onto their corresponding domain. (3) picking which of the connected paths to traverse based on accumulated information at runtime. This represents a framework that allows the accumulation of knowledge about optimization algorithms acting within it by defining a set of meta-heuristics. With these it is possible to make informed choices to build better optimization algorithms. To show the effectiveness of the framework a novel quality diversity algorithm is developed, Redu, Mechanical Engineering | Biomechanical Design - BioRobotics

The vision behind this paper looks ahead to evolutionary robot systems where morphologies and controllers are evolved together and ‘newborn’ robots undergo a learning process to optimize their inherited brain for the inherited body. The specific problem we address is learning controllers for the task of directed locomotion in evolvable modular robots. To this end, we present a test suite of robots with different shapes and sizes and compare two learning algorithms, Bayesian optimization and HyperNEAT. The experiments in simulation show that both methods obtain good controllers, but Bayesian optimization is more effective and sample efficient. We validate the best learned controllers by constructing three robots from the test suite in the real world and observe their fitness and actual trajectories. The obtained results indicate a reality gap, but overall the trajectories are adequate and follow the target directions successfully.

The social brain hypothesis proposes that enlarged brains have evolved in response to the increasing cognitive demands that complex social life in larger groups places on primates and other mammals. However, this reasoning can be challenged by evidence that brain size has decreased in the evolutionary transitions from solitary to social larger groups in the case of Neolithic humans and some eusocial insects. Different hypotheses can be identified in the literature to explain this reduction in brain size. We evaluate some of them from the perspective of recent approaches to cognitive science, which support the idea that the basis of cognition can span over brain, body, and environment. Here we show through a minimal cognitive model using an evolutionary robotics methodology that the neural complexity, in terms of neural entropy and degrees of freedom of neural activity, of smaller-brained agents evolved in social interaction is comparable to the neural complexity of larger-brained agents evolved in solitary conditions. The nonlinear time series analysis of agents' neural activity reveals that the decoupled smaller neural network is intrinsically lower dimensional than the decoupled larger neural network. However, when smaller-brained agents are interacting, their actual neural complexity goes beyond its intrinsic limits achieving results comparable to those obtained by larger-brained solitary agents. This suggests that the smaller-brained agents are able to enhance their neural complexity through social interaction, thereby offsetting the reduced brain size., source:https://www.frontiersin.org/articles/10.3389/fnbot.2021.634085/full

The social brain hypothesis proposes that enlarged brains have evolved in response to the increasing cognitive demands that complex social life in larger groups places on primates and other mammals. However, this reasoning can be challenged by evidence that brain size has decreased in the evolutionary transitions from solitary to social larger groups in the case of Neolithic humans and some eusocial insects. Different hypotheses can be identified in the literature to explain this reduction in brain size. We evaluate some of them from the perspective of recent approaches to cognitive science, which support the idea that the basis of cognition can span over brain, body, and environment. Here we show through a minimal cognitive model using an evolutionary robotics methodology that the neural complexity, in terms of neural entropy and degrees of freedom of neural activity, of smaller-brained agents evolved in social interaction is comparable to the neural complexity of larger-brained agents evolved in solitary conditions. The nonlinear time series analysis of agents' neural activity reveals that the decoupled smaller neural network is intrinsically lower dimensional than the decoupled larger neural network. However, when smaller-brained agents are interacting, their actual neural complexity goes beyond its intrinsic limits achieving results comparable to those obtained by larger-brained solitary agents. This suggests that the smaller-brained agents are able to enhance their neural complexity through social interaction, thereby offsetting the reduced brain size., source:https://www.frontiersin.org/articles/10.3389/fnbot.2021.634085/full

The joint evolution of morphologies and controllers of robots leads to a problem: Even if the parents have well-matching bodies and brains, the stochastic recombination can break this match and cause a body-brain mismatch in their offspring. This can be mitigated by having newborn robots perform a learning process that optimizes their inherited brain quickly after birth. An adequate learning method should work on all possible robot morphologies and be efficient. In this paper we apply Bayesian Optimization and Differential Evolution as learning algorithms and compare them on a test suite of different robot bodies.

Most evolutionary robotics studies focus on evolving some targeted behavior without considering energy usage. In this paper, we extend our simulator with a battery model to take energy consumption into account in a system where robot morphologies and controllers evolve simultaneously. The results show that including the energy consumption in the fitness in a multi-objective fashion reduces the average size of robot bodies while reducing their speed. However, robots generated without size reduction can achieve speeds comparable to robots from the baseline set.

The vision behind this paper looks ahead to evolutionary robot systems where morphologies and controllers are evolved together and ‘newborn’ robots undergo a learning process to optimize their inherited brain for the inherited body. The specific problem we address is learning controllers for the task of directed locomotion in evolvable modular robots. To this end, we present a test suite of robots with different shapes and sizes and compare two learning algorithms, Bayesian optimization and HyperNEAT. The experiments in simulation show that both methods obtain good controllers, but Bayesian optimization is more effective and sample efficient. We validate the best learned controllers by constructing three robots from the test suite in the real world and observe their fitness and actual trajectories. The obtained results indicate a reality gap, but overall the trajectories are adequate and follow the target directions successfully.

Previous research has shown automated robotic mechanism design to be both deceptive (prone to local minima) and rife with linkage problems (having highly interdependent parameters). This results in a barrier to optimization that is unable to be breached by simply applying more iterations and computational power. The research also indicates that a graph structure model of the robot in combination with an evolutionary algorithm yields useful robotic mechanisms for a limited set of simple problems. This thesis expands on this pre-existing representation by introducing an indirect model that can be used to include both controllers, motors and other new elements in the representation. Besides this extension of the mechanism model, a framework for the automated design optimization task itself is introduced. This thesis shows an equivalence between an operator based representation of the mechanisms and the graph based representation. These operators represent modifications on the mechanism structure and/or parameters. By recognizing the operators as paths in this model a graph of the search space itself can be constructed. In this graph the vertices are mechanisms and the edges are operators. Using the the operator-mechanism equivalence it is shown that designing an optimization algorithm is equivalent to (1) choosing how vertices in the space are grouped together. (2) choosing how the vertices of this search space are connected beforehand by either implicitly or explicitly picking operators and projecting onto their corresponding domain. (3) picking which of the connected paths to traverse based on accumulated information at runtime. This represents a framework that allows the accumulation of knowledge about optimization algorithms acting within it by defining a set of meta-heuristics. With these it is possible to make informed choices to build better optimization algorithms. To show the effectiveness of the framework a novel quality diversity algorithm is developed, Redu, Mechanical Engineering | Biomechanical Design - BioRobotics

[Abstract]: RealTimeBattle is an environment in which robots controlled by programs fight each other. Programs control the simulated robots using low-level messages (e.g., turn radar, accelerate). Unlike other tools like Robocode, each of these robots can be developed using different programming languages. Our purpose is to generate, without human programming or other intervention, a robot that is highly competitive in RealTimeBattle. To that end, we implemented an Evolutionary Computation technique: Genetic Programming. The robot controllers created in the course of the experiments exhibit several different and effective combat strategies such as avoidance, sniping, encircling and shooting. To further improve their performance, we propose a function-set that includes short-term memory mechanisms, which allowed us to evolve a robot that is superior to all of the rivals used for its training. The robot was also tested in a bout with the winner of the previous “RealTimeBattle Championship,” which it won. Finally, our robot was tested in a multi-robot battle arena, with five simultaneous opponents, and obtained the best results among the contenders.

We investigate the possibilities, challenges, and limitations that arise from the use of behavior trees in the context of the automatic modular design of collective behaviors in swarm robotics. To do so, we introduce Maple, an automatic design method that combines predefined modules—low-level behaviors and conditions— into a behavior tree that encodes the individual behavior of each robot of the swarm. We present three empirical studies based on two missions: AGGREGATION and FORAGING. To explore the strengths and weaknesses of adopting behavior trees as a control architecture, we compare Maple with Chocolate, a previously proposed automatic design method that uses probabilistic finite state machines instead. In the first study, we assess Maple's ability to produce control software that crosses the reality gap satisfactorily. In the second study, we investigate Maple's performance as a function of the design budget, that is, the maximum number of simulation runs that the design process is allowed to perform. In the third study, we explore a number of possible variants of Maple that differ in the constraints imposed on the structure of the behavior trees generated. The results of the three studies indicate that, in the context of swarm robotics, behavior trees might be appealing but in many settings do not produce better solutions than finite state machines., Robot Dynamics

This paper summarizes the keynote I gave on the SEAMS 2020 conference. Noting the power of natural evolution that makes living systems extremely adaptive, I describe how artificial evolution can be employed to solve design and optimization problems in software. Thereafter, I discuss the Evolution of Things, that is, the possibility of evolving physical artefacts and zoom in on a (r)evolutionary way of creating 'bodies' and 'brains' of robots for engineering and fundamental research.

We elaborate on (future) evolutionary robot systems where morphologies and controllers of real robots are evolved in the real-world. We argue that such systems must contain a learning component where a newborn robot refines its inherited controller to align with its body, which will inevitably be different from its parents.

We investigate an evolutionary robot system where (simulated) modular robots can reproduce and create robot children that inherit the parents’ morphologies by crossover and mutation. Within this system we compare two approaches to creating good controllers, i.e., evolution only and evolution plus learning. In the first one the controller of a robot child is inherited, so that it is produced by applying crossover and mutation to the controllers of its parents. In the second one the controller of the child is also inherited, but additionally, it is enhanced by a learning method. The experiments show that the learning approach does not only lead to different fitness levels, but also to different (bigger) robots. This constitutes a quantitative demonstration that changes in brains, i.e., controllers, can induce changes in the bodies, i.e., morphologies.

In evolutionary robotics, the representation of the robot is of primary importance. Often indirect encodings are used, whereby a complex developmental process grows a body and a brain from a genotype. In this work, we aim at improving the interpretability of robot morphologies and behaviours resulting from indirect encoding. We develop and use a methodology that focuses on the analysis of evolutionary attractors, represented in what we call the trait space: Using trait descriptors defined in the literature, we define morphological and behavioural Cartesian planes where we project the phenotype of the final population. In our experiments we show that, using this analysis method, we are able to better discern the effect of encodings that differ only in minor details.

This thesis is motivated by evolutionary robot systems where robot bodies and brains evolve simultaneously. In such a robot system, `birth' must be followed by `infant learning' by a learning method that works for various morphologies evolution may produce. Here we address the task of directed locomotion in modular robots with controllers based on Central Pattern Generators. We present a bio-inspired adaptive feedback mechanism that uses a forward model and an inverse model that can be learned on-the-fly. We compare two versions (a simple and a sophisticated one) of this concept to a traditional (open-loop) controller using Bayesian Optimization as a learning algorithm. The experimental results show that the sophisticated version outperforms the simple one and the traditional controller. It leads to improvement in performance and more robust controllers that cope better with noise.

Automatic optimization of robotic behavior has been the long-standing goal of Evolutionary Robotics. Allowing the problem at hand to be solved by automation often leads to novel approaches and new insights. A common problem encountered with this approach is that when this optimization occurs in a simulated environment, the optimized policies are subject to the reality gap when implemented in the real world. This often results in sub-optimal behavior, if it works at all. This paper investigates the automatic optimization of neurocontrollers to perform quick but safe landing maneuvers for a quadrotor micro air vehicle using the divergence of the optical flow field of a downward looking camera. The optimized policies showed that a piece-wise linear control scheme is more effective than the simple linear scheme commonly used, something not yet considered by human designers. Additionally, we show the utility in using abstraction on the input and output of the controller as a tool to improve the robustness of the optimized policies to the reality gap by testing our policies optimized in simulation on real world vehicles. We tested the neurocontrollers using two different methods to generate and process the visual input, one using a conventional CMOS camera and one a dynamic vision sensor, both of which perform significantly differently than the simulated sensor. The use of the abstracted input resulted in near seamless transfer to the real world with the controllers showing high robustness to a clear reality gap., Control & Simulation

Swarm robotics, inspired by swarms in nature, has great potential. The resilience, scalability, robustness and redundancy of having many robots collectively perform tasks such as mapping, disaster recovery, pollution control, and cleaning make for a compelling vision. To achieve this, we need to design swarm robot systems to have a desired collective behaviour, but the design of controllers for the individual robots of a swarm such that this behaviour emerges from the interaction of the individual robots is difficult. Current solutions often use off-line automatic discovery by artificial evolution of robot controllers, which are then transferred into the swarm. This is problematic for two important reasons. Firstly, since there is the need for additional supporting infrastructure, both to evolve the new controllers and to communicate them to the swarm, the swarm is not self-sufficient. Secondly, the evolved controllers are often opaque and hard to understand, an important consideration for safety and explainability reasons. In this work we tackle both of these issues. We build a swarm of robots with very high computing performance using recently available mobile computation devices. This high performance allows us to move the evolutionary process, dependent on processing power for simulation, into the swarm. Because the computational power of the swarm grows with the size of the swarm, it is both autonomous and scalable. We use behaviour trees as the individual robot controller architecture. They are modular, hierarchical and human readable. By developing automatic tools to simplify large evolved trees, we can understand, explain, and even improve the evolved controllers. By moving the evolutionary process into the swarm, and by using understandable controllers, we make the swarm autonomous, scalable, and understandable, necessary steps towards their real-world deployment.

This paper summarizes the keynote I gave on the SEAMS 2020 conference. Noting the power of natural evolution that makes living systems extremely adaptive, I describe how artificial evolution can be employed to solve design and optimization problems in software. Thereafter, I discuss the Evolution of Things, that is, the possibility of evolving physical artefacts and zoom in on a (r)evolutionary way of creating 'bodies' and 'brains' of robots for engineering and fundamental research.

We elaborate on (future) evolutionary robot systems where morphologies and controllers of real robots are evolved in the real-world. We argue that such systems must contain a learning component where a newborn robot refines its inherited controller to align with its body, which will inevitably be different from its parents.

In evolutionary robotics, the representation of the robot is of primary importance. Often indirect encodings are used, whereby a complex developmental process grows a body and a brain from a genotype. In this work, we aim at improving the interpretability of robot morphologies and behaviours resulting from indirect encoding. We develop and use a methodology that focuses on the analysis of evolutionary attractors, represented in what we call the trait space: Using trait descriptors defined in the literature, we define morphological and behavioural Cartesian planes where we project the phenotype of the final population. In our experiments we show that, using this analysis method, we are able to better discern the effect of encodings that differ only in minor details.

We investigate an evolutionary robot system where (simulated) modular robots can reproduce and create robot children that inherit the parents’ morphologies by crossover and mutation. Within this system we compare two approaches to creating good controllers, i.e., evolution only and evolution plus learning. In the first one the controller of a robot child is inherited, so that it is produced by applying crossover and mutation to the controllers of its parents. In the second one the controller of the child is also inherited, but additionally, it is enhanced by a learning method. The experiments show that the learning approach does not only lead to different fitness levels, but also to different (bigger) robots. This constitutes a quantitative demonstration that changes in brains, i.e., controllers, can induce changes in the bodies, i.e., morphologies.

We investigate the possibilities, challenges, and limitations that arise from the use of behavior trees in the context of the automatic modular design of collective behaviors in swarm robotics. To do so, we introduce Maple, an automatic design method that combines predefined modules—low-level behaviors and conditions— into a behavior tree that encodes the individual behavior of each robot of the swarm. We present three empirical studies based on two missions: AGGREGATION and FORAGING. To explore the strengths and weaknesses of adopting behavior trees as a control architecture, we compare Maple with Chocolate, a previously proposed automatic design method that uses probabilistic finite state machines instead. In the first study, we assess Maple's ability to produce control software that crosses the reality gap satisfactorily. In the second study, we investigate Maple's performance as a function of the design budget, that is, the maximum number of simulation runs that the design process is allowed to perform. In the third study, we explore a number of possible variants of Maple that differ in the constraints imposed on the structure of the behavior trees generated. The results of the three studies indicate that, in the context of swarm robotics, behavior trees might be appealing but in many settings do not produce better solutions than finite state machines., Robot Dynamics

Automatic optimization of robotic behavior has been the long-standing goal of Evolutionary Robotics. Allowing the problem at hand to be solved by automation often leads to novel approaches and new insights. A common problem encountered with this approach is that when this optimization occurs in a simulated environment, the optimized policies are subject to the reality gap when implemented in the real world. This often results in sub-optimal behavior, if it works at all. This paper investigates the automatic optimization of neurocontrollers to perform quick but safe landing maneuvers for a quadrotor micro air vehicle using the divergence of the optical flow field of a downward looking camera. The optimized policies showed that a piece-wise linear control scheme is more effective than the simple linear scheme commonly used, something not yet considered by human designers. Additionally, we show the utility in using abstraction on the input and output of the controller as a tool to improve the robustness of the optimized policies to the reality gap by testing our policies optimized in simulation on real world vehicles. We tested the neurocontrollers using two different methods to generate and process the visual input, one using a conventional CMOS camera and one a dynamic vision sensor, both of which perform significantly differently than the simulated sensor. The use of the abstracted input resulted in near seamless transfer to the real world with the controllers showing high robustness to a clear reality gap., Control & Simulation

This thesis is motivated by evolutionary robot systems where robot bodies and brains evolve simultaneously. In such a robot system, `birth' must be followed by `infant learning' by a learning method that works for various morphologies evolution may produce. Here we address the task of directed locomotion in modular robots with controllers based on Central Pattern Generators. We present a bio-inspired adaptive feedback mechanism that uses a forward model and an inverse model that can be learned on-the-fly. We compare two versions (a simple and a sophisticated one) of this concept to a traditional (open-loop) controller using Bayesian Optimization as a learning algorithm. The experimental results show that the sophisticated version outperforms the simple one and the traditional controller. It leads to improvement in performance and more robust controllers that cope better with noise.

Research in swarm robotics has shown that automatic design is an effective approach to realize robot swarms. In automatic design methods, the collective behavior of a swarm is obtained by automatically configuring and fine-tuning the control software of individual robots. In this paper, we present TuttiFrutti: an automatic design method for robot swarms that belongs to AutoMoDe-a family of methods that produce control software by assembling preexisting software modules via optimization. The peculiarity of TuttiFrutti is that it designs control software for e-puck robots that can display and perceive colors using their RGB LEDs and omnidirectional camera. Studies with AutoMoDe have been so far restricted by the limited capabilities of the e-pucks. By enabling the use of colors, we significantly enlarge the variety of collective behaviors they can produce. We assess TuttiFrutti with swarms of e-pucks that perform missions in which they should react to colored light. Results show that TuttiFrutti designs collective behaviors in which the robots identify the colored light displayed in the environment and act accordingly. The control software designed by TuttiFrutti endowed the swarms of e-pucks with the ability to use color-based information for handling events, communicating, and navigating., SCOPUS: ar.j, info:eu-repo/semantics/published

We investigate the possibilities, challenges, and limitations that arise from the use of behavior trees in the context of the automatic modular design of collective behaviors in swarm robotics. To do so, we introduce Maple, an automatic design method that combines predefined modules—low-level behaviors and conditions— into a behavior tree that encodes the individual behavior of each robot of the swarm. We present three empirical studies based on two missions: AGGREGATION and FORAGING. To explore the strengths and weaknesses of adopting behavior trees as a control architecture, we compare Maple with Chocolate, a previously proposed automatic design method that uses probabilistic finite state machines instead. In the first study, we assess Maple's ability to produce control software that crosses the reality gap satisfactorily. In the second study, we investigate Maple's performance as a function of the design budget, that is, the maximum number of simulation runs that the design process is allowed to perform. In the third study, we explore a number of possible variants of Maple that differ in the constraints imposed on the structure of the behavior trees generated. The results of the three studies indicate that, in the context of swarm robotics, behavior trees might be appealing but in many settings do not produce better solutions than finite state machines., SCOPUS: ar.j, info:eu-repo/semantics/published

A high-level neural controller for leader-follower flight is presented. State of the art range-based relative localization schemes that rely exclusively on onboard sensors present an additional challenge to the leader-follower control problem since they restrict the flight conditions that guarantee observability. This novel controller was developed over an evolutionary process in which the simulation environment resembled the real-life constraints a group of MAVs would encounter. During the learning stage, a group of three agents is used, where one acts as a leader and flies a random trajectory, and the other two act as followers guided by a candidate controller that dictates the desired velocity commands. In the end, when equipped with the best-evolved controller, the follower agents are able to showcase a successful following behaviour that also enhances the observability of the system, although no observability metric was included in evolution., Aerospace Engineering

We compare two representations used to define the morphology of legs for a hexapod robot, which are subsequently 3D printed. A leg morphology occupies a set of voxels in a voxel grid. One method, a direct representation, uses a collection of Bezier splines. The second, an indirect method, utilises CPPN-NEAT. In our first experiment, we investigate two strategies to post-process the CPPN output and ensure leg length constraints are met. The first uses an adaptive threshold on the output neuron, the second, previously reported in the literature, scales the largest generated artefact to our desired length. In our second experiment, we build on our past work that evolves the tibia of a hexapod to provide environment-specific performance benefits. We compare the performance of our direct and indirect legs across three distinct environments, represented in a high-fidelity simulator. Results are significant and support our hypothesis that the indirect representation allows for further exploration of the design space leading to improved fitness.

We study evolutionary robot systems where not only the robot brains but also the robot bodies are evolvable. Such systems need to include a learning period right after ‘birth’ to acquire a controller that fits the newly created body. In this paper, we investigate the possibility of bootstrapping infant robot learning through employing the Lamarckian inheritance of parental controllers. In our system controllers are encoded by a combination of a morphology dependent component, a Central Pattern Generator (CPG), and a morphology independent part, a Compositional Pattern Producing Network (CPPN). This makes it possible to transfer the CPPN part of controllers between different morphologies and to create a Lamarckian system. We conduct experiments with simulated modular robots whose fitness is determined by the speed of locomotion, establish the benefits of inheriting optimized parental controllers, shed light on the conditions that influence these benefits, and observe that changing the way controllers are evolved also impacts the evolved morphologies.

Morphological evolution in a robotic system produces novel robot bodies after each reproduction event. This implies the necessity for life-time learning so that newborn robots can acquire a controller that fits their body. Thus, we obtain a system where evolution and learning are combined. This combination can be Darwinian or Lamarckian and in this paper, we compare the two. In particular, we investigate the evolved morphologies under these regimes for modular robots evolved for good locomotion. Using eight quantifiable morphological descriptors to characterize the physical properties of robots we compare the regions of attraction in the resulting 8-dimensional space. The results show prominent differences in symmetry, size, proportion, and coverage.

Automatically optimizing robotic behavior to solve complex tasks has been one of the main, long-standing goals of Evolutionary Robotics (ER). When successful, this approach will likely fundamentally change the rate of development and deployment of robots in everyday life. Performing this optimization on real robots can be risky and time consuming. As a result, much of the work in ER is done using simulations which can operate many times faster than realtime. The only downside of this, is that, due to the limited fidelity of the simulated environment, the optimized robotic behavior is typically different when transferred to a robot in the real world. This difference is referred to as the reality gap..., Control & Simulation

This paper utilizes evolutionary robotics techniques as a hypothesis generator to explore optical variables and control strategies that could be used to solve a driving-like braking task. Given such a task, humans exhibit two different braking behaviors: continuously regulated braking and impulsive braking. Based on an oft-used experimental task in human perception/action research, a series of evolutionary robotics simulations were developed to explore the space of possible braking strategies by examining how braking strategies change as the optical information is manipulated. Our results can be summarized as follows: (1) behaviors similar to human behavior were observed only when the constraints were selected correctly; (2) the optical variables and proportional rate yielded significantly better braking performance; (3) two different classes of impulsive braking behaviors were observed, including one not reported in previous studies: discrete impulsive braking and oscillatory impulsive braking; (4) the optical variable is used to initiate and terminate braking; (5) the evolved model agents use a proportional rate control strategy to regulate braking continuously. We argue that combining psychological experiments and evolutionary robotics simulations is a promising research methodology that is useful for testing existing hypotheses and generating new ones.

This paper utilizes evolutionary robotics techniques as a hypothesis generator to explore optical variables and control strategies that could be used to solve a driving-like braking task. Given such a task, humans exhibit two different braking behaviors: continuously regulated braking and impulsive braking. Based on an oft-used experimental task in human perception/action research, a series of evolutionary robotics simulations were developed to explore the space of possible braking strategies by examining how braking strategies change as the optical information is manipulated. Our results can be summarized as follows: (1) behaviors similar to human behavior were observed only when the constraints were selected correctly; (2) the optical variables and proportional rate yielded significantly better braking performance; (3) two different classes of impulsive braking behaviors were observed, including one not reported in previous studies: discrete impulsive braking and oscillatory impulsive braking; (4) the optical variable is used to initiate and terminate braking; (5) the evolved model agents use a proportional rate control strategy to regulate braking continuously. We argue that combining psychological experiments and evolutionary robotics simulations is a promising research methodology that is useful for testing existing hypotheses and generating new ones.