Search

Artificial agents required to perform non-trivial tasks are commonly controlled with Artificial Neural Networks (ANNs), which need to be carefully fine-tuned. This is where ANN optimization comes into play, often in the form of Neuroevolution (NE). Among artificial agents, the embodied ones, are characterized by a strong body-brain entanglement, i.e., a strong interdependence between the physical properties of the body and the controller. In this work, we aim at characterizing said interconnection, experimentally evaluating the impact body material properties have on NE for embodied agents. We consider the case of Voxel-based Soft Robots (VSRs), a class of simulated modular soft robots which achieve movement through the rhythmical contraction and expansion of their modules. We experiment varying several physical properties of VSRs and assess the effectiveness of the evolved controllers for the task of locomotion, together with their robustness and adaptability. Our results confirm the existence of a deep body-brain interrelationship for embodied agents, and highlight how NE fruitfully exploits the physical properties of the agents to give rise to a wide gamut of effective and adaptable behaviors.

What makes a quality-diversity (QD) algorithm effective? This question is increasingly studied as such algorithms gain in popularity. Recent work on Novelty Search linked its efficiency to the evolution of a population under a dynamic selection pressure defined by the novelty metric, which allows the population to converge to highly evolvable individuals, on which spending the evolutionary budget allows to navigate the behavior space. MAP-Elites, another popular QD algorithm, does not use an explicit population, instead sampling the parent population directly from the behavioral archive at each generation. This sampling can be uniform, or biased according to criteria such as curiosity, which favor individuals that previously generated successful offsprings. In this article, we show that such improved selection schemes are efficient because they create a dynamic pseudo-population that mimics the desirable qualities of a Novelty Search population. We do this by proposing another simpler selection scheme, youth, that simply focuses the evolutionary budget on such a dynamic pseudo-population, and by showing it exhibits similar behavior and performance as curiosity without explicitly selecting for previously successful parents. This has important implications for the design of future QD algorithms taking the best of both archive-based and population-based methods.

Organisms with different body morphology and movement dynamics have distinct abilities to move through the environment. Despite such truism, there is a lack of general principles that predict which shapes and dynamics make the organisms more fit to move. Studying a minimal yet embodied soft robot model under the influence of gravity, we find three features that predict robot locomotion fitness: (1) A larger body is better. (2) Two-point contact with the ground is better than one-point contact. (3) Out-of-phase oscillating body parts increase locomotion fitness. These design principles can guide the selection rules for evolutionary algorithms to obtain robots with higher locomotion fitness.

Locating odour sources is a hard task that has been addressed with a large variety of AI methods to produce search strategies with different levels of efficiency and robustness. However, it is still not clear how to evaluate those strategies. Simply evaluating the robot's ability to reach the goal may produce deceptive fitness values, favouring poor strategies that do not generalise. Conversely, including prior knowledge may bias the learning process. This work studies the impact of evaluation functions with various degrees of prior knowledge, in evolving search strategies. The baseline is set by performing multiple evaluations of each strategy with a function that only evaluates the task efficiency. A function was found that is able to produce strategies with equivalent performance to those of the baseline, whilst performing a single evaluation.

Robots are still missing the ability to adapt to new environments. However, biological systems are able to adapt to new environments with ease; perhaps because they have the ability to react to environmental input during a growth phase with changes not only in behaviour, but also morphology. Yet within the field of robots, environmental based development of morphology is an under researched area. In this paper we use an evolutionary algorithm to evolve neural cellular automata capable of inducing environmental based developmental plasticity in robots. We use the kinetic energy of each cell and its neighbours as an input to our network, the output of which determines the position of new cell growth. We evolve our neural cellular automata first in three individual environments and then also for performance in multiple environments. We show that the networks that use environmental feedback outperform those that do not and that by introducing environmental feedback during development, more adaptive and better performing robots are potentially possible.

We demonstrate the training of Spiking Neural Networks (SNN) in a novel multi-agent Evolutionary Robotics (ER) framework inspired by competitive evolutionary environments in nature. The topology of a SNN along with morphological parameters of the bot it controls in the ER environment is together treated as a phenotype. Rules of the framework select certain bots and their SNNs for reproduction and others for elimination based on their efficacy in capturing food in a competitive environment. While the bots and their SNNs are not explicitly trained to survive or reproduce using loss functions, these drives emerge implicitly as they evolve to hunt food and survive. Their efficiency in capturing food exhibits the evolutionary signature of punctuated equilibrium. We use this signature to compare the performances of two evolutionary inheritance algorithms on the phenotypes, Mutation and Crossover with Mutation, using ensembles of 100 experiments for each algorithm. We find that Crossover with Mutation promotes 40% faster learning in the SNN than mere Mutation with a statistically significant margin.

An open question for both natural and artificial evolutionary systems is how, and under what environmental and evolutionary conditions complexity evolves. This study investigates the impact of increasingly complex task environments on the evolution of robot complexity. Specifically, the impact of evolving body-brain couplings on locomotive task performance, where robot evolution was directed by either body-brain exploration (novelty search) or objective-based (fitness function) evolutionary search. Results indicated that novelty search enabled the evolution of increased robot body-brain complexity and efficacy given specific environment conditions. The key contribution is thus the demonstration that body-brain exploration is suitable for evolving robot complexity that enables high fitness robots in specific environments.

Quality-Diversity (QD) optimisation is a new family of learning algorithms that aims at generating collections of diverse and high-performing solutions. Among those algorithms, the recently introduced Covariance Matrix Adaptation MAP-Elites (CMA-ME) algorithm proposes the concept of emitters, which uses a predefined heuristic to drive the algorithm's exploration. This algorithm was shown to outperform MAP-Elites, a popular QD algorithm that has demonstrated promising results in numerous applications. In this paper, we introduce Multi-Emitter MAP-Elites (ME-MAP-Elites), an algorithm that directly extends CMA-ME and improves its quality, diversity and data efficiency. It leverages the diversity of a heterogeneous set of emitters, in which each emitter type improves the optimisation process in different ways. A bandit algorithm dynamically finds the best selection of emitters depending on the current situation. We evaluate the performance of ME-MAP-Elites on six tasks, ranging from standard optimisation problems (in 100 dimensions) to complex locomotion tasks in robotics. Our comparisons against CMA-ME and MAP-Elites show that ME-MAP-Elites is faster at providing collections of solutions that are significantly more diverse and higher performing. Moreover, in cases where no fruitful synergy can be found between the different emitters, ME-MAP-Elites is equivalent to the best of the compared algorithms.

Autonomous robots are increasingly used in remote and hazardous environments, where damage to sensory-actuator systems cannot be easily repaired. Such robots must therefore have controllers that continue to function effectively given unexpected malfunctions and damage to robot morphology. This study applies the Intelligent Trial and Error (IT&E) algorithm to adapt hexapod robot control to various leg failures and demonstrates the IT&E map-size parameter as a critical parameter in influencing IT&E adaptive task performance. We evaluate robot adaptation for multiple leg failures on two different map-sizes in simulation and validate evolved controllers on a physical hexapod robot. Results demonstrate a trade-off between adapted gait speed and adaptation duration, dependent on adaptation task complexity (leg damage incurred), where map-size is crucial for generating behavioural diversity required for adaptation.

This article studies the performance of different evolutionary strategies for deep reinforcement learning policy optimization. The policy will be centred in an important swarm robotic task: the aggregation of simple robots in the environment. The main inspiration for robotic swarm comes from the observation of social animals. Ants, bees, birds, and fish are some examples of how simple individuals can succeed when they gather in groups. In addition, is important to highlight that aggregation may be considered as a previous requirement to tackle another tasks.Due the design of a swarm behaviour is a comprehensive process of experimentation, one of the current solutions is learn a policy able to control a robot. Gradient descent techniques have demonstrated their learning power in the field of neural networks and reinforcement learning, among others. But some of these techniques are difficult to be applied in the field of robotics because the requirements needed to calculate the gradient to be informative are not met. For that reason, in this article we are going to use and compare the evolutionary strategies CMA-ES, PEPG, SES, GA y OpenAI-ES. A fast simulator, based in the differential robot Mbot Ranger, will be used. Once the aggregation task is learned we will compare each strategy for different swarm sizes to analyse the convergence time, the quality of the policies, its scalability and its capacity to generalize.

Few evolved robots have been tested outside simulation due to real world experiments being resource and time expensive. In this paper, we discuss different approaches to physically implement evolved robots and propose a modular robot system with external automatic reconfiguration. While the morphological space is reduced, it provides us with a fast, reusable, fully autonomous system to evolve physical robots in reality.

Multi-level evolution is a bottom-up robotic design paradigm which decomposes the design problem into layered sub-tasks that involve concurrent search for appropriate materials, component geometry and overall morphology. Each of the three layers operate with the goal of building a library of diverse candidate solutions which will be used either as building blocks for the layer above or provided to the decision maker for final use. In this paper we provide a theoretical discussion on the concepts and technologies that could potentially be used as building blocks for this framework.

An unsolved problem in both natural and artificial evolutionary systems is determining the exact environmental and evolutionary conditions that enable complexity to evolve [16]. This is especially pertinent in evolutionary robotics [4] where possible problem-solving behavior is constrained by brain (controller) and body (morphology) complexity [12]. We evaluate the impact of environments and complexity costs on robotic controller and morphology evolution across various evolutionary robotics task scenarios. This study uses evolutionary robotics as an experimental platform to investigate the arrow of complexity hypothesis [3], previously demonstrated to hold in artificial evolutionary systems given an imposed complexity cost [2]. Specifically, we test whether energy costs imposed on evolving robot controller and morphology complexity enables the evolution of increasingly complex controller and morphological designs concomitant with increasing environment complexity. Morphological complexity was equated with possible sensor configurations for a physical counterpart Khepera III mobile robot [8], and neural complexity was equated to artificial neural network topological configurations that coupled with a robot's evolved morphology.

Evolutionary computation provides methods to automatically generate controllers for swarm robotics. Many approaches rely on optimization and the targeted behavior is quantified in form of a fitness function. Other methods, like novelty search, increase exploration by putting selective pressure on unexplored behavior space using a domain-specific behavioral distance function. In contrast, minimize surprise leads to the emergence of diverse behaviors by using an intrinsic motivation as fitness, that is, high prediction accuracy. We compare a standard genetic algorithm, novelty search and minimize surprise in a swarm robotics setting to evolve diverse behaviors and show that minimize surprise is competitive to novelty search.

Voxel-based soft robots (VSRs) are aggregations of elastic, cubic blocks that have sparkled the interest of Robotics and Artificial Life researchers. VSRs can move by varying the volume of individual blocks, according to control signals dictated by a controller, possibly based on inputs coming from sensors embedded in the blocks. Neural networks (NNs) have been used as centralized processing units for those sensing controllers, with weights optimized using evolutionary computation. This structuring breaks the intrinsic modularity of VSRs: decomposing a VSR into modules to be assembled in a different way is very hard. In this work we propose an alternative approach that enables full modularity and is based on a distributed neural controller. Each block contains a small NN that outputs signals to adjacent blocks and controls the local volume, based on signals from adjacent blocks and on local sensor readings. We show experimentally for the locomotion task that our controller is as effective as the centralized one. Our experiments also suggest that the proposed framework indeed allows exploiting modularity: VSRs composed of pre-trained parts (body and controller) can be evolved more efficiently than starting from scratch.

A critical issue in evolutionary robotics is the transfer of controllers learned in simulation to reality. This is especially the case for small Unmanned Aerial Vehicles (UAVs), as the platforms are highly dynamic and susceptible to breakage. Previous approaches often require simulation models with a high level of accuracy, otherwise significant errors may arise when the well-designed controller is being deployed onto the targeted platform. Here we try to overcome the transfer problem from a different perspective, by designing a spiking neurocontroller which uses synaptic plasticity to cross the reality gap via online adaptation. Through a set of experiments we show that the evolved plastic spiking controller can maintain its functionality by self-adapting to model changes that take place after evolutionary training, and consequently exhibit better performance than its non-plastic counterpart., GECCO2020

Wearable technology is a broad discourse that has evolved over decades, growing into an industry today that represents billions of dollars in product revenue, but stagnating as products become driven by technological developments rather than human needs. However, numerous isolated developments of a new class of wearables are emerging, being both aware of their environment and function, and able to physically adapt to user needs. Examined through this paper collectively, this new class of wearables are described as awareables, representing a shift in technology towards more life-like products. The context of this shift is broadly analyzed alongside similar shifts within the fields of architecture (responsive architecture), additive manufacturing (4D printing) and robotics (evolutionary robotics). The intent of this paper is to encourage new discourse and practical work, calling for researchers and product designers to think beyond gizmos, and instead consider more natural interactions between people and products inspired by nature.

To investigate how encodings influence evolving the morphology and control of modular robots, we compared three encodings: a direct encoding and two generative encodings-a compositional pattern producing network (CPPN) and a Lindenmayer System (L-System). The evolutionary progression and final performance of the direct encoding and the L-System was significantly better than the CPPN. The generative encodings converge quicker than the direct encoding in terms of morphological and controller diversity.

International audience; Novelty Search is an exploration algorithm driven by the novelty of a behavior. The same individual evaluated at different generations has different fitness values. The corresponding fitness landscape is thus constantly changing and if, at the scale of a single generation , the metaphor of a fitness landscape with peaks and valleys still holds, this is not the case anymore at the scale of the whole evolutionary process. How does this kind of algorithms behave? Is it possible to define a model that would help understand how it works? This understanding is critical to analyse existing Novelty Search variants and design new and potentially more efficient ones. We assert that Novelty Search asymptotically behaves like a uniform random search process in the behavior space. This is an interesting feature, as it is not possible to directly sample in this space: the algorithm has a direct access to the genotype space only, whose relationship to the behavior space is complex. We describe the model and check its consistency on a classical Novelty Search experiment. We also show that it sheds a new light on results of the literature and suggests future research work.

Quality diversity (QD) algorithms such as MAP-Elites have emerged as a powerful alternative to traditional single-objective optimization methods. They were initially applied to evolutionary robotics problems such as locomotion and maze navigation, but have yet to see widespread application. We argue that these algorithms are perfectly suited to the rich domain of video games, which contains many relevant problems with a multitude of successful strategies and often also multiple dimensions along which solutions can vary. This paper introduces a novel modification of the MAP-Elites algorithm called MAP-Elites with Sliding Boundaries (MESB) and applies it to the design and rebalancing of Hearthstone, a popular collectible card game chosen for its number of multidimensional behavior features relevant to particular styles of play. To avoid overpopulating cells with conflated behaviors, MESB slides the boundaries of cells based on the distribution of evolved individuals. Experiments in this paper demonstrate the performance of MESB in Hearthstone. Results suggest MESB finds diverse ways of playing the game well along the selected behavioral dimensions. Further analysis of the evolved strategies reveals common patterns that recur across behavioral dimensions and explores how MESB can help rebalance the game., Comment: Accepted to the Genetic and Evolutionary Computation Conference (GECCO-2019)

Quality-diversity algorithms focus on discovering multiple diverse and high-performing solutions. MAP-elites is such an algorithm, as it partitions the solution space into bins and searches for the best solution possible for each bin. In this paper, multi-behavior variants of MAP-Elites are tested where the MAP-Elites grid partitions the solution space based on a certain dimension, while selection is guided by measures of diversity on another dimension. Four divergent search algorithms are tested for this selection process, targeting novelty or surprise or their combination, and their performance on a soft robot evolution task is discussed.

Quality-Diversity optimization is a new family of optimization algorithms that, instead of searching for a single optimal solution to solving a task, searches for a large collection of solutions that all solve the task in a different way. This approach is particularly promising for learning behavioral repertoires in robotics, as such a diversity of behaviors enables robots to be more versatile and resilient. However, these algorithms require the user to manually define behavioral descriptors, which is used to determine whether two solutions are different or similar. The choice of a behavioral descriptor is crucial, as it completely changes the solution types that the algorithm derives. In this paper, we introduce a new method to automatically define this descriptor by combining Quality-Diversity algorithms with unsupervised dimensionality reduction algorithms. This approach enables robots to autonomously discover the range of their capabilities while interacting with their environment. The results from two experimental scenarios demonstrate that robot can autonomously discover a large range of possible behaviors, without any prior knowledge about their morphology and environment. Furthermore, these behaviors are deemed to be similar to handcrafted solutions that uses domain knowledge and significantly more diverse than when using existing unsupervised methods., The Genetic and Evolutionary Computation Conference 2019

Evolution in nature has allowed biological systems to develop and survive in many different environments. It can be attributed to one of the major features of natural evolution: its open-endedness, that can be considered as the ability to continuously produce novelty and/or complexity [1]. This feature is critical to allow an agent to continuously adapt to its environment. We propose here an extension of Quality Diversity algorithms to make it more open-ended. Quality Diversity algorithms aim at generating a large set of diverse solutions [3, 10]. They can be used in a two-steps process in which (1) a diverse set of solutions is generated offline before (2) the agent can exploit it online to find the behavior that fits to the situation [2, 4--6]. QD algorithms main goal is to fill a behavior space whose dimensions are defined by behavior descriptors. Current QD algorithms can generate novelty and complexity, but within the frame of these behavior descriptors. To make the evolutionary process more open-ended, we extend Cully and Demiris framework [3] and introduce Multi-Container Quality Diversity algorithms (MCQD). MCQD are QD algorithms relying on multiple behavior spaces that can be added on the fly. MCQD thus aims at implementing an open-ended evolutionary process that would explore any new behavior space that could be found of interest during the search process.

Evolutionary robotics researchers often need to share results that may be too difficult to describe in text and too complex to show using images. Many researchers include links to videos as supplementary materials, but videos have a predefined view of the scene and do not allow watchers to adjust the viewing angle to their preference. In this paper we present a web-based application (based on three.js) for sharing interactive animations. Specifically, our tool (called Review) enables researchers to generate simple animation log data that can be loaded in any modern web browser on a computer or mobile device. The camera in these animations is controlled by the user such that they can pan, tilt, rotate, and zoom in and out of the scene. Review is meant to improve the ability of researchers to share their evolved results with one another.

Muscle and tendon elasticity enables animals to interact with their environment softly, reducing ground impact force and increasing efficiency of locomotion. Traditional rigid body robots remain the commercially viable option, but incorporating flexibility can harness the benefits exhibited by natural organisms. In this paper, we examine how the addition of passive flexibility impacts performance and locomotive efficiency in a quadruped animat. Results show that the addition of flexibility in the spine and lower limbs of a quadruped animat significantly increases the distance traveled compared to a fully rigid-body animat. However, replacing these passively flexibile joints with actively controlled joints results in the farthest traveling individuals while maintaining similar efficiency. It appears that increases in DOF and joint configuration are the drivers of performance increases rather than passive flexibility.

In this paper, we describe the Evo-ROS framework, which is intended to help bridge the gap between the evolutionary and traditional robotics communities. Evo-ROS combines an evolutionary algorithm with individual physics-based evaluations conducted using the Robot Operating System (ROS) and the Gazebo simulation environment. Our goals in developing Evo-ROS are to (1) provide researchers in evolutionary robotics with access to the extensive support for real-world components and capabilities developed by the ROS community and (2) enable ROS developers, and more broadly robotics researchers, to take advantage of evolutionary search during design and testing. We describe the details of the Evo-ROS structure and operation, followed by presentation of a case study using Evo-ROS to optimize placement of sonar sensors on unmanned ground vehicles that can experience reduced sensing capability due to component failures and physical damage. The case study provides insights into the current capabilities and identifies areas for future enhancements.

Traditional Evolutionary Robotics (ER) employs evolutionary techniques to search for a single monolithic controller which can aid a robot to learn a desired task. These techniques suffer from bootstrap and deception issues when the tasks are complex for a single controller to learn. Behaviour-decomposition techniques have been used to divide a task into multiple subtasks and evolve separate subcontrollers for each subtask. However, these subcontrollers and the associated subcontroller arbitrator(s) are all evolved off-line. A distributed, fully embodied and evolutionary version of such approaches will greatly aid online learning and help reduce the reality gap. In this paper, we propose an immunology-inspired embodied action-evolution cum selection algorithm that can cater to distributed ER. This algorithm evolves different subcontrollers for different portions of the search space in a distributed manner just as antibodies are evolved and primed for different antigens in the antigenic space. Experimentation on a collective of real robots embodied with the algorithm showed that a repertoire of antibody-like subcontrollers was created, evolved and shared on-the-fly to cope up with different environmental conditions. In addition, instead of the conventionally used approach of broadcasting for sharing, we present an Intelligent Packet Migration scheme that reduces energy consumption., Comment: 8 pages, 5 figures, GECCO 18, Kyoto, Japan

The Evolutionary Robotics process requires the evaluation of large numbers of robot controllers in order to determine their relative fitnesses. The evaluation of many controllers is typically performed in simulation instead of real-world hardware in order to speed up the evolutionary process and avoid damage to robot hardware. Physics-based simulators are traditionally used in robotics for evaluating controllers. Effective traditional simulators may require a high level of accuracy and their creation requires specialised knowledge of the dynamics of the robotic system. Alternatively Artificial Neural Network based simulators are relatively simple to construct, are highly accurate, efficient and assume little specialised knowledge of the dynamics involved. This paper proposes a novel controller evolution method that integrates uncertainty information from an ensemble of Artificial Neural Network based simulators. By estimating the accuracy of simulator predictions during the controller evolution process, robot behaviours that are not accurately simulated are likely to be avoided. A performance comparison of a single versus ensembles configuration of Artificial Neural Network based simulators is investigated. The ensemble approach developed in this work has so far outperformed all prior controller evolution methods investigated for the presented problem.

The presence of functional diversity within a group has been demonstrated to lead to greater robustness, higher performance and increased problem-solving ability in a broad range of studies that includes insect groups, human groups and swarm robotics. Evolving group diversity however has proved challenging within Evolutionary Robotics, requiring reproductive isolation and careful attention to population size and selection mechanisms. To tackle this issue, we introduce a novel, decentralised, variant of the MAP-Elites illumination algorithm which is hybridised with a well-known distributed evolutionary algorithm (mEDEA). The algorithm simultaneously evolves multiple diverse behaviours for multiple robots, with respect to a simple token-gathering task. Each robot in the swarm maintains a local archive defined by two pre-specified functional traits which is shared with robots it come into contact with. We investigate four different strategies for sharing, exploiting and combining local archives and compare results to mEDEA. Experimental results show that in contrast to previous claims, it is possible to evolve a functionally diverse swarm without geographical isolation, and that the new method outperforms mEDEA in terms of the diversity, coverage and precision of the evolved swarm., In GECCO 2018

Bio-hybrid systems---close couplings of natural organisms with technology---are high potential and still underexplored. In existing work, robots have mostly influenced group behaviors of animals. We explore the possibilities of mixing robots with natural plants, merging useful attributes. Significant synergies arise by combining the plants' ability to efficiently produce shaped material and the robots' ability to extend sensing and decision-making behaviors. However, programming robots to control plant motion and shape requires good knowledge of complex plant behaviors. Therefore, we use machine learning to create a holistic plant model and evolve robot controllers. As a benchmark task we choose obstacle avoidance. We use computer vision to construct a model of plant stem stiffening and motion dynamics by training an LSTM network. The LSTM network acts as a forward model predicting change in the plant, driving the evolution of neural network robot controllers. The evolved controllers augment the plants' natural light-finding and tissue-stiffening behaviors to avoid obstacles and grow desired shapes. We successfully verify the robot controllers and bio-hybrid behavior in reality, with a physical setup and actual plants.

© 2017 ACM. We construct and investigate a strongly embodied evolutionary system, where not only the controllers but also the morphologies undergo evolution in an on-line fashion. In these studies, we have been using various types of robot morphologies and controller architectures in combination with several learning algorithms, e.g. evolutionary algorithms, reinforcement learning, simulated annealing, and HyperNEAT. This hands-on experience provides insights and helps us elaborate on interesting research directions for future development.

Due to the large number of evaluations required, evolutionary robotics experiments are generally conducted in simulated environments. One way to increase the generality of a robot's behavior is to evolve it in multiple environments. These environment spaces can be defined by the number of free parameters (f) and the number of variations each free parameter can take (n). Each environment space then has nf individual environments. For a robot to be fit in the environment space it must perform well in each of the nf environments. Thus the number of environments grows exponentially as n and f are increased. To mitigate the problem of having to evolve a robot in each environment in the space we introduce the concept of ecological modularity. Ecological modularity is here defined as the robot's modularity with respect to free parameters in the its environment space. We show that if a robot is modular along m of the free parameters in its environment space, it only needs to be evolved in nf-m+1 environments to be fit in all of the nf environments. This work thus presents a heretofore unknown relationship between the modularity of an agent and its ability to generalize evolved behaviors in new environments.

In evolutionary robotics, evolutionary methods are used to optimize robots to different tasks. Because using physical robots is costly in terms of both time and money, simulated robots are generally used instead. Most physics engines are written in C++ which can be a barrier for new programmers. In this paper we present two Python wrappers, Pyrosim and Evosoro, around two well used simulators, Open Dynamics Engine (ODE) and Voxelyze/VoxCAD, which respectively handle rigid and soft bodied simulation. Python is an easier language to understand so more time can be spent on developing the actual experiment instead of programming the simulator.

In this work we investigate the possibility to exploit environmental differentiation to promote evolvability in artificial evolution. More specifically we propose a new algorithm and demonstrate how agents evolved for the ability to solve the double-pole balancing problem in differentiated environmental conditions among the population outperform agents evolved in homogeneous environmental conditions. The algorithm operates by evolving the agents on multiple environmental niches with randomly varying environmental characteristics and by enabling agents displaying superior performance in other niches to colonize them. Agents evolved through the proposed algorithm outperform agents evolved in homogeneous environments, either on stable or temporally varying environments.

The application of evolutionary robotics [1] to swarm robotics gives evolutionary swarm robotics [8]. The evolution or learning of multi-agent behaviors is known to be challenging [7]. Hence, new approaches still need to be explored. Examples are innovative methods to explore environment-driven, distributed evolution [2, 4]. Here, we are inspired to evolve collective behaviors following a mathematical framework by Friston et al. [3], which defines an information-theoretic analogon to thermodynamic (Helmholtz) free energy. This free energy is basically an error in the predictions that our brain makes about our environment. Evolution is related by the rationale that minimal prediction errors are achieved by limiting an agent's reactions to sensory input. This results, in turn, in better adapted behaviors: "By sampling [...] the environment selectively they restrict their exchange with it within bounds that preserve their physical integrity and allow them to last longer" [3]. The previously investigated evolution of swarm behaviors by minimizing surprisal [5, 6, 9] is subject to this study. Previous studies were limited to artificial 1-d environments, here, we report first results for 2-d. Although adding one dimension may seem a minor step, there are qualitative changes in the emergent behaviors (e.g., flocking is a collective decision with infinitely many options) and the future transition to real robots will be easier starting from 2-d.

Divergent search is a recent trend in evolutionary computation that does not reward proximity to the objective of the problem it tries to solve. Traditional evolutionary algorithms tend to converge to a single good solution, using a fitness proportional to the quality of the problem's solution, while divergent algorithms aim to counter convergence by avoiding selection pressure towards the ultimate objective. This paper explores how a recent divergent algorithm, surprise search, can affect the evolution of soft robot morphologies, comparing the performance and the structure of the evolved robots.

A common aim across evolutionary search is to skillfully navigate complex search spaces, which requires search algorithms that exploit search space structure. This paper focuses on evolutionary robotics (ER) in particular, wherein controllers for robots are evolved to produce complex behavior. One productive approach for probing search space structure is to analyze properties of fitness landscapes; however, this paper argues that ER may require a fresh perspective for landscape analysis, because ER often goes beyond the black-box setting, i.e. evaluations provide useful information about how robots behave, beyond scalar performance heuristics. Indeed, some ER algorithms explicitly exploit such behavioral information, e.g. to follow gradients of behavioral novelty rather than to climb gradients of increasing performance. Thus well-motivated behavior-aware metrics may aid probing search-space structure in ER. In particular, this paper argues that behavioral conceptions of deception, evolvability, and rarity may help to understand ER landscapes, and seeks to quantify and explore them within a common ER benchmark task. To help this investigation, an expressive but limited encoding is designed, such that the behavior of all possible individuals in the domain can be precomputed. The result is an efficient platform for experimentation that facilitates (1) probing exact quantifications of deception, evolvability, and rarity in the chosen domain, and (2) the ability to efficiently drive search through idealistic ground-truth measures. The results help develop intuitions and suggest possible new ER algorithms. The hope is that the extensible open-source framework enables quick experimentation and idea generation, aiding brainstorming of new search algorithms and measures.

We investigate the possibilities of incorporating user feedback at run-time in an embodied evolution setting. User feedback in this case consists of a user ranking a small sample from the population in an ongoing evolutionary process according to some criterion. We describe an approach to develop a surrogate model, disseminate that model across the evolving robot collective and incorporate it as an additional objective during the evolutionary run. We evaluate the method in a number of scenarios, showing that this approach does to some extent cause the collective to absorb new objectives, including objectives to refrain from particular behaviour, but not to the level that is achieved by explicitly adding the true objective function.

A common aim in evolutionary search is to skillfully navigate complex search spaces. Achieving this aim requires creating search algorithms that exploit the structure of such spaces. Yet studying such structure directly is challenging because of the expansiveness of most search spaces. In the context of evolutionary robotics, this paper suggests a middle-ground approach that combines a full-fledged domain with an expressive but limited encoding, and then precomputes the behavior of all possible individuals, enabling evaluation as a look-up table. The product is an experimental playground in which search is non-trivial yet which offers extreme computational efficiency and ground truth about search-space structure. This paper describes the approach and demonstrates a range of its applications, directly exploring deception, behavioral rarity, and generalizations of evolvability in a popular benchmark task. The hope is that the extensible framework enables quick experimentation and idea generation, aiding brainstorming of new search algorithms and measures.

This work presents the hardware design and implementation of the EMeRGE open modular robot platform. EMeRGE (Easy Modular Embodied Robot Generation) modules are designed to be cheap and easy to build and their hardware is open for anyone to use and modify. Four magnetic connectors enable the quick assembly of different complex robot morphologies like the ones generated by evolutionary robotics experiments. Non-human agents, like robotic manipulators, can also take advantage of the magnetic connectors to assemble and disassemble morphologies.

Most modern applications of Evolutionary Robotics (ER) rely upon computer-based physics simulations in order to model the behavior of the systems in question. One of the greatest challenges in the field of ER, therefore, is the development of robust, high-precision and accurate physics simulators that can model all necessary and relevant real-world interactions in an computationally efficient manner. Up until now, most popular ER simulators are nonetheless deficient in one or many of these properties. Here we introduce a new competitive simulator, the Baseline-Realistic Objective Open-Ended Kinematics Simulator (BROOKS) that outperforms other off-the-shelf simulators in most criteria. Our simulator is free, open-sourced, and easy to modify. It can model a wide range of robotic platforms, substrates and environments. Moreover, we claim solutions produced within the BROOKS simulator perform almost identically in the real-world, thereby helping to address one of the most challenging aspects of simulation in Evolutionary Robotics: the Reality Gap. Ultimately, we believe that BROOKS will establish a new baseline against which all other simulators should be compared.

In this paper we report ongoing work evolving bio-hybrid societies. We develop robots that are integrated into an animal society and accepted as conspecifics. We are using evolutionary algorithms to optimise robot controllers to affect the behaviour of animals. Fitness evaluation is the result of measuring the effect a robot controller has on these animals. Animal habituation and heterogeneous response are two factors that have a major role in this fitness evaluation. We discuss our choices in designing a fitness evaluation procedure and how using animals as fitness function providers impacts this.

Social learning enables multiple robots to share learned experiences while completing a task. The literature offers examples where robots trained with social learning reach a higher performance compared to their individual learning counterparts [e.g, 2, 4]. No explanation has been advanced for that observation. In this research, we present experimental results suggesting that a lack of tuning of the parameters in social learning experiments could be the cause. In other words: the better the parameter settings are tuned, the less social learning can improve the system performance.

Recent research in small populations of Thymio II robots illustrated the relative benefits of populations distinguishing heritable and learning features in robots for a simple obstacle avoidance task. Here we scientifically validate these results by repeating them using a simulation. An additional benefit of this work is to provide confidence in the simulation model. This is important because evolutionary swarm robotics experiments can be very time consuming to run in real robots. Having a reliable simulation allows many more experiments to be run in simulation with only the most interesting results needing to be verified with real robots. We describe the development of a simulation using RoboRobo that's using the same three-tier learning framework that was demonstrated in the real-world. The simulation is shown to replicate the real-world results in terms of illustrating the relative benefits of each type of learning, and if anything, indicates that social learning can be more powerful than orieinallv thought.

Implementing lifetime learning by means of on-line evolution, we establish an indirect encoding scheme that combines Compositional Pattern Producing Networks (CPPNs) and Central Pattern Generators (CPGs) as a relevant learner and controller for open-loop gait controllers in modular robots which have evolving morphologies. Experimental validation on the morphologically evolved robots shows that a Lamarckian setup with CPPN-CPG provides substantial benefits compared to controllers learned from scratch.

The fields of artificial life and evolutionary robotics have seen growing interest in evolution as a source of creativity, as opposed to a tool for optimization. New intentionally divergent algorithms such as novelty search with local competition (NSLC) and MAP-Elites accordingly attempt to harness evolution's aptitude for divergence in a new search paradigm called quality diversity (QD), which aims to find a wide variety of possible solutions spread across a behavior space. To date, QD has mainly been studied in domains where potential diversity is limited. In anticipation of future, more open-ended applications of QD algorithms, this paper introduces a new domain inspired by the popular Minecraft video game featuring a larger behavior space that is substantially more difficult to exhaust. Preliminary results are presented, showcasing sample block structures built by evolved neural network controllers.

We, as well as others, have already shown in previous works that reproductive isolation and a large population size are critical to achieve behavioral specialization in embodied evolutionary robotics. Here, we extend our previous work from [3] by experimentally demonstrating that fitness-proportionate and rank-based selection operators largely outperform other selection operators when it comes to evolving behavioral specialization.

Different subsystems of organisms adapt over many time scales, such as rapid changes in the nervous system (learning), slower morphological and neurological change over the lifetime of the organism (postnatal development), and change over many generations (evolution). Much work has focused on instantiating learning or evolution in robots, but relatively little on development. Although many theories have been forwarded as to how development can aid evolution, it is difficult to isolate each such proposed mechanism. Thus, here we introduce a minimal yet embodied model of development: the body of the robot changes over its lifetime, yet growth is not influenced by the environment. We show that even this simple developmental model confers evolvability because it allows evolution to sweep over a larger range of body plans than an equivalent non-developmental system, and subsequent heterochronic mutations 'lock in' this body plan in more morphologically-static descendants. Future work will involve gradually complexifying the developmental model to determine when and how such added complexity increases evolvability.