Search

The ability of robots to write Chinese strokes, which is recognized as a sophisticated task, involves complicated kinematic control algorithms. The conventional approaches for robotic writing of Chinese strokes often suffer from limited font generation methods, which limits the ability of robots to perform high-quality writing. This paper instead proposes a developmental evolutionary learning framework that enables a robot to learn to write fundamental Chinese strokes. The framework first considers the learning process of robotic writing as an evolutionary easy-to-difficult procedure. Then, a developmental learning mechanism called “Lift-constraint, act and saturate” that stems from developmental robotics is used to determine how the robot learns tasks ranging from simple to difficult by building on the learning results from the easy tasks. The developmental constraints, which include altitude adjustments, number of mutation points, and stroke trajectory points, determine the learning complexity of robot writing. The developmental algorithm divides the evolutionary procedure into three developmental learning stages. In each stage, the stroke trajectory points gradually increase, while the number of mutation points and adjustment altitudes gradually decrease, allowing the learning difficulties involved in these three stages to be categorized as easy, medium, and difficult. Our robot starts with an easy learning task and then gradually progresses to the medium and difficult tasks. Under various developmental constraint setups in each stage, the robot applies an evolutionary algorithm to handle the basic shapes of the Chinese strokes and eventually acquires the ability to write with good quality. The experimental results demonstrate that the proposed framework allows a calligraphic robot to gradually learn to write five fundamental Chinese strokes and also reveal a developmental pattern similar to that of humans. Compared to an evolutionary algorithm without the developmental mechanism, the proposed framework achieves good writing quality more rapidly.

Robotics currently lacks fully autonomous capabilities, especially where task knowledge is incomplete and optimal robotic solutions cannot be pre-engineered. The intersection of evolutionary robotics, artificial life and embodied artificial intelligence presents a promising paradigm for generating multitask problem-solvers suitable for adapting over extended periods in unexplored, remote and hazardous environments. To address the automation of evolving robotic systems, we propose fully autonomous, embodied artificial-life factories and laboratories, situated in various environments as multi-task problem-solvers. Such integrated factories and laboratories would be adaptive solution designers, producing fit-for-purpose physical robots with accelerated artificial evolution that experiment to continually discover new tasks. Such tasks would be stepping-stones towards accomplishing given mission objectives over extended periods (days to decades). Rather than being purely speculative, prerequisite technologies to realize such factories have been experimentally demonstrated. Currently, vast scientific and enterprise opportunities await in applications such as asteroid mining, terraforming, space and deep sea exploration, though no suitable solution exists. The proposed embodied artificial-life factories and laboratories, termed: AutoFac, use robot production equipment run by artificial evolution controllers to collect and synthesize environmental information (from robotic sensory systems). Such information is merged with current needs and mission objectives to create new robot embodiment and task definitions that are environmentally adapted and balance task-oriented behavior with exploration. AutoFac is thus generalist (deployable in many environments) but continually produces specialist solutions within such environments a perpetual robot machine.

The challenge of robotic reproduction -- making of new robots by recombining two existing ones -- has been recently cracked and physically evolving robot systems have come within reach. Here we address the next big hurdle: producing an adequate brain for a newborn robot. In particular, we address the task of targeted locomotion which is arguably a fundamental skill in any practical implementation. We introduce a controller architecture and a generic learning method to allow a modular robot with an arbitrary shape to learn to walk towards a target and follow this target if it moves. Our approach is validated on three robots, a spider, a gecko, and their offspring, in three real-world scenarios., 12 pages

We introduce a method that permits to co-evolve the body and the control properties of robots. It can be used to adapt the morphological traits of robots with a hand-designed morphological bauplan or to evolve the morphological bauplan as well. Our results indicate that robots with co-adapted body and control traits outperform robots with fixed hand-designed morphologies. Interestingly, the advantage is not due to the selection of better morphologies but rather to the mutual scaffolding process that results from the possibility to co-adapt the morphological traits to the control traits and vice versa. Our results also demonstrate that morphological variations do not necessarily have destructive effects on robots’ skills.

In evolutionary robotics, robot controllers are often evolved in simulation, as using the physical robot for fitness evaluation can take a prohibitively long time. Simulators provide a quick way to evaluate controller fitness. A simulator is tasked with providing appropriate sensor information to the controller. If the robot has an on-board camera, an entire virtual visual environment is needed to simulate the camera’s signal. In the past, these visual environments have been constructed by hand, requiring the use of hand-crafted models, textures and lighting, which is a tedious and time-consuming process. This paper proposes a deep neural network-based architecture for simulating visual environments. The neural networks are trained exclusively from images captured from the robot, creating a 3-dimensional visual environment without using hand-crafted models, textures or lighting. It does not rely on any external domain specific datasets, as all training data is captured in the physical environment. Robot controllers were evolved in simulation to discern between objects with different colours and shapes, and they successfully completed the same task in the real world.

Theory of mind (ToM) is the ability to understand others’ mental states (e.g., intentions). Studies on human ToM show that the way we understand others’ mental states is very efficient, in the sense that observing only some portion of others’ behaviors can lead to successful performance. Recently, ToM has gained interest in robotics to build robots that can engage in complex social interactions. Although it has been shown that robots can infer others’ internal states, there has been limited focus on the data utilization of ToM mechanisms in robots. Here we show that robots can infer others’ intentions based on limited information by selectively and flexibly using behavioral cues similar to humans. To test such data utilization, we impaired certain parts of an actor robot’s behavioral information given to the observer, and compared the observer’s performance under each impairment condition. We found that although the observer’s performance was not perfect compared to when all information was available, it could infer the actor’s mind to a degree if the goal-relevant information was intact. These results demonstrate that, similar to humans, robots can learn to infer others’ mental states with limited information.

Rapid developments in evolutionary computation, robotics, 3D-printing, and material science are enabling advanced systems of robots that can autonomously reproduce and evolve. The emerging technology of robot evolution challenges existing AI ethics because the inherent adaptivity, stochasticity, and complexity of evolutionary systems severely weaken human control and induce new types of hazards. In this paper we address the question how robot evolution can be responsibly controlled to avoid safety risks. We discuss risks related to robot multiplication, maladaptation, and domination and suggest solutions for meaningful human control. Such concerns may seem far-fetched now, however, we posit that awareness must be created before the technology becomes mature.

Quality-Diversity optimization algorithms such as MAP-Elites, aim to generate collections of both diverse and high-performing solutions to an optimization problem. MAP-Elites has shown promising results in a variety of applications. In particular in evolutionary robotics tasks targeting the generation of behavioral repertoires that highlight the versatility of robots. However, for most robotics applications MAP-Elites is limited to using simple open-loop or low-dimensional controllers. Here we present Policy Gradient Assisted MAP-Elites (PGA-MAP-Elites), a novel algorithm that enables MAP-Elites to efficiently evolve large neural network controllers by introducing a gradient-based variation operator inspired by Deep Reinforcement Learning. This operator leverages gradient estimates obtained from a critic neural network to rapidly find higher-performing solutions and is paired with a traditional genetic variation to maintain a divergent search behavior. The synergy of these operators makes PGA-MAP-Elites an efficient yet powerful algorithm for finding diverse and high-performing behaviors. We evaluate our method on four different tasks for building behavioral repertoires that use deep neural network controllers. The results show that PGA-MAP-Elites significantly improves the quality of the generated repertoires compared to existing methods.

International audience; We present a behavioral diversity selection scheme that favors reproductive isolation to promote the learning of multiple task in on-line embodied evolutionary robotics (EER). The scheme estimates the behavior of the controllers without the need to access the agent experience, respecting thus the online, distributed properties EER. Reproductive isolation is assessed through coalescence trees and task specialization is tested on a concurrent foraging setting.

What makes a quality-diversity (QD) algorithm effective? This question is increasingly studied as such algorithms gain in popularity. Recent work on Novelty Search linked its efficiency to the evolution of a population under a dynamic selection pressure defined by the novelty metric, which allows the population to converge to highly evolvable individuals, on which spending the evolutionary budget allows to navigate the behavior space. MAP-Elites, another popular QD algorithm, does not use an explicit population, instead sampling the parent population directly from the behavioral archive at each generation. This sampling can be uniform, or biased according to criteria such as curiosity, which favor individuals that previously generated successful offsprings. In this article, we show that such improved selection schemes are efficient because they create a dynamic pseudo-population that mimics the desirable qualities of a Novelty Search population. We do this by proposing another simpler selection scheme, youth, that simply focuses the evolutionary budget on such a dynamic pseudo-population, and by showing it exhibits similar behavior and performance as curiosity without explicitly selecting for previously successful parents. This has important implications for the design of future QD algorithms taking the best of both archive-based and population-based methods.

Organisms with different body morphology and movement dynamics have distinct abilities to move through the environment. Despite such truism, there is a lack of general principles that predict which shapes and dynamics make the organisms more fit to move. Studying a minimal yet embodied soft robot model under the influence of gravity, we find three features that predict robot locomotion fitness: (1) A larger body is better. (2) Two-point contact with the ground is better than one-point contact. (3) Out-of-phase oscillating body parts increase locomotion fitness. These design principles can guide the selection rules for evolutionary algorithms to obtain robots with higher locomotion fitness.

Locating odour sources is a hard task that has been addressed with a large variety of AI methods to produce search strategies with different levels of efficiency and robustness. However, it is still not clear how to evaluate those strategies. Simply evaluating the robot's ability to reach the goal may produce deceptive fitness values, favouring poor strategies that do not generalise. Conversely, including prior knowledge may bias the learning process. This work studies the impact of evaluation functions with various degrees of prior knowledge, in evolving search strategies. The baseline is set by performing multiple evaluations of each strategy with a function that only evaluates the task efficiency. A function was found that is able to produce strategies with equivalent performance to those of the baseline, whilst performing a single evaluation.

This work presents a platform for evolution of morphology in full cycle reconfigurable hardware: The EMERGE (Easy Modular Embodied Robot Generator) modular robot platform. Three parts necessary to implement a full cycle process, i.e., assembling the modules in morphologies, testing the morphologies, disassembling modules and repeating, are described as a previous step to testing a fully autonomous system: the mechanical design of the EMERGE module, extensive tests of the modules by first assembling them manually, and automatic assembly and disassembly tests. EMERGE modules are designed to be easy and fast to build, one module is built in half an hour and is constructed from off-the-shelf and 3D printed parts. Thanks to magnetic connectors, modules are quickly attached and detached to assemble and reconfigure robot morphologies. To test the performance of real EMERGE modules, 30 different morphologies are evolved in simulation, transferred to reality, and tested 10 times. Manual assembly of these morphologies is aided by a visual guiding tool that uses AprilTag markers to check the real modules positions in the morphology against their simulated counterparts and provides a color feedback. Assembly time takes under 5 min for robots with fewer than 10 modules and increases linearly with the number of modules in the morphology. Tests show that real EMERGE morphologies can reproduce the performance of their simulated counterparts, considering the reality gap. Results also show that magnetic connectors allow modules to disconnect in case of being subjected to high external torques that could damage them otherwise. Module tracking combined with their easy assembly and disassembly feature enable EMERGE modules to be also reconfigured using an external robotic manipulator. Experiments demonstrate that it is possible to attach and detach modules from a morphology, as well as release the module from the manipulator using a passive magnetic gripper. This shows that running a completely autonomous, evolution of morphology in full cycle reconfigurable hardware of different topologies for robots is possible and on the verge of being realized. We discuss EMERGE features and the trade-off between reusability and morphological variability among different approaches to physically implement evolved robots.

Evolving morphologies and controllers of robots simultaneously leads to a problem: Even if the parents have well-matching bodies and brains, the stochastic recombination can break this match and cause a body-brain mismatch in their offspring. We argue that this can be mitigated by having newborn robots perform a learning process that optimizes their inherited brain quickly after birth. We compare three different algorithms for doing this. To this end, we consider three algorithmic properties, efficiency, efficacy, and the sensitivity to differences in the morphologies of the robots that run the learning process., Comment: Associated code: https://github.com/fudavd/revolve/tree/learning

SUMMARYThis paper investigates artificial neural network (ANN)-based simulators as an alternative to physics-based approaches for evolving controllers in simulation for a complex snake-like robot. Prior research has been limited to robots or controllers that are relatively simple. Benchmarks are performed in order to identify effective simulator topologies. Additionally, various controller evolution strategies are proposed, investigated and compared. Using ANN-based simulators for controller fitness estimation during controller evolution is demonstrated to be a viable approach for the high-dimensional problem specified in this work.

It has often been found that students appreciate hands-on work, and find that they learn more with courses that include a project than those relying solely on conventional lectures and tests. This type of project driven learning is a key component of “Inquiry-based learning” (IBL), which aims at teaching methodology as well as content by incorporating the student as an actor rather than a spectator. Robotics applications are especially well-suited for IBL due to the value of trial and error experience, the multiple possibilities for students to implement their own ideas, and the importance of programming, problem-solving and electro-mechanical skills in real world engineering and science jobs. Furthermore, robotics platforms can be useful teaching media and learning tools for a variety of topics. Here, we present RoboGen: an open-source, web-based, software and hardware platform for Robotics and Artificial Intelligence with a particular focus on Evolutionary Robotics. We describe the platform in detail, compare it to existing alternatives, and present results of its use as a platform for Inquiry-based learning within a master's level course at the Ecole Polytechnique Fédérale de Lausanne (EPFL).

SummaryMission planning is a complex motion planning problem specified by using Temporal Logic constituting of Boolean and temporal operators, typically solved by model verification algorithms with an exponential complexity. The paper proposes co-evolutionary optimization thus building an iterative solution to the problem. The language for mission specification is generic enough to represent everyday missions, while specific enough to design heuristics. The mission is broken into components which cooperate with each other. The experiments confirm that the robot is able to outperform the search, evolutionary and model verification techniques. The results are demonstrated by using a Pioneer LX robot.

We demonstrate the training of Spiking Neural Networks (SNN) in a novel multi-agent Evolutionary Robotics (ER) framework inspired by competitive evolutionary environments in nature. The topology of a SNN along with morphological parameters of the bot it controls in the ER environment is together treated as a phenotype. Rules of the framework select certain bots and their SNNs for reproduction and others for elimination based on their efficacy in capturing food in a competitive environment. While the bots and their SNNs are not explicitly trained to survive or reproduce using loss functions, these drives emerge implicitly as they evolve to hunt food and survive. Their efficiency in capturing food exhibits the evolutionary signature of punctuated equilibrium. We use this signature to compare the performances of two evolutionary inheritance algorithms on the phenotypes, Mutation and Crossover with Mutation, using ensembles of 100 experiments for each algorithm. We find that Crossover with Mutation promotes 40% faster learning in the SNN than mere Mutation with a statistically significant margin.

An open question for both natural and artificial evolutionary systems is how, and under what environmental and evolutionary conditions complexity evolves. This study investigates the impact of increasingly complex task environments on the evolution of robot complexity. Specifically, the impact of evolving body-brain couplings on locomotive task performance, where robot evolution was directed by either body-brain exploration (novelty search) or objective-based (fitness function) evolutionary search. Results indicated that novelty search enabled the evolution of increased robot body-brain complexity and efficacy given specific environment conditions. The key contribution is thus the demonstration that body-brain exploration is suitable for evolving robot complexity that enables high fitness robots in specific environments.

One of the most active fields of research in evolutionary robotics is the development of autonomous robots with the ability to interact with the physical world and to communicate with each other, in “robot societies”. Interactions may involve a range of different motor actions, motivational forces and cognitive processes. Actions, in turn directly affect the agent’s perceptions of the world. In the “Action/Perception-Cycle” (see Figure 1), biological organisms are integrated sensorimotor systems. This means that intelligent processes require a body, and that symbols are grounded in the environment in which animals live (Harnad, 1990). In short, behavior is fundamentally linked to cognition. This is true for humans, animals and artificial agents. Without this grounding, artificial animals and agents cannot live and behave successfully in their artificial environments. One way of achieving it, is to use Genetic Algorithms to evolve agents’ neural architecture (Nolfi & Floreano, 2000). This creates the prospect of robots that can live in complex socially organized communities in which they communicate with humans and with each other (Cangelosi e Parisi, 2002). According to these authors, cognition is an intrinsically embedded phenomenon in which the dynamical relations between the neural system, the body and the environment play a central role. In this view, agents are dynamical systems and cognitive functioning has to be understood using tools from dynamical system theory (van Gelder, 1995, 1998a; 1998b; Bilotta et al., 2007a-2007f). This perspective on cognition has been called the Dynamical and Embodied view of Cognition (DEC) (Keijzer, 2002). In this chapter we describe our own contribution to Evolutionary Robotics, namely a proposal for a new generation of believable agents capable of life-like intelligent communicative and emotional behavior. We focus on CNNs (Cellular Neural Networks) and on the use of these networks as robot controllers. In previous work, we used Genetic Algorithms (GAs) to evolve Artificial Non-linear Networks (ANNs) displaying artificial adaptive behavior, with features similar to those observed in animals and humans. In (Bilotta et al. 2006), we replaced ANNs with a new class of dynamical system called Cellular Non-linear Networks (CNNs) and used CNNs to implement a multilayer locomotion model for six-legged artificial robots in a virtual environment with many of the characteristics of a physical environment. First invented by Chua and co-workers (1988), CNNs have been extended to create a CNN Universal Machine (CNN-UM) (Roska & Chua, 1993), the first algorithmically programmable analog computer chip suitable for the modeling of sensory

Evolutionary robotics is a good method for the generation of controllers for autonomous robots. However, up to date, evolutionary methods do not achieve the generation of behaviors for complex robots with a fixed body structure composed of lots of sensors and actuators. For such cases, no satisfactory results exist due to the large search space that the evolutionary algorithm has to face. Furthermore, the bootstrap problem does not allow convergence on the first generations, preventing the generation of simple solutions with a minimum fitness value that could guide the evolutionary path towards the final solution. Solutions like incremental evolution try to overcome the problem, but they do not scale well in complex robots with lots of devices. The question is then, why natural evolution succeeded evolving complex animals, but evolutionary robotics does not. One answer to that question may be that, while natural evolution gradually evolved at the same time the animals body plan, their sensors and actuators, their nervous system, and even their environment, artificial evolution tries to evolve the nervous system for a robot with a fixed given body, sensors and actuators, within a fixed complex environment. Our proposal states that when, as it happens in most cases, none of the evolutionary constraints can be relaxed (those are, the robot body, the sensors and actuators, the behavior to implement or the environment), then the use of external knowledge to guide the evolutionary process should be mandatory. Evolutionary approaches try to avoid the use of such knowledge, also called bias, because it directs the evolutionary search towards specific places of the space, not allowing the algorithm to find its own solutions. In this paper, we advocate instead for the use of bias as an inevitable situation when the robot body, task and environment are complex and fixed. Based on this idea, we develop a modular architecture for evolutionary controllers based on neural networks, which allows the selective introduction of bias knowledge in the neural controller during the evolutionary process. The architecture allows the introduction of external knowledge on selected stages of the evolutionary process, affecting only selected parts of the controller that need to accommodate that information. The evolutionary controller is progressively designed in a series of stages, almost in a surgical way, independently of the complexity of the robot (in terms of number of sensors and actuators). This approach allows to avoid the bootstrap problem completely, and to obtain a completely distributed neural controller for the robot using only artificial evolution.

Quality-Diversity algorithms search for large collections of diverse and high-performing solutions, rather than just for a single solution like typical optimisation methods. They are specially adapted for multi-modal problems that can be solved in many different ways, such as complex reinforcement learning or robotics tasks. However, these approaches are highly dependent on the choice of feature descriptors (FDs) quantifying the similarity in behaviour of the solutions. While FDs usually needs to be hand-designed, recent studies have proposed ways to define them automatically by using feature extraction techniques, such as PCA or Auto-Encoders, to learn a representation of the problem from previously explored solutions. Here, we extend these approaches to more complex problems which cannot be efficiently explored by relying only on a single representation but require instead a set of diverse and complementary representations. We describe MC-AURORA, a Quality-Diversity approach that optimises simultaneously several collections of solutions, each with a different set of FDs, which are, in turn, defined automatically by an ensemble of modular auto-encoders. We show that this approach produces solutions that are more diverse than those produced by single-representation approaches., Draft version. 10 pages, 4 figures, 4 tables, Accepted at the GECCO2021 Conference

There are several ubiquitous kinematic structures that are used in industrial robots, with the most prominent being a six-axis angular structure. However, researchers are experimenting with task-based mechanism synthesis that could provide higher efficiency with custom optimized manipulators. Many studies have focused on finding the most efficient optimization algorithm for task-based robot manipulators. These manipulators, however, are usually optimized from simple modular joints and links, without exploring more elaborate modules. Here, we show that link modules defined by small numbers of parameters have better performance than more complicated ones. We compare four different manipulator link types, namely basic predefined links with fixed dimensions, straight links that can be optimized for different lengths, rounded links, and links with a curvature defined by a Hermite spline. Manipulators are then built from these modules using a genetic algorithm and are optimized for three different tasks. The results demonstrate that manipulators built from simple links not only converge faster, which is expected given the fewer optimized parameters, but also converge on lower cost values. Web of Science 11 6 art. no. 2471

Animals ranging from rats to humans can demonstrate cognitive map capabilities. We evolved weights in a biologically plausible recurrent neural network (RNN) using an evolutionary algorithm to replicate the behavior and neural activity observed in rats during a spatial and working memory task in a triple T-maze. The rat was simulated in the Webots robot simulator and used vision, distance and accelerometer sensors to navigate a virtual maze. After evolving weights from sensory inputs to the RNN, within the RNN, and from the RNN to the robot's motors, the Webots agent successfully navigated the space to reach all four reward arms with minimal repeats before time-out. Our current findings suggest that it is the RNN dynamics that are key to performance, and that performance is not dependent on any one sensory type, which suggests that neurons in the RNN are performing mixed selectivity and conjunctive coding. Moreover, the RNN activity resembles spatial information and trajectory-dependent coding observed in the hippocampus. Collectively, the evolved RNN exhibits navigation skills, spatial memory, and working memory. Our method demonstrates how the dynamic activity in evolved RNNs can capture interesting and complex cognitive behavior and may be used to create RNN controllers for robotic applications.

Taking inspiration from the navigation ability of humans, this study investigated a method of providing robotic controllers with a basic sense of position. It incorporated robotic simulators into robotic controllers to provide them with a mechanism to approximate the effects their actions had on the robot. Controllers with and without internal simulators were tested and compared. The proposed controller architecture was shown to outperform the regular controller architecture. However, the longer an internal simulator was executed, the more inaccurate it became. Thus, the performance of controllers with internal simulators reduced over time unless their internal simulator was periodically corrected.

This study is motivated by evolutionary robot systems where robot bodies and brains evolve simultaneously. In such systems robot 'birth' must be followed by 'infant learning' by a learning method that works for various morphologies evolution may produce. Here we address the task of directed locomotion in modular robots with controllers based on Central Pattern Generators. We present a bio-inspired adaptive feedback mechanism that uses a forward model and an inverse model that can be learned on-the-fly. We compare two versions (a simple and a sophisticated one) of this concept to a traditional (open-loop) controller using Bayesian optimization as a learning algorithm. The experimental results show that the sophisticated version outperforms the simple one and the traditional controller. It leads to a better performance and more robust controllers that better cope with noise.

The long term goal of the Autonomous Robot Evolution (ARE) project is to create populations of physical robots, in which both the controllers and body plans are evolved. The transition of evolutionary designs from purely simulation environments into the real world creates the possibility for new types of system able to adapt to unknown and changing environments. In this paper, a system for creating robots is introduced in order to allow for their body plans to be designed algorithmically and physically instantiated using the previously introduced Robot Fabricator. This system consists of two types of components. Firstly, skeleton parts are created bespoke for each design by 3D printing, allowing the overall shape of the robot to include almost infinite variety. To allow for the shortcomings of 3D printing, the second type of component are organs which contain components such as motors and sensors, and can be attached to the skeleton to provide particular functions. Specific organ designs are presented, with discussion of the design challenges for evolutionary robotics in hardware. The Robot Fabricator is extended to allow for robots with joints, and some example body plans shown to demonstrate the diversity possible using this system of robot generation.

In many natural environments, there are different forms of living creatures that successfully accomplish the same task while being diverse in shape and behavior. This biodiversity is what made life capable of adapting to disrupting changes. Being able to reproduce biodiversity in non-biological agents, while still optimizing them for a particular task, might increase their applicability to scenarios where human response to unexpected changes is not possible. In this work, we focus on Voxel-based Soft Robots (VSRs), a form of robots that grants great freedom in the design of both body and controller and is hence promising in terms of biodiversity. We use evolutionary computation for optimizing, at the same time, body and controller of VSRs for the task of locomotion. We investigate experimentally whether two key factors - -evolutionary algorithm (EA) and representation - -impact the emergence of biodiversity and if this occurs at the expense of effectiveness. We devise a way for measuring biodiversity, systematically characterizing the robots shape and behavior, and apply it to the VSRs evolved with three EAs and two representations. The experimental results suggest that the representation matters more than the EA and that there is not a clear trade-off between diversity and effectiveness.

Artificial neural networks (ANNs) are commonly used for controlling robotic agents. For robots with many sensors and actuators, ANNs can be very complex, with many neurons and connections. Removal of neurons or connections, i.e., pruning, may be desirable because (a) it reduces the complexity of the ANN, making its operation more energy efficient, and (b) it might improve the generalization ability of the ANN. Whether these goals can actually be achieved in practice is however still not well known. On the other hand, it is widely recognized that pruning in biological neural networks plays a fundamental role in the development of brains and their ability to learn. In this work, we consider the case of Voxel-based Soft Robots, a kind of robots where sensors and actuators are distributed over the body and that can be controlled with ANNs optimized by means of neuroevolution. We experimentally characterize the effect of different forms of pruning on the effectiveness of neuroevolution, also in terms of generalization ability of the evolved ANNs. We find that, with some forms of pruning, a large portion of the connections can be pruned without strongly affecting robot capabilities. We also observe sporadic improvements in generalization ability.

Evolutionary Algorithms have been applied in robotics over the last quarter of a century to simultaneously evolve robot body morphology and controller. However, the literature shows that in this area, one is still unable to generate robots that perform better than conventional manual designs, even for simple tasks. It is noted that the main hindrance to satisfactory evolution is poor controller generated as a result of the simultaneous variation of the morphology and controller. As the controller is a result of pure evolution, it is not given a chance to improve before fitness is calculated, which is the equivalent of reproducing immediately after birth in biological evolution. Therefore, to improve co-evolution and to bring artificial robot evolution a step closer to biological evolution, this paper introduces Reinforced Co-evolution Algorithm (ReCoAl), which is a hybrid of an Evolutionary and a Reinforcement Learning algorithm. It combines the evolutionary and learning processes found in nature to co-evolve robot morphology and controller. ReCoAl works by allowing a direct policy gradient based RL algorithm to improve the controller of an evolved robot to better utilise the available morphological resources before fitness evaluation. The ReCoAl is tested for evolving mobile robots to perform navigation and obstacle avoidance. The findings indicate that the controller learning process has both positive and negative effects on the progress of evolution, similar to observations in evolutionary biology. It is also shown how, depending on the effectiveness of the learning algorithm, the evolver generates robots with similar fitness in different generations.

Evolution has been a topic of interest that has been explored extensively throughout recent history. Ever since the proposal of the evolutionary theory by Darwin, there have been attempts made to validate, extend, explore and exploit the different aspects of this theory. This study sets forth methods to explore elements of what has been proposed by Darwinian evolution using Braitenberg vehicles, simple machines with sensory-motor couplings in an environmental context. In this study, a simulation environment is set up based upon the descriptions of neurophysiologist Valentino Braitenberg. This simulation environment is then utilized to carry out the thought experiments envisioned by Braitenberg for the simplest nontrivial kinds of vehicles. The methodology and results for the carried out experiments are detailed in this paper. Apart from an understanding of whether the environmental setup affects evolution, specifically the number of light sources (stimuli) and location of these light sources, the experiments show an interesting trend regarding dynamic equilibrium of the evolutionary process, the ramifications of which might not have been understood well enough previously. It is concluded that ecological setup, as well as the initial genetic makeup of the vehicles, play a crucial role in the evolution of vehicles in scenarios laid out in this study. Further, the placement of stimuli (location of lights) and the number of the stimuli have a visible effect on the survivability of vehicle types (species).

Robots operating in the real world will experience a range of different environments and tasks. It is essential for the robot to have the ability to adapt to its surroundings to work efficiently in changing conditions. Evolutionary robotics aims to solve this by optimizing both the control and body (morphology) of a robot, allowing adaptation to internal, as well as external factors. Most work in this field has been done in physics simulators, which are relatively simple and not able to replicate the richness of interactions found in the real world. Solutions that rely on the complex interplay among control, body, and environment are therefore rarely found. In this article, we rely solely on real-world evaluations and apply evolutionary search to yield combinations of morphology and control for our mechanically self-reconfiguring quadruped robot. We evolve solutions on two distinct physical surfaces and analyze the results in terms of both control and morphology. We then transition to two previously unseen surfaces to demonstrate the generality of our method. We find that the evolutionary search finds high-performing and diverse morphology-controller configurations by adapting both control and body to the different properties of the physical environments. We additionally find that morphology and control vary with statistical significance between the environments. Moreover, we observe that our method allows for morphology and control parameters to transfer to previously unseen terrains, demonstrating the generality of our approach.

Crowd navigation with autonomous systems is a topic which has seen a rapid increase in interest recently. While it appears natural to humans, being able to reach a target can prove difficult or impossible to a mobile robot because of the safety issues related to collisions with people. In this work we propose an approach to control a robot in a crowded environment; the method employs an Artificial Neural Network (ANN) that is trained with the NeuroEvolution of Augmented Topologies (NEAT) method. Models for the kinematics, perception, and cognition of the robot are presented. In particular, perception is based on a raycasting model which is tailored on the ANN. An in-depth analysis of a number of parameters of the environment and the robot is performed and a comparative analysis is presented; finally, results of the performance of the controller trained with NEAT are compared to those of a human driver who takes over the controller itself. Results show that the intelligent controller is able to perform on par with the human, within the simulated environment.

We outline a perspective on the future of evolutionary robotics and discuss a long-term vision regarding robots that evolve in the real world. We argue that such systems offer significant potential for advancing both science and engineering. For science, evolving robots can be used to investigate fundamental issues about evolution and the emergence of embodied intelligence. For engineering, artificial evolution can be used as a tool that produces good designs in difficult applications in complex unstructured environments with (partially) unknown and possibly changing conditions. This implies a new paradigm, second-order software engineering, where instead of directly developing a system for a given application, we develop an evolutionary system that will develop the target system for us. Importantly, this also holds for the hardware; with a complete evolutionary robot system, both the software and the hardware are evolved. In this chapter, we discuss the long-term vision, elaborate on the main challenges, and present the initial results of an ongoing research project concerned with the first tangible implementation of such a robot system.

In modular robotics, modules can be reconfigured to change the morphology of the robot, making it able to adapt for specific tasks. However, optimizing both the body and control is a difficult challenge due to the intricate relationship between fine-tuning control and morphological changes that can invalidate such optimizations. To solve this challenge we compare three different Evolutionary Algorithms on their capacity to optimize morphologies in modular robotics. We compare two objective-based search algorithms, with MAP-Elites. To understand the benefit of diversity we transition the evolved populations into two difficult environments to see if diversity can have an impact on solving complex environments. In addition, we analyse the genealogical ancestry to shed light on the notion of stepping stones as key to enable high performance. The results show that MAP-Elites is capable of evolving the highest performing solutions in addition to generating the largest morphological diversity. For the transition between environments the results show that MAP-Elites is better at regaining performance by promoting morphological diversity. With the analysis of genealogical ancestry we show that MAP-Elites produces more diverse and higher performing stepping stones than the other objective-based search algorithms. Transitioning the populations to more difficult environments show the utility of morphological diversity, while the analysis of stepping stones show a strong correlation between diversity of ancestry and maximum performance on the locomotion task. The paper shows the advantage of promoting diversity for solving a locomotion task in different environments for modular robotics. By showing that the quality and diversity of stepping stones in Evolutionary Algorithms is an important factor for overall performance we have opened up a new area of analysis and results.

Advances in rapid prototyping have opened up new avenues of research within Evolutionary Robotics in which not only controllers but also the body plans (morphologies) of robots can evolve in real-time and real-space. However, this also introduces new challenges, in that robot models that can be instantiated from an encoding in simulation might not be manufacturable in practice (due to constraints associated with the 3D printing and/or automated assembly processes). We introduce a representation for evolving (wheeled) robots with a printed plastic skeleton, and evaluate three variants of a novelty-search algorithm in terms of their ability to produce populations of manufacturable but diverse robots. While the set of manufacturable robots discovered represent only a small fraction of the overall search space of all robots, all methods are shown to be capable of generating a diverse population of manufacturable robots that we conjecture is large enough to seed an evolving robotic ecosystem.

RealTimeBattle is an environment in which robots controlled by programs fight each other. Programs control the simulated robots using low-level messages (e.g., turn radar, accelerate). Unlike other tools like Robocode, each of these robots can be developed using different programming languages. Our purpose is to generate, without human programming or other intervention, a robot that is highly competitive in RealTimeBattle. To that end, we implemented an Evolutionary Computation technique: Genetic Programming. The robot controllers created in the course of the experiments exhibit several different and effective combat strategies such as avoidance, sniping, encircling and shooting. To further improve their performance, we propose a function-set that includes short-term memory mechanisms, which allowed us to evolve a robot that is superior to all of the rivals used for its training. The robot was also tested in a bout with the winner of the previous &ldquo, RealTimeBattle Championship,&rdquo, which it won. Finally, our robot was tested in a multi-robot battle arena, with five simultaneous opponents, and obtained the best results among the contenders.

We investigate the possibilities, challenges, and limitations that arise from the use of behavior trees in the context of the automatic modular design of collective behaviors in swarm robotics. To do so, we introduceMaple, an automatic design method that combines predefined modules—low-level behaviors and conditions—into a behavior tree that encodes the individual behavior of each robot of the swarm. We present three empirical studies based on two missions:aggregationandForaging. To explore the strengths and weaknesses of adopting behavior trees as a control architecture, we compareMaplewithChocolate, a previously proposed automatic design method that uses probabilistic finite state machines instead. In the first study, we assessMaple’s ability to produce control software that crosses the reality gap satisfactorily. In the second study, we investigateMaple’s performance as a function of the design budget, that is, the maximum number of simulation runs that the design process is allowed to perform. In the third study, we explore a number of possible variants ofMaplethat differ in the constraints imposed on the structure of the behavior trees generated. The results of the three studies indicate that, in the context of swarm robotics, behavior trees might be appealing but in many settings do not produce better solutions than finite state machines.

In this review we introduce the framework of reality-assisted evolution to summarize a growing trend towards combining model-based and model-free approaches to improve the design of physically embodied soft robots. In silico, data-driven models build, adapt and improve representations of the target system using real-world experimental data. By simulating huge numbers of virtual robots using these data-driven models, optimization algorithms can illuminate multiple design candidates for transference to the real world. In reality, large-scale physical experimentation facilitates the fabrication, testing and analysis of multiple candidate designs. Automated assembly and reconfigurable modular systems enable significantly higher numbers of real-world design evaluations than previously possible. Large volumes of ground-truth data gathered via physical experimentation can be returned to the virtual environment to improve data-driven models and guide optimization. Grounding the design process in physical experimentation ensures the complexity of virtual robot designs does not outpace the model limitations or available fabrication technologies. We outline key developments in the design of physically embodied soft robots under the framework of reality-assisted evolution., Comment: Final manuscript, accepted for publication in Artificial Life

This article studies the performance of different evolutionary strategies for deep reinforcement learning policy optimization. The policy will be centred in an important swarm robotic task: the aggregation of simple robots in the environment. The main inspiration for robotic swarm comes from the observation of social animals. Ants, bees, birds, and fish are some examples of how simple individuals can succeed when they gather in groups. In addition, is important to highlight that aggregation may be considered as a previous requirement to tackle another tasks.Due the design of a swarm behaviour is a comprehensive process of experimentation, one of the current solutions is learn a policy able to control a robot. Gradient descent techniques have demonstrated their learning power in the field of neural networks and reinforcement learning, among others. But some of these techniques are difficult to be applied in the field of robotics because the requirements needed to calculate the gradient to be informative are not met. For that reason, in this article we are going to use and compare the evolutionary strategies CMA-ES, PEPG, SES, GA y OpenAI-ES. A fast simulator, based in the differential robot Mbot Ranger, will be used. Once the aggregation task is learned we will compare each strategy for different swarm sizes to analyse the convergence time, the quality of the policies, its scalability and its capacity to generalize.

This paper reflects on a seminal work in the history of AI and representation: Rodney Brooks’ 1991 paper Intelligence without representation. Brooks advocated the removal of explicit representations and engineered environments from the domain of his robotic intelligence experimentation, in favour of an evolutionary-inspired approach using layers of reactive behaviour that operated independently of each other. Brooks criticised the current progress in AI research and believed that removing complex representation from AI would help address problematic areas in modelling the mind. His belief was that we should develop artificial intelligence by being guided by the evolutionary development of our own intelligence and that his approach mirrored how our own intelligence functions. Thus, the field of behaviour-based robotics emerged. This paper offers a historical analysis of Brooks’ behaviour-based robotics approach and its impact on artificial intelligence and cognitive theory at the time, as well as on modern-day approaches to AI.

Generalized behavior is a long standing goal for evolutionary robotics. Behaviors for a given task should be robust to perturbation and capable of operating across a variety of environments. We have previously shown that Lexicase selection evolves high-performing individuals in a semi-generalized wall crossing task–i.e., where the task is broadly the same, but there is variation between individual instances. Further work has identified effective parameter values for Lexicase selection in this domain but other factors affecting and explaining performance remain to be identified. In this paper, we expand our prior investigations, examining populations over evolutionary time exploring other factors that might lead to generalized behavior. Results show that genomic clusters do not correspond to performance, indicating that clusters of specialists do not form within the population. While early individuals gain a foothold in the selection process by specializing on a few wall heights, successful populations are ultimately pressured towards generalized behavior. Finally, we find that this transition from specialists to generalists also leads to an increase in tiebreaks, a mechanism in Lexicase, during selection providing a metric to assess the performance of individual replicates.

Few evolved robots have been tested outside simulation due to real world experiments being resource and time expensive. In this paper, we discuss different approaches to physically implement evolved robots and propose a modular robot system with external automatic reconfiguration. While the morphological space is reduced, it provides us with a fast, reusable, fully autonomous system to evolve physical robots in reality.

An unsolved problem in both natural and artificial evolutionary systems is determining the exact environmental and evolutionary conditions that enable complexity to evolve [16]. This is especially pertinent in evolutionary robotics [4] where possible problem-solving behavior is constrained by brain (controller) and body (morphology) complexity [12]. We evaluate the impact of environments and complexity costs on robotic controller and morphology evolution across various evolutionary robotics task scenarios. This study uses evolutionary robotics as an experimental platform to investigate the arrow of complexity hypothesis [3], previously demonstrated to hold in artificial evolutionary systems given an imposed complexity cost [2]. Specifically, we test whether energy costs imposed on evolving robot controller and morphology complexity enables the evolution of increasingly complex controller and morphological designs concomitant with increasing environment complexity. Morphological complexity was equated with possible sensor configurations for a physical counterpart Khepera III mobile robot [8], and neural complexity was equated to artificial neural network topological configurations that coupled with a robot's evolved morphology.

International audience; The MAP-Elites quality-diversity algorithm has been successful in robotics because it can create a behaviorally diverse set of solutions that later can be used for adaptation, for instance to unanticipated damages. In MAP-Elites, the choice of the behaviour space is essential for adaptation, the recovery of performance in unseen environments , since it defines the diversity of the solutions. Current practice is to hand-code a set of behavioural features, however, given the large space of possible behaviour-performance maps, the designer does not know a priori which behavioural features maximise a map's adaptation potential. We introduce a new meta-evolution algorithm that discovers those behavioural features that maximise future adaptations. The proposed method applies Covari-ance Matrix Adaptation Evolution Strategy to evolve a population of behaviour-performance maps to maximise a meta-fitness function that rewards adaptation. The method stores solutions found by MAP-Elites in a database which allows to rapidly construct new behaviour-performance maps on-the-fly. To evaluate this system , we study the gait of the RHex robot as it adapts to a range of damages sustained on its legs. When compared to MAP-Elites with user-defined behaviour spaces, we demonstrate that the meta-evolution system learns high-performing gaits with or without damages injected to the robot.

Evolutionary computation provides methods to automatically generate controllers for swarm robotics. Many approaches rely on optimization and the targeted behavior is quantified in form of a fitness function. Other methods, like novelty search, increase exploration by putting selective pressure on unexplored behavior space using a domain-specific behavioral distance function. In contrast, minimize surprise leads to the emergence of diverse behaviors by using an intrinsic motivation as fitness, that is, high prediction accuracy. We compare a standard genetic algorithm, novelty search and minimize surprise in a swarm robotics setting to evolve diverse behaviors and show that minimize surprise is competitive to novelty search.

Voxel-based soft robots (VSRs) are aggregations of elastic, cubic blocks that have sparkled the interest of Robotics and Artificial Life researchers. VSRs can move by varying the volume of individual blocks, according to control signals dictated by a controller, possibly based on inputs coming from sensors embedded in the blocks. Neural networks (NNs) have been used as centralized processing units for those sensing controllers, with weights optimized using evolutionary computation. This structuring breaks the intrinsic modularity of VSRs: decomposing a VSR into modules to be assembled in a different way is very hard. In this work we propose an alternative approach that enables full modularity and is based on a distributed neural controller. Each block contains a small NN that outputs signals to adjacent blocks and controls the local volume, based on signals from adjacent blocks and on local sensor readings. We show experimentally for the locomotion task that our controller is as effective as the centralized one. Our experiments also suggest that the proposed framework indeed allows exploiting modularity: VSRs composed of pre-trained parts (body and controller) can be evolved more efficiently than starting from scratch.

This paper summarizes the keynote I gave on the SEAMS 2020 conference. Noting the power of natural evolution that makes living systems extremely adaptive, I describe how artificial evolution can be employed to solve design and optimization problems in software. Thereafter, I discuss the Evolution of Things, that is, the possibility of evolving physical artefacts and zoom in on a (r)evolutionary way of creating 'bodies' and 'brains' of robots for engineering and fundamental research.

The field of Evolutionary Robotics addresses the challenge of automatically designing robotic systems. Furthermore, the field can also support biological investigations related to evolution. In this paper, we evolve (simulated) modular robots under diverse environmental conditions and analyze the influences that these conditions have on the evolved morphologies, controllers, and behavior. To this end, we introduce a set of morphological, controller, and behavioral descriptors that together span a multi-dimensional trait space. Using these descriptors, we demonstrate how changes in environmental conditions induce different levels of differentiation in this trait space. Our main goal is to gain deeper insights into the effect of the environment on a robotic evolutionary process.

For the first time, a field programmable transistor array (FPTA) was used to evolve robot control circuits directly in analog hardware. Controllers were successfully incrementally evolved for a physical robot engaged in a series of visually guided behaviours, including finding a target in a complex environment where the goal was hidden from most locations. Circuits for recognising spoken commands were also evolved and these were used in conjunction with the controllers to enable voice control of the robot, triggering behavioural switching. Poor quality visual sensors were deliberately used to test the ability of evolved analog circuits to deal with noisy uncertain data in realtime. Visual features were coevolved with the controllers to automatically achieve dimensionality reduction and feature extraction and selection in an integrated way. An efficient new method was developed for simulating the robot in its visual environment. This allowed controllers to be evaluated in a simulation connected to the FPTA. The controllers then transferred seamlessly to the real world. The circuit replication issue was also addressed in experiments where circuits were evolved to be able to function correctly in multiple areas of the FPTA. A methodology was developed to analyse the evolved circuits which provided insights into their operation. Comparative experiments demonstrated the superior evolvability of the transistor array medium.