Search

The ability of robots to write Chinese strokes, which is recognized as a sophisticated task, involves complicated kinematic control algorithms. The conventional approaches for robotic writing of Chinese strokes often suffer from limited font generation methods, which limits the ability of robots to perform high-quality writing. This paper instead proposes a developmental evolutionary learning framework that enables a robot to learn to write fundamental Chinese strokes. The framework first considers the learning process of robotic writing as an evolutionary easy-to-difficult procedure. Then, a developmental learning mechanism called “Lift-constraint, act and saturate” that stems from developmental robotics is used to determine how the robot learns tasks ranging from simple to difficult by building on the learning results from the easy tasks. The developmental constraints, which include altitude adjustments, number of mutation points, and stroke trajectory points, determine the learning complexity of robot writing. The developmental algorithm divides the evolutionary procedure into three developmental learning stages. In each stage, the stroke trajectory points gradually increase, while the number of mutation points and adjustment altitudes gradually decrease, allowing the learning difficulties involved in these three stages to be categorized as easy, medium, and difficult. Our robot starts with an easy learning task and then gradually progresses to the medium and difficult tasks. Under various developmental constraint setups in each stage, the robot applies an evolutionary algorithm to handle the basic shapes of the Chinese strokes and eventually acquires the ability to write with good quality. The experimental results demonstrate that the proposed framework allows a calligraphic robot to gradually learn to write five fundamental Chinese strokes and also reveal a developmental pattern similar to that of humans. Compared to an evolutionary algorithm without the developmental mechanism, the proposed framework achieves good writing quality more rapidly.

Robotics currently lacks fully autonomous capabilities, especially where task knowledge is incomplete and optimal robotic solutions cannot be pre-engineered. The intersection of evolutionary robotics, artificial life and embodied artificial intelligence presents a promising paradigm for generating multitask problem-solvers suitable for adapting over extended periods in unexplored, remote and hazardous environments. To address the automation of evolving robotic systems, we propose fully autonomous, embodied artificial-life factories and laboratories, situated in various environments as multi-task problem-solvers. Such integrated factories and laboratories would be adaptive solution designers, producing fit-for-purpose physical robots with accelerated artificial evolution that experiment to continually discover new tasks. Such tasks would be stepping-stones towards accomplishing given mission objectives over extended periods (days to decades). Rather than being purely speculative, prerequisite technologies to realize such factories have been experimentally demonstrated. Currently, vast scientific and enterprise opportunities await in applications such as asteroid mining, terraforming, space and deep sea exploration, though no suitable solution exists. The proposed embodied artificial-life factories and laboratories, termed: AutoFac, use robot production equipment run by artificial evolution controllers to collect and synthesize environmental information (from robotic sensory systems). Such information is merged with current needs and mission objectives to create new robot embodiment and task definitions that are environmentally adapted and balance task-oriented behavior with exploration. AutoFac is thus generalist (deployable in many environments) but continually produces specialist solutions within such environments a perpetual robot machine.

The elephant in the room for evolutionary robotics is the reality gap. In the history of the field, several studies investigated this phenomenon on fixed robot morphologies where only the controllers evolved. This paper addresses the reality gap in a wider context, in a system where both morphologies and controllers evolve. In this context the morphology of the robots becomes a variable with a currently unknown influence. To examine this influence, we construct a test suite of robots with various morphologies and evolve their controllers for an effective gait. Comparing the simulated and the real-world performance of evolved controllers sampled at different generations during the evolutionary process, we gain new insights into the factors that influence the reality gap.

The challenge of robotic reproduction -- making of new robots by recombining two existing ones -- has been recently cracked and physically evolving robot systems have come within reach. Here we address the next big hurdle: producing an adequate brain for a newborn robot. In particular, we address the task of targeted locomotion which is arguably a fundamental skill in any practical implementation. We introduce a controller architecture and a generic learning method to allow a modular robot with an arbitrary shape to learn to walk towards a target and follow this target if it moves. Our approach is validated on three robots, a spider, a gecko, and their offspring, in three real-world scenarios., 12 pages

We introduce a method that permits to co-evolve the body and the control properties of robots. It can be used to adapt the morphological traits of robots with a hand-designed morphological bauplan or to evolve the morphological bauplan as well. Our results indicate that robots with co-adapted body and control traits outperform robots with fixed hand-designed morphologies. Interestingly, the advantage is not due to the selection of better morphologies but rather to the mutual scaffolding process that results from the possibility to co-adapt the morphological traits to the control traits and vice versa. Our results also demonstrate that morphological variations do not necessarily have destructive effects on robots’ skills.

In evolutionary robotics, robot controllers are often evolved in simulation, as using the physical robot for fitness evaluation can take a prohibitively long time. Simulators provide a quick way to evaluate controller fitness. A simulator is tasked with providing appropriate sensor information to the controller. If the robot has an on-board camera, an entire virtual visual environment is needed to simulate the camera’s signal. In the past, these visual environments have been constructed by hand, requiring the use of hand-crafted models, textures and lighting, which is a tedious and time-consuming process. This paper proposes a deep neural network-based architecture for simulating visual environments. The neural networks are trained exclusively from images captured from the robot, creating a 3-dimensional visual environment without using hand-crafted models, textures or lighting. It does not rely on any external domain specific datasets, as all training data is captured in the physical environment. Robot controllers were evolved in simulation to discern between objects with different colours and shapes, and they successfully completed the same task in the real world.

This paper focuses on the collective cognition by robotic swarms. The robotic swarms expected to perform tasks that are beyond the capability of a single robot by collective behavior that emerge from local interactions, similar to biological swarms. However, the robotic swarms have to rely on the collective cognition more than biological swarms when considering the limitation in sensory capabilities and the cost of each robot. In this paper, we develop controllers for a robotic swarm to accomplish a foraging task that requires collective cognition. In this task, robots have to both collectively distinguish two objects, namely food and poison, and cooperatively transport food objects to the nest. We applied an evolutionary robotics approach with the covariance matrix adaptation evolution strategy to develop controllers for robotic swarms. The results of computer simulations show that collective cognition was successfully developed, which allows the robots to transport only food objects. In addition, we also perform experiments to examine the scalability and the flexibility of the developed controllers.

Theory of mind (ToM) is the ability to understand others’ mental states (e.g., intentions). Studies on human ToM show that the way we understand others’ mental states is very efficient, in the sense that observing only some portion of others’ behaviors can lead to successful performance. Recently, ToM has gained interest in robotics to build robots that can engage in complex social interactions. Although it has been shown that robots can infer others’ internal states, there has been limited focus on the data utilization of ToM mechanisms in robots. Here we show that robots can infer others’ intentions based on limited information by selectively and flexibly using behavioral cues similar to humans. To test such data utilization, we impaired certain parts of an actor robot’s behavioral information given to the observer, and compared the observer’s performance under each impairment condition. We found that although the observer’s performance was not perfect compared to when all information was available, it could infer the actor’s mind to a degree if the goal-relevant information was intact. These results demonstrate that, similar to humans, robots can learn to infer others’ mental states with limited information.

The concept of an artificial corporeal machine that can reproduce has attracted the attention of researchers from various fields over the past century. Some have approached the topic with a desire to understand biological life and develop artificial versions; others have examined it as a potentially practical way to use material resources from the moon and Mars to bootstrap the exploration and colonization of the solar system. This review considers both bodies of literature, with an emphasis on the underlying principles required to make self-replicating robotic systems from raw materials a reality. We then illustrate these principles with machines from our laboratory and others and discuss how advances in new manufacturing processes such as 3-D printing can have a synergistic effect in advancing the development of such systems.

Not all generate-and-test search algorithms are created equal. Bayesian Optimization (BO) invests a lot of computation time to generate the candidate solution that best balances the predicted value and the uncertainty given all previous data, taking increasingly more time as the number of evaluations performed grows. Evolutionary Algorithms (EA) on the other hand rely on search heuristics that typically do not depend on all previous data and can be done in constant time. Both the BO and EA community typically assess their performance as a function of the number of evaluations. However, this is unfair once we start to compare the efficiency of these classes of algorithms, as the overhead times to generate candidate solutions are significantly different. We suggest to measure the efficiency of generate-and-test search algorithms as the expected gain in the objective value per unit of computation time spent. We observe that the preference of an algorithm to be used can change after a number of function evaluations. We therefore propose a new algorithm, a combination of Bayesian optimization and an Evolutionary Algorithm, BEA for short, that starts with BO, then transfers knowledge to an EA, and subsequently runs the EA. We compare the BEA with BO and the EA. The results show that BEA outperforms both BO and the EA in terms of time efficiency, and ultimately leads to better performance on well-known benchmark objective functions with many local optima. Moreover, we test the three algorithms on nine test cases of robot learning problems and here again we find that BEA outperforms the other algorithms., Comment: 13 pages, 10 Figures

Rapid developments in evolutionary computation, robotics, 3D-printing, and material science are enabling advanced systems of robots that can autonomously reproduce and evolve. The emerging technology of robot evolution challenges existing AI ethics because the inherent adaptivity, stochasticity, and complexity of evolutionary systems severely weaken human control and induce new types of hazards. In this paper we address the question how robot evolution can be responsibly controlled to avoid safety risks. We discuss risks related to robot multiplication, maladaptation, and domination and suggest solutions for meaningful human control. Such concerns may seem far-fetched now, however, we posit that awareness must be created before the technology becomes mature.

© Copyright © 2021 Eiben.This paper takes a critical look at the concept of real-world robot evolution discussing specific challenges for making it practicable. After a brief review of the state of the art several enablers are discussed in detail. It is noted that sample efficient evolution is one of the key prerequisites and there are various promising directions towards this in different stages of maturity, including learning as part of the evolutionary system, genotype filtering, and hybridizing real-world evolution with simulations in a new way. Furthermore, it is emphasized that an evolutionary system that works in the real world needs robots that work in the real world. Obvious as it may seem, to achieve this significant complexification of the robots and their tasks is needed compared to the current practice. Finally, the importance of not only building but also understanding evolving robot systems is emphasised, stating that in order to have the technology work we also need the science behind it.

Quality-Diversity optimization algorithms such as MAP-Elites, aim to generate collections of both diverse and high-performing solutions to an optimization problem. MAP-Elites has shown promising results in a variety of applications. In particular in evolutionary robotics tasks targeting the generation of behavioral repertoires that highlight the versatility of robots. However, for most robotics applications MAP-Elites is limited to using simple open-loop or low-dimensional controllers. Here we present Policy Gradient Assisted MAP-Elites (PGA-MAP-Elites), a novel algorithm that enables MAP-Elites to efficiently evolve large neural network controllers by introducing a gradient-based variation operator inspired by Deep Reinforcement Learning. This operator leverages gradient estimates obtained from a critic neural network to rapidly find higher-performing solutions and is paired with a traditional genetic variation to maintain a divergent search behavior. The synergy of these operators makes PGA-MAP-Elites an efficient yet powerful algorithm for finding diverse and high-performing behaviors. We evaluate our method on four different tasks for building behavioral repertoires that use deep neural network controllers. The results show that PGA-MAP-Elites significantly improves the quality of the generated repertoires compared to existing methods.

As open-ended learning based on divergent search algorithms such as Novelty Search (NS) draws more and more attention from the research community, it is natural to expect that its application to increasingly complex real-world problems will require the exploration to operate in higher dimensional Behavior Spaces which will not necessarily be Euclidean. Novelty Search traditionally relies on k-nearest neighbours search and an archive of previously visited behavior descriptors which are assumed to live in a Euclidean space. This is problematic because of a number of issues. On one hand, Euclidean distance and Nearest-neighbour search are known to behave differently and become less meaningful in high dimensional spaces. On the other hand, the archive has to be bounded since, memory considerations aside, the computational complexity of finding nearest neighbours in that archive grows linearithmically with its size. A sub-optimal bound can result in "cycling" in the behavior space, which inhibits the progress of the exploration. Furthermore, the performance of NS depends on a number of algorithmic choices and hyperparameters, such as the strategies to add or remove elements to the archive and the number of neighbours to use in k-nn search. In this paper, we discuss an alternative approach to novelty estimation, dubbed Behavior Recognition based Novelty Search (BR-NS), which does not require an archive, makes no assumption on the metrics that can be defined in the behavior space and does not rely on nearest neighbours search. We conduct experiments to gain insight into its feasibility and dynamics as well as potential advantages over archive-based NS in terms of time complexity., Author version of the paper accepted at GECCO 21

International audience; We present a behavioral diversity selection scheme that favors reproductive isolation to promote the learning of multiple task in on-line embodied evolutionary robotics (EER). The scheme estimates the behavior of the controllers without the need to access the agent experience, respecting thus the online, distributed properties EER. Reproductive isolation is assessed through coalescence trees and task specialization is tested on a concurrent foraging setting.

What makes a quality-diversity (QD) algorithm effective? This question is increasingly studied as such algorithms gain in popularity. Recent work on Novelty Search linked its efficiency to the evolution of a population under a dynamic selection pressure defined by the novelty metric, which allows the population to converge to highly evolvable individuals, on which spending the evolutionary budget allows to navigate the behavior space. MAP-Elites, another popular QD algorithm, does not use an explicit population, instead sampling the parent population directly from the behavioral archive at each generation. This sampling can be uniform, or biased according to criteria such as curiosity, which favor individuals that previously generated successful offsprings. In this article, we show that such improved selection schemes are efficient because they create a dynamic pseudo-population that mimics the desirable qualities of a Novelty Search population. We do this by proposing another simpler selection scheme, youth, that simply focuses the evolutionary budget on such a dynamic pseudo-population, and by showing it exhibits similar behavior and performance as curiosity without explicitly selecting for previously successful parents. This has important implications for the design of future QD algorithms taking the best of both archive-based and population-based methods.

Organisms with different body morphology and movement dynamics have distinct abilities to move through the environment. Despite such truism, there is a lack of general principles that predict which shapes and dynamics make the organisms more fit to move. Studying a minimal yet embodied soft robot model under the influence of gravity, we find three features that predict robot locomotion fitness: (1) A larger body is better. (2) Two-point contact with the ground is better than one-point contact. (3) Out-of-phase oscillating body parts increase locomotion fitness. These design principles can guide the selection rules for evolutionary algorithms to obtain robots with higher locomotion fitness.

Locating odour sources is a hard task that has been addressed with a large variety of AI methods to produce search strategies with different levels of efficiency and robustness. However, it is still not clear how to evaluate those strategies. Simply evaluating the robot's ability to reach the goal may produce deceptive fitness values, favouring poor strategies that do not generalise. Conversely, including prior knowledge may bias the learning process. This work studies the impact of evaluation functions with various degrees of prior knowledge, in evolving search strategies. The baseline is set by performing multiple evaluations of each strategy with a function that only evaluates the task efficiency. A function was found that is able to produce strategies with equivalent performance to those of the baseline, whilst performing a single evaluation.

Robots are still missing the ability to adapt to new environments. However, biological systems are able to adapt to new environments with ease; perhaps because they have the ability to react to environmental input during a growth phase with changes not only in behaviour, but also morphology. Yet within the field of robots, environmental based development of morphology is an under researched area. In this paper we use an evolutionary algorithm to evolve neural cellular automata capable of inducing environmental based developmental plasticity in robots. We use the kinetic energy of each cell and its neighbours as an input to our network, the output of which determines the position of new cell growth. We evolve our neural cellular automata first in three individual environments and then also for performance in multiple environments. We show that the networks that use environmental feedback outperform those that do not and that by introducing environmental feedback during development, more adaptive and better performing robots are potentially possible.

This work presents a platform for evolution of morphology in full cycle reconfigurable hardware: The EMERGE (Easy Modular Embodied Robot Generator) modular robot platform. Three parts necessary to implement a full cycle process, i.e., assembling the modules in morphologies, testing the morphologies, disassembling modules and repeating, are described as a previous step to testing a fully autonomous system: the mechanical design of the EMERGE module, extensive tests of the modules by first assembling them manually, and automatic assembly and disassembly tests. EMERGE modules are designed to be easy and fast to build, one module is built in half an hour and is constructed from off-the-shelf and 3D printed parts. Thanks to magnetic connectors, modules are quickly attached and detached to assemble and reconfigure robot morphologies. To test the performance of real EMERGE modules, 30 different morphologies are evolved in simulation, transferred to reality, and tested 10 times. Manual assembly of these morphologies is aided by a visual guiding tool that uses AprilTag markers to check the real modules positions in the morphology against their simulated counterparts and provides a color feedback. Assembly time takes under 5 min for robots with fewer than 10 modules and increases linearly with the number of modules in the morphology. Tests show that real EMERGE morphologies can reproduce the performance of their simulated counterparts, considering the reality gap. Results also show that magnetic connectors allow modules to disconnect in case of being subjected to high external torques that could damage them otherwise. Module tracking combined with their easy assembly and disassembly feature enable EMERGE modules to be also reconfigured using an external robotic manipulator. Experiments demonstrate that it is possible to attach and detach modules from a morphology, as well as release the module from the manipulator using a passive magnetic gripper. This shows that running a completely autonomous, evolution of morphology in full cycle reconfigurable hardware of different topologies for robots is possible and on the verge of being realized. We discuss EMERGE features and the trade-off between reusability and morphological variability among different approaches to physically implement evolved robots.

Evolving morphologies and controllers of robots simultaneously leads to a problem: Even if the parents have well-matching bodies and brains, the stochastic recombination can break this match and cause a body-brain mismatch in their offspring. We argue that this can be mitigated by having newborn robots perform a learning process that optimizes their inherited brain quickly after birth. We compare three different algorithms for doing this. To this end, we consider three algorithmic properties, efficiency, efficacy, and the sensitivity to differences in the morphologies of the robots that run the learning process., Comment: Associated code: https://github.com/fudavd/revolve/tree/learning

Most evolutionary robotics studies focus on evolving some targeted behavior without taking the energy usage into account. This limits the practical value of such systems because energy efficiency is an important property for real-world autonomous robots. In this paper, we mitigate this problem by extending our simulator with a battery model and taking energy consumption into account during fitness evaluations. Using this system we investigate how energy awareness affects the evolution of robots. Since our system is to evolve morphologies as well as controllers, the main research question is twofold: (i) what is the impact on the morphologies of the evolved robots, and (ii) what is the impact on the behavior of the evolved robots if energy consumption is included in the fitness evaluation? The results show that including the energy consumption in the fitness in a multi-objective fashion (by NSGA-II) reduces the average size of robot bodies while at the same time reducing their speed. However, robots generated without size reduction can achieve speeds comparable to robots from the baseline set.

Engineers, control theorists, and neuroscientists often view the delay imposed by finite signal propagation velocities as a problem that needs to be compensated for or avoided. In this article, we consider the alternative possibility that in some cases, signal delay can be used functionally, that is, as an essential component of a cognitive system. To investigate this idea, we evolve a minimal robot controller to solve a basic stimulus-distinction task. The controller is constrained so that the solution must utilize a delayed recurrent signal. Different from previous evolutionary robotics studies, our controller is modeled using delay differential equations, which (unlike the ordinary differential equations of conventional continuous-time recurrent neural networks) can accurately capture delays in signal propagation. We analyze the evolved controller and its interaction with its environment using classical dynamical systems techniques. The analysis shows what kinds of invariant sets underlie the various successful and unsuccessful performances of the robot, and what kinds of bifurcations produce these invariant sets. In the second phase of our analysis, we turn our attention to the parameter θ, which describes the amount of signal delay included in the model. We show how the delay destabilizes certain attractors that would exist if there were no delay and creates other stable attractors, resulting in an agent that performs well at the target task.

This work presents a genetic programming control design methodology that extends the traditional behavior–based control strategy towards a synthetic-analytic perspective. The proposed approach considers the internal and external dynamics of the system, providing solutions to a general structure, and including analytic functions, which can be studied within the Control Theory framework. The method is illustrated for the tracking control problem under bounded velocity restrictions of a nonholonomic wheeled mobile robot . A classic Control Theory (CT) based controller that solves the tracking problem (but not the velocity constraint requirement) is chosen from the literature; based on its stability properties , a modified structure where the search of suitable analytic basis behaviors, fulfilling both control objectives simultaneously, can be introduced. The proposed framework takes the form of a learning process based on Genetic Programming (GP) which generates a set of nonlinear tracking controllers satisfying pre-specified velocity bounds. A collection of 9113 suitable nonlinear solutions were obtained to augment the ground controller. Simulations and real–time experiments are performed to illustrate the effectiveness of the methodology through the testing of the models with the best performance, as well as those with lower structural complexity.

I present criteria or benchmarks of personhood that robotic agents must pass for them to match us in social intelligence and indicate some current successes and difficulties for the future with res...

It has often been found that students appreciate hands-on work, and find that they learn more with courses that include a project than those relying solely on conventional lectures and tests. This type of project driven learning is a key component of “Inquiry-based learning” (IBL), which aims at teaching methodology as well as content by incorporating the student as an actor rather than a spectator. Robotics applications are especially well-suited for IBL due to the value of trial and error experience, the multiple possibilities for students to implement their own ideas, and the importance of programming, problem-solving and electro-mechanical skills in real world engineering and science jobs. Furthermore, robotics platforms can be useful teaching media and learning tools for a variety of topics. Here, we present RoboGen: an open-source, web-based, software and hardware platform for Robotics and Artificial Intelligence with a particular focus on Evolutionary Robotics. We describe the platform in detail, compare it to existing alternatives, and present results of its use as a platform for Inquiry-based learning within a master's level course at the Ecole Polytechnique Fédérale de Lausanne (EPFL).

SummaryMission planning is a complex motion planning problem specified by using Temporal Logic constituting of Boolean and temporal operators, typically solved by model verification algorithms with an exponential complexity. The paper proposes co-evolutionary optimization thus building an iterative solution to the problem. The language for mission specification is generic enough to represent everyday missions, while specific enough to design heuristics. The mission is broken into components which cooperate with each other. The experiments confirm that the robot is able to outperform the search, evolutionary and model verification techniques. The results are demonstrated by using a Pioneer LX robot.

This research proposed augmenting Evolutionary Robotics controllers with their own robotic simulator. The motivation behind this idea is that the controller could use the simulator to approximate the effects of its actions. This would allow the controller to maintain an approximation of the robot’s location and orientation. It was hypothesised that this information could enhance a controller’s navigation ability. Both regular and augmented controllers were tested and compared on a task that required them to employ adaptive navigation strategies to efficiently solve the task. Augmented controllers were shown to perform better and possess superior behaviours in comparison to regular controllers.

We demonstrate the training of Spiking Neural Networks (SNN) in a novel multi-agent Evolutionary Robotics (ER) framework inspired by competitive evolutionary environments in nature. The topology of a SNN along with morphological parameters of the bot it controls in the ER environment is together treated as a phenotype. Rules of the framework select certain bots and their SNNs for reproduction and others for elimination based on their efficacy in capturing food in a competitive environment. While the bots and their SNNs are not explicitly trained to survive or reproduce using loss functions, these drives emerge implicitly as they evolve to hunt food and survive. Their efficiency in capturing food exhibits the evolutionary signature of punctuated equilibrium. We use this signature to compare the performances of two evolutionary inheritance algorithms on the phenotypes, Mutation and Crossover with Mutation, using ensembles of 100 experiments for each algorithm. We find that Crossover with Mutation promotes 40% faster learning in the SNN than mere Mutation with a statistically significant margin.

An open question for both natural and artificial evolutionary systems is how, and under what environmental and evolutionary conditions complexity evolves. This study investigates the impact of increasingly complex task environments on the evolution of robot complexity. Specifically, the impact of evolving body-brain couplings on locomotive task performance, where robot evolution was directed by either body-brain exploration (novelty search) or objective-based (fitness function) evolutionary search. Results indicated that novelty search enabled the evolution of increased robot body-brain complexity and efficacy given specific environment conditions. The key contribution is thus the demonstration that body-brain exploration is suitable for evolving robot complexity that enables high fitness robots in specific environments.

Quality-Diversity (QD) optimisation is a new family of learning algorithms that aims at generating collections of diverse and high-performing solutions. Among those algorithms, the recently introduced Covariance Matrix Adaptation MAP-Elites (CMA-ME) algorithm proposes the concept of emitters, which uses a predefined heuristic to drive the algorithm's exploration. This algorithm was shown to outperform MAP-Elites, a popular QD algorithm that has demonstrated promising results in numerous applications. In this paper, we introduce Multi-Emitter MAP-Elites (ME-MAP-Elites), an algorithm that directly extends CMA-ME and improves its quality, diversity and data efficiency. It leverages the diversity of a heterogeneous set of emitters, in which each emitter type improves the optimisation process in different ways. A bandit algorithm dynamically finds the best selection of emitters depending on the current situation. We evaluate the performance of ME-MAP-Elites on six tasks, ranging from standard optimisation problems (in 100 dimensions) to complex locomotion tasks in robotics. Our comparisons against CMA-ME and MAP-Elites show that ME-MAP-Elites is faster at providing collections of solutions that are significantly more diverse and higher performing. Moreover, in cases where no fruitful synergy can be found between the different emitters, ME-MAP-Elites is equivalent to the best of the compared algorithms.

Autonomous robots are increasingly used in remote and hazardous environments, where damage to sensory-actuator systems cannot be easily repaired. Such robots must therefore have controllers that continue to function effectively given unexpected malfunctions and damage to robot morphology. This study applies the Intelligent Trial and Error (IT&E) algorithm to adapt hexapod robot control to various leg failures and demonstrates the IT&E map-size parameter as a critical parameter in influencing IT&E adaptive task performance. We evaluate robot adaptation for multiple leg failures on two different map-sizes in simulation and validate evolved controllers on a physical hexapod robot. Results demonstrate a trade-off between adapted gait speed and adaptation duration, dependent on adaptation task complexity (leg damage incurred), where map-size is crucial for generating behavioural diversity required for adaptation.

The social brain hypothesis proposes that enlarged brains have evolved in response to the increasing cognitive demands that complex social life in larger groups places on primates and other mammals. However, this reasoning can be challenged by evidence that brain size has decreased in the evolutionary transitions from solitary to social larger groups in the case of Neolithic humans and some eusocial insects. Different hypotheses can be identified in the literature to explain this reduction in brain size. We evaluate some of them from the perspective of recent approaches to cognitive science, which support the idea that the basis of cognition can span over brain, body, and environment. Here we show through a minimal cognitive model using an evolutionary robotics methodology that the neural complexity, in terms of neural entropy and degrees of freedom of neural activity, of smaller-brained agents evolved in social interaction is comparable to the neural complexity of larger-brained agents evolved in solitary conditions. The nonlinear time series analysis of agents' neural activity reveals that the decoupled smaller neural network is intrinsically lower dimensional than the decoupled larger neural network. However, when smaller-brained agents are interacting, their actual neural complexity goes beyond its intrinsic limits achieving results comparable to those obtained by larger-brained solitary agents. This suggests that the smaller-brained agents are able to enhance their neural complexity through social interaction, thereby offsetting the reduced brain size.

5.1 Optimization using Multi-objective GA In Section4, GA was successfully applied to optimizing the humanoid motions. However, the task is essentially multi-objective optimization. For instance, in case of kicking, we have to consider the trade-off between the stability of the robot and the ball distance. Therefore, we have apply Multi-objective GA (MOGA) (Fonseca & Fleming, 1993) with two fitness, i.e., the chest-link velocity ( Vmax ) and the ball travel distance ( r ). We empirically derive the velocity when the robot falls down and set the value as the velocity limitation of the chest link ( Vlimit ). Rank-based fitness assignment method and niche-formation method are employed to determine the fitness for maximizing ( Vlimit selection is also used according to the fitness values.

Quality-Diversity algorithms search for large collections of diverse and high-performing solutions, rather than just for a single solution like typical optimisation methods. They are specially adapted for multi-modal problems that can be solved in many different ways, such as complex reinforcement learning or robotics tasks. However, these approaches are highly dependent on the choice of feature descriptors (FDs) quantifying the similarity in behaviour of the solutions. While FDs usually needs to be hand-designed, recent studies have proposed ways to define them automatically by using feature extraction techniques, such as PCA or Auto-Encoders, to learn a representation of the problem from previously explored solutions. Here, we extend these approaches to more complex problems which cannot be efficiently explored by relying only on a single representation but require instead a set of diverse and complementary representations. We describe MC-AURORA, a Quality-Diversity approach that optimises simultaneously several collections of solutions, each with a different set of FDs, which are, in turn, defined automatically by an ensemble of modular auto-encoders. We show that this approach produces solutions that are more diverse than those produced by single-representation approaches., Draft version. 10 pages, 4 figures, 4 tables, Accepted at the GECCO2021 Conference

Multi-level evolution (MLE) is a novel robotic design paradigm which decomposes the design problem into layered sub-tasks that involve concurrent search for appropriate materials, component geometry and overall morphology. This has a number of advantages, mainly in terms of quality and scalability. In this paper, we present a hierarchical approach to robotic design based on the MLE architecture. The design problem involves finding a robotic design which can be used to perform a specific locomotion task. At the materials layer, we put together a simple collection of materials which are represented by combinations of mechanical properties such as friction and restitution. At the components layer we combine these materials with geometric design to form robot limbs. Finally, at the robot layer we introduce these evolved limbs into robotic body-plans and learn control policies to form complete robots. Quality-diversity algorithms at each level allow for the discovery of a wide variety of reusable elements. The results strongly support the initial claims for the benefits of MLE, allowing for the discovery of designs that would otherwise be difficult to achieve with conventional design paradigms.

There are several ubiquitous kinematic structures that are used in industrial robots, with the most prominent being a six-axis angular structure. However, researchers are experimenting with task-based mechanism synthesis that could provide higher efficiency with custom optimized manipulators. Many studies have focused on finding the most efficient optimization algorithm for task-based robot manipulators. These manipulators, however, are usually optimized from simple modular joints and links, without exploring more elaborate modules. Here, we show that link modules defined by small numbers of parameters have better performance than more complicated ones. We compare four different manipulator link types, namely basic predefined links with fixed dimensions, straight links that can be optimized for different lengths, rounded links, and links with a curvature defined by a Hermite spline. Manipulators are then built from these modules using a genetic algorithm and are optimized for three different tasks. The results demonstrate that manipulators built from simple links not only converge faster, which is expected given the fewer optimized parameters, but also converge on lower cost values. Web of Science 11 6 art. no. 2471

Animals ranging from rats to humans can demonstrate cognitive map capabilities. We evolved weights in a biologically plausible recurrent neural network (RNN) using an evolutionary algorithm to replicate the behavior and neural activity observed in rats during a spatial and working memory task in a triple T-maze. The rat was simulated in the Webots robot simulator and used vision, distance and accelerometer sensors to navigate a virtual maze. After evolving weights from sensory inputs to the RNN, within the RNN, and from the RNN to the robot's motors, the Webots agent successfully navigated the space to reach all four reward arms with minimal repeats before time-out. Our current findings suggest that it is the RNN dynamics that are key to performance, and that performance is not dependent on any one sensory type, which suggests that neurons in the RNN are performing mixed selectivity and conjunctive coding. Moreover, the RNN activity resembles spatial information and trajectory-dependent coding observed in the hippocampus. Collectively, the evolved RNN exhibits navigation skills, spatial memory, and working memory. Our method demonstrates how the dynamic activity in evolved RNNs can capture interesting and complex cognitive behavior and may be used to create RNN controllers for robotic applications.

Taking inspiration from the navigation ability of humans, this study investigated a method of providing robotic controllers with a basic sense of position. It incorporated robotic simulators into robotic controllers to provide them with a mechanism to approximate the effects their actions had on the robot. Controllers with and without internal simulators were tested and compared. The proposed controller architecture was shown to outperform the regular controller architecture. However, the longer an internal simulator was executed, the more inaccurate it became. Thus, the performance of controllers with internal simulators reduced over time unless their internal simulator was periodically corrected.

In nature, very few animals locomote on two legs. Static bipedalism can be found in four limbed and five limbed animals like dogs, cats, birds, monkeys and kangaroos, but it cannot be seen in hexapods or other multi-limbed animals. In this paper, we present a simulation with a novel perspective on the evolution of static bipedalism, with a virtual creature evolving its body and controllers, and we apply an evolutionary algorithm to explore the locomotion transition from octapods to bipods. We find that the presence of four limbs in the evolutionary trajectory of the creature scaffolds a parametric jump that enables bipedalism, and shows that hexapods, without undergoing such transformation, struggle to evolve into bipeds. An analysis of the transitional parameters points to the role of a shorter femur length in helping maintain the stability of the body, and the tibia length is responsible for improving the forward speed.

In evolution, the evolutionary success of individuals is influenced in equal parts by the environment they are living in and by the adapting capability that they possess. In this work, instead of analyzing the adaptability of a population, we investigate how different environments influence the evolution of morphologies and controllers of modular robots. Previous work analyzed how robot evolution is influenced by different environments while keeping fixed the robot task. However, each environment may present different tasks that influence robot success. In this work, we investigate how different tasks in the same physical environment influence the evolutionary outcome. Specifically, we compare morphologies and behaviors under two tasks, undirected locomotion, and rotation in a flat terrain environment. We perform this analysis by analyzing the morphological and behavioral traits of the final population. Results suggest that both morphology and behavior types are strongly influenced by the environment, with the rotation task producing more proportionate and balanced robots compared to the locomotion task.

One key challenge in Evolutionary Robotics (ER) is to evolve morphology and controllers of robots. Most experiments in the field converge rapidly to a single solution for the entire population. Early convergence results in a premature loss of diversity, which creates inconsistent results across multiple runs, sometimes converging to a local optimum. In Nature we can observe the opposite behavior: the more time passes, the more life becomes increasingly diverse. The increasing diversity is correlated to the formation of new species, which is catalyzed by reproductive isolation caused by physical or behavioral separation. Inspired by natural evolution, in this paper we apply artificial speciation based on morphological traits to an ER system. Individuals are forced to crossover only with individuals within the same species and a protection mechanism is applied to newly created species. In our experiments, we demonstrate that this speciation mechanism, inspired by NEAT, can evolve a population rich of many coexisting individuals, differing both in morphology and behavior.

This study is motivated by evolutionary robot systems where robot bodies and brains evolve simultaneously. In such systems robot 'birth' must be followed by 'infant learning' by a learning method that works for various morphologies evolution may produce. Here we address the task of directed locomotion in modular robots with controllers based on Central Pattern Generators. We present a bio-inspired adaptive feedback mechanism that uses a forward model and an inverse model that can be learned on-the-fly. We compare two versions (a simple and a sophisticated one) of this concept to a traditional (open-loop) controller using Bayesian optimization as a learning algorithm. The experimental results show that the sophisticated version outperforms the simple one and the traditional controller. It leads to a better performance and more robust controllers that better cope with noise.

The long term goal of the Autonomous Robot Evolution (ARE) project is to create populations of physical robots, in which both the controllers and body plans are evolved. The transition of evolutionary designs from purely simulation environments into the real world creates the possibility for new types of system able to adapt to unknown and changing environments. In this paper, a system for creating robots is introduced in order to allow for their body plans to be designed algorithmically and physically instantiated using the previously introduced Robot Fabricator. This system consists of two types of components. Firstly, skeleton parts are created bespoke for each design by 3D printing, allowing the overall shape of the robot to include almost infinite variety. To allow for the shortcomings of 3D printing, the second type of component are organs which contain components such as motors and sensors, and can be attached to the skeleton to provide particular functions. Specific organ designs are presented, with discussion of the design challenges for evolutionary robotics in hardware. The Robot Fabricator is extended to allow for robots with joints, and some example body plans shown to demonstrate the diversity possible using this system of robot generation.

In many natural environments, there are different forms of living creatures that successfully accomplish the same task while being diverse in shape and behavior. This biodiversity is what made life capable of adapting to disrupting changes. Being able to reproduce biodiversity in non-biological agents, while still optimizing them for a particular task, might increase their applicability to scenarios where human response to unexpected changes is not possible. In this work, we focus on Voxel-based Soft Robots (VSRs), a form of robots that grants great freedom in the design of both body and controller and is hence promising in terms of biodiversity. We use evolutionary computation for optimizing, at the same time, body and controller of VSRs for the task of locomotion. We investigate experimentally whether two key factors - -evolutionary algorithm (EA) and representation - -impact the emergence of biodiversity and if this occurs at the expense of effectiveness. We devise a way for measuring biodiversity, systematically characterizing the robots shape and behavior, and apply it to the VSRs evolved with three EAs and two representations. The experimental results suggest that the representation matters more than the EA and that there is not a clear trade-off between diversity and effectiveness.

Artificial neural networks (ANNs) are commonly used for controlling robotic agents. For robots with many sensors and actuators, ANNs can be very complex, with many neurons and connections. Removal of neurons or connections, i.e., pruning, may be desirable because (a) it reduces the complexity of the ANN, making its operation more energy efficient, and (b) it might improve the generalization ability of the ANN. Whether these goals can actually be achieved in practice is however still not well known. On the other hand, it is widely recognized that pruning in biological neural networks plays a fundamental role in the development of brains and their ability to learn. In this work, we consider the case of Voxel-based Soft Robots, a kind of robots where sensors and actuators are distributed over the body and that can be controlled with ANNs optimized by means of neuroevolution. We experimentally characterize the effect of different forms of pruning on the effectiveness of neuroevolution, also in terms of generalization ability of the evolved ANNs. We find that, with some forms of pruning, a large portion of the connections can be pruned without strongly affecting robot capabilities. We also observe sporadic improvements in generalization ability.

Evolutionary Algorithms have been applied in robotics over the last quarter of a century to simultaneously evolve robot body morphology and controller. However, the literature shows that in this area, one is still unable to generate robots that perform better than conventional manual designs, even for simple tasks. It is noted that the main hindrance to satisfactory evolution is poor controller generated as a result of the simultaneous variation of the morphology and controller. As the controller is a result of pure evolution, it is not given a chance to improve before fitness is calculated, which is the equivalent of reproducing immediately after birth in biological evolution. Therefore, to improve co-evolution and to bring artificial robot evolution a step closer to biological evolution, this paper introduces Reinforced Co-evolution Algorithm (ReCoAl), which is a hybrid of an Evolutionary and a Reinforcement Learning algorithm. It combines the evolutionary and learning processes found in nature to co-evolve robot morphology and controller. ReCoAl works by allowing a direct policy gradient based RL algorithm to improve the controller of an evolved robot to better utilise the available morphological resources before fitness evaluation. The ReCoAl is tested for evolving mobile robots to perform navigation and obstacle avoidance. The findings indicate that the controller learning process has both positive and negative effects on the progress of evolution, similar to observations in evolutionary biology. It is also shown how, depending on the effectiveness of the learning algorithm, the evolver generates robots with similar fitness in different generations.

Designing robots by hand can be costly and time consuming, especially if the robots have to be created with novel materials, or be robust to internal or external changes. In order to create robots automatically, without the need for human intervention, it is necessary to optimise both the behaviour and the body design of the robot. However, when co-optimising the morphology and controller of a locomoting agent the morphology tends to converge prematurely, reaching a local optimum. Approaches such as explicit protection of morphological innovation have been used to reduce this problem, but it might also be possible to increase exploration of morphologies using a more indirect approach. We explore how changing the environment, where the agent locomotes, affects the convergence of morphologies. The agents’ morphologies and controllers are co-optimised, while the environments the agents locomote in are evolved open-endedly with the Paired Open-Ended Trailblazer (POET). We compare the diversity, fitness and robustness of agents evolving in environments generated by POET to agents evolved in handcrafted curricula of environments. Our agents each contain of a population of individuals being evolved with a genetic algorithm. This population is called the agent-population. We show that agent-populations evolving in open-endedly evolving environments exhibit larger morphological diversity than agent-populations evolving in hand crafted curricula of environments. POET proved capable of creating a curriculum of environments which encouraged both diversity and quality in the populations. This suggests that POET may be capable of reducing premature convergence in co-optimisation of morphology and controllers.

Evolution has been a topic of interest that has been explored extensively throughout recent history. Ever since the proposal of the evolutionary theory by Darwin, there have been attempts made to validate, extend, explore and exploit the different aspects of this theory. This study sets forth methods to explore elements of what has been proposed by Darwinian evolution using Braitenberg vehicles, simple machines with sensory-motor couplings in an environmental context. In this study, a simulation environment is set up based upon the descriptions of neurophysiologist Valentino Braitenberg. This simulation environment is then utilized to carry out the thought experiments envisioned by Braitenberg for the simplest nontrivial kinds of vehicles. The methodology and results for the carried out experiments are detailed in this paper. Apart from an understanding of whether the environmental setup affects evolution, specifically the number of light sources (stimuli) and location of these light sources, the experiments show an interesting trend regarding dynamic equilibrium of the evolutionary process, the ramifications of which might not have been understood well enough previously. It is concluded that ecological setup, as well as the initial genetic makeup of the vehicles, play a crucial role in the evolution of vehicles in scenarios laid out in this study. Further, the placement of stimuli (location of lights) and the number of the stimuli have a visible effect on the survivability of vehicle types (species).

In evolutionary robotics, we evaluate individuals by placing them in an initial configuration in the environment, and then measure their fitness over a period of time. The choice of initial configuration has a direct impact on the fitness of an individual and thereby also the overarching evolutionary process. In this paper, we propose the concept of dynamic initial configurations, which is an initial configuration that is neither random nor fixed, but develops dynamically in response to the evolutionary process. As an example we have implemented a competitive co-evolutionary algorithm where initial configurations and controllers are evolved together to solve an obstacle avoidance task of a mobile robot. We show that, while a evolutionary approach taken from literature consistently fails, the co-evolutionary approach succeeds in 22 out of 25runs. This example demonstrates the benefit of dynamic initial configurations, but more work is needed to establish if the concept generalizes to more complex tasks, environments and morphologies.