Search

The ability of robots to write Chinese strokes, which is recognized as a sophisticated task, involves complicated kinematic control algorithms. The conventional approaches for robotic writing of Chinese strokes often suffer from limited font generation methods, which limits the ability of robots to perform high-quality writing. This paper instead proposes a developmental evolutionary learning framework that enables a robot to learn to write fundamental Chinese strokes. The framework first considers the learning process of robotic writing as an evolutionary easy-to-difficult procedure. Then, a developmental learning mechanism called “Lift-constraint, act and saturate” that stems from developmental robotics is used to determine how the robot learns tasks ranging from simple to difficult by building on the learning results from the easy tasks. The developmental constraints, which include altitude adjustments, number of mutation points, and stroke trajectory points, determine the learning complexity of robot writing. The developmental algorithm divides the evolutionary procedure into three developmental learning stages. In each stage, the stroke trajectory points gradually increase, while the number of mutation points and adjustment altitudes gradually decrease, allowing the learning difficulties involved in these three stages to be categorized as easy, medium, and difficult. Our robot starts with an easy learning task and then gradually progresses to the medium and difficult tasks. Under various developmental constraint setups in each stage, the robot applies an evolutionary algorithm to handle the basic shapes of the Chinese strokes and eventually acquires the ability to write with good quality. The experimental results demonstrate that the proposed framework allows a calligraphic robot to gradually learn to write five fundamental Chinese strokes and also reveal a developmental pattern similar to that of humans. Compared to an evolutionary algorithm without the developmental mechanism, the proposed framework achieves good writing quality more rapidly.

Robotics currently lacks fully autonomous capabilities, especially where task knowledge is incomplete and optimal robotic solutions cannot be pre-engineered. The intersection of evolutionary robotics, artificial life and embodied artificial intelligence presents a promising paradigm for generating multitask problem-solvers suitable for adapting over extended periods in unexplored, remote and hazardous environments. To address the automation of evolving robotic systems, we propose fully autonomous, embodied artificial-life factories and laboratories, situated in various environments as multi-task problem-solvers. Such integrated factories and laboratories would be adaptive solution designers, producing fit-for-purpose physical robots with accelerated artificial evolution that experiment to continually discover new tasks. Such tasks would be stepping-stones towards accomplishing given mission objectives over extended periods (days to decades). Rather than being purely speculative, prerequisite technologies to realize such factories have been experimentally demonstrated. Currently, vast scientific and enterprise opportunities await in applications such as asteroid mining, terraforming, space and deep sea exploration, though no suitable solution exists. The proposed embodied artificial-life factories and laboratories, termed: AutoFac, use robot production equipment run by artificial evolution controllers to collect and synthesize environmental information (from robotic sensory systems). Such information is merged with current needs and mission objectives to create new robot embodiment and task definitions that are environmentally adapted and balance task-oriented behavior with exploration. AutoFac is thus generalist (deployable in many environments) but continually produces specialist solutions within such environments a perpetual robot machine.

The elephant in the room for evolutionary robotics is the reality gap. In the history of the field, several studies investigated this phenomenon on fixed robot morphologies where only the controllers evolved. This paper addresses the reality gap in a wider context, in a system where both morphologies and controllers evolve. In this context the morphology of the robots becomes a variable with a currently unknown influence. To examine this influence, we construct a test suite of robots with various morphologies and evolve their controllers for an effective gait. Comparing the simulated and the real-world performance of evolved controllers sampled at different generations during the evolutionary process, we gain new insights into the factors that influence the reality gap.

The challenge of robotic reproduction -- making of new robots by recombining two existing ones -- has been recently cracked and physically evolving robot systems have come within reach. Here we address the next big hurdle: producing an adequate brain for a newborn robot. In particular, we address the task of targeted locomotion which is arguably a fundamental skill in any practical implementation. We introduce a controller architecture and a generic learning method to allow a modular robot with an arbitrary shape to learn to walk towards a target and follow this target if it moves. Our approach is validated on three robots, a spider, a gecko, and their offspring, in three real-world scenarios., 12 pages

We introduce a method that permits to co-evolve the body and the control properties of robots. It can be used to adapt the morphological traits of robots with a hand-designed morphological bauplan or to evolve the morphological bauplan as well. Our results indicate that robots with co-adapted body and control traits outperform robots with fixed hand-designed morphologies. Interestingly, the advantage is not due to the selection of better morphologies but rather to the mutual scaffolding process that results from the possibility to co-adapt the morphological traits to the control traits and vice versa. Our results also demonstrate that morphological variations do not necessarily have destructive effects on robots’ skills.

In evolutionary robotics, robot controllers are often evolved in simulation, as using the physical robot for fitness evaluation can take a prohibitively long time. Simulators provide a quick way to evaluate controller fitness. A simulator is tasked with providing appropriate sensor information to the controller. If the robot has an on-board camera, an entire virtual visual environment is needed to simulate the camera’s signal. In the past, these visual environments have been constructed by hand, requiring the use of hand-crafted models, textures and lighting, which is a tedious and time-consuming process. This paper proposes a deep neural network-based architecture for simulating visual environments. The neural networks are trained exclusively from images captured from the robot, creating a 3-dimensional visual environment without using hand-crafted models, textures or lighting. It does not rely on any external domain specific datasets, as all training data is captured in the physical environment. Robot controllers were evolved in simulation to discern between objects with different colours and shapes, and they successfully completed the same task in the real world.

This paper focuses on the collective cognition by robotic swarms. The robotic swarms expected to perform tasks that are beyond the capability of a single robot by collective behavior that emerge from local interactions, similar to biological swarms. However, the robotic swarms have to rely on the collective cognition more than biological swarms when considering the limitation in sensory capabilities and the cost of each robot. In this paper, we develop controllers for a robotic swarm to accomplish a foraging task that requires collective cognition. In this task, robots have to both collectively distinguish two objects, namely food and poison, and cooperatively transport food objects to the nest. We applied an evolutionary robotics approach with the covariance matrix adaptation evolution strategy to develop controllers for robotic swarms. The results of computer simulations show that collective cognition was successfully developed, which allows the robots to transport only food objects. In addition, we also perform experiments to examine the scalability and the flexibility of the developed controllers.

Theory of mind (ToM) is the ability to understand others’ mental states (e.g., intentions). Studies on human ToM show that the way we understand others’ mental states is very efficient, in the sense that observing only some portion of others’ behaviors can lead to successful performance. Recently, ToM has gained interest in robotics to build robots that can engage in complex social interactions. Although it has been shown that robots can infer others’ internal states, there has been limited focus on the data utilization of ToM mechanisms in robots. Here we show that robots can infer others’ intentions based on limited information by selectively and flexibly using behavioral cues similar to humans. To test such data utilization, we impaired certain parts of an actor robot’s behavioral information given to the observer, and compared the observer’s performance under each impairment condition. We found that although the observer’s performance was not perfect compared to when all information was available, it could infer the actor’s mind to a degree if the goal-relevant information was intact. These results demonstrate that, similar to humans, robots can learn to infer others’ mental states with limited information.

To fully understand the evolution of complex morphologies, analyses cannot stop at selection: It is essential to investigate the roles and interactions of multiple processes that drive evolutionary outcomes. The challenges of undertaking such analyses have affected both evolutionary biologists and evolutionary roboticists, with their common interests in complex morphologies. In this paper, we present analytical techniques from evolutionary biology, selection gradient analysis and morphospace walks, and we demonstrate their applicability to robot morphologies in analyses of three evolutionary mechanisms: randomness (genetic mutation), development (an explicitly implemented genotype-to-phenotype map), and selection. In particular, we applied these analytical techniques to evolved populations of simulated biorobots—embodied robots designed specifically as models of biological systems, for the testing of biological hypotheses—and we present a variety of results, including analyses that do all of the following: illuminate different evolutionary dynamics for different classes of morphological traits; illustrate how the traits targeted by selection can vary based on the likelihood of random genetic mutation; demonstrate that selection on two selected sets of morphological traits only partially explains the variance in fitness in our biorobots; and suggest that biases in developmental processes could partially explain evolutionary dynamics of morphology. When combined, the complementary analytical approaches discussed in this paper can enable insight into evolutionary processes beyond selection and thereby deepen our understanding of the evolution of robotic morphologies.

According to Hebbian theory, synaptic plasticity is the ability of neurons to strengthen or weaken the synapses among them in response to stimuli. It plays a fundamental role in the processes of learning and memory of biological neural networks. With plasticity, biological agents can adapt on multiple timescales and outclass artificial agents, the majority of which still rely on static Artificial Neural Network (ANN) controllers. In this work, we focus on Voxel-based Soft Robots (VSRs), a class of simulated artificial agents, composed as aggregations of elastic cubic blocks. We propose a Hebbian ANN controller where every synapse is associated with a Hebbian rule that controls the way the weight is adapted during the VSR lifetime. For a given task and morphology, we optimize the controller for the task of locomotion by evolving, rather than the weights, the parameters of the Hebbian rules. Our results show that the Hebbian controller is comparable, often better than a non-Hebbian baseline and that it is more adaptable to unforeseen damages. We also provide novel insights into the inner workings of plasticity and demonstrate that "true" learning does take place, as the evolved controllers improve over the lifetime and generalize well.

The framework of Simulation-to-real learning, i.e, learning policies in simulation and transferring those policies to the real world is one of the most promising approaches towards data-efficient learning in robotics. However, due to the inevitable reality gap between the simulation and the real world, a policy learned in the simulation may not always generate a safe behaviour on the real robot. As a result, during adaptation of the policy in the real world, the robot may damage itself or cause harm to its surroundings. In this work, we introduce a novel learning algorithm called SafeAPT that leverages a diverse repertoire of policies evolved in the simulation and transfers the most promising safe policy to the real robot through episodic interaction. To achieve this, SafeAPT iteratively learns a probabilistic reward model as well as a safety model using real-world observations combined with simulated experiences as priors. Then, it performs Bayesian optimization on the repertoire with the reward model while maintaining the specified safety constraint using the safety model. SafeAPT allows a robot to adapt to a wide range of goals safely with the same repertoire of policies evolved in the simulation. We compare SafeAPT with several baselines, both in simulated and real robotic experiments and show that SafeAPT finds high-performance policies within a few minutes in the real world while minimizing safety violations during the interactions., Comment: Under review. For video of the paper http://tiny.cc/safeAPT

Artificial neural networks (ANNs) can be employed as controllers for robotic agents. Their structure is often complex, with many neurons and connections, especially when the robots have many sensors and actuators distributed across their bodies and/or when high expressive power is desirable. Pruning (removing neurons or connections) reduces the complexity of the ANN, thus increasing its energy efficiency, and has been reported to improve the generalization capability, in some cases. In addition, it is well-known that pruning in biological neural networks plays a fundamental role in the development of brains and their ability to learn. In this study, we consider the evolutionary optimization of neural controllers for the case study of Voxel-based soft robots, a kind of modular, bio-inspired soft robots, applying pruning during fitness evaluation. For a locomotion task, and for centralized as well as distributed controllers, we experimentally characterize the effect of different forms of pruning on after-pruning effectiveness, life-long effectiveness, adaptability to new terrains, and behavior. We find that incorporating some forms of pruning in neuroevolution leads to almost equally effective controllers as those evolved without pruning, with the benefit of higher robustness to pruning. We also observe occasional improvements in generalization ability.

Rapid developments in evolutionary computation, robotics, 3D-printing, and material science are enabling advanced systems of robots that can autonomously reproduce and evolve. The emerging technology of robot evolution challenges existing AI ethics because the inherent adaptivity, stochasticity, and complexity of evolutionary systems severely weaken human control and induce new types of hazards. In this paper we address the question how robot evolution can be responsibly controlled to avoid safety risks. We discuss risks related to robot multiplication, maladaptation, and domination and suggest solutions for meaningful human control. Such concerns may seem far-fetched now, however, we posit that awareness must be created before the technology becomes mature.

Simultaneously evolving morphologies (bodies) and controllers (brains) of robots can cause a mismatch between the inherited body and brain in the offspring. To mitigate this problem, the addition of an infant learning period by the so-called Triangle of Life framework has been proposed relatively long ago. However, an empirical assessment is still lacking to-date. In this paper we investigate the effects of such a learning mechanism from different perspectives. Using extensive simulations we show that learning can greatly increase task performance and reduce the number of generations required to reach a certain fitness level compared to the purely evolutionary approach. Furthermore, although learning only directly affects the controllers, we demonstrate that the evolved morphologies will be also different. This provides a quantitative demonstration that changes in the brain can induce changes in the body. Finally, we examine the concept of morphological intelligence quantified by the ability of a given body to learn. We observe that the learning delta, the performance difference between the inherited and the learned brain, is growing throughout the evolutionary process. This shows that evolution is producing robots with an increasing plasticity, that is, consecutive generations are becoming better and better learners which in turn makes them better and better at the given task. All in all, our results demonstrate that the Triangle of Life is not only a concept of theoretical interest, but a system architecture with practical benefits., Comment: Frontiers in Robotics and AI. arXiv admin note: text overlap with arXiv:2107.08249

© Copyright © 2021 Eiben.This paper takes a critical look at the concept of real-world robot evolution discussing specific challenges for making it practicable. After a brief review of the state of the art several enablers are discussed in detail. It is noted that sample efficient evolution is one of the key prerequisites and there are various promising directions towards this in different stages of maturity, including learning as part of the evolutionary system, genotype filtering, and hybridizing real-world evolution with simulations in a new way. Furthermore, it is emphasized that an evolutionary system that works in the real world needs robots that work in the real world. Obvious as it may seem, to achieve this significant complexification of the robots and their tasks is needed compared to the current practice. Finally, the importance of not only building but also understanding evolving robot systems is emphasised, stating that in order to have the technology work we also need the science behind it.

Quality-Diversity optimization algorithms such as MAP-Elites, aim to generate collections of both diverse and high-performing solutions to an optimization problem. MAP-Elites has shown promising results in a variety of applications. In particular in evolutionary robotics tasks targeting the generation of behavioral repertoires that highlight the versatility of robots. However, for most robotics applications MAP-Elites is limited to using simple open-loop or low-dimensional controllers. Here we present Policy Gradient Assisted MAP-Elites (PGA-MAP-Elites), a novel algorithm that enables MAP-Elites to efficiently evolve large neural network controllers by introducing a gradient-based variation operator inspired by Deep Reinforcement Learning. This operator leverages gradient estimates obtained from a critic neural network to rapidly find higher-performing solutions and is paired with a traditional genetic variation to maintain a divergent search behavior. The synergy of these operators makes PGA-MAP-Elites an efficient yet powerful algorithm for finding diverse and high-performing behaviors. We evaluate our method on four different tasks for building behavioral repertoires that use deep neural network controllers. The results show that PGA-MAP-Elites significantly improves the quality of the generated repertoires compared to existing methods.

International audience; We present a behavioral diversity selection scheme that favors reproductive isolation to promote the learning of multiple task in on-line embodied evolutionary robotics (EER). The scheme estimates the behavior of the controllers without the need to access the agent experience, respecting thus the online, distributed properties EER. Reproductive isolation is assessed through coalescence trees and task specialization is tested on a concurrent foraging setting.

What makes a quality-diversity (QD) algorithm effective? This question is increasingly studied as such algorithms gain in popularity. Recent work on Novelty Search linked its efficiency to the evolution of a population under a dynamic selection pressure defined by the novelty metric, which allows the population to converge to highly evolvable individuals, on which spending the evolutionary budget allows to navigate the behavior space. MAP-Elites, another popular QD algorithm, does not use an explicit population, instead sampling the parent population directly from the behavioral archive at each generation. This sampling can be uniform, or biased according to criteria such as curiosity, which favor individuals that previously generated successful offsprings. In this article, we show that such improved selection schemes are efficient because they create a dynamic pseudo-population that mimics the desirable qualities of a Novelty Search population. We do this by proposing another simpler selection scheme, youth, that simply focuses the evolutionary budget on such a dynamic pseudo-population, and by showing it exhibits similar behavior and performance as curiosity without explicitly selecting for previously successful parents. This has important implications for the design of future QD algorithms taking the best of both archive-based and population-based methods.

Organisms with different body morphology and movement dynamics have distinct abilities to move through the environment. Despite such truism, there is a lack of general principles that predict which shapes and dynamics make the organisms more fit to move. Studying a minimal yet embodied soft robot model under the influence of gravity, we find three features that predict robot locomotion fitness: (1) A larger body is better. (2) Two-point contact with the ground is better than one-point contact. (3) Out-of-phase oscillating body parts increase locomotion fitness. These design principles can guide the selection rules for evolutionary algorithms to obtain robots with higher locomotion fitness.

Locating odour sources is a hard task that has been addressed with a large variety of AI methods to produce search strategies with different levels of efficiency and robustness. However, it is still not clear how to evaluate those strategies. Simply evaluating the robot's ability to reach the goal may produce deceptive fitness values, favouring poor strategies that do not generalise. Conversely, including prior knowledge may bias the learning process. This work studies the impact of evaluation functions with various degrees of prior knowledge, in evolving search strategies. The baseline is set by performing multiple evaluations of each strategy with a function that only evaluates the task efficiency. A function was found that is able to produce strategies with equivalent performance to those of the baseline, whilst performing a single evaluation.

This work presents a platform for evolution of morphology in full cycle reconfigurable hardware: The EMERGE (Easy Modular Embodied Robot Generator) modular robot platform. Three parts necessary to implement a full cycle process, i.e., assembling the modules in morphologies, testing the morphologies, disassembling modules and repeating, are described as a previous step to testing a fully autonomous system: the mechanical design of the EMERGE module, extensive tests of the modules by first assembling them manually, and automatic assembly and disassembly tests. EMERGE modules are designed to be easy and fast to build, one module is built in half an hour and is constructed from off-the-shelf and 3D printed parts. Thanks to magnetic connectors, modules are quickly attached and detached to assemble and reconfigure robot morphologies. To test the performance of real EMERGE modules, 30 different morphologies are evolved in simulation, transferred to reality, and tested 10 times. Manual assembly of these morphologies is aided by a visual guiding tool that uses AprilTag markers to check the real modules positions in the morphology against their simulated counterparts and provides a color feedback. Assembly time takes under 5 min for robots with fewer than 10 modules and increases linearly with the number of modules in the morphology. Tests show that real EMERGE morphologies can reproduce the performance of their simulated counterparts, considering the reality gap. Results also show that magnetic connectors allow modules to disconnect in case of being subjected to high external torques that could damage them otherwise. Module tracking combined with their easy assembly and disassembly feature enable EMERGE modules to be also reconfigured using an external robotic manipulator. Experiments demonstrate that it is possible to attach and detach modules from a morphology, as well as release the module from the manipulator using a passive magnetic gripper. This shows that running a completely autonomous, evolution of morphology in full cycle reconfigurable hardware of different topologies for robots is possible and on the verge of being realized. We discuss EMERGE features and the trade-off between reusability and morphological variability among different approaches to physically implement evolved robots.

Evolving morphologies and controllers of robots simultaneously leads to a problem: Even if the parents have well-matching bodies and brains, the stochastic recombination can break this match and cause a body-brain mismatch in their offspring. We argue that this can be mitigated by having newborn robots perform a learning process that optimizes their inherited brain quickly after birth. We compare three different algorithms for doing this. To this end, we consider three algorithmic properties, efficiency, efficacy, and the sensitivity to differences in the morphologies of the robots that run the learning process., Comment: Associated code: https://github.com/fudavd/revolve/tree/learning

Engineers, control theorists, and neuroscientists often view the delay imposed by finite signal propagation velocities as a problem that needs to be compensated for or avoided. In this article, we consider the alternative possibility that in some cases, signal delay can be used functionally, that is, as an essential component of a cognitive system. To investigate this idea, we evolve a minimal robot controller to solve a basic stimulus-distinction task. The controller is constrained so that the solution must utilize a delayed recurrent signal. Different from previous evolutionary robotics studies, our controller is modeled using delay differential equations, which (unlike the ordinary differential equations of conventional continuous-time recurrent neural networks) can accurately capture delays in signal propagation. We analyze the evolved controller and its interaction with its environment using classical dynamical systems techniques. The analysis shows what kinds of invariant sets underlie the various successful and unsuccessful performances of the robot, and what kinds of bifurcations produce these invariant sets. In the second phase of our analysis, we turn our attention to the parameter θ, which describes the amount of signal delay included in the model. We show how the delay destabilizes certain attractors that would exist if there were no delay and creates other stable attractors, resulting in an agent that performs well at the target task.

This work presents a genetic programming control design methodology that extends the traditional behavior–based control strategy towards a synthetic-analytic perspective. The proposed approach considers the internal and external dynamics of the system, providing solutions to a general structure, and including analytic functions, which can be studied within the Control Theory framework. The method is illustrated for the tracking control problem under bounded velocity restrictions of a nonholonomic wheeled mobile robot . A classic Control Theory (CT) based controller that solves the tracking problem (but not the velocity constraint requirement) is chosen from the literature; based on its stability properties , a modified structure where the search of suitable analytic basis behaviors, fulfilling both control objectives simultaneously, can be introduced. The proposed framework takes the form of a learning process based on Genetic Programming (GP) which generates a set of nonlinear tracking controllers satisfying pre-specified velocity bounds. A collection of 9113 suitable nonlinear solutions were obtained to augment the ground controller. Simulations and real–time experiments are performed to illustrate the effectiveness of the methodology through the testing of the models with the best performance, as well as those with lower structural complexity.

I present criteria or benchmarks of personhood that robotic agents must pass for them to match us in social intelligence and indicate some current successes and difficulties for the future with res...

It has often been found that students appreciate hands-on work, and find that they learn more with courses that include a project than those relying solely on conventional lectures and tests. This type of project driven learning is a key component of “Inquiry-based learning” (IBL), which aims at teaching methodology as well as content by incorporating the student as an actor rather than a spectator. Robotics applications are especially well-suited for IBL due to the value of trial and error experience, the multiple possibilities for students to implement their own ideas, and the importance of programming, problem-solving and electro-mechanical skills in real world engineering and science jobs. Furthermore, robotics platforms can be useful teaching media and learning tools for a variety of topics. Here, we present RoboGen: an open-source, web-based, software and hardware platform for Robotics and Artificial Intelligence with a particular focus on Evolutionary Robotics. We describe the platform in detail, compare it to existing alternatives, and present results of its use as a platform for Inquiry-based learning within a master's level course at the Ecole Polytechnique Fédérale de Lausanne (EPFL).

SummaryMission planning is a complex motion planning problem specified by using Temporal Logic constituting of Boolean and temporal operators, typically solved by model verification algorithms with an exponential complexity. The paper proposes co-evolutionary optimization thus building an iterative solution to the problem. The language for mission specification is generic enough to represent everyday missions, while specific enough to design heuristics. The mission is broken into components which cooperate with each other. The experiments confirm that the robot is able to outperform the search, evolutionary and model verification techniques. The results are demonstrated by using a Pioneer LX robot.

In swimming virtual creatures, there is often a disparity between the level of detail in simulating a swimmer’s body and that of the fluid it moves in. In order to address this disparity, we have developed a new approach to modelling swimming virtual creatures using pseudo-soft bodies and particle-based fluids, which has sufficient realism to investigate a larger range of body-environment interactions than are usually included. As this comes with increased computational costs, which may be severe, we have also developed a means of reducing the volume of fluid that must be simulated.

The Evolutionary Robotics (ER) process has been applied extensively to developing control programs to achieve locomotion in legged robots, as an automated alternative to the arduous task of manually creating control programs for such robots. The evolution of such controllers is typically performed in simulation by making use of a physics engine-based robotic simulator. Making use of such physics-based simulators does, however, have certain challenges associated with it, such as these simulators’ computational inefficiency, potential issues with lack of accuracy and the human effort required to construct such simulators. The current study therefore proposed and investigated an alternative method of simulation for a hexapod (six-legged) robot in the ER process, and directly compared this newly-proposed simulation method to traditional physics-based simulation. This alternative robotic simulator was built based solely on experimental data acquired directly from observing the behaviour of the robot. This data was used to construct a simulator for the robot based on Artificial Neural Networks (ANNs). To compare this novel simulation method to traditional physics simulation, the ANN-based simulators were used to evolve simple open-loop locomotion controllers for the robot in simulation. The real-world performance of these controllers was compared to that of controllers evolved in a more traditional physics-based simulator. The obtained results indicated that the use of ANN-based simulators produced controllers which could successfully perform the required locomotion task on the real-world robot. In addition, the controllers evolved using the ANN-based simulators allowed the real-world robot to move further than those evolved in the physics-based simulator and the ANN-based simulators were vastly more computationally efficient than the physics-based simulator. This study thus decisively indicated that ANN-based simulators offer a superior alternative to widely-used physics simulators in ER for the locomotion task considered.

Evolutionary robotics is concerned with optimizing autonomous robots for one or more specific tasks. Remarkably, the energy needed to operate autonomously is hardly ever considered. This is quite striking because energy consumption is a crucial factor in real-world applications and ignoring this aspect can increase the reality gap. In this paper, we aim to mitigate this problem by extending our robot simulator framework with a model of a battery module and studying its effect on robot evolution. The key idea is to include energy efficiency in the definition of fitness. The robots will need to evolve to achieve high gait speed and low energy consumption. Since our system evolves the robots’ morphologies as well as their controllers, we investigate the effect of the energy extension on the morphologies and on the behavior of the evolved robots. The results show that by including the energy consumption, the evolution is not only able to achieve higher task performance (robot speed), but it reaches good performance faster. Inspecting the evolved robots and their behaviors discloses that these improvements are not only caused by better morphologies, but also by better settings of the robots’ controller parameters.

We demonstrate the training of Spiking Neural Networks (SNN) in a novel multi-agent Evolutionary Robotics (ER) framework inspired by competitive evolutionary environments in nature. The topology of a SNN along with morphological parameters of the bot it controls in the ER environment is together treated as a phenotype. Rules of the framework select certain bots and their SNNs for reproduction and others for elimination based on their efficacy in capturing food in a competitive environment. While the bots and their SNNs are not explicitly trained to survive or reproduce using loss functions, these drives emerge implicitly as they evolve to hunt food and survive. Their efficiency in capturing food exhibits the evolutionary signature of punctuated equilibrium. We use this signature to compare the performances of two evolutionary inheritance algorithms on the phenotypes, Mutation and Crossover with Mutation, using ensembles of 100 experiments for each algorithm. We find that Crossover with Mutation promotes 40% faster learning in the SNN than mere Mutation with a statistically significant margin.

An open question for both natural and artificial evolutionary systems is how, and under what environmental and evolutionary conditions complexity evolves. This study investigates the impact of increasingly complex task environments on the evolution of robot complexity. Specifically, the impact of evolving body-brain couplings on locomotive task performance, where robot evolution was directed by either body-brain exploration (novelty search) or objective-based (fitness function) evolutionary search. Results indicated that novelty search enabled the evolution of increased robot body-brain complexity and efficacy given specific environment conditions. The key contribution is thus the demonstration that body-brain exploration is suitable for evolving robot complexity that enables high fitness robots in specific environments.

The social brain hypothesis proposes that enlarged brains have evolved in response to the increasing cognitive demands that complex social life in larger groups places on primates and other mammals. However, this reasoning can be challenged by evidence that brain size has decreased in the evolutionary transitions from solitary to social larger groups in the case of Neolithic humans and some eusocial insects. Different hypotheses can be identified in the literature to explain this reduction in brain size. We evaluate some of them from the perspective of recent approaches to cognitive science, which support the idea that the basis of cognition can span over brain, body, and environment. Here we show through a minimal cognitive model using an evolutionary robotics methodology that the neural complexity, in terms of neural entropy and degrees of freedom of neural activity, of smaller-brained agents evolved in social interaction is comparable to the neural complexity of larger-brained agents evolved in solitary conditions. The nonlinear time series analysis of agents' neural activity reveals that the decoupled smaller neural network is intrinsically lower dimensional than the decoupled larger neural network. However, when smaller-brained agents are interacting, their actual neural complexity goes beyond its intrinsic limits achieving results comparable to those obtained by larger-brained solitary agents. This suggests that the smaller-brained agents are able to enhance their neural complexity through social interaction, thereby offsetting the reduced brain size.

One of the most active fields of research in evolutionary robotics is the development of autonomous robots with the ability to interact with the physical world and to communicate with each other, in “robot societies”. Interactions may involve a range of different motor actions, motivational forces and cognitive processes. Actions, in turn directly affect the agent’s perceptions of the world. In the “Action/Perception-Cycle” (see Figure 1), biological organisms are integrated sensorimotor systems. This means that intelligent processes require a body, and that symbols are grounded in the environment in which animals live (Harnad, 1990). In short, behavior is fundamentally linked to cognition. This is true for humans, animals and artificial agents. Without this grounding, artificial animals and agents cannot live and behave successfully in their artificial environments. One way of achieving it, is to use Genetic Algorithms to evolve agents’ neural architecture (Nolfi & Floreano, 2000). This creates the prospect of robots that can live in complex socially organized communities in which they communicate with humans and with each other (Cangelosi e Parisi, 2002). According to these authors, cognition is an intrinsically embedded phenomenon in which the dynamical relations between the neural system, the body and the environment play a central role. In this view, agents are dynamical systems and cognitive functioning has to be understood using tools from dynamical system theory (van Gelder, 1995, 1998a; 1998b; Bilotta et al., 2007a-2007f). This perspective on cognition has been called the Dynamical and Embodied view of Cognition (DEC) (Keijzer, 2002). In this chapter we describe our own contribution to Evolutionary Robotics, namely a proposal for a new generation of believable agents capable of life-like intelligent communicative and emotional behavior. We focus on CNNs (Cellular Neural Networks) and on the use of these networks as robot controllers. In previous work, we used Genetic Algorithms (GAs) to evolve Artificial Non-linear Networks (ANNs) displaying artificial adaptive behavior, with features similar to those observed in animals and humans. In (Bilotta et al. 2006), we replaced ANNs with a new class of dynamical system called Cellular Non-linear Networks (CNNs) and used CNNs to implement a multilayer locomotion model for six-legged artificial robots in a virtual environment with many of the characteristics of a physical environment. First invented by Chua and co-workers (1988), CNNs have been extended to create a CNN Universal Machine (CNN-UM) (Roska & Chua, 1993), the first algorithmically programmable analog computer chip suitable for the modeling of sensory

Evolutionary robotics is a good method for the generation of controllers for autonomous robots. However, up to date, evolutionary methods do not achieve the generation of behaviors for complex robots with a fixed body structure composed of lots of sensors and actuators. For such cases, no satisfactory results exist due to the large search space that the evolutionary algorithm has to face. Furthermore, the bootstrap problem does not allow convergence on the first generations, preventing the generation of simple solutions with a minimum fitness value that could guide the evolutionary path towards the final solution. Solutions like incremental evolution try to overcome the problem, but they do not scale well in complex robots with lots of devices. The question is then, why natural evolution succeeded evolving complex animals, but evolutionary robotics does not. One answer to that question may be that, while natural evolution gradually evolved at the same time the animals body plan, their sensors and actuators, their nervous system, and even their environment, artificial evolution tries to evolve the nervous system for a robot with a fixed given body, sensors and actuators, within a fixed complex environment. Our proposal states that when, as it happens in most cases, none of the evolutionary constraints can be relaxed (those are, the robot body, the sensors and actuators, the behavior to implement or the environment), then the use of external knowledge to guide the evolutionary process should be mandatory. Evolutionary approaches try to avoid the use of such knowledge, also called bias, because it directs the evolutionary search towards specific places of the space, not allowing the algorithm to find its own solutions. In this paper, we advocate instead for the use of bias as an inevitable situation when the robot body, task and environment are complex and fixed. Based on this idea, we develop a modular architecture for evolutionary controllers based on neural networks, which allows the selective introduction of bias knowledge in the neural controller during the evolutionary process. The architecture allows the introduction of external knowledge on selected stages of the evolutionary process, affecting only selected parts of the controller that need to accommodate that information. The evolutionary controller is progressively designed in a series of stages, almost in a surgical way, independently of the complexity of the robot (in terms of number of sensors and actuators). This approach allows to avoid the bootstrap problem completely, and to obtain a completely distributed neural controller for the robot using only artificial evolution.

Quality-Diversity algorithms search for large collections of diverse and high-performing solutions, rather than just for a single solution like typical optimisation methods. They are specially adapted for multi-modal problems that can be solved in many different ways, such as complex reinforcement learning or robotics tasks. However, these approaches are highly dependent on the choice of feature descriptors (FDs) quantifying the similarity in behaviour of the solutions. While FDs usually needs to be hand-designed, recent studies have proposed ways to define them automatically by using feature extraction techniques, such as PCA or Auto-Encoders, to learn a representation of the problem from previously explored solutions. Here, we extend these approaches to more complex problems which cannot be efficiently explored by relying only on a single representation but require instead a set of diverse and complementary representations. We describe MC-AURORA, a Quality-Diversity approach that optimises simultaneously several collections of solutions, each with a different set of FDs, which are, in turn, defined automatically by an ensemble of modular auto-encoders. We show that this approach produces solutions that are more diverse than those produced by single-representation approaches., Draft version. 10 pages, 4 figures, 4 tables, Accepted at the GECCO2021 Conference

The joint optimisation of body-plan and control via evolutionary processes can be challenging in rich morphological spaces in which offspring can have body-plans that are very different from either of their parents. This causes a potential mismatch between the structure of an inherited controller and the new body. To address this, we propose a framework that combines an evolutionary algorithm to generate body-plans and a learning algorithm to optimise the parameters of a neural controller. The topology of this controller is created once the body-plan of each offspring body-plan is generated. The key novelty of the approach is to add an external archive for storing learned controllers that map to explicit `types' of robots (where this is defined with respect the features of the body-plan). By learning from a controller with an appropriate structure inherited from the archive, rather than from a randomly initialised one, we show that both the speed and magnitude of learning increases over time when compared to an approach that starts from scratch, using two tasks and three environments. The framework also provides new insights into the complex interactions between evolution and learning., 15 pages including 2 pages of supplementary materials, 16 figures, 1 table. Currently under review for the special issue of IEEE TCDS on Towards autonomous evolution, (re)production and learning in robotic eco-systems. https://www.york.ac.uk/robot-lab/are/ieee_special_issue_2020/

Multi-level evolution (MLE) is a novel robotic design paradigm which decomposes the design problem into layered sub-tasks that involve concurrent search for appropriate materials, component geometry and overall morphology. This has a number of advantages, mainly in terms of quality and scalability. In this paper, we present a hierarchical approach to robotic design based on the MLE architecture. The design problem involves finding a robotic design which can be used to perform a specific locomotion task. At the materials layer, we put together a simple collection of materials which are represented by combinations of mechanical properties such as friction and restitution. At the components layer we combine these materials with geometric design to form robot limbs. Finally, at the robot layer we introduce these evolved limbs into robotic body-plans and learn control policies to form complete robots. Quality-diversity algorithms at each level allow for the discovery of a wide variety of reusable elements. The results strongly support the initial claims for the benefits of MLE, allowing for the discovery of designs that would otherwise be difficult to achieve with conventional design paradigms.