> Evolutionary intelligence & evolutionary robotics

We use simulated evolution to automatically design "artificial nervous systems" for robots.

Evolution has been one of the prominent processes to shape the animal brain; imitating this process by employing evolution-inspired algorithms to design "artificial nervous systems" therefore drew considerable attention in artificial intelligence over the last two decades. This line of thought drives us to propose many methods to exploit evolutionary algorithms (EA) to design neural networks (neuro-evolution). We especially work on:

Funding

Funding provided by the ANR (projects Evoneuro and Creadapt).

(images: S. Magenat & R. Wheeler)

Illumination algorithms / Quality diversity (MAP-Elites)

Nearly all science and engineering fields use search algorithms, which automatically explore a search space to find high-performing solutions: chemists search through the space of molecules to discover new drugs; engineers search for stronger, cheaper, safer designs, scientists search for models that best explain data, etc. The goal of search algorithms has traditionally been to return the single highest-performing solution in a search space.

Illumination algorithms (also called Quality Diversity algorithms) are new, fundamentally different type of algorithm that is more useful because it provides a holistic view of how high-performing solutions are distributed throughout a search space.

The MAP-Elites (Multi-dimensional Archive of Phenotypic Elites) algorithm creates a map of high-performing solutions at each point in a space defined by dimensions of variation that a user gets to choose. For example, a drug company may wish to understand how performance changes as the size of molecules and their cost-to-produce vary. MAP-Elites produces a large diversity of high-performing, yet qualitatively different solutions, which can be more helpful than a single, high-performing solution.

Pre-prints / arxiv


2023

P. Kent, A. Gaier, J.-B Mouret, J. Branke (2023). BOP-Elites, a Bayesian Optimisation Approach to Quality Diversity Search with Black-Box descriptor functions.
arXiv preprint arXiv:2307.09326v2. [url]

2015

J.-B Mouret, J. Clune (2015). Illuminating search spaces by mapping elites.
arXiv preprint. [url]

Articles in peer-reviewed journals


2023

Jacques Zhong and Vincent Weistroffer and Jean-Baptiste Mouret and Francis Colas and Pauline Maurice (2023). Workstation Suitability Maps: Generating Ergonomic Behaviors on a Population of Virtual Humans with Multi-task Optimization.
IEEE Robotics and Automation Letters.
→ [pdf] [url] [video]

2020

Lehman, Joel and others (2020). The surprising creativity of digital evolution: A collection of anecdotes from the evolutionary computation and artificial life research communities.
Artificial Life.
→ [pdf] [url]

R. Kaushik, P. Desreumaux, J.-B Mouret (2020). Adaptive Prior Selection for Repertoire-based Online Learning in Robotics.
Frontiers in Robotics and AI.
→ [pdf] [url] [video]

J.-B Mouret (2020). Evolving the behavior of machines: from micro to macroevolution.
iScience. 101731. Elsevier.[url]

2019

D. Howard, A. Eiben, D. Kennedy, J.-B Mouret, P. Valencia, D. Winkler (2019). Evolving embodied intelligence from materials to machines.
Nature Machine Intelligence. 1. (1) 12--19. Nature Publishing Group.
→ [pdf] [url]

2018

A. Gaier, A. Asteroth, J.-B Mouret (2018). Data-Efficient Design Exploration through Surrogate-Assisted Illumination.
Evolutionary Computation. MIT Press. 10.1162/evco_a_00231
→ [pdf] [url]

2017

V. Vassiliades, K. Chatzilygeroudis, J.-B Mouret (2017). Using Centroidal Voronoi Tessellations to Scale Up the Multi-dimensional Archive of Phenotypic Elites Algorithm.
IEEE Transactions on Evolutionary Computation. 10.1109/TEVC.2017.2735550
→ [pdf] [url] [source code]

K. Chatzilygeroudis, V. Vassiliades, J.-B Mouret (2017). Reset-free Trial-and-Error Learning for Robot Damage Recovery.
Robotics and Autonomous Systems. 1-19. Elsevier. 10.1016/j.robot.2017.11.010
→ [pdf] [url] [source code] [video]

2016

A. Cully, J.-B Mouret (2016). Evolving a Behavioral Repertoire for a Walking Robot.
Evolutionary Computation. 24. (1) 59-88. MIT Press. 10.1162/EVCO_a_00143
→ [pdf] [url] [video]

2015

R. Reuillon, C. Schmitt, R. De Aldama, J.-B Mouret (2015). A New Method to Evaluate Simulation Models: The Calibration Profile (CP) Algorithm.
Journal of Artificial Societies and Social Simulation. 18. (1) SimSoc Consortium. 10.18564/jasss.2675[url]

A. Cully, J. Clune, D. Tarapore, J.-B Mouret (2015). Robots that can adapt like animals.
Nature. 521. (7553) 503-507. Nature Publishing Group. 10.1038/nature14422
→ [pdf] [url] [source code] [video] [video]

Articles in peer-reviewed conferences


2020

A. Gaier, A. Asteroth, J.-B Mouret (2020). Discovering Representations for Black-box Optimization.
Proc. of GECCO 103--111. 10.1145/3377930.3390221
→ [pdf] [url]

D. Bossens, J.-B Mouret, D. Tarapore (2020). Learning behaviour-performance maps with meta-evolution.
Proc. of GECCO
→ [pdf] [url]

J.-B Mouret, G. Maguire (2020). Quality Diversity for Multi-task Optimization.
Proc. of GECCO 121--129. 10.1145/3377930.3390203
→ [pdf] [url]

2018

R. Pautrat, K. Chatzilygeroudis, J.-B Mouret (2018). Bayesian Optimization with Automatic Prior Selection for Data-Efficient Direct Policy Search.
IEEE International Conference on Robotics and Automation (ICRA)
→ [pdf] [url] [video]

V. Vassiliades, J.-B Mouret (2018). Discovering the Elite Hypervolume by Leveraging Interspecies Correlation.
Proc. of GECCO 10.1145/3205455.3205602
→ [pdf] [url]

2017

A. Gaier, A. Asteroth, J.-B Mouret (2017). Aerodynamic Design Exploration through Surrogate-Assisted Illumination.
18th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference (best student paper) 10.2514/6.2017-3330
→ [pdf]

A. Gaier, A. Asteroth, J.-B Mouret (2017). Data-Efficient Exploration, Optimization, and Modeling of Diverse Designs through Surrogate-Assisted Illumination.
Proc. of GECCO 10.1145/3071178.3071282
→ [pdf] [url]

2016

D. Tarapore, J. Clune, A. Cully, J.-B Mouret (2016). How Do Different Encodings Influence the Performance of the MAP-Elites Algorithm?.
Proc. of GECCO ACM. 10.1145/2908812.2908875
→ [pdf] [url] [source code]

Workshops, book chapters, and minimally reviewed conferences


2021

K. Chatzilygeroudis, A. Cully, V. Vassiliades, J.-B Mouret (2021). Quality-Diversity Optimization: a novel branch of stochastic optimization.
Black Box Optimization, Machine Learning, and No-Free Lunch Theorems 109--135. Springer.[url]

Encouraging behavioral diversity: making EAs faster and better

Evolutionary Robotics (ER) aims at automatically designing robots or controllers of robots without having to describe their inner workings. To reach this goal, ER researchers primarily employ phenotypes that can lead to an infinite number of robot behaviors and fitness functions that only reward the achievement of the task---and not how to achieve it. These choices make ER particularly prone to premature convergence.

To tackle this problem, several papers recently proposed to explicitly encourage the diversity of the robot's behaviors, rather than the diversity of the genotypes as in classic evolutionary optimization. Such an approach avoids the need to compute distances between structures and the pitfalls of the non-injectivity of the phenotype/behavior relation; however, it also introduces new questions: how to compare behavior? should this comparison be task-specific? and what is the best way to encourage diversity in this context?

In our work, we proposed several methods to encourage behavioral diversity, in particular using multi-objective evolutionary algorithms (multi-objectivization). We compared each approach on three different tasks and two different genotypes. Results show that fostering behavioral diversity substantially improves the evolutionary process in the investigated experiments, regardless of genotype or task. Among the benchmarked approaches, multi-objective methods were the most efficient and the generic, Hamming-based, behavioral distance was at least as efficient as task-specific behavioral metrics.




Related publications

Articles in peer-reviewed journals


2016

A. Cully, J.-B Mouret (2016). Evolving a Behavioral Repertoire for a Walking Robot.
Evolutionary Computation. 24. (1) 59-88. MIT Press. 10.1162/EVCO_a_00143
→ [pdf] [url] [video]

2014

S. Doncieux, J.-B Mouret (2014). Beyond black-box optimization: a review of selective pressures for evolutionary robotics.
Evolutionary Intelligence. 7. (2) 71-93. 10.1007/s12065-014-0110-x[url]

2012

J.-B Mouret, S. Doncieux (2012). Encouraging Behavioral Diversity in Evolutionary Robotics: an Empirical Study.
Evolutionary Computation. 20. (1) 91-133. MIT Press. 10.1162/EVCO_a_00048
→ [pdf] [url]

Articles in peer-reviewed conferences


2018

V. Vassiliades, J.-B Mouret (2018). Discovering the Elite Hypervolume by Leveraging Interspecies Correlation.
Proc. of GECCO 10.1145/3205455.3205602
→ [pdf] [url]

R. Kaushik, K. Chatzilygeroudis, J.-B Mouret (2018). Multi-objective Model-based Policy Search for Data-efficient Learning with Sparse Rewards.
Conference on Robot Learning (CoRL)
→ [pdf] [url]

2013

A. Cully, J.-B Mouret (2013). Behavioral Repertoire Learning in Robotics.
Proc. of GECCO 175-182. ACM. 10.1145/2463372.2463399
→ [pdf] [url] [video]

S. Doncieux, J.-B Mouret (2013). Behavioral Diversity with Multiple Behavioral Distances.
15th IEEE Congress on Evolutionary Computation (CEC'13) 1-8. IEEE. 10.1109/CEC.2013.6557731
→ [pdf]

2010

S. Doncieux, J.-B Mouret (2010). Behavioral diversity measures for Evolutionary Robotics.
12th IEEE Congress on Evolutionary Computation (CEC'10) 1303-1310. IEEE. 10.1109/CEC.2010.5586100WCCI
→ [pdf] [url]

2009

J.-B Mouret, S. Doncieux (2009). Overcoming the bootstrap problem in evolutionary robotics using behavioral diversity.
11th IEEE Congress on Evolutionary Computation (CEC'09) 1161--1168. IEEE. 10.1109/CEC.2009.4983077
→ [pdf] [url]

J.-B Mouret, S. Doncieux (2009). Using Behavioral Exploration Objectives to Solve Deceptive Problems in Neuro-evolution.
Proc. of GECCO 627--634. ACM. 10.1145/1569901.1569988
→ [pdf] [url]

J.-B Mouret, S. Doncieux (2009). Evolving modular neural-networks through exaptation.
11th Congress on Evolutionary Computation (CEC'09) (Best Student Paper) 1570--1577. IEEE. 10.1109/CEC.2009.4983129
→ [pdf] [url]

[ top]

Crossing the reality gap: the transferability approach

How to make sure that results obtained in simulation work on the robot? In robotics, gradient-free optimization algorithms (e.g. evolutionary algorithms) are often used only in simulation because they require the evaluation of many candidate solutions. Nevertheless, solutions obtained in simulation often do not work well on the real device. The transferability approach aims at crossing this gap between simulation and reality by automatically making the optimization algorithm aware of the limits of the simulation.

Related publications

Articles in peer-reviewed journals


2018

J. Rieffel*, J.-B Mouret* (2018). Adaptive and Resilient Soft Tensegrity Robots.
Soft Robotics. (* J. Rieffel and J.-B. Mouret contributed equally to this work). https://doi.org/10.1089/soro.2017.0066[url] [video]

2016

A. Cully, J.-B Mouret (2016). Evolving a Behavioral Repertoire for a Walking Robot.
Evolutionary Computation. 24. (1) 59-88. MIT Press. 10.1162/EVCO_a_00143
→ [pdf] [url] [video]

2015

A. Cully, J. Clune, D. Tarapore, J.-B Mouret (2015). Robots that can adapt like animals.
Nature. 521. (7553) 503-507. Nature Publishing Group. 10.1038/nature14422
→ [pdf] [url] [source code] [video] [video]

2013

S. Koos, A. Cully, J.-B Mouret (2013). Fast Damage Recovery in Robotics with the T-Resilience Algorithm.
International Journal of Robotics Research (IJRR). 32. (14) 1700-1723. SAGE Publications. 10.1177/0278364913499192
→ [pdf] [url]

S. Koos, J.-B Mouret, S. Doncieux (2013). The Transferability Approach: Crossing the Reality Gap in Evolutionary Robotics.
IEEE Transactions on Evolutionary Computation (TEC). 17. (1) 122--145. IEEE. 10.1109/TEVC.2012.2185849
→ [pdf] [url] [video] [video]

Articles in peer-reviewed conferences


2013

M. Oliveira, S. Doncieux, J.-B Mouret, C. Peixoto Santos (2013). Optimization of Humanoid Walking Controller: Crossing the Reality Gap.
Proc. of IEEE Humanoids 1-7. 10.1109/HUMANOIDS.2013.7029963
→ [pdf]

2011

S. Koos, J.-B Mouret (2011). Online Discovery of Locomotion Modes for Wheel-Legged Hybrid Robots: a Transferability-based Approach.
14th International Conference on Climbing and Walking Robots (CLAWAR). Highly Recommended Paper, category ``control of CLAWAR'' 70-77. 10.1142/9789814374286_0008
→ [pdf] [url] [video] [video] [video]

2010

S. Koos, J.-B Mouret, S. Doncieux (2010). Crossing the Reality Gap in Evolutionary Robotics by Promoting Transferable Controllers.
Proc. of GECCO 119-126. ACM. 10.1145/1830483.1830505
→ [pdf] [url]

2009

S. Koos, J.-B Mouret, S. Doncieux (2009). Automatic system identification based on coevolution of models and tests.
11th IEEE Congress on Evolutionary Computation (CEC'09) 560--567. IEEE. 10.1109/CEC.2009.4982995
→ [pdf] [url]

[ top]

Encoding large neural networks: generative and developmental systems

EvoNeuro encoding

Neuro-evolution and computational neuroscience are two scientific domains that produce surprisingly different artificial neural networks. Inspired by the "toolbox" used by neuroscientists to create their models, this work investigate two main points: (1) neural maps (spatially-organized identical neurons) should be the building blocks to evolve neural networks able to perform cognitive functions and (2) well-identified modules of the brain for which there exists computational neuroscience models provide well-defined benchmarks for neuro-evolution.

Relationships between synaptic plasticity and generative and developmental systems (GDS)

We analyzed the mutual relationships between generative and developmental systems (GDS) and sy-naptic plasticity when evolving plastic artificial neural networks (ANNs) in reward-based scenarios. We first introduce the concept of synaptic Transitive Learning Abilities (sTLA), which reflects how well an evolved plastic ANN can cope with learning scenarios not encountered during the evolution process. We subsequently report results of a set of experiments designed to check that (1) synaptic plasticity can help a GDS to fine-tune synaptic weights and (2) that with the investigated generative encoding (EvoNeuro), only a few learning scenarios are necessary to evolve a general learning system, which can adapt itself to reward-based scenarios not tested during the fitness evaluation.

Related publications

Articles in peer-reviewed journals


2015

D. Tarapore, J.-B Mouret (2015). Evolvability signatures of generative encodings: Beyond standard performance benchmarks.
Information Sciences. 313. 43-61. Elsevier. 10.1016/j.ins.2015.03.046[url]

2013

P. Tonelli, J.-B Mouret (2013). On the Relationships between Generative Encodings, Regularity, and Learning Abilities when Evolving Plastic Artificial Neural Networks.
PLoS One. 8. (11) e79138. 10.1371/journal.pone.0079138
→ [pdf]

2008

J.-B Mouret, S. Doncieux (2008). MENNAG: a modular, regular and hierarchical encoding for neural-networks based on attribute grammars.
Evolutionary Intelligence. 1. (3) 187--207. 10.1007/s12065-008-0015-7
→ [pdf] [html]

Articles in peer-reviewed conferences


2016

D. Tarapore, J. Clune, A. Cully, J.-B Mouret (2016). How Do Different Encodings Influence the Performance of the MAP-Elites Algorithm?.
Proc. of GECCO ACM. 10.1145/2908812.2908875
→ [pdf] [url] [source code]

2014

D. Tarapore, J.-B Mouret (2014). Comparing the evolvability of generative encoding schemes.
Artificial Life 14: Proc. of the Fourteenth International Conference on the Synthesis and Simulation of Living Systems 55-62. MIT Press. 10.7551/978-0-262-32621-6-ch011
→ [pdf]

J. Huizinga, J.-B Mouret, J. Clune (2014). Evolving Neural Networks That Are Both Modular and Regular: HyperNeat Plus the Connection Cost Technique.
Proc. of GECCO 1-8. ACM. 10.1145/2576768.2598232
→ [pdf] [source code] [video]

2011

P. Tonelli, J.-B Mouret (2011). Using a Map-Based Encoding to Evolve Plastic Neural Networks.
Proc. of IEEE Symposium Series on Computational Intelligence 9-16. 10.1109/EAIS.2011.5945909
→ [pdf]

2010

J.-B Mouret, S. Doncieux, B. Girard (2010). Importing the Computational Neuroscience Toolbox into Neuro-Evolution---Application to Basal Ganglia.
Proc. of GECCO 587-594. ACM. 10.1145/1830483.1830592
→ [pdf] [url]

[ top]