> Evolutionary intelligence & evolutionary robotics

We use simulated evolution to automatically design "artificial nervous systems" for robots.

Evolution has been one of the prominent processes to shape the animal brain; imitating this process by employing evolution-inspired algorithms to design "artificial nervous systems" therefore drew considerable attention in artificial intelligence over the last two decades. This line of thought drives us to propose many methods to exploit evolutionary algorithms (EA) to design neural networks (neuro-evolution). We especially work on:

Funding

Funding provided by the ANR (projects Evoneuro and Creadapt).

(images: S. Magenat & R. Wheeler)

Illumination algorithms / Quality diversity (MAP-Elites)

Nearly all science and engineering fields use search algorithms, which automatically explore a search space to find high-performing solutions: chemists search through the space of molecules to discover new drugs; engineers search for stronger, cheaper, safer designs, scientists search for models that best explain data, etc. The goal of search algorithms has traditionally been to return the single highest-performing solution in a search space.

Illumination algorithms (also called Quality Diversity algorithms) are new, fundamentally different type of algorithm that is more useful because it provides a holistic view of how high-performing solutions are distributed throughout a search space.

The MAP-Elites (Multi-dimensional Archive of Phenotypic Elites) algorithm creates a map of high-performing solutions at each point in a space defined by dimensions of variation that a user gets to choose. For example, a drug company may wish to understand how performance changes as the size of molecules and their cost-to-produce vary. MAP-Elites produces a large diversity of high-performing, yet qualitatively different solutions, which can be more helpful than a single, high-performing solution.

Pre-prints / arxiv


2015

J.-B Mouret, J. Clune (2015). Illuminating search spaces by mapping elites.
arXiv preprint. [url]

Articles in peer-reviewed journals


2017

V. Vassiliades, K. Chatzilygeroudis, J.-B Mouret (2017). Scaling Up MAP-Elites Using Centroidal Voronoi Tessellations.
IEEE Transactions on Evolutionary Computation. [url]

2016

A. Cully, J.-B Mouret (2016). Evolving a Behavioral Repertoire for a Walking Robot.
Evolutionary Computation. 24. (1) 59-88. MIT Press. 10.1145/2463372.2463399
→ [pdf] [url] [video]

2015

A. Cully, J. Clune, D. Tarapore, J.-B Mouret (2015). Robots that can adapt like animals.
Nature. 521. (7553) 503-507. Nature Publishing Group. 10.1038/nature14422
→ [pdf] [url] [source code] [video] [video]

Articles in peer-reviewed conferences


2017

A. Gaier, A. Asteroth, J.-B Mouret (2017). Aerodynamic Design Exploration through Surrogate-Assisted Illumination.
18th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference (best student paper) 10.2514/6.2017-3330
→ [pdf]

A. Gaier, A. Asteroth, J.-B Mouret (2017). Data-Efficient Exploration, Optimization, and Modeling of Diverse Designs through Surrogate-Assisted Illumination.
Proc. of GECCO 10.1145/3071178.3071282
→ [pdf] [url]

Encouraging behavioral diversity: making EAs faster and better

Evolutionary Robotics (ER) aims at automatically designing robots or controllers of robots without having to describe their inner workings. To reach this goal, ER researchers primarily employ phenotypes that can lead to an infinite number of robot behaviors and fitness functions that only reward the achievement of the task---and not how to achieve it. These choices make ER particularly prone to premature convergence.

To tackle this problem, several papers recently proposed to explicitly encourage the diversity of the robot's behaviors, rather than the diversity of the genotypes as in classic evolutionary optimization. Such an approach avoids the need to compute distances between structures and the pitfalls of the non-injectivity of the phenotype/behavior relation; however, it also introduces new questions: how to compare behavior? should this comparison be task-specific? and what is the best way to encourage diversity in this context?

In our work, we proposed several methods to encourage behavioral diversity, in particular using multi-objective evolutionary algorithms (multi-objectivization). We compared each approach on three different tasks and two different genotypes. Results show that fostering behavioral diversity substantially improves the evolutionary process in the investigated experiments, regardless of genotype or task. Among the benchmarked approaches, multi-objective methods were the most efficient and the generic, Hamming-based, behavioral distance was at least as efficient as task-specific behavioral metrics.




Related publications

Articles in peer-reviewed journals


2016

A. Cully, J.-B Mouret (2016). Evolving a Behavioral Repertoire for a Walking Robot.
Evolutionary Computation. 24. (1) 59-88. MIT Press. 10.1145/2463372.2463399
→ [pdf] [url] [video]

2014

S. Doncieux, J.-B Mouret (2014). Beyond black-box optimization: a review of selective pressures for evolutionary robotics.
Evolutionary Intelligence. 7. (2) 71-93. 10.1007/s12065-014-0110-x[url]

2012

J.-B Mouret, S. Doncieux (2012). Encouraging Behavioral Diversity in Evolutionary Robotics: an Empirical Study.
Evolutionary Computation. 20. (1) 91-133. MIT Press. 10.1162/EVCO_a_00048
→ [pdf] [url]

Articles in peer-reviewed conferences


2013

A. Cully, J.-B Mouret (2013). Behavioral Repertoire Learning in Robotics.
Proc. of GECCO 175-182. ACM. 10.1145/2463372.2463399
→ [pdf] [url] [video]

S. Doncieux, J.-B Mouret (2013). Behavioral Diversity with Multiple Behavioral Distances.
15th IEEE Congress on Evolutionary Computation (CEC'13) 1-8. IEEE. 10.1109/CEC.2013.6557731
→ [pdf]

2010

S. Doncieux, J.-B Mouret (2010). Behavioral diversity measures for Evolutionary Robotics.
12th IEEE Congress on Evolutionary Computation (CEC'10) 1303-1310. IEEE. 10.1109/CEC.2010.5586100WCCI
→ [pdf] [url]

2009

J.-B Mouret, S. Doncieux (2009). Overcoming the bootstrap problem in evolutionary robotics using behavioral diversity.
11th IEEE Congress on Evolutionary Computation (CEC'09) 1161--1168. IEEE. 10.1109/CEC.2009.4983077
→ [pdf] [url]

J.-B Mouret, S. Doncieux (2009). Using Behavioral Exploration Objectives to Solve Deceptive Problems in Neuro-evolution.
Proc. of GECCO 627--634. ACM. 10.1145/1569901.1569988
→ [pdf] [url]

J.-B Mouret, S. Doncieux (2009). Evolving modular neural-networks through exaptation.
11th Congress on Evolutionary Computation (CEC'09) (Best Student Paper) 1570--1577. IEEE. 10.1109/CEC.2009.4983129
→ [pdf] [url]

[ top]

Crossing the reality gap: the transferability approach

How to make sure that results obtained in simulation work on the robot? In robotics, gradient-free optimization algorithms (e.g. evolutionary algorithms) are often used only in simulation because they require the evaluation of many candidate solutions. Nevertheless, solutions obtained in simulation often do not work well on the real device. The transferability approach aims at crossing this gap between simulation and reality by automatically making the optimization algorithm aware of the limits of the simulation.

Related publications

Pre-prints / arxiv


2017

J. Rieffel*, J.-B Mouret* (2017). Soft Tensegrity Robots.
arXiv preprint. (* J. Rieffel and J.-B. Mouret contributed equally to this work).[url]

2016

K. Chatzilygeroudis, V. Vassiliades, J.-B Mouret (2016). Reset-free Trial-and-Error Learning for Data-Efficient Robot Damage Recovery.
arXiv preprint. [url]

Articles in peer-reviewed journals


2016

A. Cully, J.-B Mouret (2016). Evolving a Behavioral Repertoire for a Walking Robot.
Evolutionary Computation. 24. (1) 59-88. MIT Press. 10.1145/2463372.2463399
→ [pdf] [url] [video]

2015

A. Cully, J. Clune, D. Tarapore, J.-B Mouret (2015). Robots that can adapt like animals.
Nature. 521. (7553) 503-507. Nature Publishing Group. 10.1038/nature14422
→ [pdf] [url] [source code] [video] [video]

2013

S. Koos, A. Cully, J.-B Mouret (2013). Fast Damage Recovery in Robotics with the T-Resilience Algorithm.
International Journal of Robotics Research (IJRR). 32. (14) 1700-1723. SAGE Publications. 10.1177/0278364913499192
→ [pdf] [url]

S. Koos, J.-B Mouret, S. Doncieux (2013). The Transferability Approach: Crossing the Reality Gap in Evolutionary Robotics.
IEEE Transactions on Evolutionary Computation (TEC). 17. (1) 122--145. IEEE. 10.1109/TEVC.2012.2185849
→ [pdf] [url] [video] [video]

Articles in peer-reviewed conferences


2013

M. Oliveira, S. Doncieux, J.-B Mouret, C. Peixoto Santos (2013). Optimization of Humanoid Walking Controller: Crossing the Reality Gap.
Proc. of IEEE Humanoids 1-7. 10.1109/HUMANOIDS.2013.7029963
→ [pdf]

2010

S. Koos, J.-B Mouret, S. Doncieux (2010). Crossing the Reality Gap in Evolutionary Robotics by Promoting Transferable Controllers.
Proc. of GECCO 119-126. ACM. 10.1145/1830483.1830505
→ [pdf] [url]

2009

S. Koos, J.-B Mouret, S. Doncieux (2009). Automatic system identification based on coevolution of models and tests.
11th IEEE Congress on Evolutionary Computation (CEC'09) 560--567. IEEE. 10.1109/CEC.2009.4982995
→ [pdf] [url]

[ top]

Encoding large neural networks: generative and developmental systems

EvoNeuro encoding

Neuro-evolution and computational neuroscience are two scientific domains that produce surprisingly different artificial neural networks. Inspired by the "toolbox" used by neuroscientists to create their models, this work investigate two main points: (1) neural maps (spatially-organized identical neurons) should be the building blocks to evolve neural networks able to perform cognitive functions and (2) well-identified modules of the brain for which there exists computational neuroscience models provide well-defined benchmarks for neuro-evolution.

Relationships between synaptic plasticity and generative and developmental systems (GDS)

We analyzed the mutual relationships between generative and developmental systems (GDS) and sy-naptic plasticity when evolving plastic artificial neural networks (ANNs) in reward-based scenarios. We first introduce the concept of synaptic Transitive Learning Abilities (sTLA), which reflects how well an evolved plastic ANN can cope with learning scenarios not encountered during the evolution process. We subsequently report results of a set of experiments designed to check that (1) synaptic plasticity can help a GDS to fine-tune synaptic weights and (2) that with the investigated generative encoding (EvoNeuro), only a few learning scenarios are necessary to evolve a general learning system, which can adapt itself to reward-based scenarios not tested during the fitness evaluation.

Related publications

Articles in peer-reviewed journals


2015

D. Tarapore, J.-B Mouret (2015). Evolvability signatures of generative encodings: Beyond standard performance benchmarks.
Information Sciences. 313. 43-61. Elsevier. 10.1016/j.ins.2015.03.046[url]

2013

P. Tonelli, J.-B Mouret (2013). On the Relationships between Generative Encodings, Regularity, and Learning Abilities when Evolving Plastic Artificial Neural Networks.
PLoS One. 8. (11) e79138. 10.1371/journal.pone.0079138
→ [pdf]

2008

J.-B Mouret, S. Doncieux (2008). MENNAG: a modular, regular and hierarchical encoding for neural-networks based on attribute grammars.
Evolutionary Intelligence. 1. (3) 187--207. 10.1007/s12065-008-0015-7
→ [pdf] [html]

Articles in peer-reviewed conferences


2014

D. Tarapore, J.-B Mouret (2014). Comparing the evolvability of generative encoding schemes.
Artificial Life 14: Proc. of the Fourteenth International Conference on the Synthesis and Simulation of Living Systems 55-62. MIT Press. 10.7551/978-0-262-32621-6-ch011
→ [pdf]

J. Huizinga, J.-B Mouret, J. Clune (2014). Evolving Neural Networks That Are Both Modular and Regular: HyperNeat Plus the Connection Cost Technique.
Proc. of GECCO 1-8. ACM. 10.1145/2576768.2598232
→ [pdf] [source code] [video]

2011

P. Tonelli, J.-B Mouret (2011). Using a Map-Based Encoding to Evolve Plastic Neural Networks.
Proc. of IEEE Symposium Series on Computational Intelligence 9-16. 10.1109/EAIS.2011.5945909
→ [pdf]

2010

J.-B Mouret, S. Doncieux, B. Girard (2010). Importing the Computational Neuroscience Toolbox into Neuro-Evolution---Application to Basal Ganglia.
Proc. of GECCO 587-594. ACM. 10.1145/1830483.1830592
→ [pdf] [url]

[ top]