Meta-Learning an Evolvable Developmental Encoding
2406.09020
0
0
Abstract
Representations for black-box optimisation methods (such as evolutionary algorithms) are traditionally constructed using a delicate manual process. This is in contrast to the representation that maps DNAs to phenotypes in biological organisms, which is at the hear of biological complexity and evolvability. Additionally, the core of this process is fundamentally the same across nearly all forms of life, reflecting their shared evolutionary origin. Generative models have shown promise in being learnable representations for black-box optimisation but they are not per se designed to be easily searchable. Here we present a system that can meta-learn such representation by directly optimising for a representation's ability to generate quality-diversity. In more detail, we show our meta-learning approach can find one Neural Cellular Automata, in which cells can attend to different parts of a DNA string genome during development, enabling it to grow different solvable 2D maze structures. We show that the evolved genotype-to-phenotype mappings become more and more evolvable, not only resulting in a faster search but also increasing the quality and diversity of grown artefacts.
Create account to get full access
Overview
- Presents a novel approach to meta-learning an "evolvable developmental encoding" for training artificial neural networks
- Explores how learning algorithms can be designed to produce neural networks that are more evolvable, i.e., better able to adapt and change over time
- Combines concepts from developmental biology, meta-learning, and neuroevolution to create a flexible framework for generating adaptable neural network architectures
Plain English Explanation
The paper focuses on developing a new way to train artificial neural networks that allows them to continuously evolve and adapt over time, much like biological organisms. The key idea is to create a "developmental encoding" - a set of rules that govern how the neural network grows and changes during the training process.
By meta-learning this developmental encoding, the researchers aim to produce neural networks that are more "evolvable" - meaning they can easily adapt to new environments or challenges without needing to be completely retrained from scratch. This is inspired by how biological organisms are able to evolve and develop new capabilities over generations.
The paper combines insights from developmental biology, meta-learning, and neuroevolution to create a flexible framework for generating adaptable neural network architectures. The goal is to move beyond rigid, hand-designed networks towards more open-ended, evolvable systems that can continually learn and improve over time.
Technical Explanation
The key technical contribution of the paper is a novel meta-learning algorithm for training an "evolvable developmental encoding" (EDE) that can be used to generate adaptable neural network architectures.
The EDE is represented as a set of differentiable rules that govern how the neural network's structure and parameters evolve during training. By meta-learning the parameters of this developmental encoding, the researchers aim to produce networks that are more evolvable and adaptable to new tasks or environments.
The EDE is implemented using a differentiable neural computer that can dynamically modify the network's architecture and weights. During meta-training, the developmental encoding is optimized to maximize the network's performance and evolvability on a set of training tasks.
The paper presents experiments demonstrating that neural networks trained using the EDE approach are able to adapt more quickly and effectively to new challenges compared to traditionally trained networks. This suggests the EDE framework may be a promising direction for developing more flexible and adaptable artificial intelligence systems.
Critical Analysis
The paper presents a compelling vision for enhancing the evolvability of artificial neural networks, drawing on insights from developmental biology and meta-learning. However, the experimental results are somewhat limited in scope, focusing primarily on simple toy problems.
More research would be needed to evaluate how well the EDE approach scales to larger, more complex tasks and to assess its robustness to different types of challenges or environmental changes. The authors also acknowledge that the developmental encoding can be computationally expensive to train, which may limit its practical applicability in certain domains.
Additionally, while the paper discusses the potential for the EDE to produce more open-ended, continuously learning systems, the current work still operates within a fairly constrained set of predefined tasks and environments. Extending this research towards true open-ended evolution and lifelong learning remains an important area for further exploration.
Conclusion
This paper presents a novel approach to meta-learning an "evolvable developmental encoding" for training more adaptable and flexible artificial neural networks. By drawing on insights from developmental biology, meta-learning, and neuroevolution, the researchers have developed a flexible framework for generating neural architectures that can continuously evolve and improve over time.
While the experimental results are promising, significant further research will be needed to fully realize the potential of this approach. Scaling the EDE to larger, more complex domains, enhancing its computational efficiency, and exploring pathways towards genuine open-ended learning and adaptation are all important next steps.
Nonetheless, this work represents an important step towards the development of artificial intelligence systems that can continually grow, learn, and adapt - much like their biological counterparts. As the field of AI continues to advance, techniques like the evolvable developmental encoding may prove crucial for creating truly flexible and capable intelligent systems.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
New!Semantically Rich Local Dataset Generation for Explainable AI in Genomics
Pedro Barbosa, Rosina Savisaar, Alcides Fonseca
0
0
Black box deep learning models trained on genomic sequences excel at predicting the outcomes of different gene regulatory mechanisms. Therefore, interpreting these models may provide novel insights into the underlying biology, supporting downstream biomedical applications. Due to their complexity, interpretable surrogate models can only be built for local explanations (e.g., a single instance). However, accomplishing this requires generating a dataset in the neighborhood of the input, which must maintain syntactic similarity to the original data while introducing semantic variability in the model's predictions. This task is challenging due to the complex sequence-to-function relationship of DNA. We propose using Genetic Programming to generate datasets by evolving perturbations in sequences that contribute to their semantic diversity. Our custom, domain-guided individual representation effectively constrains syntactic similarity, and we provide two alternative fitness functions that promote diversity with no computational effort. Applied to the RNA splicing domain, our approach quickly achieves good diversity and significantly outperforms a random baseline in exploring the search space, as shown by our proof-of-concept, short RNA sequence. Furthermore, we assess its generalizability and demonstrate scalability to larger sequences, resulting in a $approx$30% improvement over the baseline.
7/4/2024
🤖
Evolution and learning in differentiable robots
Luke Strgar, David Matthews, Tyler Hummer, Sam Kriegman
0
0
The automatic design of robots has existed for 30 years but has been constricted by serial non-differentiable design evaluations, premature convergence to simple bodies or clumsy behaviors, and a lack of sim2real transfer to physical machines. Thus, here we employ massively-parallel differentiable simulations to rapidly and simultaneously optimize individual neural control of behavior across a large population of candidate body plans and return a fitness score for each design based on the performance of its fully optimized behavior. Non-differentiable changes to the mechanical structure of each robot in the population -- mutations that rearrange, combine, add, or remove body parts -- were applied by a genetic algorithm in an outer loop of search, generating a continuous flow of novel morphologies with highly-coordinated and graceful behaviors honed by gradient descent. This enabled the exploration of several orders-of-magnitude more designs than all previous methods, despite the fact that robots here have the potential to be much more complex, in terms of number of independent motors, than those in prior studies. We found that evolution reliably produces ``increasingly differentiable'' robots: body plans that smooth the loss landscape in which learning operates and thereby provide better training paths toward performant behaviors. Finally, one of the highly differentiable morphologies discovered in simulation was realized as a physical robot and shown to retain its optimized behavior. This provides a cyberphysical platform to investigate the relationship between evolution and learning in biological systems and broadens our understanding of how a robot's physical structure can influence the ability to train policies for it. Videos and code at https://sites.google.com/view/eldir.
5/28/2024
Toward Artificial Open-Ended Evolution within Lenia using Quality-Diversity
Maxence Faldor, Antoine Cully
0
0
From the formation of snowflakes to the evolution of diverse life forms, emergence is ubiquitous in our universe. In the quest to understand how complexity can arise from simple rules, abstract computational models, such as cellular automata, have been developed to study self-organization. However, the discovery of self-organizing patterns in artificial systems is challenging and has largely relied on manual or semi-automatic search in the past. In this paper, we show that Quality-Diversity, a family of Evolutionary Algorithms, is an effective framework for the automatic discovery of diverse self-organizing patterns in complex systems. Quality-Diversity algorithms aim to evolve a large population of diverse individuals, each adapted to its ecological niche. Combined with Lenia, a family of continuous cellular automata, we demonstrate that our method is able to evolve a diverse population of lifelike self-organizing autonomous patterns. Our framework, called Leniabreeder, can leverage both manually defined diversity criteria to guide the search toward interesting areas, as well as unsupervised measures of diversity to broaden the scope of discoverable patterns. We demonstrate both qualitatively and quantitatively that Leniabreeder offers a powerful solution for discovering self-organizing patterns. The effectiveness of unsupervised Quality-Diversity methods combined with the rich landscape of Lenia exhibits a sustained generation of diversity and complexity characteristic of biological evolution. We provide empirical evidence that suggests unbounded diversity and argue that Leniabreeder is a step toward replicating open-ended evolution in silico.
6/7/2024
📉
Mutation-Bias Learning in Games
Johann Bauer, Sheldon West, Eduardo Alonso, Mark Broom
0
0
We present two variants of a multi-agent reinforcement learning algorithm based on evolutionary game theoretic considerations. The intentional simplicity of one variant enables us to prove results on its relationship to a system of ordinary differential equations of replicator-mutator dynamics type, allowing us to present proofs on the algorithm's convergence conditions in various settings via its ODE counterpart. The more complicated variant enables comparisons to Q-learning based algorithms. We compare both variants experimentally to WoLF-PHC and frequency-adjusted Q-learning on a range of settings, illustrating cases of increasing dimensionality where our variants preserve convergence in contrast to more complicated algorithms. The availability of analytic results provides a degree of transferability of results as compared to purely empirical case studies, illustrating the general utility of a dynamical systems perspective on multi-agent reinforcement learning when addressing questions of convergence and reliable generalisation.
5/29/2024