How morphological development can guide evolution
For the results above, as in nature24, the mutation rate of each voxel was left under evolutionary control (self-adaptation). In an effort to assess the generality of our results, we replicated the experiment described above for various fixed mutation rates (Supplementary Fig. S3). Without development, as in Hinton and Nowlan’s case4, the search space has a single spike of high fitness. One can not do better than random search in such a space. At the highest mutation rate, optimizing Evo morphologies reduces to random search, and this is the only mutation rate where Evo does not require significantly more generations than Evo-Devo to find the faster design. This can be observed in Supplementary Fig. S3 by comparing the generation at which the slopes of the fitness curves increase dramatically. However, the best two treatments, as measured by the highest median speed at the end of optimization, have development, and the robots they produced are significantly faster than those produced by random search (Evo with the highest mutation rate) (p < 0.01).
To further test the sensitivity of our results to the various settings of our particular system, we transcribed the main experiment for a different class of morphologies (rigid bodies) and controllers (neural networks). Details are provided in Supplementary Fig. S5 and Supplementary Methods. The results of this test indicate that differential canalization exists elsewhere, but it does not always increase evolvability.
In these experiments, the intersection of two time scales—slow linear development and rapid oscillatory actuation, as from a central pattern generator—generates positive and negative feedback in terms of instantaneous velocity: the robot speeds up and slows down during various points in its lifetime (Supplementary Fig. S2J). Prior to canalization, unless all of the phenotypes swept over by an individual in development keep the robot motionless, there will be intervals of relatively superior and inferior performance. Evolution can thus improve overall fitness in a descendant by lengthening the time intervals containing superior phenotypes and reducing the intervals of inferior phenotypes. However, this is only possible if such mutations exist.
We have found here that such mutations do exist in cases where evolutionary changes to one trait do not disrupt the successful behavior contributed by other traits. For example, robots that exhibited the locally optimal trotting behavior (Fig. 2A) exhibited a tight coupling between morphology and control, and thus evolution was unable to canalize development in either one, since mutations to one subsystem tended to disrupt the other. Brief ontogenetic periods of rolling behavior (Fig. 2C), on the other hand, could be temporally extended by evolution through canalization of the morphology alone (Fig. 2D), since these morphologies are generally robust to the pattern of actuation. The key observation here is that only phenotypic traits that render the agent robust to changes in other traits become assimilated, a phenomenon we term differential canalization.
This insight was exposed by modeling the development of simulated robots as they interacted with a physically realistic environment. Differential canalization may be possible in disembodied agents as well, if they conform to appropriate conditions described in Supplementary Discussion.
This finding of differential canalization has important implications for the evolutionary design of artificial and embodied agents such as robots. Computational and engineered systems generally maintain a fixed form as they behave and are evaluated. However, these systems are also extremely brittle when confronted with slight changes in their internal structure, such as damage, or in their external environment such as moving onto a new terrain25–27. Indeed, a perennial problem in robotics and AI is finding general solutions which perform well in novel environments28,29. Our results demonstrate how incorporating morphological development in the optimization of robots can reveal, through differential canalization, characters which are robust to internal changes. Robots that are robust to internal changes in their controllers may also be robust to external changes in their environment13. Thus, allowing robots to change their structure as they behave might facilitate evolutionary improvement of their descendants, even if these robots will be deployed with static phenotypes or in relatively unchanging environments.
These results are particularly important for the nascent field of soft robotics in which engineers cannot as easily presuppose a robot’s body plan and optimize controllers for it because designing such machines manually is unintuitive18,30. Our approach addresses this challenge, because differential canalization provides a mechanism whereby static yet robust soft robot morphologies may be automatically discovered using evolutionary algorithms for a given task environment. Furthermore, future soft robots could potentially alter their shape to best match the current task by selecting from previously trained and canalized forms. This change might occur pneumatically, as in Shepherd et al.31, or it could modulate other material properties such as stiffness (e.g. using a muscular hydrostat).
We have shown that for canalization to occur in our developmental model, some form of paedomorphosis must also occur. However, there are at least two distinct methods by which such heterochrony can proceed: progenesis and neoteny. Progenesis could occur through mutations which move initial parameter values (ℓ,ϕ) toward their final values (ℓ⁎,ϕ⁎). Neoteny could instead occur through mutations which move final values (ℓ⁎,ϕ⁎) toward their initial values (ℓ,ϕ). Although a superior phenotype can materialize anywhere along the ontogenetic timeline, late onset mutations are less likely to be deleterious than early onset mutations. This is because our developmental model is linear in terms of process, and interfering with any step affects all temporally-downstream steps. Since the probability of a mutation being beneficial is inversely proportional to its phenotypic magnitude32, mutational changes in the terminal stages of development require the smallest change to the developmental program. Hence, late-onset discoveries of superior traits are more likely to occur without breaking functionality at other points in ontogeny, and these traits can become canalized by evolution through progenesis: mutations which reduce the amount of ontogenetic time prior to realizing the superior trait (by moving ℓ→ℓ⁎ and/or ϕ→ϕ⁎). Indeed progenesis was observed most often in our trials (Fig. 6): late onset mutations which transform a walking robot into a rolling one are discovered by the evolutionary process, and are then moved back toward the birth of the robots’ descendants through subsequent mutations.
Finally, we would like to note the observed phenomenon of increased plasticity prior to genetic assimilation. Models of the Baldwin effect usually assume that phenotypic plasticity itself does not evolve, although it has been shown how major changes in the environment can select for increased plasticity in a character that is initially canalized33. In our experiments however, there is no environmental change. There is also a related concept known as ‘sensitive periods’ of development in which an organism’s phenotype is more responsive to experience34. Despite great interest in sensitive periods, the adaptive reasons why they have evolved are unclear35. In our model, increasing the amount of morphological development increases the chance of capturing an advantageous static phenotype, which can then be canalized, once found. However, a phenotype will not realize the globally optimal solution by simply maximizing development. This would merely lengthen the line on which development unfolds in phenotypic hyperspace (n-dimensional real space).
The developmental model described herein is intentionally minimalistic in order to isolate the effect of morphological and neurological change in the evolutionary search for embodied agents. The simplifying assumptions necessary to do so make it difficult to assess the biological implications. For example, we model development as an open loop process and thus ignore environmental queues and sensory feedback36,37. We also disregard the costs and constraints of phenotypic plasticity38,39. By removing these confounding factors, we hope these results will help generate novel hypotheses about morphological development, heterochrony, modularity and evolvability in biological systems.
Ballistic development ℬ(t) is simply a linear function from a starting value a to a final value b, in ontogenetic time t∈(0,τ),
Because a and b are constants set by the genotype and τ=10 (sec) is a fixed hyperparameter, development is predetermined, monotonic and irreversible—in a word: ballistic.
For smaller voxels, it is necessary to implement damping into their actuation to avoid simulation instability. Actuation amplitude is limited by a linear damping factor ξ(x)=min{1,(4x−1)/3}, which only affects voxels with resting length less than one centimeter, and approaches zero (no actuation) as the resting length goes to its lower bound of 0.25 cm.
Actuation of the k-th voxel ψk(t) therefore depends on the starting and final phase offsets (ϕk,ϕk⁎) for relative displacement, and on the starting and final resting lengths (ℓk,ℓk⁎) for amplitude. With maximum amplitude A = 0.14 cm and a fixed frequency f = 4 Hz, we have:
The current length of the k-th voxel at time t, denoted by ℒk(t), is the resting length plus the offset and damped signal ψk(t).
Current length is broken down into its constituent parts for a single voxel, under each treatment, in Supplementary Fig. S2.
We employed a standard evolutionary algorithm, Age-Fitness-Pareto Optimization40, which uses the concept of Pareto dominance and an objective of age (in addition to fitness) intended to promote diversity among candidate designs. A single evolutionary trial maintains a population of thirty robots, for ten thousand generations. Every generation, the population is first doubled by creating modified copies of each individual in the population. The age of each individual is then incremented by one. Next, an additional random individual (with age zero) is injected into the population (which now consists of 61 robots). Finally, selection reduces the population down to its original size (30 robots) according to the two objectives of fitness (maximized) and age (minimized).
We performed the above procedure thirty times to produce thirty independent evolutionary trials. That the number of trials is the same as the population size within each trial is an admittedly confusing coincidence.
The mutation rate is also evolved for each voxel, independently, and slightly modified every time a genotype is copied from parent to child. These 48 independent mutation rates are initialized such that only a single voxel is mutated on average. Mutations follow a normal distribution (σℓ=0.75cm;σϕ=π/2) and are applied by first selecting what parameter types to mutate (ϕk,ϕk⁎,ℓk,ℓk⁎), and then choosing, for each parameter separately, which voxels to mutate. In Supplemental Materials we provide exact derivations of the expected genotypic impact of mutations, in terms of the proportions of voxels and parameters modified, for a given fixed mutation rate λ. There is a negligible difference between Evo and Evo-Devo in terms of the expected number of parent voxels modified during mutation (Supplementary Fig. S4).
The amount of development in a particular voxel can range from zero (in the case that starting and final values are equal) to 1.5 cm for the morphology (which ranges from 0.25 cm to 1.75 cm) and π for the controller (which ranges from −π/2toπ/2). The morphological development window, WL, is the sum of the absolute difference of starting and final resting lengths across the robot’s 48 voxels, divided by the total amount of possible morphological development.
The controller development window, WΦ, is the sum of the absolute difference of starting and final phase offsets across the robot’s 48 voxels, divided by the total amount of possible controller development.
We used the Mann-Whitney U test to calculate the p-values reported in this paper.
This work was supported by Army Research Office award W911NF-16-1-10304 and DARPA contract HR0011-18-2-0022. The computational resources provided by the Vermont Advanced Computing Core are gratefully acknowledged.
S.K. conducted the experiments and wrote the manuscript. All authors conceived the experiments, analyzed the results and reviewed the manuscript.
github.com/skriegman/how-devo-can-guide-evo contains the source code necessary for reproducing the results reported in this paper.
github.com/skriegman/how-devo-can-guide-evo contains the source code necessary for reproducing the results reported in this paper.