How morphological development can guide evolution

Organisms result from adaptive processes interacting across different time scales. One such interaction is that between development and evolution. Models have shown that development sweeps over several traits in a single agent, sometimes exposing promising static traits. Subsequent evolution can then canalize these rare traits. Thus, development can, under the right conditions, increase evolvability. Here, we report on a previously unknown phenomenon when embodied agents are allowed to develop and evolve: Evolution discovers body plans robust to control changes, these body plans become genetically assimilated, yet controllers for these agents are not assimilated. This allows evolution to continue climbing fitness gradients by tinkering with the developmental programs for controllers within these permissive body plans. This exposes a previously unknown detail about the Baldwin effect: instead of all useful traits becoming genetically assimilated, only traits that render the agent robust to changes in other traits become assimilated. We refer to this as differential canalization. This finding also has implications for the evolutionary design of artificial and embodied agents such as robots: robots robust to internal changes in their controllers may also be robust to external changes in their environment, such as transferal from simulation to reality or deployment in novel environments.

The shape of life changes on many different time scales. From generation to generation, populations gradually increase in complexity and relative competency. At the individual level, organisms grow from a single-celled egg and exhibit extreme postnatal change as they interact with the outside world during their lifetimes. At a faster time scale still, organisms behave such as to survive and reproduce.

Many organisms manifest different traits as they interact with their environment. It seems wasteful not to utilize this extra exploration to speed the evolutionary search for good genotypes. However, to communicate information from these useful but temporary traits to the genotype requires inverting the generally very complex, nonlinear and stochastic mapping from DNA to phenotype. Inverting such a function would be exceedingly difficult to compute. Organisms can, however, pass on their particular capacity to acquire certain characteristics. Thus phenotypic plasticity can affect the direction and rate of evolutionary change by influencing selection pressures. Although this phenomenon was originally described by Baldwin1, Morgan2 and Waddington3, among others, it has become known as ‘the Baldwin effect’. In Baldwin’s words: ‘the most plastic individuals will be preserved to do the advantageous things for which their variations show them to be the most fit, and the next generation will show an emphasis of just this direction in its variations’1. In a fixed environment, when the ‘advantageous thing’ to do is to stay the same, selection can favor genetic variations which more easily, reliably, or quickly produce these traits. This can lead to the genetic determination of a character which in previous generations needed to be developed or learned.

Thirty years ago, Hinton and Nowlan4 provided a simple computational model of the Baldwin effect that clearly demonstrated how phenotypic plasticity could, under certain conditions, speed evolutionary search without communication to the genotype. They considered the evolution of a bitstring that is only of value when perfectly matching a predefined target string. The search space therefore has a single spike of high fitness with no slope leading to the summit. In such a space, evolution is no better than random search.

Hinton and Nowlan then allowed part of the string to randomly change at an additional (and faster) developmental time scale. When the genetically specified (nonplastic) portion of the string is correct, there is a chance of discovering the remaining portion in development. The speed at which such individuals tend to find the good string will be proportional to the number of genetically determined bits. When the target string is found, development stops and the individual is rewarded for the amount of remaining developmental time. This has the effect of creating a gradient of increasing fitness surrounding the correct specification that natural selection can easily climb by incrementally assimilating more correct bits to the genotype.

Hinton and Nowlan imagined the bitstring as specifying the connections of a neural network in a very harsh environment. We are also interested in this interaction of subsystems unfolding at different time scales, but consider an embodied agent situated in a physically-realistic environment rather than an abstract control system. This distinction is important as it grounds our hypotheses in the constraints and opportunities afforded by the physical world. It also allows us to investigate how changes in morphology and control might differentially affect the direction or rate of evolutionary search. More specifically, it exposes the previously unknown phenomenon of differential canalization reported here.

Inspired by Hinton and Nowlan, Floreano and Mondada5 explored the interaction between learning and evolution in mobile robots with a fixed body plan but plastic neural control structure. They noted that the acquisition of stable behavior in ontogeny did not correspond to stability (no further change) of individual synapses, but rather was regulated by continuously changing synapses which were dynamically stable. In other words, agents exploited this ontogenetic change for behavior, and this prevented its canalization. In this paper, we structure development in a way that restricts its exploitation for behavior and thus promotes the canalization of high performing static phenotypes. Also, the robot’s body plan was fixed in Floreano and Mondada’s experiments5, whereas in the work reported here, evolution and development may modify body plans.

Several models that specifically address morphological development of embodied agents have been reported in the literature6–10. However, the relationship between morphological development and evolvability is seldom investigated in such models. Moreover, there are exceedingly few cases that considered postnatal change to the body plan of an agent (its resting structural form) as it behaves and interacts with the environment through physiological functioning (at a faster time scale).

We are only aware of four cases reported in the literature in which a simulated robot’s body was allowed to change while it was behaving. In the first two cases11,12, it was not clear whether this ontogenetic morphological change facilitated the evolution of behavior. Later, Bongard13 demonstrated how such change could lead to a form of self-scaffolding that smoothed the fitness landscape and thus increased evolvability. This ontogenetic change also exposed evolution to a wider range of sensor-motor contingencies, which increased robustness to novel environments. More recently, Kriegman et al.14 showed how development can sweep over a series of body plans in a single agent, and subsequent heterochronic mutations canalize the most promising body plan in more morphologically-static descendants.

We are not aware of any cases reported in the literature to date in which a simulated robot’s body and control are simultaneously allowed to change while it is behaving. In this paper, we investigate such change in the morphologies and controllers of soft robots as they are evolved for coordinated action in a simulated 3D environment. By morphology we mean the current state of a robot’s shape, which is slowly changed over the course of its lifetime by a developmental process. We distinguish this from the controller, which sends propagating waves of actuation throughout the individual, which also affects the instantaneous shape of the robot but to a much smaller degree. We here refer to these two processes as ‘morphology’ and ‘control’. As both processes change the shape, and thus behavior, of the robot, this distinction is somewhat arbitrary. However, the central claim of this paper, which is that some traits become canalized while others do not, is not reliant on this distinction.

We use soft robots because they provide many more degrees of morphological freedom compared to traditional robots composed of rigid links connected by rotary or linear actuators. This flexibility allows soft robots to accomplish tasks that would be otherwise impossible for their rigid-bodied counterparts, such as squeezing through small apertures15 or continuously morphing to meet different tasks. Recent advancements in materials science are enabling the fabrication of 3D-printed muscles16 and nervous systems17. However, there are several challenges to the field of soft robotics, including an overall lack of design intuition: What should a robot with nearly unbounded morphological possibility look like, and how can it be controlled? Controllability often depends on precision actuation and feedback authority, but these properties are difficult to maintain in soft materials in which motion in one part of the robot can propagate in unanticipated ways throughout its body18.

We present here a minimally complex but embodied model of morphological and neurological development. This new model represents an alternative approach to the challenging problem of soft robot design and presents an in silico testbed for hypotheses about evolving and developing embodied systems. This model led to the discovery of differential canalization and how it can increase evolvability.

We consider a locomotion task, over flat terrain, for soft robots composed of a 4 × 4 × 3 grid of voxels (Fig. 1). Robots are evaluated for 40 actuation cycles at 4 Hz, yielding a lifetime of ten seconds. Fitness is taken to be distance traveled measured in undeformed body lengths (four unit voxels, i.e. 4 cm). Example robots are shown in Figs 1 and 2, and Supplementary Video S1.

All experiments were performed in the open-source soft-body physics simulator Voxelyze19. Voxelyze simulates soft materials using two elements: particles and beams. A particle is a point mass with rotational inertia. A spring-like beam (with translational and rotational stiffness) connects two adjacent particles. Each particle is connected to at most six neighbors (above, below, front, back, left, and right), on a cartesian grid. Local material properties are stored at particles and averaged across shared beams. Finally, a voxel mesh is drawn around each particle for visualization, such that adjacent voxels touch at the center of their shared beam. More details about how this is actually implemented are given by Hiller and Lipson19.

The morphology of a robot is given by the resting (beam) length stored at each voxel (Fig. 1). However the shape and volume of each voxel is changed by external forces from the environment and internal forces via behavior. The morphology of a robot is denoted by the 4 × 4 × 3 = 48-element vector ℓ, where each element is the resting length stored at that voxel (with possible values within 1.0 ± 0.75 cm). Like most animals, our robots are bilaterally symmetrical. We built this constraint into our robots because bilateral symmetry is known to help with forward locomotion20. The lefthand 2 × 4 × 3 = 24 resting voxel lengths are reflected on the other, righthand side of the midsagittal line, yielding 24 independent resting lengths.

The controller, however, is not constrained to be symmetrical since many behaviors, even for symmetric morphologies, consist of asymmetric gaits, and is given by the phase offset of each voxel from a global oscillating signal with an amplitude of 0.14 cm. The controller is denoted by the 48-element vector ϕ, where each element is the phase offset of that voxel (with possible values within 0 ± π/2).

We investigated the impact of development in this model by comparing two experimental variants: Evo and Evo-Devo (schematized in Supplementary Fig. S2). The control treatment, Evo, lacks development and therefore maintains a fixed morphology and control policy in a robot as it behaves over its lifetime. Two parameters per voxel are sufficient to specify an evolved robot at any time t in its lifetime: its morphology ℓk, and controller ϕk. An evolutionary algorithm optimizes 24 morphological and 48 control parameters.

The experimental treatment Evo-Devo evolves a developmental program rather than a static phenotype (Fig. 1). For each parameter in an Evo robot, an Evo-Devo robot has two: its starting and final value. The evolutionary algorithm associated with the Evo-Devo treatment thus optimizes 48 morphological and 96 control parameters. The morphology and controller of the k-th voxel change linearly from starting to final values, throughout the lifetime of a developing robot. The endpoint parameters are denoted by asterisks: the controller develops from ϕ to ϕ⁎, the morphology develops from ℓ to ℓ⁎. The starting and final points of development are predetermined by a genome which in turn fixes the direction (compression or expansion) and rate of change for each voxel. Development is thus ballistic in nature rather than adaptive, as it cannot be influenced by the environment (Equation 1).

For both treatments we conducted 30 independent evolutionary trials. At the end of evolutionary optimization, the non-developmental robots (Evo) tend to move on average with a speed of 10 body lengths in 10 seconds, or 1 length/sec. The evolved and developing robots (Evo-Devo) tend to move at over 5 lengths/sec (Fig. 3A). To ensure evolved and developing robots are not exploiting some unfair advantage conferred by changing body plans and control policies unavailable to non-developmental robots, we manually remove their development by setting ℓ⁎=ℓ and ϕ⁎=ϕ, which fixes the structure of their morphologies and controllers at birth (t=0) (Equation 1). That is to say, we convert the evolved Evo-Devo robots into Evo robots (Equation 1 reduces to Equation 2). The resulting reduced robots suffer only a slight (and statistically non-significant) decrease in median speed and still tend to be almost five times faster than the systems evolved without development (Fig. 3B, treatment ‘Evo-Devo removed’). Ballistic development is therefore beneficial for search but does not provide a behavioral advantage in this task environment.

To investigate this apparent search advantage, we trace development and fitness across the 30 lineages which produced a ‘run champion’: the robot with highest fitness at the termination of a given evolutionary trial (Supplementary Fig. S1). We measure the amount of ballistic change in each robot—its ‘ballistic plasticity’—by a statistic we call the developmental window. The developmental window is defined separately for morphology (Equation 5) and control (Equation 6) as the absolute difference in starting and final values summed across the robot and divided by the total amount of possible development, such that 0 and 1 indicate no and maximal developmental change, respectively. Evo robots by definition have development windows of zero, as do Evo-Devo robots that have had development manually removed. An Evo-Devo robot with a small developmental window has thus become canalized3.

In terms of fitness, there were two observed basins of attraction in average velocity: a slower design type which either trots or gallops at a speed of less than 1 length/sec (Figs 2A,B and 4A∗), and a faster design type that rolls at 5–6 lengths/sec (Fig. 2C). After ten thousand generations, 25 out of a total of 30 Evo-Devo trials (83.3%) find the faster design, compared to just 6 out of 30 Evo trials (20%).

Modular systems are more evolvable than non-modular systems because they allow evolution to improve one subsystem without disrupting others21,22. Modularity may be a property of the way a system is built, or it may be an evolved property. The robots evolved here are by definition modular because the genes which affect morphology are independent of those which affect its control. However the more successful Evo-Devo lineages evolved an additional form of modularity, which we term differential canalization: Some initially developmentally plastic traits become integrated and canalized, while other traits remain plastic.

In the successful Evo-Devo trials, morphological traits were canalized while control traits were not. Evidence for this is provided in Supplementary Fig. S1, which is summarized by Fig. 4A. Trajectories of controller development (green curves) do not follow any discernible pattern in phylogenetic time, and appear upon visual inspection to be consistent with a random walk or genetic drift. The trajectories of morphological development (red curves), however, follow a consistent pattern. The magnitude of morphological development increases slightly, but significantly (p < 0.001), before decreasing all the way to the most recent descendant, which is the most fit robot from that trial (the run champion). Run champions tend to have much less morphological development than their most distant ancestor (p < 0.001), but there is not a significant difference between champion and ancestral controller windows. Furthermore, this pattern tends to correlate with high fitness: in trials in which this pattern did not appear (runs 6, 8, 16–18), fitness did not increase appreciably over evolutionary time.

This process within the lineages of the run champions is consistent with a more general correlation found in all designs explored during optimization across all runs: Individuals with the highest fitness values tend to have very small amounts of morphological development, while their control policies are free to develop (Fig. 4B). However, despite the fact that morphological development tends to be canalized in the most fit individuals, it cannot simply be discarded as the non-developmental systems have by definition small morphological windows, and small controller windows, but also low fitness.

To test the sensitivity of the evolved morphologies to changes in their control policies, we applied a random series of control mutations to the Evo and Evo-Devo run champions from each evolutionary trial. For each run champion, we perform 1000 subsequent random controller mutations that build upon each other in series (a Brownian trajectory in the space of controllers)—and repeat this process ten times for each run champion, each with a unique random seed. It was found that optimized Evo-Devo robots tend to possess body plans that are much more robust to control mutations than those of Evo robots (Fig. 5A). The first control mutation to optimized Evo robots tends to immediately render them immobile, whereas optimized Evo-Devo robots tend to retain most of their functionality even after 1000 successive random changes to their controllers. Within Evo-Devo designs, the functionality of the 25 fast designs are minimally affected by changes to their control, whereas the five slow designs also tend to break after the first control mutation (Fig. 5B). Thus it can be concluded that these five robots are non-modular: their non-canalized morphologies evolved a strong dependency on their controllers. The Evo robots are similarly non-modular: they are brittle to control mutations.

To test the sensitivity of the evolved controllers to changes in their morphologies, we applied the same procedure described in the previous paragraph but with random morphological mutations rather than control mutations. It was found that both developmental and non-developmental systems tend to evolve controllers that are very sensitive to morphological mutations (Fig. 5C). These findings are consistent with those of Cheney et al.23, who also reported that robots were more sensitive to changes in their morphology than in their controllers. Here, the first few morphological mutations to optimized robots, in both treatments, tend to immediately render them immobile. Within Evo-Devo design types, neither of which canalized development in their controllers (Supplementary Fig. S1), both the fast and slow designs possess controllers sensitive to changes in their morphologies (Fig. 5D). Thus it can be concluded that the non-canalized controllers evolved a strong dependency on their morphologies. Therefore the only trait to be successfully canalized was also the only trait that rendered the agent robust to changes in other traits.

The evolutionary algorithm can rapidly discover an actuation pattern that elicits a very small amount of forward movement in these soft robots regardless of the morphology. There is then an incremental path of increasing locomotion speed that natural selection can climb by gradually growing legs to reduce the surface area touching the floor and thus friction, and simultaneously refining controller actuation patterns to better match and exploit the morphology (Fig. 2A,B).

There is, however, a vastly superior design partially hidden from natural selection—a ‘needle in the haystack’, to use Hinton and Nowlan’s metaphor4. On flat terrain, rolling can be much faster and more efficient than walking, but finding such a design is difficult because the fitness landscape is deceptive. Rolling over once is much less likely to occur in a random individual than shuffling forwards slightly. And as a population continues to refine walking morphologies and gaits, lineages containing rocking individuals which are close to rolling over, or roll over just once, do not survive long enough to eventually produce a true rolling descendant.

Development can alter the search space evolution operates in because individuals sweep over a continuum of phenotypes, with different velocities, rather than single static phenotype that travels at a constant speed (Supplementary Fig. S2E,J). The lineages which ultimately evolved the faster rolling design initially increased their morphological plasticity in phylogenetic time as evidenced by the initial upward trends in the red curves in Supplementary Fig. S1 (summarized by Fig. 4A) which contain a statistically significant difference between their starting and maximum developmental windows (p < 0.001). This exposes evolution to a wider range of body plans and thus increases the chance of randomly rolling at least once at some point during the evaluation period.

The peak of morphological plasticity in Supplementary Fig. S1 (summarized by Fig. 4A) generally lines up with the start of an increasing trend in fitness (blue curves) and marks the onset of differential canalization. Rolling just once allows an individual to move further (1 body length) than some early walking behaviors but they incur the fitness penalty of having fallen over and thus not being able to subsequently walk for the rest of the trial. Therefore this tends to happen at the very end of ontogeny (Fig. 6), as individuals evolve to ‘dive’ in the last few time steps of the simulation of their behavior, thus incurring an additional increase of fitness over their parent, which does not exhibit this behavior. Since more rolling incurs more fitness than less rolling, a form of progenesis occurs as heterochronic mutations move ℓk closer to ℓk⁎, for each voxel. This gradually earlifies rolling from a late onset behavior to one that arises increasing earlier in ontogeny (Supplementary Video S1). As more individuals in the population discover and earlify this rolling behavior, the competition stiffens until eventually individuals which are not born rolling from the start are not fast enough to compete (Fig. 2C,D).