Evolutionary Connectionism: Algorithmic Principles Underlying the Evolution of Biological Organisation in Evo-Devo, Evo-Eco and Evolutionary Transitions

Importantly, the reason that a new connection is made is not, and cannot be, because this connection will, at some point in the future, provide jumps in configuration space that are adaptive; a benefit that has not yet happened cannot be the reason the connection was made. The connections that are made simply canalise the co-occurrence of variables that already co-occur (and are therefore close to neutral at the time they are made). Over a distribution of local optima, the connections that are most robust (most often neutral) are retained by the learning algorithm and those that are less robust (occasionally deleterious) are removed. This deep optimisation technique thus exploits the unsupervised correlation learning principle of deep learning: connections that canalise existing correlations reduce the dimensionality of the problem space in a way that is effective in enabling adaptive jumps in configuration space (or movements in a higher-level representation of the problem), even though those higher-level jumps have not yet occurred.

Together these works show that learning can improve optimisation. This can be achieved by using learning to bias selection, i.e. altering the effective energy function (or the fitness function) that controls the dynamics of the system. Or it can be achieved by using learning to bias the movements that are sampled in configuration space, e.g. by exploiting modularity or, in particular, by collapsing subsets of variables into higher-level emergent variables (Watson et al. 2011b). The deep optimisation provided by changing the generate function has optimisation capabilities that cannot be exhibited by changing the selection function. Next we argue that these different types of connectionist learning are implemented in the evolution of different types of biological organisations, and that they have analogous consequences for the adaptive capabilities of evolution by natural selection.

We argue that specific but different types of correlation learning are directly relevant to understanding the evolution of developmental and ecological organisations, and to understanding how they work together with the evolution of reproductive organisations in the evolutionary transitions.

The evolution of developmental and ecological organisations are very different from each other because of the level of organisation that constitutes the evolutionary unit in these two cases. Developmental networks, controlling the combinations of features that co-occur in phenotypes, are selected as a single evolutionary unit; selection applies at the level of the whole network and changes in the organisation of the network can be selected because of the differences they confer on the phenotypes that are produced. This is the analogue of reinforcement correlation learning. In contrast, ecological networks are not evolutionary units, and when we talk about the evolution of the ecological network we just mean the changes, caused by evolution, that occur as a consequence of natural selection acting on the composition of each species within the ecological community individually not as a composite unit (in short, no group selection here). Because selection occurs at the individual level and not the system level, this is the analogue of unsupervised correlation learning—selection does not act to increase collective fitness (in general) but in acting to increase individual fitness (by changing community interaction terms) it can have the (side-)effect of stabilising the current ecological composition or increasing ecological robustness. Interestingly, the observation that selection for robustness at the system level also follows unsupervised learning principles seems to imply an underlying equivalence between selection for robustness at the system level and selection that acts on components below the system level—e.g. selection that favours modularity or features that are independently beneficial is closely related to selection that favours robustness at the system level.

But importantly, system robustness produced by changes to interactions does not simply remove variability, but shapes or directs variability at the system level—in the same way that unsupervised correlation learning mimics correlations in the input.

Our work described below suggests that the evolution of developmental and ecological interactions are both analogous to self-modelling systems that alter the selection function—but whereas changing ecological interactions explicitly alters selection on evolutionary units by altering the relationships they have with other units (ecological relationships between species in the community), changing developmental interactions alters ‘selection’ on components that are internal to the evolutionary unit (e.g. on gene-expression potentials within the organism). The latter has the effect of biasing the phenotypic variation that is produced at the system level, i.e. by changing the co-occurrence of traits in the phenotype.

The evolution of reproductive relationships is not captured by any one level of evolutionary unit—neither the system as a whole nor the individual components. Rather, the evolution of reproductive relationships is precisely concerned with changes in the evolutionary unit—changes that convert multiple evolutionary units at one level of organisation into a new evolutionary unit at a higher level of organisation. These changes must be driven by the lower level evolutionary unit—the higher level unit cannot be driving the evolutionary process before it exists (Ryan et al. 2015). We argue that this is analogous to the incremental addition of new layers of organisation in deep learning (combining unsupervised and supervised correlation learning mechanisms applied at successive scales of organisation) or to the rescaling of the optimisation process as per the distributed deep optimisation models. This biases the combinations of particles that are created in a more radical (multi-scale) manner than the evolution of developmental interactions (Watson et al. 2011b). Specifically, whereas developmental organisations bias phenotypic variability (by recreating specific phenotypic patterns through the organisation of internal selection or context-sensitive differential growth between components) (Laland et al. 2014), reproductive organisations can bias genetic variability (by enabling the combination of genetic differences in a collective to be inherited to descendent collectives as a unit and supressing internal differential selection between them).

Accordingly, although evo-devo, evo-eco and evo-ego domains each have their own special characteristics, there are methodological reasons to study them as a set:They have common underlying principles that can be captured by adaptive networks. In particular, a positive feedback between topology and behaviour (i.e. correlated behaviours result in the evolution of stronger network interactions which in turn produce more strongly correlated behaviours, and so on).Learning theory offers detailed knowledge about the consequences of this type of positive feedback in networks, namely correlation learning—offering unifying principles for studying the evolution of these three classes of biological networks.Learning theory also offers well-studied variants of correlation learning (reinforcement correlation learning, unsupervised correlation learning and deep correlation learning) that apply to developmental, ecological and reproductive organisations, respectively—offering principles that help us understand the algorithmic distinctions between the three cases.

Studying them as a set is also motivated by their complementary roles in the major evolutionary transitions:(4)Taken together, developmental, ecological and reproductive organisations cover the evolution of the necessary and sufficient components of the Darwinian machine, namely, variation, selection and inheritance, respectively.(5)The three organisations are complementary in the level of evolutionary unit they address: evolution of a network as a single evolutionary unit (evo-devo), evolution of multiple evolutionary units within a network (evo-eco), and evolution that changes the evolutionary units (subsets of nodes become single nodes) (evo-ego).(6)Together they control how information about the selective environment (e.g. what things ‘go together’) is broken-up into smaller pieces (evolution of developmental modularity or ecological independence/community structure) and put together into larger systems (evolution of developmental and ecological dependencies), and ultimately re-scaled to enable evolutionary adaptation to be reinstantiated at a higher level of biological organisation (via evolutionary transitions).

A summary of the unifying themes and their relation to principles of connectionist learning is given in Table 1.Table 1Outline of a connectionist theoretical framework for the evolution of developmental, ecological and reproductive organisationsThe simple principle of positive feedback between behaviour on a network and changes in network topology, known as correlation learning in neural networks, aka. “Neurons that fire together wire together”, is analogous to the evolution of developmental, ecological and reproductive organisations

In the following sections we briefly discuss how our work using this framework is beginning to answer the motivating questions in evo-devo and evo-eco. We then discuss our preliminary work on the evolution of new evolutionary units and major transitions in more detail.

Rupert Riedl, one of the founding pioneers of the field we now know as evo-devo, suggested that body plans and developmental constraints evolve to mimic the structure of the constraints imposed on the phenotype by the environment (Riedl 1977; Wagner and Laubichler 2004). Only recently has theoretical work caught-up with this intuitive idea. Toussaint and von Seelen (2007) show that natural selection necessarily favours phenotypic variation distributions that are structurally similar to the pattern of selection. Pavlicev et al. (2011) analyse the population genetics of selection on an allele controlling the correlation of two phenotypic traits. They show that it evolves under natural selection to align phenotypic variation with the direction of selection, such that if two traits are selected together positive developmental correlations evolve, and if one is selected for when the other is selected against negative developmental correlations evolve. This is functionally equivalent to reinforcement correlation learning. In larger systems we see the same pattern occurring in work where developmental interactions evolve modularity that to mimics the modular structure of variation in a selective environment (Lipson et al. 2002; Watson et al. 2014; Kashtan et al. 2009).

Building on these observations, our work shows that the regulatory interactions of gene-regulation networks evolve under natural selection in exactly the same manner as correlation learning modifies the synaptic connections of a neural network (Watson et al. 2014). This enables us to show that gene-regulation networks can exhibit a “developmental memory” able to store and recall multiple phenotypes that have been selected for in the past, exactly like the associative memory of a Hopfield network. Using knowledge of how correlation learning can generalise over a set of training patterns, we also showed that gene-networks can produce novel phenotypes that have not been selected for in the past but have common structural regularities (e.g. new combinations of phenotypic modules). This provides a more formal basis with which to understand Parter et al.’s results (2008) showing that genotype-phenotype maps can generalise from past environments to facilitate evolvability in novel environments.

The analogy between generalisation in learning systems and evolvability in novel environments has considerable technical depth. For example, generalisation in learning systems is not mysterious, and accordingly, the evolution of evolvability that facilitates long-term adaptation in previously unseen environments is possible (Watson et al. 2010a; Kounios et al., in prep.). But neither is generalisation for granted in learning systems, and learning theory can help us understand the capabilities and limitations of such generalisation and evolvability in evolved systems. In particular, in learning systems, overfitting occurs when improved performance on the training set decreases generalisation on the test set. This is the analogue of past selection that fails to facilitate future evolvability. However, conditions that alleviate overfitting and improve generalisation in machine learning, such as the application of a parsimony pressure that favours simple models, also confer better generalisation in evolved G-P maps (e.g. applied via a cost of connections; Clune et al. 2013) (Kouvaris et al. 2015).

Evolutionary processes exhibit different outcomes in structured populations than they do in freely-mixed populations. Often, this is modelled with games played on networks (Jackson and Zenou 2014). Recently there has been a rapid expansion of work investigating what happens when agents on a network have behaviours that can alter the topology of the network (Pacheco et al. 2006; Traulsen et al. 2008; Van Segbroeck et al. 2010; Santos et al. 2006), i.e. adaptive networks (Gross and Sayama 2009). Different scenarios involve different assumptions about exactly how agents modify their connections to others, but most cases are consistent with the assumption that they change connections to maximise their individual utility (e.g., leaving connections unchanged when they are winning, and re-wiring connections when they are losing; Nowak and Sigmund 1993; Santos et al. 2006).

Usually these models are intended to represent the interaction structure of a single population where nodes represent individuals and edges represent fitness-affecting interactions between individuals. If instead, we utilise a network model to represent ecological interactions, then nodes represent different species and edges represent ecological fitness interactions between species. When the network represents a single population, the reproduction of an individual at one node may replace the individual of a connected node (because they are members of the same species). But in an ecological network this is not the case; reproduction of individuals simply modifies the strategy of the species at that node (and/or the connections the species has with others). We have shown that a games-on-networks model of this type behaves exactly like unsupervised correlation learning (Davies et al. 2011; Watson et al. 2010a). Moreover, since the total community welfare of a social attractor is correlated with the basin size of the attractor, the changes to connections produced under an individual utility-maximisation principle have the emergent effect of increasing global community welfare (collective fitness) without selection at the system level (Davies et al. 2011; Watson et al. 2010a)—as per a self-modelling dynamical system.

We can also model the evolution of ecological organisations in a more conventional Lotka-Volterra system where a community matrix represents the network of ecological interactions (Poderoso and Fontanari 2007; Wilson 1992). We then let individuals within each species carry traits that can modify their ecological interactions with others under individual selection (i.e., in whatever manner increases their representation within their own species). This work shows that interactions evolve according to the principles of unsupervised correlation learning. This is consistent with character displacement (Brown and Wilson 1956; Dayan and Simberloff 2005) which alleviates competitive interactions or reinforces mutualistic interactions in proportion to the contact a pair of species has experienced. Accordingly, an ecosystem can form a distributed associative memory of past environmental conditions, storing and recalling multiple community composition patterns in the resultant population dynamics, despite the fact that there is no selection for such behaviour at the system level (Power et al. 2015). This suggests a significant expansion for the role of ecological memory observed in natural ecosystems (Thompson et al. 2001) and provides the possibility of being able to characterise the conditions under which eco-evo dynamics lead to self-regulation/homeostasis rather than self-destruction (Lenton 2004; Lenton and van Oijen 2002).

Conventionally, it is assumed that the products of the Darwinian machine do not alter how the machine operates, i.e. that fixed mechanisms of variation and selection are applied to a fixed reproductive unit. Above we have discussed how changes to developmental organisation and ecological organisation can alter variation and selection. But in an evolutionary transition, all three of these component mechanisms (i.e., variation, selection and inheritance) are transformed or re-created at a higher level of organisation. This results in a Darwinian machine that operates via heritable variation in reproductive success at a higher level of biological organisation. There are many complex issues involved, and the details are different in different types of transition. Nonetheless, we discuss how the basic positive feedback principle applies in this context, i.e. evolutionary units that reproduce together (at the same time or under the same conditions) become ‘wired’ together into a new evolutionary unit, and how connectionist learning principles help us understand the consequence of this feedback in larger systems.

First, we discuss the evolutionary challenges involved in different types of transition and, in particular, collectives containing particles of complementary functional types. We explain why coordinating these complementary functional types is fundamental to creating fitness differences that belong to the collective. Second, we discuss our work thus far in the evolution of pairwise relationships, and third, our preliminary work on the evolution of new reproductive units in larger networks.

Evolutionary transitions can be classified into two types (Queller 1997): Fraternal transitions, e.g., the transition to multi-cellularity, involve controlled aggregations of related individuals (‘like kinds’). Their origination is motivated primarily by economies of scale. This involves the evolution of traits that modify relatedness by evolving parameters of population structure that control the amount of mixing (such as initial group size/life cycle bottleneck size, germline segregation, dispersal parameters/time in groups) (Jackson and Watson 2013; Powers et al. 2011; Ryan et al. 2015; Johnson and Gaines 1990) and hence the likelihood of meeting others like oneself (assortment). Egalitarian transitions, e.g., the symbiotic origin of eukaryote organelles (Margulis and Fester 1991; Margulis 1993) and the origin of chromosomes (Maynard Smith and Szathmáry 1993), involve the union of previously unrelated lineages (‘unlike kinds’). Their origination is motivated primarily by the provision of complementary functions (Bouchard and Huneman 2013). This involves the evolution of reproductive mechanisms that change the co-dispersal of lineages (e.g. changing from horizontal to vertical transmission of symbionts), linking lineages that were previously inherited independently such that they are subsequently inherited together.

These differences suggest that these types of transition do not share common mechanisms or motives (Queller 1997). However, both types of transition, when considered more fully, involve both a change in the level of the evolutionary unit (from particles to collectives) and the origination of heterogeneous functional roles, but in different orders (Fig. 1). In egalitarian transitions, evolutionary entities differentiate functional roles first (e.g. via speciation) and then form a new evolutionary unit, whereas in fraternal transitions, entities change the scale of the evolutionary unit first and then differentiate in their functional roles. Both types of transition thereby result in higher-level groups with internally differentiated roles, and organisational changes controlling the diversification of roles and changes in the level of selection are involved in both types of transition. In egalitarian transitions this diversification of roles and the coexistence of complementary types (occurring before the transition) involves changes in relationships that are ecological—i.e. between multiple existing evolutionary units. In fraternal transitions the diversification of roles (occurring after the transition in the evolutionary unit) involves changes in relationships that are developmental—i.e. within a single unit of selection. The evolution of organisational networks is central in both cases, however. An important difference between fraternal and egalitarian transitions is that in the differentiate-first egalitarian case the differentiation of roles can be controlled by the original inheritance mechanism (e.g. mitochondria are different from the nucleus because they have their own genetic lineages), whereas in the differentiate-second fraternal case, the differentiation of roles must be controlled by a new inheritance system (e.g.

liver cells are different from skin cells because of their epigenetic states) (Jablonka 1994; Jablonka and Lamb 2006).Fig. 1Two dimensions of change in evolutionary transitions. Fraternal transitions (e.g. to multicellularity) involve first a change in the level of selection, then diversification of functional roles within this new unit (1 → 2a → 3). Egalitarian transitions (e.g. to eukaryote cells with organelles) involve first a diversification of functional roles between multiple units then a change in the level of selection (1 → 2b → 3). Whereas conventional views of the transitions focus on changes in the level of selection (vertical axis), this view emphasises how the evolution of developmental and ecological relationships (horizontal axis) creates organisations that govern the complex phenotype of the new unit/collective. This coordination of diverse functional roles between particles within a collective is essential to create fitness differences between collectives whilst simultaneously eliminating or suppressing fitness differences between particles

The observation that all transitions involve both changes in the level of evolutionary unit and diversification of roles is logical given what is required to instantiate the Darwinian machine at a higher-level of organisation, in particular, the suppression of fitness differences at one level of organisation and the creation of heritable variability that confers fitness differences at another (Clarke 2010, Ryan, in prep.). Much work on the evolution of individuality focusses on factors that suppress fitness differences between particles within a collective in order to prevent particle competition within groups from subverting group heritability (Okasha 2006; Bourke 2011). For example, the origin of multicellularity (in most cases) involves the evolution of a population bottleneck and sequestration of the germ line that creates high-relatedness among particles within a collective and facilitates high frequencies of cooperative particles (Godfrey-Smith 2009; Buss 1987; Okasha 2006). But this is only one aspect of a transition. It is also necessary that these cooperative particles have the capability to take on heterogeneous roles in the group phenotype (Jablonka 1994; Jablonka and Lamb 2006), enabling fitness advantages to arise from the coordination of those roles (e.g. via division of labour). In multicellular organisms this is facilitated by epigenetic differentiation that enables cells to adopt heterogeneous phenotypes despite being genetically homogeneous (Jablonka 1994; Jablonka and Lamb 2006). This separation of genotype from phenotype (i.e. requiring a plastic developmental process) is necessary to create a situation where fitness variance within collectives can be suppressed without also eliminating fitness variance between collectives (Tudge et al. 2013; Jablonka and Lamb 2006).

That is, for there to be selection at a higher level there must be fitness differences between collectives, yet collectives must contain particles that all have the same replication rate (to prevent individual selection from undermining the collective). If collectives only contain one type of particle (i.e. fraternal), then this removes within-collective variation but also makes it more difficult after the transition for descendent collectives to be different from each other (unless particles have coordinated plastic phenotypes). Conversely, if collectives contain more than one type of particle (i.e. egalitarian), then descendent collectives can be different because of the particular combinations or arrangement of particles they contain but selection on within-collective variation must be controlled (e.g. by ‘policing’ mitochondrial replication).

In order to identify natural selection at a new level of organisation we want to distinguish it from changes that are driven by natural selection at the existing (lower) level of organisation (Okasha 2006). Only if collective interests and particle interests act in opposition to one another could an adaptation be observed at the collective level that could not be explained as the result of adaptation at the particle level. However, at the same time, it seems counter intuitive to assert that particles belonging to fit collectives must be individually unfit. Indeed, we might expect that the fitness of a particle might be directly derived by its membership in a fit collective—implying that particle fitness is proportional to collective fitness, and collective fitness is a linear sum of particle fitness. There are conflicting ideas here that need to be disentangled. Let us first make the issues more concrete with an example.

Consider an example where particles attain fitness benefits by having phenotypes that are coordinated with the phenotypes of others (complementary roles), rather than benefits that arise from their intrinsic individual characteristics. For example, suppose that a proto-multicellular organism must be both motile (in order to gather resources to survive) and fecund, and that individual cells cannot be in the motile state and in the reproductive state simultaneously (Solari et al. 2006). Both roles might be provided initially in a single-celled organism via phenotypic plasticity and a lifecycle that moves reversibly between one state and the other. But if two cells work together to allow specialisation in these roles, there are efficiencies to be gained in, for example, the time and energy required to switch between phenotypic states. Clearly, the fitness of an immotile reproductive cell or a non-reproductive motile cell alone may be zero. But the fitness of cells that belong to a collective (of two) with complementary roles is non-zero (for the non-reproductive cell, it is its inclusive fitness that is relevant; Ryan, in prep.). Accordingly, their individual fitness in this case is not depressed in order to serve the fitness of the collective; quite the opposite, they are fit because they belong to a fit collective (Ryan, in prep.). Yet, if collective and individual fitnesses are in alignment, this seems to undermine the significance of adaptation at the collective level (Okasha 2006).

Okasha (2006) points to the resolution. Specifically, even if collective fitness is a linear sum of particle fitnesses, collective fitness can nonetheless be a non-linear function of particle phenotypes. In the previous example, the reproductive cell state is a phenotype that has a fitness that can be very high or very low depending on whether the other cell in the partnership adopts the motile cell state or reproductive cell state, respectively. The cell phenotypes thus have a highly non-linear relationship to collective fitness, but individual fitness (fitness of the particle, not fitness of the particle-phenotype) may nonetheless be proportional to collective fitness. Thus when particles have diverse, potentially synergistic, functional roles, good coordination between them can create fitness benefits at the collective level that cannot be accounted for by phenotypes that confer fitness differences at the individual level. It is thus the coordination ability itself that is both fit for the particle and fit for the collective, whereas neither of the two cell-phenotypes are individually fit (more exactly, fitness differences between these particle phenotypes do not explain the fitness differences that can arise between collectives). Accordingly, the ability to coordinate particle phenotypes with one another is not just a useful ‘add-on’ in evolutionary transitions, but actually essential in creating fitness differences that belong to the collective and not to the lower level of biological organisation. Accordingly, the evolution of individual particle phenotypes is inadequate to explain collective-level adaptations, and it is in exactly this case where a connectionist approach, i.e. addressing the evolution of relationships that coordinate particle phenotypes, comes into its own.

In egalitarian transitions this requires coordinating the reproduction and inheritance of diverse types within collectives. In fraternal transitions this requires particles that are plastic; capable of producing more than one phenotype, i.e. a context-sensitive developmental process, possibly including parental effects (Tudge et al., submitted, Tudge et al. 2013).

Collective fitness thus requires the presence of more than one mutually-exclusive particle phenotypes that are complementary or synergistic (e.g. specialisation in survivability and fecundity). Let us denote these particle phenotypes as 0 and 1, such that collective phenotypes may be 00, 01, 10, or 11 (01 and 10 may have the same fitness). For each of these particles, neither the 0 phenotype nor the 1 phenotype is successful on its own, nor is it any more fit than the other phenotype on average over the possible contexts provided by the other particle. Nonetheless, by evolving to coordinate particle phenotypes (or anti-coordinate them) a collective containing complementary particles may be fitter than other collectives. This will thereby confer a fitness benefit to the particles it contains even though these fitness benefits cannot be attributed to either particle phenotype. In machine learning, a function with this property is called a non-linearly separable function—and is a well-known touchstone in neural learning test cases. The non-linearly separable functions for two Boolean variables are logical XOR (inequality, 01 or 10) and IFF (equality, 00 and 11). Other functions do not have this property. For example, under logical AND (11), a 0 phenotype is never superior to a 1 phenotype, but a 1 phenotype is superior to a 0 phenotype in some contexts (i.e. contexts where the other particle is a 1); accordingly, particle-level selection alone can produce the 11 collective.

Interestingly, when groups are built of individuals from two different species (i.e. egalitarian transitions), XOR and IFF are logically isomorphic because the labelling of each variable (0 or 1) is arbitrary and independent of the other. But when groups are built of two instantiations of the same individual (i.e. fraternal transition) IFF is trivially easy to satisfy and XOR is not. For equality (IFF), duplication of an individual of either type that passes through a single-cell bottleneck, or some other mechanism that enforces genetic assortment, is sufficient. But inequality (XOR) is not so simple in fraternal groups. Which is just to say that groups of heterogeneous phenotypes cannot be built from homogeneous genotypes unless there is also some mechanism of context-sensitive differentiation (e.g. plasticity).