Evolutionary Connectionism: Algorithmic Principles Underlying the Evolution of Biological Organisation in Evo-Devo, Evo-Eco and Evolutionary Transitions
These distinctions can also be formalised in game theoretic terms. In conventional social dilemmas, such as the Prisoner’s Dilemma, the Snowdrift game or the Stag Hunt, the social composition that maximises community welfare is 100 % cooperation (a homogeneous collective). In contrast, in division of labour games, the community welfare of a group is maximised by a particular combination of multiple strategies (Tudge et al. 2013; Tudge et al., submitted). This kind of social dilemma cannot be solved by genetic assortment alone (i.e. homogenous groups); some form of developmental process, through which particles may coordinate with one another to adopt complementary roles (West-Eberhard 2003; Jablonka 1994), is required to solve this type of dilemma (Tudge et al., submitted).
Given this view of the evolutionary challenges involved in an evolutionary transition, next we want to understand the conditions under which natural selection acting on the existing, i.e. lower, evolutionary units will meet these challenges. First we address how the action of natural selection modifies the evolutionary unit in dilemmas that can be resolved by positive assortment or homogeneous groups (e.g. the first stage of a fraternal transition). Then we address a minimal case of heterogeneous groups, i.e. two-player division of labour games, given particles with plastic phenotypes, as per a fraternal transition with minimal diversification of roles. This describes our work on social niche construction. For homogeneous groups, scaling-up to larger collectives is unproblematic. In contrast, for groups with multiple heterogeneous functional roles, it becomes necessary to describe the evolution of reproductive relationships in a network. Our models thus far address how natural selection modifies the evolutionary unit in networks of genetically heterogeneous components, i.e. egalitarian transitions. This work illustrates how the positive feedback principle, in common with the feedback in developmental and ecological networks, also applies to the evolution of reproductive relationships.
Social evolution theory explains the evolution of cooperation by showing that strategy assortment makes cooperators fitter than defectors even when the reverse is true in a well-mixed population (Frank 1998; Bourke 2011). Social assortment (where similar phenotypes meet one another with higher probability than would be expected from their global frequency) and relatedness (where similar genotypes meet one another with higher probability than would be expected from their global frequency; Michod and Hamilton 1980) are exogenous parameters to this type of explanation (Ryan et al. 2015). In natural populations, however, there are many ways in which individual characters affect strategy assortment (e.g. seed dispersal radii, quorum sensing dispersal behaviours, habitat preferences, context-sensitive phenotypes) (Powers et al. 2011). We seek to endogenise parameters affecting population structures and genetic assortment into evolutionary explanations; to thereby explain the evolution of cooperation at a deeper level. For example, instead of concluding that cooperation prevails because genotypes are positively assorted, we ask why population structures that provide such genetic assortment evolved—and in particular, whether these structures evolved precisely because they enabled greater cooperation.
Social niche construction (Powers 2010) thus studies the concurrent evolution of social behaviours with behaviours that modify the social niche, e.g. via traits that modify population structure (Powers et al. 2011; Powers 2010; Ryan et al. 2015). This work shows various conditions under which group structure that supports cooperation can evolve under individual natural selection (Szathmáry 2011). In game theoretic terms, one way to formalise this feedback is with the concept of a meta-game, i.e. where individuals have traits that modify the game they are playing, in addition to social strategies (Jackson and Watson 2013; Jackson and Watson, submitted; Jackson 2011; Doncaster et al. 2013). This work enables us to characterise the conditions under which individual natural selection can transform social interactions to remove a social dilemma (moving the effective game from a Prisoner’s Dilemma to a Harmony Game, for example) and enabling cooperation to prevail (Jackson and Watson, submitted). This is a simple example of a feedback between a social structure and a social behaviour, echoing the other examples of feedback between an evolutionary process and the parameters of that process we have discussed. We find that natural selection can, via social niche construction, increase the level of cooperation in a population (e.g. by evolving group size in the direction that favours cooperation).
However, this does not necessarily demonstrate a new evolutionary unit that exhibits heritable variation at the collective level. This depends on whether we are modelling a type-1 or type-2 group selection process (Okasha 2006; Wilson 1989). In type-1 group selection, individual fitness is affected by a group context but groups do not have heritable lineages. Here reproduction of groups may be via an aggregation and dispersal process, where individuals reproduce via a migrant pool (Maynard Smith 1964). In type-2 group selection models, groups do have heritable lineages and fitness is thus meaningful at the group level. Here, group reproduction may be via a group-fissioning process or propagule reproduction such as in the stochastic corrector model (Szathmáry and Demeter 1987). In both cases, social niche construction can change the balance of particle-level and group-level selection by, for example, modifying the initial size of groups (Powers et al. 2011), or the initial size of group propagules (Ryan et al. 2015, Ryan, in prep.). But the latter is particularly relevant to the evolution of individuality at a new level of organisation because only in this case do collectives have lineages and heritable properties (Ryan et al. 2015; Ryan, in prep.). In this case (unlike the migrant pool model) it is possible to show that collectives exhibit heritable variation in reproductive success and that this increases over evolutionary time via social niche construction.
Those works have addressed conventional social dilemmas that are resolved by positive assortment (genetic and phenotypic). Other work is investigating social niche construction in models that address division of labour games and fraternal transitions. These models ask when individual natural selection (on the original evolutionary units) will utilise phenotypic plasticity, and when it will utilise context sensitive plasticity (e.g. parental effects), on the assumption that these characteristics are controlled by particle genotypes. We find that positive genetic assortment alone does not resolve such dilemmas, but positive genetic assortment is necessary to enable the evolution of negatively-assorted phenotypes via plasticity. This is because this scenario creates a new (conventional) social dilemma on genotypes between ‘co-operators’ that have the ability to coordinate phenotypes correctly, and defectors that do not. Purging such defectors is a proviso for the evolution of negative phenotypic assortment, i.e., the expression of complementary functional roles via phenotypic plasticity (Tudge et al. 2013; Tudge et al., submitted). Accordingly, positive genetic assortment creates a situation where an allele that produces coordinated phenotypic differentiation is then favoured and can thereby maximise collective fitness (Tudge et al., submitted).
Encapsulating pairs of particle-phenotypes into a new evolutionary unit causes them to be reproduced as an indivisible pair (i.e. one particle-phenotype cannot reproduce without the other if particle reproduction is appropriately policed or if the two particle phenotypes share the same genotype). Like the connections in the deep optimisation model, this changes the way combinations of particles are substituted under selection. For example, suppose a 11 pair (e.g. cooperative pair) has higher collective fitness than a 00 pair (e.g. defect pair). Before a transition, a 11 pair cannot competitively exclude a 00 pair because when particles reproduce as individuals, 01 pairs will also be created and individual selection favours 0 in this case. Put differently, before the transition the pair is not the relevant evolutionary unit. But after the transition, 11 can competitively exclude 00 when reproducing as a unit because it has higher collective fitness. Thus by eliminating fitness differences within groups, the remaining effect of selection derives entirely from between-group selection, i.e. the relevant evolutionary unit is now the pair. Putting together the ideas of social niche construction with a minimal demonstration of the between-group selection it enables, we show that individual selection creates groups that facilitate higher-level adaptation (Snowdon et al. 2011). This is analogous to the jumps in configuration space that are facilitated by the new units in deep optimisation.
These works (and others, Ghang and Nowak 2014; Szathmáry 2011) indicate that it is possible to endogenise the evolution of reproductive organisations into evolutionary theory (Ryan et al. 2015). We find that when natural selection is given the chance to do so, at least in small systems (e.g. a single pairwise relationship between two particles), it favours reproductive organisations that follow correlation learning principles—either reinforcing positive correlations or negative correlations, depending on what the social game requires. The new evolutionary units thus created can change evolutionary outcomes by creating selection that operates at the collective level rather than the particle level.
In dilemmas that are solved by positive assortment, it is easy to imagine that economies of scale provide even greater benefits to larger collectives (greater than size two). But scaling-up from pairwise relationships to large numbers of particles is more challenging when it involves complementary functional roles. Intuitively, homogeneity scales easily, but heterogeneity does not—there are many ways to be different and organising these differences is, for example, what makes multi-cellular organisms interestingly different from simple colonies of cells.
In large heterogeneous systems, the relevant question changes from ‘how do individuals create reproductive structures that change the level of selection?’ (e.g. by social niche construction) to ‘given that individuals can create reproductive structures that change the level of selection, which individuals do so and with whom?’ Here our work focusses on egalitarian transitions where genetically diverse particles are placed together into a new reproductive unit. This work seeks to identify conditions where particle-level natural selection is successful in creating new evolutionary units that are fit given that there is variation in the membership of such units. The evolution of such units or partnerships enables individual natural selection to control which individuals reproduce independently and which are ‘in the same boat’, i.e. which individuals have shared reproductive fate/create a single vertical lineage. A particle that forms such a reproductive partnership must forgo within-collective fitness differences but may gain fitness differences at the collective level through the combinations of multiple diverse types that are enabled by these units.
Our early work in this area used a simple approach. It created higher-level evolutionary units at random and let natural selection retain those that were adaptive (Watson 2006; Watson and Pollack 2002). The symbiotic evolutionary adaptation model could correctly identify which composite units corresponded to sub-problems in a combinatorial optimisation problem (Watson 2006). However, this approach did not scale well. In short, there are too many possible partnerships to try and there are many partnerships that are far from optimal but fitter than not having a partnership at all. This makes it very difficult for selection to find partnerships that enable effective adaptation at higher levels of organisation. This is analogous to the problem of how supervised learning under-determines intermediate representations in deep correlation learning—there are many intermediate representations that locally improve the output but are not optimal for the higher-level task.
In hindsight, we were missing a trick; this work overlooked the value of unsupervised correlation learning in reducing the dimensionality of the search space before new units are created. More recent work rectifies this by exploiting principles of unsupervised learning introduced into the new approaches to deep learning. This uses individual-based simulations where, as before, individuals have traits that define symbiotic partnerships controlling who they co-disperse with during reproduction, thus creating new heritable units. But this is now combined with ecological dynamics (particle level selection) such that selection for new evolutionary units occurs mostly at local ecological equilibria. Under these conditions, new evolutionary units that join two species together must be at least as good as the combinations of species that already co-occur under individual selection at ecological equilibria—otherwise individuals that are not partnered will be fitter. By occasionally perturbing the ecological dynamics, we cause the system to visit many different ecological equilibria, and under these conditions, only partnerships that are robust over the distribution of ecological equilibria visited will survive selection. Accordingly, the partnerships that are favoured by selection are those that evolve to canalise the combinations of species that already co-occur most frequently under particle-level selection. This implements the unsupervised correlation learning principle, i.e. evolutionary units that reproduce together (at the same ecological equilibria) become ‘wired’ together into new evolutionary units. This greatly reduces the number of partnerships that are favoured under individual selection, and these more stringent selective conditions prove to be extremely effective in focussing selection onto partnerships that are adaptive (Watson et al.
2009a, b; Watson et al., submitted). The partnerships that evolve under these conditions subsequently enable groups of species to invade ecological equilibria as a unit, allowing the ecological dynamics to jump out of local equilibria and thereby find new ecological equilibria of higher collective fitness.
Again, this is analogous to the jumps in configuration space that are facilitated by the new units in deep optimisation. In the size-two collectives of the previous section, the computational optimisation capabilities of forming new units are not significant; assuming that there is heritable variability in the membership of egalitarian groups, forming groups at random would be sufficient to find groups that maximise collective welfare. In these larger systems, this is not the case. Forming large collectives at random is not sufficient to find collectives with high collective welfare. Moreover, incrementally modifying the membership of groups is not sufficient to find collectives with high collective welfare even when selection is enforced at the collective level (Watson et al., in prep., Watson et al. 2009a, b). In other words, finding fit collectives is a computationally difficult problem that cannot be solved by selection at the particle level nor by simply enforcing selection at the global level. However, under the selective conditions described, implementing deep optimisation with unsupervised learning principles, natural selection picks out particular subsets of species that have synergistic fitness interactions to form new units. This enables these intermediate-sized units to invade as a unit and competitively exclude other subsets of species (even if those species are not favoured under individual selection). This enables the evolving ecosystem to rescale the evolutionary process, and thereby find larger collectives with especially high collective fitness, and so on. This multi-scale process enables natural selection to find solutions that cannot be found by single-level evolutionary processes at either the individual or global level (Watson et al. 2009a, b).
The reason that these partnerships evolve is not, and cannot be, because they will enable effective adaptation in future. These partnerships evolve because they are the partnerships that exhibit the immediate fitness benefits of robustness. Specifically, partnerships between particles that already frequently co-occur under particle-level selection are close to neutral with respect to their effect on collective fitness (i.e. they are partnerships between types that already co-occur). Nonetheless, it is exactly these partnerships that later enable selection at the collective level to escape local optima and discover higher fitness solutions. We can understand why this is the case by analogy with deep learning and deep optimisation: partnerships that amplify existing correlations reduce the dimensionality of the search space in a non-arbitrary way. The evolution of reproductive correlations under individual selection, shown in these models, has the effect of canalising combinations of particles that commonly co-occur at social equilibria. This occurs without any information about what combinations of particles will be adaptive at the higher level of selection that is thus created. By mimicking the structure of co-occurrence produced at ecological equilibria in this way, the evolution of new reproductive relationships is analogous to the use of unsupervised learning to reduce the dimensionality of a data set in deep optimisation. Optimisation algorithms inspired by this multi-scale process resulted in the technique of multi-scale search (Mills et al. 2014) which is able to solve formally difficult optimisation problems that provably cannot be solved by conventional (i.e. single scale) evolutionary models.
Taken together these models suggest that ‘deep evolution’, involving transitions through multiple scales of biological organisation, implements a sort of multi-scale correlation learning machine, or deep optimisation, that is quite different from the conventional micro-evolutionary (i.e. hill-climbing) model of evolutionary adaptation (Watson 2012).
Our work illustrates several different but complementary processes by which the Darwinian Machine changes as a result of its own products. The evolution of developmental networks modifies the distribution of phenotypic variants that selection can act on, and the evolution of ecological networks modifies the selection acting on those variants.
The major evolutionary transitions involve more radical transformations in the underlying processes of the Darwinian Machine. We have illustrated several of the components that are involved in a mechanistic account of such transitions in separate models. Our models of social niche construction consider small collectives (pairwise games) and larger homogeneous collectives. Our work on large heterogeneous collectives thus far address complementary research questions by assuming that social niche construction capable of instantiating higher-level evolutionary units of genetically heterogeneous units is provided. A more integrated model should bring these works together. Our models also do not yet attempt to address fraternal transitions in large collectives of many functional roles. Further, it would be desirable to present a model where particles have the evolutionary option to employ mechanisms that either create egalitarian or fraternal transitions (or potentially a mixture of the two). This would enable us to examine the conditions under which natural selection takes different routes through the space described in Fig. 1.
From greater perspective, although the work on the evolution of developmental and ecological organisations has theoretical and conceptual unifying themes, we do not yet have a model that integrates changes in developmental, ecological and reproductive organisations freely in the same model. This would be ambitious, of course, but we believe that this is necessary to properly understand the interaction of developmental and ecological organisations in the major transitions (Fig. 1). In an egalitarian transition, the relationships between the diverse components are originally ecological relationships, i.e. between multiple species. After the transition, these ecological relationships are encapsulated within a new single unit (O’Toole et al. 2000; Bouchard and Huneman 2013), but the original functional roles and particle-level selection dynamics are retained within the new unit, at least initially. In a fraternal transition, e.g. the transition to multicellularity, the transition to a new unit occurs before functional roles diversify. Thus the functional relationships between these different types are ‘developmental’ from the outset, i.e. within a single evolutionary unit. However, such transitions create a new ‘internal selection’ ecology implemented in a new inheritance system, e.g., during ontogenesis, different cell types grow into tissues through (context sensitive) replication of epigenetic states (Jablonka 1994; Laland et al. 2014; Huang et al. 2005). The population dynamics of these cell populations is affected by inter-cellular interactions and competition for resources within the embryo that determine which types grow where, when and how much (Buss 1987).
Accordingly, either changes in ecological organisations (capable of controlling inter-specific dependencies to enable co-existence of complementary functions), or changes in developmental organisations (capable of controlling phenotypic plasticity to coordinate differential growth of particle phenotypes with one another), are necessary components of evolutionary transitions. An integrated model of these dynamics, enabling the bottom-up evolution of new evolutionary units through successive scales of organisation, remains as the ambitious aim of this research programme.
Connectionism recognises that the cleverness of cognition does not derive from the cleverness of the individual neural parts but from the organisation of the relationships between them. We have introduced the term evolutionary connectionism to recognise that, in the same way, evolutionary innovation need not originate from the adaptation of the evolutionary parts per se but from the evolution of the relationships between them. We have argued that this is much more than a superficial analogy between learning and evolution. Specific, but simple, organisational principles are common to correlation learning systems and the evolution of organisations. Selection for correlations that improve fitness at the system level (the evolution of developmental organisation) is equivalent to reinforcement correlation learning mechanisms. And when selection for correlations that improve fitness at the component level (the evolution of ecological organisations) act to stabilise the system state this is equivalent to unsupervised correlation learning at the system level. In an evolutionary transition, where the level of selection changes from one scale of organisation to another higher level of organisation, the ecological attractors reinforced by unsupervised learning are converted into developmental attractors of the new unit. Then, by evolving new interactions with other units at this higher level, they may enact an unsupervised correlation learning process at the next level of organisation, and so on. This deep evolution process shares algorithmic properties with deep correlation learning, and optimisation techniques based on these principles (multi-scale search, deep optimisation) have adaptive capabilities that are in a provably different class from conventional single-scale microevolution.
These connections with learning theory suggest that the special problems of each biological domain have solutions. Specifically, they suggest that;In evo-devo, it is not impossible for short-term selection to discover developmental organisations that facilitate future evolution. This can occur in the same way, and with the same limitations, as the ability of correlation learning to generalise from past data to perform well on previously unseen data. This requires the ability to form a model of the data (e.g. a correlation model) that captures structural regularities that are invariant over time, even if superficial structures are novel. This provides a formal basis for the evolution of evolvability.In evo-eco, it is not impossible for a community to be organised to reflect the structural properties of its environment (e.g. the structure of abiotic, constraints and resources in which the ecological community resides) without selection at the community level. This can occur in the same way, and with the same limitations, as the process by which unsupervised correlation learning learns the structure of a data set without performance feedback. Such an organisation arises as a consequence of selection at the level of individuals within component species, and facilitates system stability and robustness to perturbations (if the perturbations have similar structure to those experienced in the past) even though there is no notion of Darwinian fitness at the system level. This provides a formal basis for the evolution of community organisation without community selection.In evo-ego, it is not impossible for natural selection to favour reproductive organisations that facilitate higher-level adaptation before evolutionary units at that higher level exist.
This can occur in the same way, and with the same limitations, as the way in which unsupervised learning can reduce the dimensionality of a problem in deep learning even though it has no performance metric that knows what the new representation might be used for at the next level of organisation. This implies that the evolutionary transitions may not be simply a collection of extraordinary accidents (even though each has its own unique features) but the result of systematic adaptive processes.
Although we argue that it is not impossible for biological systems to exhibit such behaviours, clearly they do not always do so. Our aim in making these connections with learning theory is so that we have potential to identify the conditions when biological systems will produce these behaviours and when they will not (Watson and Szathmáry, 2015). For example, the transfer of learning theory to evo-devo suggests that the evolution of evolvability will be sensitive to the match between the deep structural regularities of the environment and the intrinsic inductive biases of developmental processes (i.e. the kind of regularities that are easy to ‘learn’ in that ‘model space’), and to the costs and benefits of ‘overfitting’ the environment (Kounios et al., in prep., Kouvaris et al. 2015). The transfer of learning theory to evo-eco suggests that the evolution of community organisation will be sensitive to the presence of ecological constraints that cause species to coevolve dependencies with one another rather than simply evolve toward independence (Power et al. 2015). And the transfer of learning theory to evo-ego suggests that the evolution of new evolutionary units will exhibit limitations analogous to those of deep learning.
Together the evolution of developmental, ecological and reproductive organisations modifies the mechanisms of variation, selection and inheritance that drive evolution by natural selection. The evolutionary connectionism framework sheds light on how the Darwinian Machine can thereby be rescaled from one level of biological organisation to another. The results thus far demonstrate that connectionist learning principles provide a productive methodological approach to important biological questions and offer numerous new insights that expand our understanding of evolutionary processes (Watson and Szathmáry 2015). Regardless of how the exact alignment between the evolutionary and learning models discussed in this paper develops with future research, the algorithmic territory covered by learning algorithms is, we argue, the right conceptual territory for developing our understanding of how evolutionary processes change over evolutionary time (Watson and Szathmáry 2015).