A variational approach to niche construction

On this view, one can now consider the ultimate (non-equilibrium) steady state of reciprocal exchange between the agent and its environment. We know that the dynamics of both external and internal states can be cast as gradient flows on the respective free energies of the agent and environment. So, under what conditions would their free energies be minimized? The following lemma suggests that this non-equilibrium steady state corresponds to a generalized synchrony between the agent and niche.

If the internal and external states of a random dynamical system exhibit generalized synchrony, then their variational free energy is jointly minimized with respect to internal and external states.

Here, are blanket states that comprise action and sensory states, respectively, while the external and internal states are denoted by . Generalized synchronization implies a synchronization manifold: , which has been conditioned on the blanket state. The existence of this (conditional) manifold is assured because for every given blanket state, there is an expected external and internal state. The equations above imply that the existence of a synchronization manifold renders the expected states minimizers of their respective variational free energies.

(heuristic): for simplicity, we will deal with the case of identical synchronization. Identical synchronization implies that there is an identity mapping between the external and internal states and their influences on each other [61,62]. This can be expressed as a synchronization in which the sensory states of the agent (i.e. active states of the environment) become the active states of the agent (i.e. the sensory states of the environment) and both are perfectly correlated. The synchronization manifold can be regarded as a surface that would be traced out if we plotted the states of the agent and environment against each other (figure 2 for example).

The free-energy principle says that the flow of internal can be expressed as a gradient flow on a free-energy functional of posterior beliefs about external states, encoded by internal states, and vice versa for the external states:

The free energy has been written here (for internal states) in several ways that provide a useful interpretation. The first decomposes the free energy into an energy and entropy term by analogy with statistical thermodynamics. The second rearrangement is more common in Bayesian statistics and machine learning. It shows that when the posterior beliefs are the true posterior density , their KL divergence disappears and the free energy becomes negative log evidence , namely the negative log probability of sensory states, under the generative model of hidden causes and sensed consequences: . This is why the variational free energy is sometimes called an evidence bound—because it is always greater than negative log evidence [64,65]. The final rearrangement shows that minimizing free energy is effectively the same as finding accurate explanations that are minimally complex; i.e. that conform to Ockham's principle.

Under identical synchronization, the gradient flows in equation (4.2) are identical everywhere. This implies that the underlying free-energy gradients are isomorphic on either side of the Markov blanket. In turn, this means that the generative model entertained by the agent about its environment is exactly the same as the generative model entertained by the environment about the agent. Furthermore, both agent and environment generate sensory and active states that are congruent with their shared generative model. This implies that the posterior beliefs (i.e. densities) must be veridical:

Substituting this into the expressions for free energy above shows that the divergence term disappears, and the free energy attains its lower bound.

In other words, the generative model of both agent and niche are the veridical explanations of their respective sensory fluctuations and the variational free energies of both are jointly minimized with respect to the posterior beliefs parametrized by their respective states.

For simplicity, we have just dealt with the situation of identical synchrony. Clearly, there will be formal asymmetries in the external structure of the environment and the internal structure of the agent. The above arguments must, therefore, be generalized (heuristically), such that the synchronization between internal and external states is generalized. Usually, in dynamical systems theory, generalized synchronization is considered in the light of skew product systems (i.e. master–slave systems) [61]. The twist here is that there is a bidirectional coupling that is actively maintained. This active maintenance rests upon a corollary of generalized synchrony; namely, that there is a generalized synchrony between the active and sensory states from the point of view of both the agent and niche. In other words, for generalized synchrony to emerge (i.e. for the minimization of joint free energy) it is necessary that both systems supply the right sort of sensory impressions that enable them to implicitly infer each other.

Clearly, the above construction starts to look like a model of communication. Indeed, the notion of generalized synchrony across a shared Markov blanket has been used previously to model communication in computational neuroscience—and resolve the problem of neuronal hermeneutics (i.e. inferring what was meant by reference to a shared narrative). Figure 2 illustrates the generalized (identical) synchrony that emerges when two synthetic songbirds can hear each other singing [63].

In the present setting, this can be regarded as a minimal simulation of the coupling between an agent and its niche, where the niche is another agent. Niche construction, in a broad sense, involves the modification of both biotic and abiotic ‘agents’. As discussed above, those modifications amount to learning via sensory and active (active inference) exchanges, also in a broad sense, among agents populating the niche.

This perspective on mutual influence—that is an emergent property of minimizing (joint) free energy—can be applied to multiple agents. A nice example of this is the joint free-energy minimization exhibited by simulations of cells that have a common (generative) model of their place within a tissue or organ; see [48,60] for details. In short, the same form of generalized synchrony (to a point attractor) can be used to model morphogenesis, where the environment is constituted by other agents with Markov blankets (i.e. other cells), who share the same generative model. This example is particularly interesting, because the evidence for the shared model now becomes the lower bound for the joint free energy [48].

The mathematical image of generalized synchrony, cast in terms of variational free-energy minimization, resonates nicely with earlier formulations of self-organization in cybernetics; namely, the good regulator theorem [66]. Here, the good regulator theorem applies not just to the regulator (agent) but also the (environmental) system that is being regulated (i.e. niche), conferring them both the status of agent. Clearly, to become a good model of one's niche, one needs to infer its causal structure.

In terms of the Bayesian mechanics above, this corresponds to a slow gradient flow on variational free energy averaged over time (in a path integral sense) [67,68]. In other words, there are certain states within and beyond the Markov blanket that constitute order parameters [50,69], or parameters of the generative model that are subject to the same free-energy minimizing pressures as fast fluctuations normally interpreted in terms of inference. The distinction between phenotypic states and traits maps to the distinction between the internal states that fluctuate quickly and slowly, where it is tempting to associate phenotypic traits with the order parameters of synergetics [50,70]—evolving over very slow (somatic or developmental) timescales. In relation to the statistical conception of the phenotype described earlier (cf. §3.1.), and the organism–niche synchrony described above, we can associate some of these phenotypic traits to order parameters that pertain to the material setting of the physical environment. These undergo even slower fluctuations caused, notably, by the niche construction.

We introduce the distinction between fast inference and slow learning because simulations suggest that important asymmetries in the way that the agent learns about the environment—and the environment learns about the agent—can have profound effects on niche construction. In brief, if the (implicit) generative model of the environment holds very precise beliefs about certain dynamics, then the organism will come to learn the environment more quickly than the environment will learn about the agent. Conversely, when action upon the environment effectively teaches the environment about the agent—and the environment holds imprecise beliefs about the agent—the environment will yield to the agent's beliefs more readily (cf. results summarized in figure 3 [71]). The asymmetry in learning between organisms and their environment will be central to our discussion of ecological inheritance in §5.2. In brief, the asymmetry in learning rate between the environment and organisms allows material features of the environment to ‘retain’ regularities in behavioural patterns of groups of organisms unfolding over long timescales (e.g. cultural practices). These are ‘encoded’ through fast inference and niche construction by individual organisms. These may be viewed as order parameters encoded in the material features of the environmental, which are part of the joint phenotypic space (including slowly fluctuating traits, and quickly fluctuating states), and which are passed on through ecological inheritance.

In the present example of the two birds singing to each other, the (order) parameters of the two respective generative models were optimized with respect to their respective variational free energies. Crucially, one bird had relatively precise beliefs about the autonomous dynamics generating birdsong, while the other had less precise beliefs. The first bird, therefore, essentially learned from the second. On the current argument, we can associate the agent and environment with either bird, to introduce the notion that the environment may learn about the agent or the agent can learn about the environment, or that both can occur at the same time. The simulations of generalized synchrony (using birdsong) reported in this paper can be reproduced using the academic software available from http://www.fil.ion.ucl.ac.uk/spm/software/. Typing DEM at the MATLAB prompt will invoke a graphical user interface. The simulations can be reproduced by selecting Birdsong duet button. This allows users to examine the code (DEM_demo_duet.m) and subroutines—and customize them at their discretion.

Here, the ensemble that realizes the generalized synchrony consists in the entire population (the two birds) along with their local environment (for each bird, the other bird). In the simulation, the free energy of one bird is a functional of (sensory) signals from the environment (the other bird), while the free energy of the environment depends upon (action) signals from the bird. In principle, this equips the bird–environment system, or niche, with a measure of complementarity in relation to the phenotypes it hosts. Because the free energies of each bird and their respective environment (the other bird) are extensive quantities (i.e. they can be added together), we have, in principle, a way of scoring the coevolutionary complementarity among the various biotic and abiotic agents of a niche, which is simply the sum of their respective variational free energies. Increase in complementarity correlates with decrease in free energy. Crucially, this measure transcends any particular context, and can be compared quantitatively at different spatio-temporal scales (e.g. between two agents, two groups, two species).

This concludes our formal analysis. For further details, see [47] and [72] for a complementary treatment of self-organized criticality (i.e. critical slowing), variational free-energy minimization and generalized synchrony. We now turn the key role of precision estimation in determining the level of symmetry of exchange between agents and environment.

The FEP understands precision estimation (the estimation of the reliability of sensory information) as a meta-learning mechanism, because it allows the organism to know what worth learning [73]. Learning precision is a crucial part of the attunement to causal regularities in the environment. For instance, humans have evolved a host of phenotypic expressions, from highly visible white sclera and gaze tracking abilities, to cultural ‘prestige cues’, which optimize the process of learning from whom and what to learn [37].

Constructed aspects of the niche can play an important role in learning, and estimating precision. Consider ‘desire paths’ in humans [74]. Such paths can be the by-products of people cutting through the grass rather than following the paved road. Over time, a dirt trail might form, thereby attracting additional pedestrians and reinforcing the trail in a looping manner, which will further steer future interactions between the users and their niche.

In this example, niche construction is mostly implicit: pedestrians need not be aware that they are constructing a path, or aware of how their action on the environment can benefit them, or impact others. They merely act in accordance both with local constraints, and with respect to their expectations (e.g. the desire to reach the extremity of the park). This highlights the fact that in general, we can expect physical changes produced through active inference to be consistent with, or complement the organisms' expectations.

Lasting physical changes to the environment produced via niche construction function as high fidelity, precise sources of information. By actively engaging their environment, organisms—and indeed, entire populations—fit their niches to their prior expectations, e.g. in desire paths. Information relevant to the generation of adaptive responses is encoded in the physical environment (cf. [18]), thereby directing the learning of causal regularities, and aiding the organism in managing uncertainty (cf. [75–78]) (e.g. there is a high probability that I end up faster to the other extremity of the park if I follow this trail). Salient aspects of the niche modulate the attention of organisms and direct active inference [42,79,80] by grounding learning in the socio-material fabric: constructed aspects of the niche function as meta-learning mechanisms and encode salience.

We use ‘salience’ here in a very particular and formal way that is consistent with the active inference scheme. Salience is the information gain, reduction of uncertainty, Bayesian surprise, or epistemic value that constitutes an important part of expected free energy. In brief, certain environmental cues (e.g. signs and semiotics) attain salience or epistemic affordance because they indicate the actions that will minimize expected free energy or, more simply, resolve uncertainty (see [81] for details). This facilitates substantially the modelling of the environment by the organism.

An important function of environment-based meta-learning is that it enables organisms to reduce cognitive demands, and in a sense ‘upload’ the computational burden of salience estimation to the environment itself. Hence, it is no stretch to say that learning what is salient or has epistemic affordance does not rest solely upon the learning of brain, or body-based priors, but also on carving out a specifically constructed niche through the traces left by reiterated action (of self and others, past and present). This could be an implicit strategy by which organisms save on metabolic resources. In the parlance of free-energy minimization, uploading information into the niche translates to minimizing complexity and associated thermodynamic costs of computation via the Jarzynski inequality [82,83]. Heuristically, it eases the decision-making process by reducing uncertainty as well as the need for metabolically expensive, internal (neural) information processing. Uploading also enables the reduction of model complexity over time by constraining the hypothesis space that the organism has to model, narrowing the range of possible priors, and thereby increasing thermodynamic efficiency [84].

In this section, we discuss the variational approach to niche construction in relation to situated learning in developmental psychology and ecological inheritance. We revisit the standard critiques of NCT under the lens of the variational approach.

The idea of the niche as supporting learning is consonant with new perspectives on niche construction in developmental psychology, especially with the situated learning paradigm (e.g. [26]). As Flynn and colleagues point out, ‘the niche into which we are born … in part, dictates what we learn’ [26]. One of the functions of the developmental niche is to support situated learning, defined as the process by which an organism comes to acquire the knowledge necessary to integrate a ‘community of practice’ [85].

This requires of organisms that they engage conventionalized patterns of collective activity, grounded in material artefacts and other physical aspects of the niche, which have been transmitted to allow for the progressive integration of the organism as a legitimate member of the community [42,85]. From the point of view of NCT, the engagement of the constructed environment inherited from the community of practice enables situated learning, and influences the selection of norms, habits, and values modulating attention ([26,41], cf. the concept of regime of attention [79]).

By providing opportunities for learning how to learn, the developmental niche does not merely indicate what information ought to be learned, but also how best to learn it [26]. As a meta-learning device, the information encoded in the material states of the local environment function to guide active inference by weighting sensory inputs according to their reliability, or salience, and constitute a more general device that allow organisms to learn how to learn, by guiding action–perception cycles (active inference).

A good example of simple DNC and situated learning is the process of learning technical skills, for instance the learning of nut cracking skills by young bearded capuchin monkeys (Sapajus libidinosus) in Boa-Vista forest (Brazil) [16]. Bearded capuchin monkeys feed on palm nuts, which they must crack using stones as hammers, and other stones or logs as anvils [86]. To learn nut cracking for youngsters is no small business, as it requires all sorts of complex actions, e.g. properly placing the nut on the anvil, maintaining the nut stable throughout the striking action, learning the adequate kind of striking action that keeps the nut in place while also have enough velocity to crack open the nut, etc.