Variational ecology and the physics of sentient systems

In the second step (green panel – Policy selection), the approximate posterior beliefs from the first step are used to evaluate expected free energy F(π,τ) and subsequent beliefs about action. These beliefs correspond to the epistemic and pragmatic affordances that underwrite policy selection. Note that the free energy per se is a function of sensory states, given a policy. In contrast, the expected free energy is a function of the policy. The construct of affordance in active inference corresponds to inferences about action on the environment, which are selected in terms of competing policies via the minimisation of expected free energy. The variables in this figure correspond to those in Fig. 1. Here, a policy π comprises a sequence of actions; the expression Q(η|π) denotes beliefs about hidden states given a particular policy; and Q(π) denotes posterior beliefs about the policy that is currently being pursued by the agent. Free energy is the difference between complexity and accuracy, while expected free energy can be decomposed into expected complexity (i.e., complexity cost or risk) and expected inaccuracy (i.e., ambiguity). Risk can be regarded as the (KL) divergence (D) between beliefs about future states under a particular policy and prior preferences about states. Ambiguity denotes the loss of a definitive mapping between external states and observed sensory states (quantified as entropy, H). Alternatively, expected free energy can be decomposed into epistemic and pragmatic affordance. Posterior beliefs about policies depend on their expected free energy. Crucially, these posterior beliefs include the free energy evaluated during perceptual inference. This has several interesting consequences from our perspective.

This construction means that the agent has to infer the policy that it is currently pursuing and verify its predictions in light of sensory evidence. This is possible because the beliefs about actions that are encoded by internal states are distinct from the active states of the agent's MB. Free energy per se provides evidence that a particular policy is being pursued. In this scheme, agents (will appear to) entertain beliefs about their own behaviour, endowing them with what is defined as intentionality of goal directed behaviour under active inference. In effect, this enables agents to author their own sensorium in a fashion that has close connections with niche construction: see main text and [48]. See [94] for technical discussion. Figure from [95].Fig. 5

This is where the variational approach to niche construction (VANC) comes in. VANC exploits the expression of expected free energy as expected cost plus ambiguity, to propose that agents upload, as it were, much of the leg work in computing expected free energy to their self-tailored, constructed environment. More precisely, it argues that niche construction can be cast as the collective activity by which organisms act on their material environment to create unambiguous structure, which can be leveraged (via active inference). Lasting changes to the niche capture the fact that environmental cues function as an unambiguous indicator of the affordance of action possibilities. These can be cast in terms of epistemic resources that flag those actions that resolve the ambiguity associated with future observations [71], while conforming to the (prior) preferences of the organism entailed by its generative model.

VANC casts DNC as the joint optimisation of the niche – and its constituent ensemble of agents – over ontogeny through dense histories of active inference, and casts SNC as Bayesian model selection [29], [40]. By engaging ecologically inherited exogenetic resources of the niche, living organisms – especially those, like humans, that depend on large-scale coordination – can reliably track, predict, and model the unfolding of causal regularities at and across large spatiotemporal scales [54]. We will argue that this is so because niches themselves (including the organisms and their material setting) track those regularities. For instance, a complex system of agriculture that employs irrigation techniques enables groups of humans to smoothly cope with climate fluctuations that could otherwise jeopardise food production. In Section 4, we explain how the existence of higher-order, large-scale human ensembles depend on specific exogenetic resources of the niche – namely, epistemic resources of the kind just discussed – and how this emerges from ensembles of agents engaging in active inference.

The skilled intentionality framework (SIF) [60] provides an account of the origin of intentionality or directed purposiveness in cognitive systems. We appeal to the SIF in offering an account of how to draw higher-order MBs; namely, by mobilising the notion of affordances. The SIF recasts cognition as the engagement by organisms of the affordances that make up their local niche, thereby providing a real-time dynamics for the engagement of exogenetic (epistemic) resources that the niche affords, and enabling the stabilisation of the local niche.

What are affordances, and how are they to be interpreted under the free energy formulation? The SIF defines affordances as possibilities for engagement that obtain between a set of abilities at an organism's disposal and relevant or salient features of the material environment [33], [48], [72], [73]. Here, ‘engagement’ refers to structured, skillful patterns of action and perception5 – what we have covered under the rubric of active inference. In effect, the SIF provides an organism-centred dynamics that explains, concretely, what it means for an agent or group of agents and their ecological niche to engage in active inference and niche construction. Moreover, it does this by considering explicitly how the ecological niche is disclosed to the organism – as a field of affordances.

The SIF tells us what is special about living systems in terms of their being directed towards meaningful worlds (or strictly speaking, outcomes). Namely, it points out something special about the way living systems self-organise. The dynamics of most (non-living) self-organising systems emerge and stabilise around an energy gradient, which those same dynamics then typically resolve or consume; e.g., as a lightning bolt strikes, the charge gradient around which it organised dissipates. Unlike other self-organising systems, living organisms are unique in that they actively generate and maintain the gradients that sustain them, through adaptive actions. In other words, an organism's self-evidencing underwrites self-organisation and the very ergodicity upon which both rest [48], [74].

What are the gradients around which living systems organise? The variational framework suggests that these gradients are variational free energy gradients resolved through active inference. Consistent with the FEP, under the SIF, affordances are cast as expected free energy gradients or differences. These differences are in the expected free energy associated with the repertoire of actions or abilities available to an organism under its generative model and the learned niche [33], [48]. In formulations of active inference for generative models of discrete states (i.e., Markov decision processes), these abilities correspond to the policies and their affordance is quantified in terms of the expected free energy under each policy. Given this equivalence, affordance can be decomposed into complexity cost and ambiguity (as in Fig. 5). Alternatively, by rearranging its terms, expected free energy can be expressed in terms of epistemic (i.e., intrinsic) and pragmatic (i.e., extrinsic) affordance (see [71], [75] for details). The ensuing variational formulation of affordances uses the path integral of free energy from the current point in time to a future time point, where the only difference between expected free energy and free energy per se is that sensory states have yet to be realised. This means the expectation in expected free energy is over sensory states in the future, based upon posterior beliefs informed by sensory states in the past. Put simply, the best action is the next action that belongs to the policy (i.e., sequence of actions) with the greatest affordance – or the least expected free energy. This is formally related to Hamilton's principle of least Action6 that translates here into a variational principle of greatest ‘Affordance’.

For organisms to engage the affordances offered by their niche is the variational ‘tissue’ that connects dynamics to the niche in which those organisms exist. The expected free energy of each policy can be cast as constituting the set of affordances that an organism can entertain. Action and policy selection integrate ensemble dynamics, by minimising expected free energy directly and learning the affordances on offer from the niche by selecting courses of action (i.e. policies) with the greatest epistemic affordance. This has close relationships with intrinsic motivation and exploration in ethology [76], [77], [78], [79] where, perhaps, the most important exploration is “what can I do with my body?” (i.e., the body as niche). This is most clearly seen in development and neurorobotics [78]. Things get more interesting when we appreciate that the niche itself is subject to exactly the same normative principles. In other words, as each agent is trying to learn about and infer its niche, the niche – through the collective inferences, actions and material artifacts of its constituent agents – appears to learn about and predict the behaviour of each agent. This must be true, because each MB that comprises the eco-niche is itself trying to minimise variational free energy. Heuristically, this means that because every agent is trying to predict their niche, they collectively shape their field of affordances in such a way that their niche appears to infer the behaviour of its agents and therefore becomes inherently more predictable (i.e., less ambiguous).

This prompts a revision of the ontology of affordances under the SIF [48]. The ‘field of affordances’ is the set of affordances that solicit the organism at a given time. This field is constituted by expected free energy gradients that are induced by the entire ensemble. The engagement of the niche by the organism then corresponds to active inference; in the sense that these dynamics are simply the resolution of a local free energy landscape – a path of least (Hamiltonian) Action over the field of affordances. ‘Solicitations’ are those affordances that effectively engage the organism in action–perception loops at a given time. The ‘landscape of affordances’ is thus the set of affordances available in a niche at a given time. On this view, the landscape of affordances is a product of inference about “what would happen if I did that?” [79]. Affordance is therefore an attribute of active, if counterfactual, engagement with the niche. Yet at the same time, it is a statement about the learned (and therefore lived) world.

The picture that emerges by integrating a variational approach to niche construction with the ontology of affordances could be summarised as follows. When you select actions with the greatest affordance, you learn about your niche. However, your niche comprises other MBs that must be learning about you. These can be other ‘creatures like you’, ‘cognitive gadgets’, and ‘desire paths’, and so on. In other words, your action on the environment constitutes sensory evidence for a niche (i.e., landscape of affordances) that is trying to model you, while the niche acts on you via sensory impressions. From the perspective of some ‘Godhead’ looking down on your niche, a self-organisation would emerge at a higher spatiotemporal scale – that itself looks exactly like a self-organising, free energy minimising process. This has to be the case if your niche and all of its constituents attain some form of ergodicity, in virtue of the FEP. In other words, the network of conspecifics, their ‘desire paths’, and ‘cognitive gadgets’ would look like the internal states surrounded by a MB, separating your niche from another. We now develop this argument in the final section.

How can we describe the sort of relationships that induce conditional independence among remote subsystems of ensembles (e.g., other animals as part of a larger niche)? Can we make sense of the self-organisation of large-scale systems using MBs, to constitute robust and enduring sets of conditionally independent subsystems? How are MBs implemented at higher scales, and how can we define the relations among their states?

The internal states of a MB can be separated by a great deal of distance, while retaining some form of dependency. When looking at phenomena unfolding across longer temporal scales, imagining internal states as physically isolated in a literal sense from external states becomes conceptually constraining. Two states clearly do not need to be (and as a matter of fact cannot be) in direct contact to be part of the same system; e.g., nodes of the Internet, soldiers in a battalion. What matters is the statistical relationship between states; i.e., to form a MB, the right kinds of statistical relations and partitions need to be in play. In other words, ensembles sharing the same MB, whether cells or organisms, need not be ‘spatially’ compartmentalised, but ought to be ‘statistically’ or ‘behaviourally’ segregated; that is, some recurrent patterns of behaviours should govern the way different sets of internal states maintain their conditional independence. Practically speaking, the only thing one needs to partition states into an ensemble of MBs is their adjacency matrix. This matrix (from graph theory) encodes directed dependencies in a weighted or unweighted fashion. One can then identify a subset of internal states, their MB, and the resulting external states [3]. This process can be repeated by identifying a second set of internal states within the external states and continuing until all external states have been exhausted. Note that, to pursue this partition into an ensemble of MBs, one only needs the adjacency matrix describing ‘who is connected to whom’. This can easily be applied to differential equations describing evolutionary dynamics – or links in social media networks. See Fig. 6 for an illustrative example.

The question then is to determine how the active and sensory states are manifest, where the partition of states is dispersed over high dimensions of abstract state spaces. In other words, we need to understand what sort of ‘behaviour’ their states exhibit.Fig. 6A particular partition. This schematic figure illustrates a partition of vectors states (small coloured balls) into particles (comprising nine vectors), where each particle (π) has six blanket states (red and magenta for active and sensory states respectively) and three internal states (cyan). The upper panel summarises the operators used to create a particular partition. We start by forming an adjacency matrix that characterises the coupling between different vectors states. In this example, the adjacency matrix is based upon the Jacobian and implicitly the flow of vector states. The resulting adjacency matrix defines a MB forming matrix (B), which identifies the children, parents, and parents of the children. The same adjacency matrix is used to form a graph Laplacian (L) that is used to define neighbouring (i.e., coupled) internal states. One first identifies a set of internal states using the graph Laplacian. Here, the j-th subset of internal states at the level i are chosen based upon dense coupling with the vector state with the largest graph Laplacian. Closely coupled internal states are then selected from the columns of the graph Laplacian that exceed some threshold. In practice, the examples used later specify the number of internal states desired for each level of the hierarchical decomposition. Having identified a new set of internal states (that are not members of any particle that has been identified so far) its MB is recovered using the MB forming matrix. The internal and blanket states then constitute a new particle, which is added to the list of particles identified.

This procedure is repeated until all vector states have been accounted for. In the example here, we have already identified four particles and the procedure adds a fifth (top) particle to the list of particles; thereby accounting for nine of the remaining vector states.Fig. 6

Under the MB formalism, sensory and active states are to be interpreted in their minimal sense; that is, as instantiating statistical relations. What it means for a system to have active states is for it to have states the dynamics of which change as a function of some systemic quantities (namely, as a function of internal and sensory states). In turn, what it means for something to have sensory states is for it to have states that change as a function of other systemic quantities (external and active states). On this view, both sensing and acting assume a statistical definition. Indeed, one could wonder about the difference between an agent sensing things in the world with a visual saccade and a photon hitting the retina of the agent. In the parlance of the MB formalism, whether a state counts as an active or a sensory state depends on whether or not it is influenced by internal and external states in the right way [3]. Thus, an active state is just one that is influenced by the system's defining states, and a sensory state is one that is not. Accordingly, a sensory state – at any level of description – might just mean a state that reliably covaries with, and conveys information about, some distal causes in the external system to which it is coupled (e.g., a receptionist taking calls on an external telephone line); and active states are just those states that enable the system to effect changes on the ‘outside’ (e.g., a public relations officer emitting press releases). An ecological niche could thus have sensory and active states (and thus, have a MB) in a sufficient and minimal statistical sense.

For a number of reasons, elements of the ontology of affordances are interesting candidates for the implementation of the sort of large scale behaviour under consideration. First, similar to the cell membrane, the sensory signals that actively engage an organism at a given time (i.e., solicitations) mediate the functional relationship between internal and external states of both the organism and its niche. In the parlance of the FEP, solicitations secure adaptive exchanges among the internal and external states by engaging the organism in loops of active inference that conform to internal expectations (e.g., they are perceived relative to the needs of the system), while enabling the learning of the causal structure of some external states (i.e., they are perceived relative to the actual causal structure). In this respect, similar to cell membranes, we can say that a solicitation “points both ways, to the environment and to the observer” [80]. In Fig. 1, this ‘pointing both ways’ is established in virtue of the circular causality induced by the MB and ensuing active inference. This brings affordances into play, in terms of the expected free energy attributed to different policies of courses of action on the environment (see Fig. 5).

Second, affordances, especially those enabled by epistemic resources, carry cultural knowledge that can be acquired in ontogeny [81], thereby securing the reproduction of adaptive patterned practices when transmitted across generations [40]. As such, affordances – especially the cultural affordances [33] that characterise a given local group – play a role in the coordination of the adaptive behaviour of members in a group over large spatial and temporal scales. Briefly, cultural affordances are the affordances with which human agents interact, and which depend on shared sets of cultural expectations, internalised through immersive practices [33]. Affordances thus allow for adaptive behavioural self-organisation among groups of agents, independent of their spatial proximity. In effect, cultural information encoded in the physical states of the ecological niche (in epistemic resources) enables the recognition and diffusion of epistemic affordances among groups over larger spatiotemporal scales (e.g., the intergenerational scale, via the passing on of affordances via cultural inheritance).

Third, defining a MB for niche systems involves the description of the conditional independence among the partitions of the system. The ontology of affordances, especially the layering it entails (e.g., organism and field, solicitation and landscape of affordances) can, in principle, be used to define the statistical compartments (partition) of the niche, and by the same token explain the conditional independence among the states of the niche.

Given these very reasons, we suggest that the field of affordances (and especially the solicitations that actually engage the agent at a given time) functions as the ‘surface’ that allows the niche to ‘sense’ agents via the agent's action on the niche [82]. The agent's actions when repeated over time encode regularities about niche-agent interactions via changes to the structure of the ecological niche. In virtue of the circular causality discussed above, this structure is determined by – and indeed, determines – the affordances that underlie each agent's action. The niche then ‘acts’ on the agent as it is sensed by the agent. We can say that an agent will be acted upon when engaging an affordance, as the affordance is a possible action to be selected, and the selection of which will entail changes in the agent. The action of the niche thus takes the form of ‘offering possibilities for engagement’. Crucially, a niche that would not offer a variety of possible engagements could not ‘act’ in any meaningful sense. The ‘active’ property of the niche rests on their ability to conform to the changing needs of the agent(s), that is, to solicit the agent by providing the right sort of sensory cues.

So far, we have discussed the relevant quantities to define the niche's MB. These are captured by the dynamics of the agent's field of solicitation, which involves patterns of action and sensation for the agent, which coincide with the action and perception of the niche. The key point here is that the agent's field of affordances (and solicitations) emerges from the dynamics of the MB of the niche, and thereby enables the niche to model sensory causal regularities, or underlying structure of the form of life it constitutes [83], [84].

Now, what are the internal states of the niche? And what are the causal regularities that they model? We suggest that internal states of the niche are a subset of the physical states of the material environment. Namely, the internal states of the niche are the physical states of the environment, which have been modified by the dense histories of different organisms interacting in their shared niche (i.e., histories of active inference). This subset of physical states comes to encode information that is used in the self-evidencing dynamics of organisms. In other words, the ecological niche becomes part of the embodied model parameters that encode the variational or recognition density that the agent uses in active inference. These model parameters encode causal regularities about agents' behaviour, which function as the external, hidden states of the niche, which it models in niche construction.

The internal states of the niche encode information about regularities of the niche-agent(s) relation [40]. This means that the internal states of the niche encode organism-specific information (e.g., affordances), not merely any changes to the physical layout. Otherwise, it would mean that the internal states would also encode random changes to the environment (e.g., volcanic eruptions and tsunamis). Hence, the ecological niche is a subset of the physical environment that the organism constructs through reiterated action over time (i.e., active inference). We can see the propensity of the environment to encode organism-specific information as the propensity to change its structure as a function of the actions of the organism (i.e., sensations of the niche) [82]. For instance, a grass patch has a much higher propensity to encode organisms-specific information than a sidewalk made of concrete – hence a grass patch after some time, and recurring actions, might encode a desire path (i.e., will turn into a cultural affordance). The point here is that the niche is always organisms, or species-specific, an Umwelt of sorts [85], whose relational structure consists of affordances, encoded through niche construction (cf. section 3).

The final quantity to define is that of the causal regularities modelled by the niche, and transcribed by its physical layout (the internal states). We have seen that for groups of enculturated agents like humans, internal states of the niche encode affordances that pertain to group behaviour; e.g., what the desire path models is the action possibility to ‘cut corners’. Now, it is a small step from this point to the one that the internal states of the niche track statistical regularities and fluctuations that underwrite the meaning of shared intentionality and normative group behaviour (cf. [86], [87], [88]). These regularities are highly abstract, and can only exist – as causes – for large-scales ensembles like groups of agents. With the example of the desire path, one can start to appreciate the continuity that obtains between the physics of self-organising systems, the form of pragmatic engagements afforded by constructed niches, and the sort of meanings and intentions that people derive from them.

In summary, the niche transcribes regularities that pertain to group behaviour – they are transcribed in the physical layout of the niche – through the active and perceptual dynamics of the agent, which coincides with the perceptual and active dynamics of the niche. These same dynamics entail the landscape of action possibilities (i.e., affordances), which maintains the structural integrity of the agent-niche system – niche dynamics resolve the free energy gradients that are induced by the physical structures of organisms and their niche, and their history of dense interaction and dynamic coupling. In this sense, the robustness of patterns of shared intentionality and enaction of shared meanings is inherited from the robustness of the niche, and vice versa.

Variational ecology (VE) is a synthesis of VNE [4], the VANC [40], and the SIF [73], and provides an explanation of collective purposive action and intentionality of living systems – a physics of sentient systems. An ecological niche ‘just is’ a structured set of affordances that are shared by agents, which enables its denizens to coordinate purposive action over sometimes vast spatial and temporal distances. The sort of affordances that emerge from niche construction, and that constitute large scale ensembles, carry semiotic and axiological meaning like moral states held in common (cf. [89]) (e.g., this dirt trail ‘means’ cutting through the park, and you shall not be late to your appointment). In effect, there is a deep sense in which affordances are what meanings are [80], and a sense in which meaning is entailed by the existence of groups of agents that share a niche (cf., sense making, [90], [91]). As they engage with the affordances of their niche, ensembles of agents: (i) maintain the structural integrity of the niche through niche construction (and active inference); and (ii) collectively enact a generative model of their relation to the niche, thereby providing an account of the physics of intentionality, and especially of shared intentionality. VE, then, is also a physics of interacting minds.