Variational ecology and the physics of sentient systems

This paper addresses the challenges faced by multiscale formulations of the variational (free energy) approach to dynamics that obtain for large-scale ensembles. We review a framework for modelling complex adaptive control systems for multiscale free energy bounding organism–niche dynamics, thereby integrating the modelling strategies and heuristics of variational neuroethology with a broader perspective on the ecological nestedness of biotic systems. We extend the multiscale variational formulation beyond the action–perception loops of individual organisms by appealing to the variational approach to niche construction to explain the dynamics of coupled systems constituted by organisms and their ecological niche. We suggest that the statistical robustness of living systems is inherited, in part, from their eco-niches, as niches help coordinate dynamical patterns across larger spatiotemporal scales. We call this approach variational ecology. We argue that, when applied to cultural animals such as humans, variational ecology enables us to formulate not just a physics of individual minds, but also a physics of interacting minds across spatial and temporal scales – a physics of sentient systems that range from cells to societies.

Recent decades have witnessed the emergence of a new project for a physics of the mind. This effort has leveraged the constructs, principles, and methods of theoretical and experimental physics to investigate and understand what we call sentience and the ‘mind’. At the turn of the century, a new evolutionary systems theory of complex adaptive systems was proposed, made possible by the advent of computational modelling, variational methods of inference, and machine learning: the (variational) free energy principle (FEP) [1], [2], [3].

According to the FEP, the physics of sentient systems follows from the statistical mechanics of life. This variational formulation stems from the observation that living systems, over time and on average, tend to revisit the same set of attracting or characteristic states. These can be cast as the characteristic phenotypic states (and traits) of the organism. The FEP explains the dynamics of any random dynamical system that appears to resist decay through adaptive action [3], [4]. Under the FEP, organisms engage the environment in a self-fulfilling prophecy of sorts; ‘surfing’ up probability gradients towards their most probable phenotypic states [5]. The FEP was originally proposed in computational neuroscience to explain neural dynamics [6], [7], where it coheres broadly with predictive coding approaches [8], [9], [10], [11], and is widely recognised as a unifying theory of the function, structure, and dynamics of the brain [5], [12], [13], [14]. It has since been extended to explain the dynamics of biological systems within and beyond the brain, ranging from the cellular level [15] and action–perception loops [16], to psychiatry [17], [18], [19], [20], [21], [22], psychology [14], [23], [24], [25] and embodied cognitive science [26], through to evolutionary dynamics (cf. [7], [27], [28], [29]) and, recently, to intersubjective and sociocultural dynamics [30], [31], [32], [33].

The FEP has recently been leveraged to furnish a fully generalisable metatheory for adaptive behaviour in sentient systems across spatial and temporal scales, called variational neuroethology (VNE) [4]. VNE synthesises the FEP with Tinbergen's [34] four levels of explanation in biology (i.e., adaptation, phylogeny, ontogeny, and mechanism) to provide a systematic guide for theorising and research in the sciences of life and mind. As a metatheory, VNE comprises two principal components: a multiscale theoretical ontology for living systems based on a recursively nested formulation of Markov blankets (MBs); and a multidisciplinary research heuristic in the biological and cognitive sciences [35]. In their response to peer commentaries, Ramstead and colleagues [4] address the promise and limitations of VNE as a heuristic for scientific inquiry, and review its scope as a research programme. This paper will complement this discussion by concentrating on criticisms of VNE as a multiscale ontology.

To date, VNE has only provided a principled method of analysing nested and mutually constraining biological systems and their complex adaptive dynamics across spatiotemporal scales [4], [35]. It has yet to offer a way to individuate systems at scales beyond that of the organism acting in its environment; e.g., large-scale ensembles such as societies or ecosystems that realise free energy bounding dynamics. This issue was cogently articulated by Bruineberg and Hesp [36] and Kirmayer [37] in their critique of VNE. They asked whether the MB formalism leveraged by VNE to individuate systems is adequate for modelling phenomena at scales beyond those of a single organism (e.g., sociocultural dynamics); since phenomena at these scales may be too transient or not sufficiently robust to license the MB formalism (which is defined in terms of conditional independencies in weakly mixing random dynamical systems). Could this reflect a fundamental distinction between the FEP as an explanatory principle for clearly bounded, ergodic, biological systems, and the greater, complementary forces at play that constrain complex adaptive systems in general (including groups of organisms and their ecological niches)? Should we restrict the FEP to the level of the organism, and then explore how this model connects meaningfully to other key concepts in evolutionary systems theory about the agent-niche relation?

The aim of this paper is to address the challenges faced by VNE with regard to dynamics that obtain for large-scale ensembles. Specifically, we review a framework for modelling complex adaptive systems for multiscale free energy bounding organism–niche dynamics, thereby integrating the modelling strategies and heuristics of VNE with a broader perspective on the ecological nestedness of biotic systems. We extend VNE beyond the action–perception loops of individual organisms (i.e., active inference, [38], [39]) by appealing to the variational approach to niche construction (VANC) [40] to explain the dynamics of coupled systems constituted by organisms and their ecological niche. We suggest that the statistical robustness of living systems is inherited, in part, from their eco-niches, as niches help coordinate dynamical patterns across larger spatiotemporal scales. We call this approach variational ecology (VE), which subsumes VNE and the VANC.2 When applied to cultural animals such as humans, VE has the important consequence of allowing us to formulate not just a physics of individual minds, but also a physics of interacting minds across spatial and temporal scales – a physics of sentient systems that range from cells to societies.

The free energy formulation appeals to a statistical conception of life. It rests on the fact that on average and over time, living systems endure as bounded, self-organising partitions of dynamical systems. In other words, organisms appear to counter dissipative environmental perturbations by resisting the weathering effects of entropic decay dictated by the fluctuation theorems that hold at nonequilibrium steady-state [41], [42]. Technically, biological systems revisit the same set of characteristic states that constitute a random dynamical attractor. This attracting set means they have properties that can be measured (i.e., they are locally ergodic). Put another way, an organism possesses an attracting set of states that it tends to occupy with a much higher frequency than others. The FEP provides a formal description of how organisms resist entropic erosion and maintain themselves within their phenotypic bounds. More exactly, it describes the dynamics they must exhibit, if they possess characteristic or attracting states.

For an organism to exist as a bounded system means that it must be able to maintain itself as a whole. By definition, for a system to exist at all, it must evince a robust form of conditional independence with respect to external (non-systemic) states. The variational framework addresses the individuation of relevant systems by operationalising the notion of conditional independence using the Markov blanket (MB) formalism. For a set of states to be enshrouded by a MB means that the dynamics of that system induce a statistical partition of its states [3], [4], [43]. A MB is the set of states that statistically isolates (insulates) internal (systemic) from external (non-systemic) states, such that changes in internal states are mediated by the states of the MB. The MB itself can be partitioned into active and sensory states, which are defined by the following relations: internal states do not influence sensory states, and external states do not influence active states.

Now, we should note that the terms ‘active’ and ‘sensory’ are potentially misleading. They are only meant to capture relations of statistical dependence between random variables. This will be crucial to our argument below, as things that we would not readily describe as literally acting or sensing in any meaningful sense can still be captured with this formalism, since it entails only a statistical enshrouding of systemic states from external ones, and the systematic statistical partition of the whole organism–niche system [4].

To keep the MB in play is not a trivial matter. The FEP explains the emergence and maintenance of this existential boundary. Under the FEP, the Markov boundary is dynamically enabled by adaptive action. The idea behind the FEP is that the active behaviour of organisms maintains them in states of viable, adaptive coupling with their ecological niche. Active inference is the process whereby living systems act on the world – and update their internal states – so as to embody or encode the statistical structure of their local environment, leading to the adaptive control of behaviour [38], [44]. In other words, over time and on average, biological systems come to fit their environment, via active optimisation procedures; e.g., action, perception, and learning [45]. From a developmental perspective, this can be cast as phenotypic accommodation and developmental plasticity [46], [47].

Crucially, this functionalist interpretation is underwritten by the gradient flows implied by the existence of a random dynamical attractor (and implicit ergodicity). This is fairly straightforward to show using either a path integral or Fokker Planck formulation of density dynamics; in which the flow of states must climb probability gradients to counter the dispersive effects of random fluctuations. When we put the MB in play, the same gradient flows reveal an (en)active and adaptive interpretation of active states, which are informed by, and only by, sensory and internal states. This follows because the flows of active and internal states are special, in the sense that their flow does not depend upon external states. These structured dependencies, which necessarily follow from the existence of random dynamical attractors with MBs, lead to active inference.

In active inference, active and internal states of the organism's MB can always be expressed as minimising or bounding a quantity called variational free energy (see Fig. 1). We can cast variational free energy as the disattunement between the statistical structure of the ecological niche and that of the organism (i.e., its phenotype and behavioural dynamics) [26], [48]. Technically, variational free energy is a proxy for a quantity called surprise (a.k.a. surprisal), which reflects the improbability of finding an organism in some sensory state (technically, surprise is the self-information or negative log probability of sensory samples encountered by an agent) [2]. The organism cannot evaluate this quantity directly. Instead, it (or rather, its behaviour or dynamics) bounds a proxy quantity, which is a variational bound on surprise, in the sense that surprise can never be greater than this bound. This is variational free energy and is exactly the same quantity used in machine learning and variational Bayes (often referred to as an evidence bound). While surprise depends only on states of the world, variational free energy depends on a Bayesian belief or probability density that is encoded by its internal states. Through active inference, the probability densities entailed by organismic (internal) states tune themselves to effectively infer the process by which sensory states were generated; i.e., the dynamics of external, unobserved states that cause fluctuations in sensory states and are hidden behind the MB.Fig. 1The Markov blanket. These schematics illustrate the partition of states into internal (μ) and hidden or external states (η) that are separated by a MB – comprising sensory (s) and active states (a).

The upper panel shows this partition as it would be applied to action and perception in the brain; where active and internal states minimise a free energy functional of sensory states. The ensuing self-organisation of internal states then corresponds to perception, while action couples brain states back to external states. The lower panel shows exactly the same dependencies but rearranged so that the internal states are associated with the intracellular states of a cell, while the sensory states become the surface states of the cell membrane overlying active states (e.g., the actin filaments of the cytoskeleton).Fig. 1

Under the FEP, to function as a control system for the organism, and ensure it remains within phenotypic states, the dynamics (i.e., adaptive behaviour) of the organism entails a statistical model (that is, a generative model) of itself acting in its ecological niche. In this framework, a generative model is a probabilistic mapping from causes in the environment (including, crucially, the actions of the organism itself) to sensory states (observations); while the generative process is the actual causal structure that generates sensed consequences from external causes that are hidden behind the MB [49]. This inference is a necessary consequence of gradient flows that minimise variational free energy. This follows because minimising variational free energy is – mathematically – the same as maximising the evidence bound in machine learning and Bayesian statistics. In this instance, the evidence is for a generative model entailed by the internal states of a creature – technically, if the internal states parameterise a posterior density over external states, then internal (and active) states will maximise the evidence for a generative model of external, niche dynamics. Active inference therefore means that the dynamics of living systems entails a generative model. The optimisation of this model corresponds to bounding or minimising variational free energy, which can be summarised as self-evidencing [50]. This notion of self-evidencing places the MB centre stage as both an evidentiary and existential boundary.

In summary, active inference can be cast as the process that confirms and updates evidence for the statistical (generative) model that a living system entails and enacts in existing – thereby producing evidence for its own existence [50]. This neatly covers action and perception in a folk psychology sense; because internal states cannot, in themselves, change sensory states, but they can optimise the probabilistic explanations (i.e. posterior probability distributions or Bayesian beliefs) for sensory impressions by minimising variational free energy. This has all the look and feel of perception. Conversely, active states cannot change posterior beliefs but they can change sensory states; either directly or vicariously via external states. This corresponds to action, which is informed by perception. In short, the MB will actively seek evidence for its own existence. Over time, the states of the organism (e.g., the brain) come to encode the statistical structure of causal regularities in the world, and underwrite the generative model – the behavioural control system (cf. [51], [52]) that regulates patterns of interaction with the environment. This sort of slow perception can be regarded as learning and is generally associated with plasticity of a developmental or experience-dependent sort in internal states that comprise internal structure and connectivity.

Two recent extensions of the free energy formulation will be the focus of our attention. These are variational neuroethology (VNE) [4] and the variational approach to niche construction (VANC) [40]. VNE and the VANC are about enabling the application of the FEP to phenomena within and beyond the brain. These approaches hold the promise of extending the variational (free energy) approach to the dynamics of sentient systems (i.e., systems with sensory states) across spatial and temporal scales.

VNE provides a framework for modelling the dynamics of sentient systems across the spatiotemporal scales they manifest and an explanation or description of their self-organisation. Formally, VNE provides a principled way of scaling up active inference over: (i) ensembles of MBs; and (ii) MBs of MBs. This allows us to formulate an integrative multiscale dynamics that link the partial dynamics of phenomena at each scale (also see [53]). In what follows, we speak to the multiscale aspects of MBs and then turn to ensembles of MBs that are coupled to each other.

VNE synthesises the process theory derived from the FEP (active inference) with Tinbergen's seminal four questions in biology (i.e., adaptation, phylogeny, ontogeny, and mechanism) to propose a heuristic guide to research in the sciences of life and mind. Integrating these two paradigms allows substantive insights drawn from one to inform and constrain models and research in the other – the FEP is a non-substantive principle that can be applied to any biological system in general (much like Hamilton's principle of least action), while Tinbergen's framework can provide substantive explanations for biological phenomena drawn from four complementary levels of analysis [54]. This heuristic has already inspired an interdisciplinary theory of the human brain called the Hierarchically Mechanistic Mind ([4], Badcock, P.B., Friston, K.J., Ploeger, A., Hohwy, J., and Ramstead, M.J., in review, [23]). The HMM rests on the idea that the brain is a hierarchically structured, self-organising system that has been sculpted by natural selection [23], [25], [55], [56]. It suggests that the dynamics, structure, and function of the human brain instantiate an embodied, situated, complex adaptive system that actively minimises free energy by generating adaptive action–perception cycles via recursive interactions between hierarchically nested, functionally differentiated subsystems ([23] Badcock, P.B., Friston, K.J., and Ramstead, M.J.D. (this issue)). By synthesising the FEP with Tinbergen's four levels of analysis, this perspective can be reduced to four complementary research questions – What is the adaptive function of a phenotypic trait? How does it emerge from circular interactions between phylogenetic (resp.

intergenerational), ontogenetic and mechanistic processes? In what ways does it instantiate the FEP? And how does it manifest in hierarchical neural dynamics? This research heuristic has already been leveraged to develop a new evidence-based theory of our capacity for depression [25].

On the other hand, VNE entails a multiscale ontology for living systems as well. This ontology extends the MB formalism as a method of individuating systems under the FEP. As discussed above, a MB is a set of states that separates a system (a set of internal states) from external, non-systemic ones, which operationalises the idea that systems only exists per se if they are endowed with a robust form of conditional independence. According to VNE, the dynamics of living systems can be cast as active inference over recursively nested MBs (see Fig. 2). Indeed, the MB ontology can be reiterated recursively, such that the MBs at any one scale are composed in turn of MBs at the scale above and below – which are also made of MBs, and so on, all the way up, and all the way down [43].Fig. 2Explanatory scope of variational approach. Variational neuroethology leverages the FEP to explain the adaptive free energy bounding dynamics of living systems across spatial and temporal scales. Here we indicate some of the scales at which these dynamics unfold. Given the intrinsic correlation between spatial and temporal scales, the phase space described here is populated mostly along its diagonal. Adapted from [92].Fig. 2

The key notion here is that in moving from one level or scale of dynamics to the next, things not only get bigger, they also get much slower. The basic idea is that the states at one scale constitute microscopic states that can be partitioned into an ensemble of MBs. To move to the higher scale, one treats each MB as an entity (e.g., particle) and summarises its dynamics with mixtures of blanket states that fluctuate slowly. In this multiscale setting, a (effective) state at any scale becomes the expression of an eigenmode of blanket states; namely, the principal eigenvectors of their Jacobian (i.e., rate of change of flow with respect to state). These mixtures are formally identical to order parameters in synergetics that reflect the amplitude of slow, unstable eigenmodes [57]. In terms of centre manifold theory, they correspond to solutions on the slow (unstable or centre) manifold [35], [58]. In short, the MB of a system (or particle), at any scale, constitutes an ensemble whose order parameters subtend blanket or internal states at the scale above. Note that the constituent (microscopic) states of an ensemble are always blanket states, although their order parameters could be blanket states or internal states at the (microscopic) scale above. This follows from the fact that the only states ‘that matter’ are those that influence other (blanket) states. Effectively, all we are doing here is applying the (en)slaving principle, or centre manifold theorem [57], recursively to MBs of MBs. Fig. 3 provides a schematic illustration of this recursive decomposition.Fig. 3Blankets of blankets. This schematic figure illustrates the recursive architecture by which successively larger (and slower) scale dynamics arise from subordinate levels. Starting at the bottom of the figure (lower panel) we can consider an ensemble of vector states (here nine).

The conditional dependencies among these vector states then define a particular partition of into particles (upper panels). Crucially, this partition equips each particle with a bipartition into blanket and internal states, where blanket states comprise active (red) and sensory states (magenta). The behaviour of each particle can now be summarised in terms of (slow) eigenmodes or mixtures of its blanket states to produce vector states at the next level or scale. These constitute an ensemble of vector states and the process starts again. The upper panels illustrate the bipartition for a single particle (left panel) and an ensemble of particles; i.e., the particular partition per se (right panel). The insets on top illustrate the implicit self-similarity in moving from one scale to the next using. In this figure, Ω ⋅ b denotes a linear mixture of blanket states specified by the principal eigenvectors of their Jacobian. Because the corresponding eigenvalues play the role of Lyapunov exponents, the resulting mixtures correspond to slow or unstable dynamical modes of activity.Fig. 3

In this multiscale framework, active inference is inherently a group activity. That is, the entire ensemble of nested MBs are bound, enslaved and constrained by dynamics at higher scales, while the lower (microscopic) scales furnish the (macroscopic) states at any given level. This construction evinces exactly the same circular causality that underlies synergetics [57], [59]; however, here it is generalised to a recursive hierarchy of scales – i.e., the hierarchical composition of blankets of blankets. Intuitively, the dynamics at one scale provide constraints (technically, establish probability gradients) on dynamics at other scales. Active inference destroys free energy gradients at each scale,3 under the guidance or control of a generative model at the scale above. This guidance is exerted through influences on sensory states, where circular causality means that the action of any MB in an ensemble of MBs could be involved in sensing, action or perception; depending upon its role at the superordinate scale; i.e., has a sensory, active or internal state at the level above. Please see [43] for a worked example using simulations of biological self-assembly.

This may sound a little abstract; however, imagine you are an employee as an institution, where you transact your (microscopic) affairs with other personnel to self-evidence your prior beliefs that you are ‘good at your job’. This would entail responding to corporate or institutional goals that emerge collectively (i.e., an implicit generative model at the macroscopic level). Note that your job may be homologous to an internal state at the institutional (macroscopic) level – relating only to other employees. Alternatively, you could be working on reception (i.e., a sensory state) or issuing press releases (i.e., an active state). Another example of multiscale self-organisation is provided in Fig. 4 to illustrate a less anthropomorphic form of self-assembly at the cellular and molecular level. In this example, the extensive nature of variational free energy is laid bare: each system or agent that comprises the ensemble shares the same generative model. This means that the total free energy is composed in exactly the same way statisticians would accumulate statistical evidence through Bayesian belief updating with each new source of evidence.4 The twist here is that the (sensory) evidence for each agent's model is generated by another agent. In short, multiscale ensembles that endure, in an ergodic sense, dissolve free energy gradients, thereby integrating dynamics within and between scales. This brings us to questions about coupling among MBs within a particular scale, which will be our focus for the remainder of the paper.Fig. 4Self-assembly and active inference. This figure shows the results of a simulation of morphogenesis under active inference reported in [70]. This simulation used a gradient descent on variational free energy using a simple ensemble of eight cells; each of which had the same (pluripotential) generative model.

This generative model predicted what each cell would sense and signal (chemotactically) for any given location in a ‘target morphology’ (lower middle panel – extracellular target signal; in other words, what each agent would expect to sense if it were at a particular location). By actively moving around, all the cells minimised their variational free energy (i.e., surprise) by inferring where they were, in relation to others. Because variational free energy is an extensive quantity, the free energy minimising arrangement of the ensemble is the target morphology. In other words, every cell has to ‘find its place’, at which point each cell minimises its own surprise about the signals it senses (because it knows its place) and the ensemble minimises the total free energy. The upper panels show the time courses of expectations about its place in the morphology (upper left), the associated active states mediating migration and signal expression (upper middle) and the resulting trajectories; projected onto the first (vertical) direction – and colour-coded to show differentiation (upper right). These trajectories progressively minimise total free energy (lower left panel). The lower right panel shows the ensuing configuration. Here, the trajectory is shown in small circles (for each time step). The insert corresponds to the target configuration. Please see [70] for further details.Fig. 4

In what follows, we look more closely at the partition of states into an ensemble of MBs – and what this means for self-organisation at any scale of dynamics. A crucial point, which enables the integration of ensemble dynamics into adaptive behaviour, is that variational free energy is an extensive quantity; in other words, it increases with the size or compass of the system in question. In the context of an ensemble of MBs, this means we can add the free energy of each agent or particle to describe the behaviour of the ensemble (see Fig. 4).

Applying the MB formalism to sentient systems is intuitive when modelling systems whose conditional independence is maintained via literal sensorimotor loops, where the interaction between active and sensory states ranges over relatively small spatiotemporal scales. Note that every leap upward in the nested hierarchy implies a concomitant increase in spatial and temporal scale (see Fig. 3), which in some cases entails an increase in the distances among the states of the system, as well as a decreased rate of change and (Bayesian) optimisation. As we ascend spatiotemporal scales, it becomes increasingly unclear how the Markov boundaries are implemented. Indeed, the justification for the use of the MB formalism to describe living systems rests on the observation that if the coupling among an ensemble of dynamical subsystems is mediated by short-range forces, then the states of those remote subsystems must be conditionally independent [3]. In summary, despite providing a framework to integrate dynamics across scales – and ensemble dynamics within a scale – VNE does not tell us how large scale ensembles self-organise so as to form higher-order MBs. In the next section, we address this issue; namely, how to draw MBs for large-scale systems like ecological niches, social ensembles, and cultural dynamics, drawing on the notion of affordances from the skilled intentionality framework [48], [60].

Niche construction theory in evolutionary biology (e.g., [61], [62], [63], [64]) argues that via their bioregulatory behaviour, living organisms (explicitly and implicitly) modify their environment, so as to steer their evolutionary trajectory, and that of other species. Arguably, then, niche construction is on a par with natural selection as a bona fide evolutionary force. Niche construction can be understood in two ways: in terms of selective niche construction (SNC), or in terms of developmental niche construction (DNC) [64]. SNC refers to changes to the ecological niche induced by the action of organisms, and by which living systems come to modify selection pressures on themselves and other species that inhabit the niche. SNC operates on a phylogenetic time scale, and involves processes like ecological and cultural inheritance [65], [66], where evolutionarily significant components of an environment are passed on from one generation to another; e.g., the remains of beaver dams are reconstructed by beavers, while the concept of a ‘canoe’ is passed down generations in the form of ‘cognitive gadgets’ [67]. DNC refers to the production in ontogeny of exogenetic resources by organisms themselves; in order to change developmental inputs, and secure the reliable and flexible reproduction of the individual life cycle [64]. DNC operates on the scale of development, learning, and action–perception cycles. The set of exogenetic resources that it optimise's include evolved loops of adaptive behaviour (e.g., grooming, parental care) and physical changes to the niche itself [68].

As an example of niche construction outcome in humans, consider improvised ‘desire paths’; e.g., a dirt trail in the park carved out by recurrent actions of agents in the neighbourhood. From the point of view of the FEP, such a path can function as an exogenetic resource. It encodes precise (high certainty, reliable) information about the fact that the end of the park is at the end of the trail. An agent can rely on this information to navigate the park efficiently, without having to know the layout of the neighbourhood. While crossing a park might not be as essential a behaviour as grooming, from the point of view of the FEP, finding oneself in expected sensory states certainly is. Indeed, exogenetic resources of the niche such as desire paths can function as reliable indicators of surprise- and ambiguity-resolving actions, which, under the statistical conception of the phenotype entailed by the FEP, is crucial for maintaining the agent's continued existence. In short, the collective behaviour of an ensemble of agents provides a form of semiotics or a set of possibilities for engagement with the niche (a.k.a. affordances) that become relevant from the point of view of the needs and concerns of any single agent. So how can a variational ecology shed light on the role of affordance in selecting adaptive actions that emerge from ensemble dynamics?

Selecting adaptive actions requires an organism to evaluate expected free energy under possible action policies [69]. Expected free energy can be expressed in different ways; e.g., as expected energy minus entropy, or as a mixture of epistemic and pragmatic value (see Fig. 5). With respect to our purposes here, expected free energy can be expressed as the expected cost of an outcome, given a certain action, plus the expected ambiguity of the outcome (cf. [70]). Expected cost corresponds to the discrepancy between outcomes conditional on a given action, and expectations or preferences about outcomes (technically, the Kullback–Leibler divergence between the two beliefs). Evaluating expected cost enables the selection of actions that bring about sensory states (observations) expected by the agent – i.e., those predicted by its generative model, and by the same token, those that once brought about, have the least deleterious or ‘costly’ consequences with regard to their surprisal. In turn, the ambiguity reflects an agent's expectations about the uncertainty of outcomes, dependent upon causes in the world.Fig. 5Bayesian mechanics and active inference. This graphic summarises the belief updating scheme in the minimisation of variational free energy and expected free energy [38], [45], [93]. In the first step (circles on the left), discrete actions solicit a sensory outcome (i.e., in the parlance of the SIF, a solicitation) used to form approximate posterior beliefs about states of the world. This belief updating involves the minimisation of free energy under a set of plausible policies (blue panel – Perceptual inference). Note that free energy F(π,s) includes Markovian dependencies among hidden states. This reflects the fact that the generative model is a Markov decision process.