Dopamine, affordance and active inference

The role of dopamine in behaviour and decision-making is often cast in terms of reinforcement learning and optimal decision theory. Here, we present an alternative view that frames the physiology of dopamine in terms of Bayes-optimal behaviour. In this account, dopamine controls the precision or salience of (external or internal) cues that engender action. In other words, dopamine balances bottom-up sensory information and top-down prior beliefs when making hierarchical inferences (predictions) about cues that have affordance. In this paper, we focus on the consequences of changing tonic levels of dopamine firing using simulations of cued sequential movements. Crucially, the predictions driving movements are based upon a hierarchical generative model that infers the context in which movements are made. This means that we can confuse agents by changing the context (order) in which cues are presented. These simulations provide a (Bayes-optimal) model of contextual uncertainty and set switching that can be quantified in terms of behavioural and electrophysiological responses. Furthermore, one can simulate dopaminergic lesions (by changing the precision of prediction errors) to produce pathological behaviours that are reminiscent of those seen in neurological disorders such as Parkinson's disease. We use these simulations to demonstrate how a single functional role for dopamine at the synaptic level can manifest in different ways at the behavioural level.

This article is about set switching and action selection during the execution of cued responses. It offers a straightforward account of dopamine in optimising behaviour in the context of (Bayes-optimal) predictive coding. Our focus is on the consequences of depleting dopamine and simulating perseveration that is characteristic of Parkinson's disease, using synthetic neuronal models based upon active inference [1]. In brief, the emergent role of dopamine is to report the precision or salience of perceptual cues that portend a predictable sequence of sensorimotor events. In this sense, it mediates the affordance of cues that elicit motor behaviour [2]; in much the same way that attention mediates the salience of cues in the perceptual domain. Gibson defined affordances as action possibilities latent in the environment [3], objectively measurable and independent of the ability to recognize them but always in relation to the actor [4]. Affordance is therefore an attribute of a cue and has to be inferred. Crucially, in this paper, inferring that an object has affordance necessarily entails an action. We hope to establish a central role for dopamine in this inference and implicit action selection.

Dopamine has been implicated in a bewildering variety of processes and pathologies in the human brain; ranging from cortical excitability to attentional deficits [5], [6]; from motor control to akinesia and set switching deficits in Parkinson's disease [7], [8], [9]; from working memory to schizophrenia [10], [11]; from reinforcement learning to addiction [12], [13]; from executive function to age-related cognitive decline [14]; from reward prediction to failures of incentive salience [15], [16]; from exploration to psychomotor poverty [17]. In fact, it is difficult to find an area of neuroscience that does not implicate dopamine; for example, it has key roles in mood, sleep, nociception, and prolactin production. In terms of its functional or computational roles, it has been suggested that dopamine reports reward prediction errors, hedonic value, incentive salience, novelty, and so on. Many accounts appeal to optimal decision theory and reinforcement learning to understand the putative role of dopamine in formal terms [18], [19], [20], [21]. These treatments rest on the assumption that behaviour is optimal in relation to some reward or cost function and invoke various heuristics from control theory (e.g., dynamic programming) to explain the computational role of dopamine.

This article takes a different view and considers that behaviour is Bayes-optimal in the sense that it maximises the Bayesian evidence for an actor's model of the world or, equivalently, minimises surprise. We have formulated this as active inference in a series of previous papers [1], [22], [23]. Active inference can be seen as an embodied (enactivist) form of predictive coding, in which perception minimises exteroceptive prediction errors and action minimises proprioceptive prediction errors. Put simply, active inference is predictive coding with classical motor reflexes. In this setting, cost functions are replaced by surprise or prediction error, in the sense that the only optimal behaviour is a behaviour that brings about expected outcomes (i.e., minimises surprise as opposed to cost). This ensures that agents avoid potentially harmful or surprising exchanges with the environment and equips them with a physiological and ethological homoeostasis. Note that this does impose constraints on behaviour, since appropriate priors can replicate the effect of any cost function [1]. In short, rewards are just familiar sensory states. This perspective has three advantages over reward-based accounts: first, it resolves the tautology inherent in the notion of reward. This tautology follows from the fact that reward is used to explain reward seeking behaviour [24], while at the same time being defined in terms of its ability to elicit reward seeking behaviour. Second, it dispenses with the intractable solutions of control theory problems (e.g., Bellman optimality equations) that are necessary to optimise reward or cost functions [25], [26], [27]. Finally, and central to this paper, it provides a novel perspective on the role of dopamine that accounts for its apparently diverse roles in terms of a single mechanism, operating at different levels of the sensorimotor hierarchy.

The Bayesian perspective suggests that there are only two sorts of things that need to be inferred about the world; namely, the state of the world and uncertainty about that state. We have suggested that predicted states of the world are encoded in terms of synaptic activity, while uncertainty is encoded by synaptic gain that encodes the precision (inverse amplitude or variance) of random fluctuations about predicted states [28]. If true, this means that modulators of synaptic gain (like dopamine) do not report perceptual content but the context in which percepts are formed. In other words, dopamine reports the precision or salience of sensorimotor constructs (representations) encoded by the activity of the synapses they modulate. This leads to a view of dopaminergic projections that select salient processing channels and associated actions. Physiologically, this is compatible with short latency dopamine bursts in the basal ganglia that occur after any salient event, whether rewarding or not [29]. In this view, dopaminergic discharges do not signify reward prediction errors but are an integral part of Bayes-optimal perception and sensorimotor integration: they respond to salient or precise cues that portend a predictable sequence of sensorimotor events that will be registered by specific proprioceptive and exteroceptive processing channels. Crucially, these responses will appear to be reward-related, because they precede sensorimotor sequences that lead to a rewarding (familiar) state. In other words, if a sequence of choices is predictable they lead to unsurprising outcomes, which are, by definition, rewarding. However, the agent may have no concept or representation of reward; it is just doing what it expects to do in the context established by the pattern of dopamine firing.

The motivation and mechanisms behind the role of dopamine considered in this paper are exactly the same as we have proposed for the attentional modulation of postsynaptic gain in sensory processing [30]. Both attentional modulation and dopaminergic gating may represent a Bayes-optimal encoding of precision that enhances the processing of particular sensory representations by selectively biasing bottom-up sensory information (prediction errors). In other words, it confers salience on attended representations [31], [32], [33], [34], [35], [36], [37]. The specific role of dopaminergic neurotransmission in behaviour (as opposed to perception) may be explained by the regional specificity of its projection fields (and postsynaptic receptor subtypes) that are mainly confined to cortical and subcortical structures concerned with predicting choices and motor responses [38]. This is important because it means that dopamine may be exclusively concerned with salient representations that have affordance; in other words, sensorimotor representations that predict both perceptual and behavioural consequences. If true, this means dopamine has a crucial role in biasing sensorimotor integration and action selection. More formally, dopamine is in a position to select the proprioceptive and exteroceptive signals (prediction errors) that compete for higher level explanation by controlling their precision. This formulation sits comfortably with the affordance competition hypothesis [2], [39] and other theoretical accounts: for example, the uncertainty processing theory of motivation [40], neurobiological accounts of decision-making [41], [42] and the plurality of roles suggested by the physiology of dopamine [43].

In particular, it draws on the same notions that link dopamine to the encoding of uncertainty [44] and adaptive responses to changes in neuronal signal to noise levels [45], [46]. This is because precision represents uncertainty due to random fluctuations or noise. By associating salience with precision we can also connect to constructs like incentive salience in psychology [47] and aberrant salience in psychopathology [48]. Indeed, it has been shown that action selection can be cast as signal selection using salience to report the “propensity for selecting a given action” [49]. Note that precision or salience is an attribute of a (probabilistic) representation that determines the confidence or certainty about what is represented; where the salience of sensory representations can be manipulated experimentally, by changing signal to noise levels or contrast. We will see an example of this later. Finally, the notion that dopamine modulates synaptic gain plays a key role in several proposals. It has been argued for in [50] and used in several Parkinson's disease modelling papers [51], [52].

In this paper, the hypothesis that dopamine release reports precision or uncertainty is based purely on its synaptic physiology. However, there is definitive neurophysiological evidence for this role of dopamine [44], where, for example, dopaminergic discharges covary with the variance or precision of juice rewards [53]. More generally, nearly every experimental manipulation evoking dopaminergic responses (novelty, unexpected rewards, etc) speaks to a change in the level of precision or confidence about subsequent contingencies. In what follows, we try to substantiate the above ideas using theoretical arguments based upon active inference and then illustrate their plausibility using simulations of cued responses. These simulations are concerned with the consequences of depleting dopaminergic neurotransmission to illustrate its central role in action selection and set switching. They can therefore be regarded as a very simple model of Parkinson's disease. This means we will not address changes in precision but assume that the tonic activity of dopaminergic neurons encodes a fixed level of precision or uncertainty [44], [43], [45]. Subsequent papers will focus on the control of (phasic and tonic) dopaminergic responses per se and will try to reproduce the empirical findings of behavioural reinforcement paradigms, using phasic dopamine discharges that shift striatial neurons into an up state to increase their gain or precision [54]. Furthermore, we will restrict our discussion to the generic effect of dopamine on postsynaptic D1 receptors located on principal cells throughout the brain [55]. This necessarily precludes a proper consideration of the balance between D1 and D2 receptor function and its relationship to the functional anatomy of the basal ganglia in Parkinson's disease [56].

The models and methods section reviews the theory on which subsequent simulations are based. This section presents a brief review of the free energy principle, with a special focus on active inference and the role of synaptic gain in encoding precision. The basic theory and ensuing differential equations used to simulate neuronal responses are exactly the same as those used to illustrate perceptual inference, learning, attention and action in a series of previous papers (Table 1). This formalism is then used to model sequential cued movements, under normal and, in the results section, depleted levels of dopamine. The particular simulations used in this paper rest on prior beliefs about sensorimotor trajectories, encoded by itinerant (wandering) dynamics in premotor cortex. These dynamics are entrained by prediction errors from the superior colliculus, the parietal cortex and motor cortex, whose precision is, we assume, controlled by dopamine. Using this architecture, we can simulate visually cued sequences of movements and, crucially, responses to sequence violations. By adding a further (prefrontal) level to the model, we examine how the ability to switch from one sequence to another is compromised when sensorimotor cues lose precision. The resulting impact on set switching is characterised in terms of (synthetic) neuronal responses and behavioural (reaction time and accuracy) measures. We conclude with a discussion of the implications for dopamine in motor control and set switching generally, and for Parkinson's disease specifically.

In this section, we briefly overview the free energy principle and active inference to frame the role of dopamine examined later. The free energy principle proposes that the states and infrastructure of a self organising system, such as the brain, should minimise the free energy of the sensory states it samples [57]. Free energy is an upper bound on the surprise associated with sensory signals, where surprise is mathematically the same as the (negative log) Bayesian evidence for the system's model of its world. Evidence is just the probability of getting some data under the model of those data. This means that minimising free energy reduces surprising exchanges with the environment or, equivalently, maximises the evidence for an agent's internal model of its sensorium. This principle entails two corollaries; the Bayesian brain hypothesis [58], [59]; [60], [61] and active inference [1], [22], [23]. The Bayesian brain hypothesis means that the brain will try to predict its sensory inputs in a Bayes-optimal fashion by representing their causes in terms of hidden states of the world. Active inference equips the Bayesian brain with motor reflex arcs that ensure its predictions are fulfilled (by suppressing proprioceptive prediction errors). In active inference, behaviour emerges as natural consequence of high-level representations (sensorimotor constructs) that have both sensory (exteroceptive) and motor (proprioceptive) consequences. These constructs or representations are maintained by bottom-up prediction errors in both modalities and reciprocate top-down (proprioceptive) predictions to the peripheral motor system that drive classical motor reflexes; while top-down predictions to sensory systems play the role of corollary discharge and suppress (exteroceptive) prediction errors.

Crucially, high-level sensorimotor representations can be dynamic in nature, with itinerant dynamics (on attractor manifolds) that embody prior beliefs about the sequence of sensorimotor events or trajectories that will unfold in the near future. These can be regarded as central pattern generators or attractors that provide proprioceptive and sensory predictions for sensorimotor integration; in other words, representations of affordance [23]. The particular sequence currently active depends upon which attractor has been selected. This selection rests upon precise bottom-up prediction errors conveying salient sensory information that has yet to be explained. In this scheme, prediction errors can induce or destroy metastable attractors at higher levels to select the trajectory that best explains sensory input. This can be regarded as selecting an attractor with an affordance that best explains sensory input; cf., “affordance competition” in [2]. The potency with which ascending prediction errors can select the appropriate attractor depends upon their postsynaptic gain. This gain encodes the precision (inverse variance) of random fluctuations about predictions. In other words, the ability of bottom-up prediction errors to bias competition among high level sensorimotor representations (attractors) depends upon their precision that we presume, in this paper, is modulated by dopamine. In the results section, we will see an example of these dynamics and what happens when the precision of prediction errors is reduced by (simulating) a reduction in dopaminergic neurotransmission. We hypothesised that this would result in a failure of set switching and the perseveration of sensorimotor dynamics of the sort seen in Parkinson's disease [7], [62], [63]. In what follows, we will unpack the above summary in slightly more formal terms:

The equations and simulations used in this paper may appear a bit complicated and ad hoc; however, they are based on just three assumptions:

The brain minimises the free energy of sensory inputs defined by a generative model.

Neuronal firing rates encode the most likely state of the world, under this model.

The first assumption is the free energy principle, which leads to active inference when considering both representations and action. This principle has to be true, in the sense that a failure to minimise free energy means that the brain will entertain increasingly surprising sensations and, at some point, will cease to exist in an ergodic sense. The second assumption is motivated easily by noting that the world is both dynamic and nonlinear. Hierarchical causal structure emerges inevitably from a separation of temporal scales. This can be seen most clearly in the slaving principle from statistical physics [64], [65], where slow ordered dynamics emerge at a macroscopic scale and enslave fast fluctuations at a microscopic scale. Finally, the third assumption follows from the first, under the constraint that probabilistic representations are encoded by a minimum number of biophysical variables. This leads to something called the Laplace assumption, in which the probability density function over hidden states is Gaussian and can be encoded by its mean or expectation. In terms of neural codes, this is referred to this as the Laplace code and is arguably the simplest and most flexible of all neural codes [66].

Given these three assumptions, one can simulate a whole variety of situations and processes by simply specifying the particular equations that constitute the generative model. The resulting perception and action is specified completely by the above assumptions and can be implemented in a biologically plausible way as described in previous applications listed in Table 1. In brief, these simulations use differential equations that minimise the free energy of sensory input using a generalised gradient descent [67].(1)These coupled differential equations describe perception and action respectively and just say that internal brain states and action change in the direction that reduces free energy. The first is known as (generalised) predictive coding and has the same form as Bayesian (e.g., Kalman-Bucy) filters used in time series analysis; see also [68]. The free energy depends on three things; sensory signals , conditional expectations or representations and a model . The model defines how expectations about states of the world conspire to produce sensory input. In neurobiological formulations, these expectations or predictions are associated with neuronal activity and the model comprises a connectivity or network architecture. The∼notation denotes variables in generalised coordinates of motion that include velocity, acceleration, jerk and so on; . The first term in Equation 1 is a prediction based upon a matrix differential operator that returns the generalised motion of the expectation, such that . The second term is usually expressed as a mixture of prediction errors that ensures the changes in conditional expectations are Bayes-optimal predictions about hidden states of the world. The second differential equation says that action also minimises free energy.

The differential equations are coupled because sensory input depends upon action, which depends upon perception through the conditional expectations. This circular dependency leads to a sampling of sensory input that is both predicted and predictable, thereby minimising free energy and surprise.

To perform neuronal simulations under this framework it is only necessary to integrate or solve Equation 1 to simulate neuronal dynamics that encode the conditional predictions and ensuing action. Conditional predictions depend upon the brain's generative model of the world, which we assume has the following (hierarchical) form(2)This equation is just a way of writing down a model that relates various quantities in the world probabilistically in terms of their generalised motion. Here, are nonlinear functions of hidden states and causes that generate sensory inputs at the first (lowest) level. Random fluctuations on the motion of hidden states and causes are conditionally independent and enter each level of the hierarchy. It is these that make the model probabilistic. They play the role of sensory noise at the first level and induce uncertainty about states at higher levels. The (inverse) amplitudes of these random fluctuations are quantified by their precisions; , which we assume to be fixed in this paper and encoded by dopamine. Hidden causes link hierarchical levels, whereas hidden states link dynamics over time. Hidden states and causes are abstract quantities (like the motion of an object in the field of view) that the brain uses to explain or predict sensations. In this hierarchical model, the output of one level acts as an input to the next. This input can produce complicated (generalised) convolutions with deep (hierarchical) structure. We will see an example of this later.

Given the form of the generative model (Equation 2) we can now write down the differential equations (Equation 1) describing neuronal dynamics in terms of (precision-weighted) prediction errors on the hidden causes and states . These errors represent the difference between conditional expectations and predicted values, under the generative model:(3)At the lowest level of the hierarchy, conditional expectations are replaced by sensory input. This equation can be derived fairly easily by computing the free energy for the hierarchical model in Equation 2 and inserting its gradients into Equation 1. What we end up with is a relatively simple update scheme, in which conditional expectations are driven by a mixture of prediction errors, where prediction errors are defined by the equations of the generative model.

It is difficult to overstate the generality and importance of Equation 3: its solutions grandfather nearly every known statistical estimation scheme, under parametric assumptions about additive or multiplicative noise [69]. These range from ordinary least squares to advanced variational deconvolution schemes. The resulting scheme is called generalised filtering or generalised predictive coding [67]. In neural network terms, Equation 3 says that error-units receive predictions from the same level and the level above. Conversely, prediction-units are driven by prediction errors from the same level and the level below. These constitute bottom-up and lateral messages that drive conditional expectations towards a better prediction to reduce the prediction error in the level below. This is the essence of recurrent message passing between hierarchical levels to optimise free energy or suppress prediction error: see [70], for a more detailed discussion. In neurobiological implementations of this scheme, the sources of bottom-up prediction errors, in the cortex, are thought to be superficial pyramidal cells that send forward connections to higher cortical areas. Conversely, predictions are conveyed from deep pyramidal cells, by backward connections, to target (polysynaptically) the superficial pyramidal cells encoding prediction error [71], [70]. The laminar specificity of the cells of origin of predictions and prediction errors becomes relevant when examining the putative role of dopamine in the encoding of precision.

Equation 3 shows that precision modulates the responses of prediction error units to their presynaptic inputs. Here, we associate precision with dopaminergic neuromodulation of these responses. The action of dopamine is mediated by a family of transmembrane G protein-coupled receptors [72] encoded by at least five dopamine receptor genes [73]. Dopamine receptors are found throughout the soma and dendrites of neurons but ultra-structural and biochemical evidence suggests that they are concentrated in dendritic spines that express glutamatergic synapses [74], [75]. Postsynaptic D1 and D2 receptors are therefore strategically positioned to control the excitability and synaptic properties of spines “with remarkable precision and versatility” [76].

Although dopamine appears to be a natural candidate to modulate principal cells reporting prediction error, Equation 3 does not tell us whether prediction errors are modulated before or after they are computed. Both mechanisms are biologically plausible: for example, superficial pyramidal cells encoding prediction error could be modulated by D1 receptors on the soma or initial segment, after the integration of signals subtending prediction errors in the dendritic tree. Conversely, precision dependent modulation could be applied at a synaptic level to all presynaptic inputs. For dopamine, the balance of evidence points to the latter mechanism [77]:

Dopamine innervation in the human prefrontal cortex exhibits a distinct bilaminar distribution with dense bands of fibres in the superficial and deep layers [78], [74]. Although some evidence suggests that dopaminergic markers are more concentrated in deep layers [79], [80], other studies report a higher concentration in supragranular layers [81]. These differences may be due to regional and species differences [82]. Here, we will focus on dopaminergic modulation of cells in the supragranular layers, because superficial pyramidal cells are thought to report prediction error [71]. Dopamine axons form symmetric synapses, predominantly on the spines of pyramidal cells. In many cases, the same spine expresses an asymmetric (excitatory) synapse. In human and monkey prefrontal cortex, the dopamine D1-specific ligand, 3H-SCH23390, and the D2-specific ligand, H3-raclopride, label binding sites that mirror the densest dopamine innervation [78], [83]. This suggests that the primary role of D1 receptors is to modulate presynaptic input to pyramidal cells at the dendritic level [77], [74]. In terms of the implicit computational architecture, we can therefore assume that dopamine gates or modulates the dendritic responses of superficial pyramidal cells, such that dopamine selects afferents encoding sensory information (prediction error) in proportion to its precision.

Figure 1 provides a schematic of the neuronal circuitry implied by this assumption, in which dopamine modulates doubly-innervated spines of superficial pyramidal cells receiving excitatory and inhibitory presynaptic inputs [84]; corresponding to conditional expectations and their predictions respectively. The opposing effects of these presynaptic inputs on postsynaptic depolarisation form a prediction error signal that is modulated at the level of the dendritic spine by dopamine. In this scheme, the tonic firing of a particular dopaminergic cell or population encodes the precision or salience of the information (prediction error) conveyed by the cells that it targets. One might imagine that phasic discharges report changes in the current context and signal a change in the relative precision (uncertainty) over different sensory channels and conditional predictions. In this paper, we will assume that the precision at each hierarchical level is constant and address the (phasic) control of dopaminergic activity in terms of optimisation of (state-dependent) precision elsewhere.

Clearly, in sensorimotor hierarchies [85], the relative levels of neuromodulatory gating (precision) at different levels can have a profound effect on perception and behaviour, because it will select particular processing channels and change the balance between bottom-up sensory information and top-down prior expectations. In hierarchical models, these prior expectations are called empirical priors, because they are optimised in relation to sensory data. Furthermore, precision or neuromodulation of synaptic gain will affect action and motor control, because action is driven by (proprioceptive) prediction errors at the sensory level that have their own gain or precision:(4)This follows because the only way that action can minimise free energy is to change sensory prediction errors by selecting which sensory signals are sampled. As noted above, the ensuing suppression of proprioceptive prediction errors can be thought of in terms of classical motor reflex arcs: see [22] for details.

Affordance is generally conceived of as the opportunities for action offered by the environment to an agent [4]. This depends on both the environment and the nature of the actor. For example, an axe only affords the possibility of use when it can be wielded. In this paper, affordance is an attribute of amodal representations at higher hierarchical levels that make both sensory and motor predictions (an ‘axe’ entails predictions not only about how it looks or feels, but also the kinaesthetic consequences of wielding it). Conditional expectations or representations with affordance elicit behaviour by sending top-down predictions down the hierarchy that are unpacked into proprioceptive predictions at the level of the cranial nerve nuclei and spinal-cord. These engage classical reflex arcs to produce the predicted motor trajectory. The action of dopamine in this context is to modulate or enable the salience of representations that have affordance, and hence the probability they will be enacted.

In summary, we have derived equations for the dynamics of perception and action using a free energy formulation of adaptive (Bayes-optimal) exchanges with the world and a generative model that is both generic and biologically plausible. In what follows, we use Equations 3 and 4 to simulate neuronal and behavioural responses. A technical treatment of the material above will be found in [67], which provides the details of the scheme used to integrate (solve) Equation 1 to produce the simulations considered next.

The preceding scheme allows one to simulate (Bayes-optimal) responses in terms of neuronal activity and motor behaviour, under any plausible generative model. Here, we consider a particular model, described in terms of the functions in Equation 2 that leads to a sequence of pointing movements, elicited by a sequence of visual cues. This model of sensorimotor integration provides the basis for simple simulated lesion experiments, in which we can deplete levels of simulated dopamine (precision) in different parts of the brain, and examine the consequences.

Because the differential equations governing perception and action are coupled (Equation 1), we need to specify two mappings: the generative model used by the brain, whose inversion maps from sensations to action, and the process by which action produces sensations. To distinguish between real states generating sensory information and the hidden states assumed by the generative model, we will use bold and italic variables respectively. Sensory input was generated using the following equations, which constitute real world dynamics that are hidden from the agent:(5)This equation specifies how hidden states in the world produce sensory inputs and how those states change in response to action and hidden causes. Although it may look complicated, the implicit dynamics are very simple: The first equation expresses sensory input as a nonlinear function of true states in the world that comprise the position of a single jointed arm in terms of vertical and horizontal angles of displacement from a resting position ; and the salience of four locations in extrinsic (visual) coordinates. Salience corresponds to appearance of visual cues at the four locations that affords the possibility of reaching towards them. These hidden states produce sensory signals in a proprioceptive modality, ; the location of the arm in visual coordinates, and the salience (e.g., illumination) of the four target locations . Here, we have used a simple tangent function to model the nonlinear transformation from intrinsic (proprioceptive) to extrinsic (visual) coordinates that is inherent in real motor control [86].

The second differential equation describes how changes in arm position and target salience depend upon action and exogenous causes respectively: the direction of the arm changes as a sigmoid (hyperbolic tangent) function of action and decays back to its resting position in the absence of action. The salience of each location is increased by exogenous causes that we can use to specify the duration and sequence of (visual) cues. The motion of the states and sensory input were subject to low levels of noise in the simulations (with a log precision of 16).