Stochasticity, bistability and the wisdom of crowds: a model for associative learning in genetic regulatory networks

In order to compute the capacity of the GAM, we define the difference between the mean responses to learned and non-learned stimuli as the signal and the square root of the sum of the variances of the responses to the learned and non-learned stimuli as the noise. In the limit of large N, the ability of a binary classifier to discriminate between the learned and non-learned stimuli depends on the SNR. If the SNR is large, it is possible to achieve a high detection rate while maintaining a low level of false positives. A low SNR implies that the two stimuli are indistinguishable. Therefore, we define the capacity of the GAM to be the oldest memory such that the SNR is larger than 1 (see also [47], [48] for a similar approach in models of neural networks). Formally, the signal-to-noise-ratio for a pattern presented patterns ago is given by:(14)where(15)We compute the capacity in the limit of large N and consider the effect of the scaling of p and q with N on the capacity of the GAM. If the values of p and q are very different then the pseudo-synapses will saturate. Therefore, we consider the same scaling of p and q, . The signal in Eq. (13) depends on the product of two terms, that depends on and a prefactor, that is independent of . It is easy to see that the prefactor, . Similarly, it is easy to see that . Therefore, . Because , a necessary condition for the SNR to be larger than 1 is . The term decays exponentially fast with . However, because , as long as , . Therefore, for , as long as , . Thus, for , the capacity of the GAM is , which is maximal for . In other words, assuming that , the capacity of the GAM is .

To gain insights into this result, we consider the optimal choice of and (which minimizes the variance), in which and . In this case, in the limit of large N,(16)and Eq. (15) becomes(17)Thus, Eq. (14) becomes:(18)The requirement that yields:(19)where we used the fact that for , and therefore .

In order to find the values of and that maximize the capacity of the GAM, we compute the zeros of the partial derivatives of Eq. (19) with respect to and , resulting in and . Thus, the capacity of the GAM is(20)For , the capacity is . Note that the capacity increases as the value of f deviates from .

In this section we compute the capacity of a large population composed of Z identical GAMs. The population response to the presentation of is given by (up to constant shift and scaling):(21)where is the pseudo synapse () in the GAM () (compare to Eq. (9)).

Similar to the analysis of the capacity of a single GAM, we compute the mean and variance of . The computation of mean of in the case of the population is similar to that computation for the case of a single GAM, yielding(22)Note that is independent of the size of the population: since all GAMs are identical, their contribution, on average, is equal. Computing the variance of Eq. (21) results yields:(23)In order to evaluate Eq. (23), we consider the dynamics of a single pseudo-synapse . Similar to Eq. (3),(24)where and are Bernoulli variables with parameters p and q, respectively.

Using induction, it is easy to prove that the value of in response to the learning of an infinite sequence of CS is given by:(25)where and and and denote the values of the Bernoulli variables and , respectively, during the encoding of the CS x patterns ago.

Substituting Eq. (26) in Eq. (23) and assuming that yields:(27)Note that in the case of a single network, , only the first term contributes, yielding Eq. (16).

The capacity of the population of GAMs is defined as the oldest memory such that the SNR is larger than 1, where the signal and noise terms in Eq. (14) are given by(28)And(29)In the limit of (large number of GAMs) the contribution of the first term in Eq. (26) to the variance vanishes and the capacity depends on the second term. For a general value of , and this term dominates , resulting in . Therefore, the population capacity in this case is , similar to that of a single GAM. However, if this term vanishes and is dominated by , resulting in . Similar to the case of a single GAM, we compute the capacity in the limit of large N and consider the effect of the scaling of p and q with N on the capacity of the population of GAMs. If the values of p and q are very different then the pseudo-synapses will saturate. Therefore, we consider the same scaling of p and q, . The signal in Eq. (27) is the same as that of a single GAM (Eq. (13)), therefore the prefactor in Eq. (13) is . Similarly, it is easy to see that . Therefore, . Because , a necessary condition for the SNR to be larger than 1 is . The term decays exponentially fast with . However, because , as long as , . Therefore, for , as long as ,. Thus, for , the capacity of the GAM is , which is maximal for . In other words, assuming that , the capacity of the GAM is . In particular, assuming that , substituting Eqs. (28) and Eq. (29) in Eq. (14) yields(30)A straightforward calculation reveals that the capacity is maximal when , resulting in capacity:(31)

In our simulations, we used the following parameters: For the generalized model (Figure 3) we used: All other parameters were the same as the single pathway model (Figure 3). The derivation of the parameters from the reaction kinetic constants is provided in the Text S1 in the Supporting Information. The reaction kinetic constants that were used are provided in Table S1 in the Supporting Information. Simulations in Figures 3 and 4 were carried out using Euler method for numerical integration with step sizes and 0.5 min, respectively.