Information Theory as an Experimental Tool for Integrating Disparate Biophysical Signaling Modules

There is a growing appreciation in the fields of cell biology and developmental biology that cells collectively process information in time and space. While many powerful molecular tools exist to observe biophysical dynamics, biologists must find ways to quantitatively understand these phenomena at the systems level. Here, we present a guide for the application of well-established information theory metrics to biological datasets and explain these metrics using examples from cell, developmental and regenerative biology. We introduce a novel computational tool named after its intended purpose, calcium imaging, (CAIM) for simple, rigorous application of these metrics to time series datasets. Finally, we use CAIM to study calcium and cytoskeletal actin information flow patterns between Xenopus laevis embryonic animal cap stem cells. The tools that we present here should enable biologists to apply information theory to develop a systems-level understanding of information processing across a diverse array of experimental systems.

Keywords: information theory, embryogenesis, regeneration, cell biology, morphogenesis, calcium

It is an inherent limitation of science that experimentalists are constrained by their tools; however, the phenomena that we study do not share these limitations. When we conflate the constraints of our methodology with the constraints of our systems we lose sight of the importance of what we cannot yet measure. During the 20th century, the establishment of the central dogma and the advances in genetics and molecular biology has enabled us to perturb and quantify biological systems at a microscopic level. During the 21st century, it has become increasingly clear that biology is not so easily reduced to its microscopic component parts, which often, individually, do not retain the fundamental features that animate matter across scales. As we expand our understanding of biology from cells and pathways to tissues and systems, we must move beyond our simple linear ideas of causality such as “signaling pathways” and “genotype determining phenotype” to do justice to the complexity of biological information processing [1,2,3,4].

Three main barriers exist to the rational discovery of interventions based on simple systems-level causal models in cell and developmental biology. First, there is the problem of time. Many manipulations, including drug treatments and genetic knockouts, constitutively activate or inhibit specific pathways but in so doing remove any information encoded in the dynamics of the signal. There is a growing body of evidence that such dynamics are crucial to proper cell signaling with specific outcomes encoded by different signal dynamics [5,6,7,8,9]. Importantly, temporal interactions are not merely inconvenient artifacts, but core to the very idea of causality [10,11]. Emerging tools such as optogenetics [7,12] have allowed us to manipulate systems with high temporal resolution, but new analytical approaches are necessary to implement these tools at the systems level, in both space and time, in non-neural systems.

Second, there is the problem of integration. Due to technical limitations, it is standard to manipulate a single pathway at a time and test for effects on a targeted set of outcomes. However, biological systems are built to be robust to just this type of insult, manifesting as spontaneous mutations, and thus, we are often limited in our ability to understand and control systems governed by redundancy and integration. Further, the specificity of independent “pathways” is confounded by the tendency of multiple different pathways converging into a small set of “secondary messengers” (Figure 1) [13,14,15,16,17,18,19]. Understanding how diverse inputs are integrated would dramatically improve our ability to predict and control biological systems, but would require supplementing beyond classical reductionist methodology with new integrative approaches.

Third, there is the problem of computation. It is clear that studying an individual neuron is entirely insufficient to understand the functioning of the nervous system [20]. Neuroscientists overcame this problem by applying tools such as positron emission technology (PET), fMRI, and calcium imaging to measure systems-level behavior of neural tissues. In non-neural systems, it is also becoming increasingly clear that many aspects of tissue-level function are irreducibly complex and require new tools that go beyond simple linear pathways commonly used in molecular genetics models [21,22,23,24]. The above techniques are increasingly being adopted by non-neural biologists, but without analytical tools to deal with the data that they produce that cannot be used to their fullest potential.

Here, we propose applying information theory as a useful approach to these three problems. Information theory provides the methodology for quantifying correlative and causative interactions between different variables. Though it forms the basis of computation, information theory is not to be confused with mathematical modeling. Indeed, information theory is unique in being a model-agnostic method with respect to the implementation details of the system being analyzed. It can readily be applied to extremely different systems, including gene–gene interactions, genotype–phenotype interactions to cell–cell interactions and beyond, requiring very few prior assumptions of how the systems work (i.e., being compatible with a wide range of mechanisms). By allowing researchers to map information flow between cells and pathways, information theory shows promise to facilitate systems-level understandinga of biological phenomena for which molecular reductionism has failed, and to leverage the emergent power of tissues for synthetic biology applications [22]. Indeed, the transformative nature of information theory should not be underestimated; Shannon’s 1948 paper [25] directly ushered in the information age and the global integration that defined it, and his formalism is the basis for the analysis that we present here.

Here, we describe useful information theory metrics and provide analogies and examples from known biological systems. Our goal is to describe a variety of information theory techniques and to illustrate how they could be applied by experimental biologists (Figure 2). We explore concepts with analogy and re-evaluation of experimental systems, but do not delve too deeply into the underlying mathematics of these concepts, since each is a well-accepted technique supported by rigorous mathematical principles that are described in detail in the cited references.

Then, we introduce a novel tool, CAIM, designed to allow biologists to easily and rigorously apply several of the metrics we discuss to their own data. We demonstrate how CAIM can be applied to describe information flow dynamics at the systems level of calcium, a key signaling node, and actin, a key component of the cytoskeleton that directly effects many biological processes including morphogenesis, the early embryonic Xenopus laevis stem cells. Finally, we integrate these data to propose a model of the systems-level functions of these signaling modalities in this primitive tissue.

The real power of information theory comes from its rigorous mathematical foundation. Science is fraught with beautiful ideas such as ”memory”, ”consciousness”, and even ”life” that seem intuitively clear but are too nebulous to withstand rigorous scientific scrutiny. The concept of information is among these in some senses, but in others its meaning is not debatable. This is true where concepts of information have been mathematically formalized. In 1948, Claude Shannon famously defined information using the formula I = − Σplog(2)p, with I denoting the information content of a signal and p being the probability that the signal is active, and which we describe in greater detail in Figure 3A. This formula may seem familiar, and indeed it is very similar to the formulation of thermodynamic entropy. The two concepts are closely related. Consider two libraries, one library that is well-organized (low thermodynamic entropy) and the other library that is very cluttered (high thermodynamic entropy). Describing the location of every book in a well-organized library requires very little information (“they are alphabetical by title starting here”), while describing the location of every book in a disorganized library requires a lot of information (“Melville is there, Shakespeare over here, Seuss down in the basement” etc.) Just as thermodynamics allows us to quantify and apply a seemingly nebulous concept such as ”order”, information theory allows us to quantify reduction in our uncertainty about the state of a system, with respect to a specified syntax or language.

Quantifying information can aid in one of the greatest challenges for the experimentalist, i.e., deciding which tool to use to study a system and what intervention to perform. Imagine that we have three fluorescent reporters for the same metabolite with different sensitivities (Figure 3C). The first reporter is active 10% of the time (I = 0.47 bits), the second reporter is active 90% of the time (I = 0.47 bits), the third reporter is active 50% of the time (I = 1.00 bit). Even though we obtain the most signal from the second reporter we obtain the most information from the third. Most of the metrics that we describe below are bounded by the amount of information of a variable, and thus, it is wise to tailor our approaches to extract the maximum amount of information from our system.

Mutual information (MI) is a measure of how much our uncertainty of one variable is decreased by knowing the state of another variable [25] (Figure 4A). Imagine that Alice and Bob always arrive to work at 8:00 a.m. The arrival times of these people will have very high mutual information, i.e., we can usually tell if Bob is around by knowing whether or not Alice is around. We cannot, however, predict anything about the relationship between Alice and Bob. It may be that they live together and carpool; it may be that they live separately but arrive on the same bus; or it may be that both of their shifts start at 8:00 a.m. MI does not provide any insight into the causal reason(s) that the two arrive at the same time.

From a practical standpoint, establishing mutual information between two factors can be very important even without knowing the mechanistic interaction between the two factors, especially if one of the factors is difficult to measure. This is the core of what physicians do in diagnostic assays, i.e., measure phenomena with high mutual information to disease states. It is not necessary to understand why heart attacks cause intense pain in the left arm, it does not really matter if we can reliably use the high MI between the two to quickly diagnose heart attacks. A biological example of this utility from our own work can be seen in symmetrical effects of amputation [26]. We observed that upon amputating a developing frog limb, the contralateral (undamaged) limb would stain strongly with the potentiometric dye DiBAC4(3) at approximately the proximal-distal location as the site of amputation, consistent with these two distant sites having surprisingly high mutual information post-amputation (Figure 4B). The cells of one limb had information about the state of cells in the opposite limb; this finding revealed an unexpected phenomenon even before investigations of the mechanism of the correlation, for example, suggesting the possibility of surrogate-site diagnostics.

As a warning to the experimentalist, while MI is useful as an initial guide to forming hypotheses, as it can also reveal relationships that are “artifacts” in the sense that they do not represent interesting causal connections. For example, if the light source used in a microscopy experiment fluctuates over time, then two nearby cells will demonstrate high MI, but this MI will not reflect biologically meaningful interactions. Despite these caveats, MI is a powerful ”first step” for information theory analysis because its model agnostic nature facilitates discovery of unexpected phenomena.

An obvious limitation to the applicability of mutual information is its inability to quantify events that are highly correlated but separated in time. This prevents MI from detecting causal interactions in which, by definition, one event must precede another. A simple solution to this problem is to temporally offset the expected ”sending” variable from the ”receiving” variable. If it takes a signal four second to propagate from Cell 1 to Cell 2, then measuring the state of Cell 1 should accurately predict the state of Cell 2 four seconds later (Figure 5A). Returning to the example of the beating heart, imagine two people with similar resting heart rates. Assuming their heart rates are not in phase, there will be a high degree of delayed MI between the two hearts; we can accurately predict when Person A’s heart is going to beat if we know when Person B’s heart beats.

An example of the potential pitfalls of delayed mutual information can be found in the elegant experiments of Isabel Palmeirim and colleagues in their discovery of the segmentation clock. They noticed waves of hairy gene expression that seemed to emerge from the tip of the tail and move anteriorly until they eventually formed a new body segment [27]. In information theory terms, they essentially detected high delayed mutual information between cells along the anterior-posterior axis, which suggested that the posterior cell was directly causing the state of its anterior neighbor. They cleverly tested this hypothesis by ablating the “sender” cells in the tailbud (Figure 5B). Surprisingly, the hairy wave still manifested, demonstrating that the causal prediction from their delayed mutual information analysis was incorrect and inferring that the two cells were oscillating indepyendently. Interestingly, future work [28,29,30] showed that these cells did require some input from their neighbors to oscillate correctly. From this example it is clear that more refined tools are necessary to satisfyingly measure causality, especially in biological systems that cannot be intervened upon as readily as the segmenting tailbud.

The segmentation clock example above exemplifies the importance of knowing the mutual information between a cell’s current and past states. If a cell’s state is heavily informed by its past this suggests that it is independent and less governed by collective dynamics. We can measure this by measuring the DMI between a cell’s dynamics and a time-delayed version of these same dynamics to generate our next metric, i.e., active information storage.

Consider a beating heart (Figure 6A), we can very easily predict when contractions are going to happen by observing the previous beating rate. Indeed, periodic signals manifest throughout biology, from the vertebrate segmentation clock to circadian rhythms [31]. Active information storage (AIS) is a formulated form of MI between a time series and a past version of itself [32,33,34,35].

Moreover, quantifying changes of AIS can provide important insights into the behavior of a biological system. If the AIS of a beating heart drops slightly, this may indicate that its host organism is exercising or experiencing stress that gradually changes the beat frequency (Figure 4B). Likewise, if the AIS drops in a non-cardiac cell, this loss of autonomous predictive power suggests that the cell is being affected by an external force. We cannot infer what is causing this change, but by identifying and quantifying this interaction we can design experiments to determine its molecular mechanism. If the AIS of a heart drops dramatically this may suggest a serious pathology such as arrythmia or even fibrillation. In turn, dramatic loss of AIS in a non-cardiac cell may suggest an important transition. During differentiation, a cell may become fundamentally different from its previous state, thus, reducing the predictive power of its past for its future. AIS can also drop dramatically if a cell is integrated into a super-cellular collective, as the collective may have substantial input on the state of the cell and only observing the past states of the cell may lose predictive value.

High AIS values, therefore, may suggest that cells are behaving more independently and less collectively. If this is true, then, experimentally isolating the cells should have a minimal effect on the dynamics. Low AIS values, in contrast, suggest that the dynamics are being largely governed by other forces that can be investigated by our next metric, i.e., transfer entropy.

Inferring causality is one of the main goals of experimentation, but can also be one of the most challenging. Our standard approach is to perturb the system and see how it changes, and then, to attribute the changes to the perturbation. However, these perturbations are often of non-physiological magnitude and, despite our best efforts at control, cn be confounded with technical artifacts. Alternatively, we observe a system over time without perturbation and observe the timing of events, effectively estimating the delayed mutual information between them, but this approach is prone to conflation of correlation with causation. In information theory, the best current tool for inferring statistical causation is called transfer entropy (TE) [36,37]. Recall that the terms entropy and information are often used interchangeably, and thus, TE can be thought of as transfer of information. TE is the amount of information about one variable gained from knowing the state of a second variable beyond what is learned from the history of the first variable (Figure 5A).

Let us say that Alice always arrives to work at 7:50 a.m. and Bob always arrives at 8:00 a.m. It may well be that Alice and Bob drive in together, Alice lets Bob off at the entrance (on account of his bad knee) and goes to park. It may be that Bob hates Alice, and always makes sure to arrive later than her to avoid running into her. In both of these cases it would be fair to say that Alice’s arrival time causes Bob’s arrival time. However, it could also be that Bob just shows up at 8:00 a.m. because that is when his shift starts. Because we can predict when Bob is going to arrive without knowing anything about Alice we cannot really conclude that her arrival time is causing his arrival time, despite reliably preceding it and allowing us to accurately predict.

While transfer entropy is a powerful tool, its broad definition of ”causality” may ring hollow to an experimentalist. We spend our lives intervening upon systems and measuring outcomes to infer causal relationships. TE excels at analyzing systems that cannot be readily manipulated, and can help us decide which systems warrant development of experimental interventions or translation into tractable model systems. Thus, TE allows identifying targets that may be confirmed as causal, once experimental intervention is applied. Once we can manipulate our system, we can move to our next metric, i.e., effective information.

Effective Information (EI) is a formal way of measuring causal interactions between an experimental manipulation and a system [17]. To measure EI, one sets the state of each variable and measures the consequences on the future states of the system (Figure 7A). This technique is conceptually very similar to many experimentally implemented biological approaches. In a reverse genetic screen, one individually mutates a set of genes (thus setting their state to 0 for null mutations, <1 for hypomorphic mutations, and >1 for hypermorphic mutations) and observes the phenotypic consequences on the phenotype of the organism (Figure 7C). Importantly, there is rarely a linear one to one correlation between a single gene and the phenotypic outcome of a complex biological system. Biologists refer to this phenomenon as ”incomplete penetrance”.

The discovery of ”instructor cells” in the skin of Xenopus embryos provides a nice example of a system with effective information (Figure 7B). Depolarizing a specific subset of epidermal cells causes the entire embryo to become hyperpigmented [38]. Interestingly, this hyperpigmentation exhibits a binary ”all-or-nothing” pattern within each animal, indicating that depolarizing the instructor cells affects the system as a whole [39,40]. In information theoretic terms, they measured the EI between the ”instructor cells” and pigmentation of the embryo system by injecting noise into the instructor cells by either depolarizing them or leaving them as unmanipulated controls, and then estimating the mutual information between the state of the instructor cell and the state of the pigmentation. If they had chosen to depolarize a set of ”non-instructor cells” instead, they would have found very little mutual information between the state of the manipulated cells and pigmentation since the state of the non-instructor cells does not predict the pigmentation state.

Importantly, EI, as with all the metrics presented here, is a continuous, not binary, metric, and thus gives us a framework for quantifying the degree of causal relationship between a manipulation and a phenotypic output. This quantification has the potential to address a substantial problem that has arisen from molecular reductionism, i.e., different researchers apply different methods to similar questions, and subsequently report any statistically significant findings regardless of effect size. Then, these findings are compiled in pathway diagrams in which each experiment is recorded as an arrow, and all arrows are weighted equally regardless of whether the manipulation caused a 10% change or a 90% change and regardless of the technical differences among the techniques used. Because of its model-free nature, EI has the potential to unify these diverse experimental findings in terms of their measurement in bits, which, then, can be compared and compiled.

Consider two researchers studying the same protein. One researcher finds that an intervention increases phosphorylation of protein X 80% of the time via Western blot. The other researcher finds that the same intervention induces ectopic tails 5% of the time in mice. It is tempting to combine these data to say that ”this intervention increases protein X phosphorylation and causes phenotype Y”, but doing so strips the data of important causal information. EI can be applied just as readily to phosphate groups and tail numbers, and thus allows us to integrate these orthologous techniques.

Neuroscientists during the 1990s faced the same fundamental problem that non-neural scientists are facing today: How can we measure information dynamics between cells when the mechanisms that transmit this information are fast and subtle? Rafael Yuste and colleagues solved this problem by effectively using calcium signaling as a proxy for action potentials [41,42,43]. Calcium also functions as an important secondary messenger in many diverse non-neural systems [13,44,45,46,47]. The approach of applying information theory (IT) metrics to non-neural calcium signaling is further supported by the observation of striking long-range coordinated calcium patterns that move across many different cells [48]. Information theory measures states of variables, and as discussed in the preceding section, it is convenient for our purposes to coarse grain the complexity of a cell to a single binary on or off calcium state signal (Figure 3C). Once interactions have been identified and measured, we can employ our prodigious arsenal of molecular tools to determine the molecular mechanisms and functional consequences of these patterns.

It is unreasonable to assume that calcium levels alone are sufficient to define the state of a cell. Fortunately, the factors that make calcium an appealing cell state reporter can also be found in other secondary messengers. The RAS/ERK signaling pathway, as one example, also integrates input from many different signaling pathways and displays large scale emergent dynamical patterns [49,50]. Indeed, it may be time to consider promoting these ”secondary messengers” to ”primary nodes” that do not merely transduce signals but integrate information from many different input pathways to determine the behavior of a cell.

There is a growing body of evidence that pulsatile dynamics in the actin cytoskeleton play important roles during morphogenesis [51]. These cytoskeletal dynamics have been described in a variety of tissues, including the Xenopus laevis primitive animal cap embryonic stem cell tissue [52]. Further, pulsatile actin dynamics have been linked with calcium dynamics in several developmental systems including mast cells [53] and the developing Xenopus neural plate [54]. In order to demonstrate the profound flexibility of information theory we apply these metrics to both calcium signaling and actin pulsation. These signaling modalities are very different at the mechanistic level. Calcium is an ion that mediates communication through several signaling pathways, while actin is a structural cytoskeletal protein that contributes to a cell’s shape and facilitates its mechanical interaction with its environment [44,51]. We have chosen to analyze these very different signaling modalities to demonstrate how the model-free nature of information theory allows us to study interactions among diverse biological phenomena. We do so in explant Xenopus laevis animal cap embryonic stem cells. These cells demonstrate both endogenous actin and calcium dynamics, but it remains unclear how they interact at the systems level and what function, if any, they play during early development. By applying information theory to the dynamics of these signaling modalities between and within minimally manipulated tissue explants, we aim to explore the degree to which these cells behave collectively and what roles these modalities play in this collectivization.