AI-driven automated discovery tools reveal diverse behavioral competencies of biological networks

Many applications in biomedicine and synthetic bioengineering rely on understanding, mapping, predicting, and controlling the complex behavior of chemical and genetic networks. The emerging field of diverse intelligence investigates the problem-solving capacities of unconventional agents. However, few quantitative tools exist for exploring the competencies of non-conventional systems. Here, we view gene regulatory networks (GRNs) as agents navigating a problem space and develop automated tools to map the robust goal states GRNs can reach despite perturbations. Our contributions include: (1) Adapting curiosity-driven exploration algorithms from AI to discover the range of reachable goal states of GRNs, and (2) Proposing empirical tests inspired by behaviorist approaches to assess their navigation competencies. Our data shows that models inferred from biological data can reach a wide spectrum of steady states, exhibiting various competencies in physiological network dynamics without requiring structural changes in network properties or connectivity. We also explore the applicability of these ‘behavioral catalogs’ for comparing evolved competencies across biological networks, for designing drug interventions in biomedical contexts and synthetic gene networks for bioengineering. These tools and the emphasis on behavior-shaping open new paths for efficiently exploring the complex behavior of biological networks. For the interactive version of this paper, please visit https://developmentalsystems.org/curious-exploration-of-grn-competencies.

Developing methods to recognize, map, predict, and control the complex, context-sensitive behavior of chemical and genetic networks is an essential frontier of research in science and engineering. These systems, such as gene regulatory networks and protein pathways, are known to be instructive drivers of embryogenesis, cell behavior, and complex physiology (Sanz-Ezquerro et al., 2017; Padilla-Longoria Enrique Balleza et al., 2008; Huang et al., 2005). Understanding the control properties of these systems is critical not only for the study of evolutionary developmental biology (Davidson, 2010; Peter and Davidson, 2011; Ten Tusscher and Hogeweg, 2011; Kim and Sayama, 2018; Srivastava, 2021), but also for comprehending and intervening in various disease states, including cancer (Singh et al., 2018; Qin et al., 2019; Fazilaty et al., 2019), and for the construction of novel synthetic biologicals in bioengineering contexts (Davies and Levin, 2022; Toda et al., 2018; Toda et al., 2020; Ho and Morsut, 2021; Santorelli et al., 2019).

Thus, much work has gone into mathematical modeling and computational inference of both protein pathways and gene regulatory network models (de Jong, 2002; Schlitt and Brazma, 2007; Fetrow and Babbitt, 2018; Delgado and Gómez-Vela, 2019), which has resulted in the development of large collections of publicly-available models such as the Biomodels database (Glont, 2018; Malik-Sheriff et al., 2020). Yet, despite the wealth of available models, scientists still largely lack an effective understanding of the range of possible behaviors that these models can exhibit under different initial conditions and environmental stimuli, and are in search of systematic methods to reveal and optimize those behaviors via external interventions. The full extent of the computational and control properties of such networks are not yet well-understood; while dynamical systems theory has been extensively used to characterize their behavior (Kauffman, 1993; Kauffman, 1995), it is not known what other sets of tools might reveal and exploit interesting properties of this ubiquitous biological substrate. The field of diverse intelligence (also known as basal cognition) has suggested that strong functional symmetries between pathway networks and neural networks could imply the existence of learning and other kinds of behavior in this unconventional substrate (Abramson and Levin, 2021; Baluška and Levin, 2016; Dodig-Crnkovic, 2022; Dodig-Crnkovic, 2022; Timsit and Grégoire, 2021; Katz et al., 2018).

Specifically, it has been hypothesized that gene regulatory networks (GRNs) and other molecular networks could be endowed with surprising navigation competencies allowing them to robustly reach diverse homeostatic or allostatic states despite a wide range of perturbations (Csermely et al., 2020; Gyurkó et al., 2013; Fields and Levin, 2022; Watson et al., 2010), and that exploiting these innate competencies could provide a promising roadmap for the design of interventions in regenerative medicine and bioengineering contexts (Mathews et al., 2023; Lagasse and Levin, 2023).

However, significant challenges remain in practice for the exploration and behavior-shaping of these innate competencies, which presents a barrier to the use of these ideas in regenerative medicine and bioengineering. Because of the non-linearity and redundancy in pathway dynamics, passive exploration strategies such as random screening are likely to either fail in uncovering the full range of potential behaviors or require time and energy beyond the available resources. Here, we formalize and investigate a view of gene regulatory networks as agents navigating a problem space. We propose a framework and automated tools, leveraging (1) curiosity-driven goal-directed exploration algorithms coming from recent advances in machine learning and (2) a battery of empirical tests inspired by behaviorist approaches, for mapping the repertoire of robust goal states that GRNs can reach within this problem space despite various perturbations. A key novelty of this work is the use of AI-based exploration tools to map the space of possible behaviors in biological networks, which opens interesting avenues for efficient mapping of unfamiliar system behaviors, yielding transferable insights for diverse problem-solving once such a map is discovered.

The challenge of exploring and mapping spaces of complex and self-organized behaviors appears in many fields such as diverse intelligence in biological systems, minimal active matter, and robotics: many systems in these areas provide a rich space of evolved, engineered, and hybrid systems that offer many of the same fundamental problems of behavior and control regardless of specific composition or provenance (Clawson and Levin, 2023). These span many orders of spatio-temporal scale, from molecular assemblies to swarms of complex organisms (Timsit and Grégoire, 2021; Krist et al., 2021; Čejková et al., 2017; Hanczyc et al., 2011). One set of approaches seeks to develop tools to identify the optimal level of control, ranging from physical rewiring to various methods from cybernetics and behavioral sciences, to reveal and exploit the native competencies and computational capacities of these systems (Davies and Levin, 2022). Specifically, it is increasingly realized that the level of competency (and thus the appropriate level of control) often cannot be guessed by inspection of a system’s components, and that its position on a spectrum ranging from passive matter to complex metacognition must be determined empirically (Rosenblueth et al., 1943; Bongard and Levin, 2021Levin, 2022), (Clawson and Levin, 2023). This is critical not only for fundamental understanding of evolution of bodies and minds (Lyon, 2006; Barandiaran and Moreno, 2006; Müller and Lengeler, 2000; Baluška and Levin, 2016; McGivern, 2020; Levin, 2023b), but also for the design of interventions in biomedicine and synthetic morphology contexts (Pezzulo and Levin, 2015; Pezzulo and Levin, 2016). Yet, a common property in many of these systems is that it is expensive in time and energy to conduct experiments: empirical exploration needs to be made under limited resources.

Thus, methods for automating efficient exploration and discovery of a diversity of behaviors in these spaces may be widely useful. As explained below, we will here leverage methods from developmental artificial intelligence initially designed for the specific purpose of exploring a diversity of behaviors using a limited budget of experiments.

One especially fascinating set of systems concerns cellular molecular pathways, or GRNs. In the lab or clinic, these pathways are usually treated as simple machines, with intervention strategies focusing on rewiring their structure to achieve a desired outcome: adding or removing nodes (gene therapy), or changing connection weights (by targeting promoter sequences or protein structures) (Wong et al., 2008; Samuel et al., 2018; Krzysztoń et al., 2021; Baum, 2007). However, the emergent, generative nature of development and physiology ensures that it is often very hard to know which genes/proteins to modify, and how, in order to reach a complex desired system-level outcome (Lobo et al., 2014). Moreover, the responses of cells and tissues to drugs changes over time, making it even more difficult to infer specific interventions (e.g. drugs) that will induce a stable improvement in pathway state in vivo. Indeed, with the exception of antibiotics and surgery, most available treatment modalities do not solve the underlying problem – they seek to mitigate symptoms, which recur (or expand) once the drug is withdrawn. This is because current therapeutics function bottom-up, attempting to force specific molecular states, as it has been challenging to develop methods for shifting complex tissues and organs towards a stable health profile. Next-generation solutions, which would offer true healing (stable correction), require an understanding of the homeostatic and allostatic properties of networks with respect to how they traverse the space of transcriptional, physiological, and anatomical states. An understanding of the behavior policies of networks as they dynamically navigate these problem spaces is essential for predicting what stimuli can be used to re-set their setpoints and guide them to autonomously maintain a healthy state.

In the language of behavioral neuroscience, this strategy corresponds to exploiting their native robustness, decision-making, and navigational competencies to induce predictable, long-lasting changes in functionality.

Significant challenges remain in revealing and controlling the range of behaviors that can self-organize in these cellular and molecular pathways. To characterize steady-state concentrations and responses to small perturbations, conventional methods rely on piecewise-linear approximation of the system behavior (Stucki, 1979; Ingalls, 2004; Ingalls, 2008; Donzé et al., 2010; Dang et al., 2011), but struggle with higher-dimensional systems or wider parameter ranges which limits their applicability (Donzé et al., 2011). Other works have proposed the porting of tools from network control theory to identify sets of control nodes allowing to drive the network behavior toward target steady states (Rozum and Albert, 2022). These methods typically exploit the network topology (Rozum and Albert, 2022; Steinway et al., 2015; Zañudo and Albert, 2015; Zañudo et al., 2017; Fontanals et al., 2020) or regulatory structure (Murrugarra et al., 2016; Choo et al., 2018; Choo et al., 2019) to identify control strategies based either on permanent knockout/activation of genes or on temporary perturbations, the latter being preferable in biomedical context.

However, these approaches often require prior knowledge of target attractor states or are limited to Boolean network models. Other works have explored the use of machine learning tools, such as evolutionary search (Paladugu et al., 2006; François, 2014; Noman et al., 2015) and gradient-descent optimization (Hiscock, 2019; Shen et al., 2021), for controlling continuous ODE biomolecular networks with high-dimensional parameter spaces, mainly in the context of synthetic circuit engineering (Camacho et al., 2018; Volk et al., 2020). While providing powerful optimization tools, these approaches tend to focus on rewiring network structure and connectivity. Moreover, the choice of a predefined fitness function and parameter range initialization is not only critical to the success of optimization (François, 2014) but largely restricts the exploration of the behavior space (Shen et al., 2021).

In contrast, an alternative line of research proposes exploring and leveraging the inherent molecular mechanisms of adaptivity and robustness in cellular pathways as a promising approach for drug interventions that do not rely on genomic editing or gene therapy (Csermely et al., 2020; Kitano, 2007a). Recently, a broad, substrate-independent behavior science perspective suggests novel properties of GRNs and other biological networks (Abramson and Levin, 2021; Manicka and Levin, 2019). This perspective views GRNs as agents that convert activation levels of specific genes (inputs) to those of effector genes (outputs), with intermediate nodes in between, leading to strategies for controlling network behavior based on a specific history of inputs (experience) rather than through network rewiring. Notably, the concept of training a chemical pathway using pulsed input stimuli (node activation or suppression drugs) has been formalized, and several networks have been analyzed to establish a taxonomy of memory types found in biological GRNs and pathways (Biswas et al., 2021; Biswas et al., 2023).

Here, building upon recent research (Fields and Levin, 2022; Biswas et al., 2021; Biswas et al., 2023), we take the next step and investigate a view of gene regulatory networks as agents navigating a problem space toward target goal states with varying degrees of competency (Figure 1a). We seek to implement a definition of goal that abstracts it from conventional associations with human or other advanced brains and facilitates the use of tools from cybernetics, behavior science, and control theory to understand broader aspects of biological regulation. Here, we use the term ‘goal’ state to refer to a system’s steady state, which it expends effort to reach despite interventions or barriers - a definition appropriate to the study of basal (or minimal) proto-cognitive regulatory systems. Our definition of goal does not imply ‘purpose’ (high-level goals where an agent has the meta-cognition to think about having goals and what they might be), and we do not attribute high-level competencies (such as re-setting one’s own goals) to GRNs.

Our particular focus lies in investigating two types of navigation competencies: versatility, which refers to the capacity to reach diverse goal states under different interventions, and robustness, which refers to the ability to reach a goal state despite various perturbations. The primary scientific question we aim to address is: What is the repertoire of robust goal states that a GRN can actively reach through minimal and non-genetic interventions within a navigation task context, and can we develop systematic methods and automated tools to aid scientists in discovering this repertoire?

To address this question in practice, our experimental framework revolves around the definition of ‘problem spaces,’ which we use as tractable components of the GRN’s overall state space (Figure 1b), and on a set of methodological contributions which we organize around three sub-questions:

Automated discovery of diverse behavioral abilities with autotelic curiosity search (Figure 1c): What is the range of possible goal states that GRNs can exhibit and how can we devise efficient exploration strategies to automatically identify these goal states? Defining goal states as attractor states of the underlying gene regulatory network, we show that traditional screening methods can be very inefficient in discovering the range of possible goal states. To address this, we propose to use intrinsically-motivated goal exploration processes (IMGEP) (Baranes and Oudeyer, 2013; Forestier et al., 2022), a recent family of diversity-driven machine learning approaches also known as autotelic curiosity search which was recently shown to form a useful discovery assistant for revealing the behavioral diversity of unfamiliar systems such as chemical oil-droplet systems (Grizou et al., 2020), physical non-equilibrium systems (Falk et al., 2024) and models of continuous cellular automata (Reinke et al., 2020; Etcheverry et al., 2020; Hamon et al., 2024).

Evaluation of the navigation competencies (Figure 1d): How competent is the GRN, in terms of robustness to perturbations, in attaining the diverse previously-identified goal states? Prior studies have offered definitions of robustness in biological networks, characterized as the degree of variation in functionality (Kitano, 2007b) or phenotypic trait (Félix and Barkoulas, 2015) under specific environmental or genetic changes. However, these studies often consider a predefined functionality and random perturbations in network parameters (Ingolia, 2004; Ma et al., 2006; Noman et al., 2015) or specific gene knockouts (Deutscher et al., 2006). Environmental perturbations on the other hand are often limited to random variations in initial conditions within a predefined range (Donzé et al., 2011; von Dassow et al., 2000). Here, inspired by behaviorist approaches, we test hypotheses about non-genetic resistance with respect to various navigation competencies that living agents often exhibit, and that do not require structural changes of network properties or connectivity. Those tests assess the system’s ability to maintain robustness despite various perturbations encountered during traversal, including developmental noise in gene expression levels, sudden ‘pushes’ within transcriptional space, and the presence of energy barriers or ‘walls’ acting as force fields in the environment.

Potential reuses of the discovered ‘behavioral catalog’ and framework (Figure 1e): Can the constructed behavioral catalogs be useful for fundamental research and practical therapeutic applications, and can the framework be easily applied to other systems and problem spaces? We propose that the discovered competencies may provide valuable insights for understanding evolvability and developmental robustness, and provide a fertile source for the design of interventions in biomedicine and synthetic morphology contexts. We also suggest that the framework and automated tools, which are observer-focused and substrate-independent, could be transposed to other systems and problem spaces.

The overall framework is summarized in Figure 1. Applying it on a database of 30 continuous (ODE) models from the Biomodels website, consisting of a total of 432 systems defined as GRN model-behavior space tuples, revealed several interesting insights. First, results suggested that most of the surveyed systems are capable of reaching a surprisingly wide spectrum of steady states depending on their initial state. Interestingly, random screening strategies were not able to reveal this diversity of reachable states (or at least not in a sample efficient way), confirming the need for more advanced exploration strategies like curiosity search. Second, among the discovered steady states, we were able to identify several robust goal states i.e., ones that the system consistently reaches despite various perturbations during traversal of transcriptional space. Altogether, these findings seem to suggest that cell phenotype and functionality could be the result of a multi-step program (Steinway et al., 2015) that could be flexibly and robustly reprogrammed by appropriate stimuli (Levin, 2022). Finally, we demonstrate possible reuses of this ‘behavioral catalog’ for comparing the network’s competencies across different classes of organisms, as well as for the design of non-genetic drug interventions. We also demonstrate an alternative reuse of the framework to reveal new kinds of reachable ‘goals’ in synthetic gene networks, suggesting alternative strategies for the design of gene networks in a bioengineering context.

An interactive executable version of the paper, as well as step-by-step tutorials and notebooks, can be found online at https://developmentalsystems.org/curious-exploration-of-grn-competencies. The full codebase of the proposed automated experimentation pipeline is written end-to-end in JAX, a high-performance numerical computing library that we leverage for parallel experimentation and computational speedups of the ODE models time-course simulations.

The GRNs analyzed in this study are biological pathway networks taken from the BioModels repository (Glont, 2018; Malik-Sheriff et al., 2020). The term ‘GRN’ is used broadly to include protein interaction, gene regulatory, and metabolic networks. In these mathematical models, the dynamic interactions between nodes of the network (molecular species) are modeled with a system of ordinary differential equations, enabling to quantitatively simulate time-course behavior (model rollouts) and observe the dynamics of node activities over time (Figure 2a). Here, following a terminology which aims to integrate concepts from dynamical complex systems with concepts from behavioral sciences, we propose to conceptualize GRN behavior as a navigation task (Table 1). Model rollouts are viewed as ‘trajectories’ in transcriptional space where network steady states are ‘goal states’ (endpoints) that the ‘agent’ (GRN) can reach with varying levels of competencies. As for living agents, these competencies may range from unstable locomotion patterns to more advanced forms of goal-directed behavior like path following, obstacle avoidance, or even forms of spatial memory and foresight. In this paper, we are particularly interested in investigating two forms of navigation competencies that we refer to as versatility, the capacity to reach diverse goal states under various interventions, and robustness, the capacity to reach a goal state despite various perturbations. Note that versatility and robustness are studied with respect to different sources of incoming environmental variation, respectively interventions and perturbations.

To investigate these competencies in practice, our experimental framework is based on the definition of ‘problem spaces,’ which include the observation space (O), behavior space (Z), intervention space (I), and perturbation space (U) as defined in Table 2. To be consistent with our navigation task terminology introduced in Table 1, we refer to a behavior z as the reached ‘goal state’ of a GRN trajectory. However these ‘goals’ may lie on a continuum between complete robustness and high sensitivity, and our primary interest lies in identifying robust goals of the system. Whereas several choices could be made for the intervention space I and perturbation space U, we intentionally consider minimal and non-genetic interventions to investigate the ‘native’ goal states of the GRN, and environmental obstacles to investigate for navigation competencies classically observed in other living agents. Examples of simulations, interventions, and perturbations are illustrated in Figure 2.

Then, a typical analysis using our framework relies on a 2-step procedure, detailed in the subsequent sections. First, to assess the versatility of the GRN, we define an exploration strategy which organizes the sequence of interventions i1,…,iN used to drive the system toward a maximally diverse set of reachable endpoints {zk∈Z}k=1..N , while being given a limited budget of experiments N. Second, to assess the robustness of the discovered goal states {zk∈Z}, we conduct a battery of empirical tests to characterize their degree of sensitivity to novel perturbations, with a fixed experimental budget of P perturbations per selected behavior z. At the end of this two-step procedure, we obtain the ‘behavioral catalog’ (H) of the studied GRN, which includes the history of experiments H={ik,ok,zk,{up,op,zp,p=1...P},k=1…N}.

Following this framework, the behavioral catalog is constructed for a database of 30 biological networks consisting of a total of 432 systems, where a system is defined as a (GRN model, intervention space (I), behavior space (Z)) tuple, as described in Materials and Methods and Supplementary file 1. These catalogs provide valuable empirical observations and insights into the navigation competencies of the studied GRNs, particularly in their ability to consistently achieve diverse goal states under various tested perturbations. Statistical analyses of the results are presented in Figure 3; Figure 5; Figure 7, and specific results for the RKIP-ERK signaling pathway (Kwang-Hyun et al., 2003) are shown in Figure 2; Figure 4, Figures 6, 8.

One advantage of modeling GRN behavior within a tractable behavior space Z is that we can then deploy strategies to efficiently discover and map that space. Notably, recent diversity-driven machine learning techniques such as Novelty Search (Lehman and Stanley, 2008; Lehman and Stanley, 2011), Quality Diversity (Cully et al., 2015; Pugh et al., 2016) and Intrinsically-Motivated Goal Exploration Processes (IMGEP) (Baranes and Oudeyer, 2013; Forestier et al., 2022) are explicitly designed to efficiently explore a so-called ‘behavior space’ or ‘goal space’ which is basically a (predefined or learned) model of the overall state space. In particular IMGEPs, which were originally developed for the learning of inverse models of highly-redundant mapping in robotics context (Baranes and Oudeyer, 2013), were recently shown to successfully assist discovery in complex self-organizing systems (Grizou et al., 2020; Falk et al., 2024; Reinke et al., 2020; Etcheverry et al., 2020).

Here, we propose to use an IMGEP to control GRN initial states and maximize the diversity of discovered endpoints {z∈Z} within a limited budget of N experiments. The IMGEP operates in two phases: initially, N∫ interventions are sampled randomly from I to populate history H, then remaining interventions are generated through a goal-directed process which relies on several key internal models. Those including a goal-embedding module R that encodes observations o into the IMGEP goal space Ƭ, a goal generator module G that samples goals from the goal space based on intrinsic motivation incentives (e.g. to promote novelty or learning progress), and a goal-conditioned optimization policy Π that generates candidate intervention parameters to achieve the current goal. Given those internal models, the goal-directed phase iterates through (1) sample a target goal g∼GH, (2) infer intervention parameters to achieve that goal i∼Πg,H, (3) conduct an experiment with the intervention i, observe the outcome o, and compute the reached goal z=Ro, and (4) store the tuple i,o,z in history H. Here, we use the GRN behavior space Z as the IMGEP goal space Ƭ=Z. Hence ‘target goal’ refers to a goal sampled by IMGEP while ‘reached goal’ refers to an actual endpoint of the GRN trajectory, discovered by IMGEP while targeting a potentially different point in Z. Throughout exploration, the IMGEP dynamically refines its Z-traversal strategy based on the knowledge acquired by its discoveries. Here, we opt for a simple IMGEP variant such that the exploration process can be seen as performing novelty search in behavior space Z (Doncieux et al., 2019). The pseudocode of our IMGEP pipeline is shown in Figure 1c and details about the internal models are provided in Materials and Methods.

The final outcome is a ‘behavioral catalog’ of the GRN, containing the diverse goal states discovered by IMGEP: H={ik,ok,zk,k=1…N}.

We deploy the IMGEP, also known as ‘curiosity search,’ on all 432 systems in the biological network database. Our evaluation focuses on two related competencies: the IMGEP agent’s ability to empirically reveal a diversity of reachable goal states in the (GRN, I, Z) system, referred to as ‘discovered diversity,’ and the GRN agent’s competency to naturally reach diverse goal states, referred to as ‘versatility.’ The true versatility of the GRN is unknown and can only be inferred through empirical exploration and proxy metrics.

For evaluating diversity, we measure the area covered in Z by the IMGEP discoveries using the threshold-coverage metric (Benureau, 2015) and compare it with the area covered by the diversity of a naive random screening strategy (which uniformly samples interventions in I). In Figure 3, the diversity discovered by the two exploration variants is shown for the 432 GRN,I,Z systems, where random search is given a budget of experiments N which is twice bigger (n=900) as the one given to the curiosity-search algorithm (n=450). Interestingly we see that, on average, at n=290 the curiosity search already significantly outperforms the final diversity achieved by random search, while only utilizing one-third of its experimental budget (n=900). Whereas we are dealing with numerical systems and our codebase allow for efficient and parallel execution, each experiment still consists of TΔT=25000 model steps, where each step integrates the ODE system. Repeating that N times for each of the 432 systems starts to be very costly, which is why having efficient exploration strategies is very valuable (and would be even more valuable when scaling the framework to larger databases). Even more critical, as illustrated in Figure 3b, it seems that, for some systems, random search is not able to discover the ‘latent’ regions revealed by the IMGEP in Z, or it would need an extremely large budget of experiments. On the other hand, as illustrated in Figure 3c, there are some systems for which random search is already quite efficient in revealing diverse behaviors in Z, and for which IMGEP performs equivalently.

In fact, the goal-directed strategy of the IMGEP is particularly beneficial for GRN,I,Z systems with high nonlinearity or redundancy in their I→Z mapping, as seen in Figure 4 and studied in robotics contexts (Benureau, 2015). Redundancy implies that many interventions in I lead to similar effects in Z, as illustrated in Figure 2 where various interventions and perturbations converge to the same endpoint. In these systems, random search will preferentially discover points in areas of high redundancy in Z whereas the IMGEP, whose exploration is directed uniformly in goal space, should cover different levels of redundancy equally. In general, when dealing with large intervention spaces and limited experimental budgets, curiosity search can be particularly useful for efficiently navigating Z-space.