Search

Efficient multiple setpoint tracking can enable advanced biotechnological applications, such as maintaining desired population levels in co-cultures for optimal metabolic division of labor. In this study, we employ reinforcement learning as a control method for population setpoint tracking in co-cultures, focusing on policy-gradient techniques where the control policy is parameterized by neural networks. However, achieving accurate tracking across multiple setpoints is a significant challenge in reinforcement learning, as the agent must effectively balance the contributions of various setpoints to maximize the expected system performance. Traditional return functions, such as those based on a quadratic cost, often yield suboptimal performance due to their inability to efficiently guide the agent toward the simultaneous satisfaction of all setpoints. To overcome this, we propose a novel return function that rewards the simultaneous satisfaction of multiple setpoints and diminishes overall reward gains otherwise, accounting for both stage and terminal system performance. This return function includes parameters to fine-tune the desired smoothness and steepness of the learning process. We demonstrate our approach considering an $\textit{Escherichia coli}$ co-culture in a chemostat with optogenetic control over amino acid synthesis pathways, leveraging auxotrophies to modulate growth.

This work presents an omics-driven modeling pipeline that integrates machine-learning tools to facilitate the dynamic modeling of multiscale biological systems. Random forests and permutation feature importance are proposed to mine omics datasets, guiding feature selection and dimensionality reduction for dynamic modeling. Continuous and differentiable machine-learning functions can be trained to link the reduced omics feature set to key components of the dynamic model, resulting in a hybrid model. As proof of concept, we apply this framework to a high-dimensional proteomics dataset of $\textit{Saccharomyces cerevisiae}$. After identifying key intracellular proteins that correlate with cell growth, targeted dynamic experiments are designed, and key model parameters are captured as functions of the selected proteins using Gaussian processes. This approach captures the dynamic behavior of yeast strains under varying proteome profiles while estimating the uncertainty in the hybrid model's predictions. The outlined modeling framework is adaptable to other scenarios, such as integrating additional layers of omics data for more advanced multiscale biological systems, or employing alternative machine-learning methods to handle larger datasets. Overall, this study outlines a strategy for leveraging omics data to inform multiscale dynamic modeling in systems biology and bioprocess engineering.

Optogenetic modulation of adenosine triphosphatase (ATPase) expression represents a novel approach to maximize bioprocess efficiency by leveraging enforced adenosine triphosphate (ATP) turnover. In this study, we experimentally implement a model-based open-loop optimization scheme for optogenetic modulation of the expression of ATPase. Increasing the intracellular concentration of ATPase, and thus the level of ATP turnover, in bioprocesses with product synthesis coupled with ATP generation, can lead to increased product formation and substrate uptake. Previous simulation studies formulated optimal control problems using dynamic constraint-based models to find optimal light inputs in fermentations with optogenetically mediated ATPase expression. However, using these models poses challenges due to resulting bilevel optimizations and complex parameterization. Here, we outline a simplified unsegregated and quasi-unstructured kinetic modeling approach that reduces the number of dynamic states and leads to single-level optimizations. The models can be augmented with Gaussian processes to compensate for model uncertainties. We implement optimal control constrained by knowledge-based and hybrid models for optogenetic ATPase expression in $\textit{Escherichia coli}$ with lactate as the main product. To do so, we genetically engineer $\textit{E. coli}$ to obtain optogenetic expression of ATPase using the CcaS/CcaR system. This represents the first experimental implementation of model-based optimization of ATPase expression in bioprocesses., Comment: 24 pages, 5 figures, journal submission

Metabolic cybergenetics is a promising concept that interfaces gene expression and cellular metabolism with computers for real-time dynamic metabolic control. The focus is on control at the transcriptional level, serving as a means to modulate intracellular metabolic fluxes. Recent strategies in this field have employed constraint-based dynamic models for process optimization, control, and estimation. However, this results in bilevel dynamic optimization problems, which pose considerable numerical and conceptual challenges. In this study, we present an alternative hybrid physics-informed dynamic modeling framework for metabolic cybergenetics, aimed at simplifying optimization, control, and estimation tasks. By utilizing machine-learning surrogates, our approach effectively embeds the physics of metabolic networks into the process rates of structurally simpler macro-kinetic models coupled with gene expression. These surrogates, informed by flux balance analysis, link the domains of manipulatable intracellular enzymes to metabolic exchange fluxes. This ensures that critical knowledge captured by the system's metabolic network is preserved. The resulting models can be integrated into metabolic cybergenetic schemes involving single-level optimizations. Additionally, the hybrid modeling approach maintains the number of system states at a necessary minimum, easing the burden of process monitoring and estimation. Our hybrid physics-informed metabolic cybergenetic framework is demonstrated using a computational case study on the optogenetically-assisted production of itaconate by $\textit{Escherichia coli}$., Comment: 25 pages, 10 figures, journal submission (reviewed/accepted version)

We outline a modeling and optimization strategy for investigating dynamic metabolic engineering interventions. Our framework is particularly useful at the early stages of research and development, often constrained by limited knowledge and experimental data. Elucidating a priori optimal trajectories of manipulatable intracellular fluxes can guide the design of suitable control schemes, e.g., cyber(ge)netic or in-cell approaches, and the selection of appropriate actuators, e.g., at the transcriptional or post-translational levels. Model-based dynamic optimization is proposed to predict optimal trajectories of target manipulatable intracellular fluxes. A challenge emerges as existing models are often oversimplified, lacking insights into metabolism, or excessively complex, making them difficult to build and implement. Here, we use surrogates derived from steady-state solutions of constraint-based metabolic models to link manipulatable intracellular fluxes to the process exchange rates of structurally simple hybrid dynamic models. The latter can be conveniently used in optimal control problems of metabolism. As a proof of concept, we apply our method to a reduced metabolic network of $\textit{Escherichia coli}$ considering two different scenarios of dynamic metabolic engineering., Comment: 6 pages, 6 figures, conference paper submission

Biotechnology offers many opportunities for the sustainable manufacturing of valuable products. The toolbox to optimize bioprocesses includes \textit{extracellular} process elements such as the bioreactor design and mode of operation, medium formulation, culture conditions, feeding rates, etc. However, these elements are frequently insufficient for achieving optimal process performance or precise product composition. One can use metabolic and genetic engineering methods for optimization at the intracellular level. Nevertheless, those are often of static nature, failing when applied to dynamic processes or if disturbances occur. Furthermore, many bioprocesses are optimized empirically and implemented with little-to-no feedback control to counteract disturbances. The concept of cybergenetics has opened new possibilities to optimize bioprocesses by enabling online modulation of the gene expression of metabolism-relevant proteins via external inputs (e.g., light intensity in optogenetics). Here, we fuse cybergenetics with model-based optimization and predictive control for optimizing dynamic bioprocesses. To do so, we propose to use dynamic constraint-based models that integrate the dynamics of metabolic reactions, resource allocation, and inducible gene expression. We formulate a model-based optimal control problem to find the optimal process inputs. Furthermore, we propose using model predictive control to address uncertainties via online feedback. We focus on fed-batch processes, where the substrate feeding rate is an additional optimization variable. As a simulation example, we show the optogenetic control of the ATPase enzyme complex for dynamic modulation of enforced ATP wasting to adjust product yield and productivity.