Search

Starting with sets of disorganized observations of spatially varying and temporally evolving systems, obtained at different (also disorganized) sets of parameters, we demonstrate the data-driven derivation of parameter dependent, evolutionary partial differential equation (PDE) models capable of generating the data. This tensor type of data is reminiscent of shuffled (multidimensional) puzzle tiles. The independent variables for the evolution equations (their "space" and "time") as well as their effective parameters are all emergent , i.e. determined in a data-driven way from our disorganized observations of behavior in them. We use a diffusion map based questionnaire approach to build a smooth parametrization of our emergent space/time/parameter space for the data. This approach iteratively processes the data by successively observing them on the "space," the "time" and the "parameter" axes of a tensor. Once the data become organized, we use machine learning (here, neural networks) to approximate the operators governing the evolution equations in this emergent space. Our illustrative examples are based (i) on a simple advection-diffusion model; (ii) on a previously developed vertex-plus-signaling model of Drosophila embryonic development; and (iii) on two complex dynamic network models (one neuronal and one coupled oscillator model) for which no obvious smooth embedding geometry is known a priori. This allows us to discuss features of the process like symmetry breaking, translational invariance, and autonomousness of the emergent PDE model, as well as its interpretability., (© The Author(s) 2025. Published by Oxford University Press on behalf of National Academy of Sciences.)

Before we attempt to (approximately) learn a function between two sets of observables of a physical process, we must first decide what the inputs and outputs of the desired function are going to be. Here we demonstrate two distinct, data-driven ways of first deciding "the right quantities" to relate through such a function, and then proceeding to learn it. This is accomplished by first processing simultaneous heterogeneous data streams (ensembles of time series) from observations of a physical system: records of multiple observation processes of the system. We determine (i) what subsets of observables are common between the observation processes (and therefore observable from each other, relatable through a function); and (ii) what information is unrelated to these common observables, therefore particular to each observation process, and not contributing to the desired function. Any data-driven technique can subsequently be used to learn the input-output relation-from k-nearest neighbors and Geometric Harmonics to Gaussian Processes and Neural Networks. Two particular "twists" of the approach are discussed. The first has to do with the identifiability of particular quantities of interest from the measurements. We now construct mappings from a single set of observations from one process to entire level sets of measurements of the second process, consistent with this single set. The second attempts to relate our framework to a form of causality: if one of the observation processes measures "now," while the second observation process measures "in the future," the function to be learned among what is common across observation processes constitutes a dynamical model for the system evolution., (© The Author(s) 2024. Published by Oxford University Press on behalf of National Academy of Sciences.)

Background: E. coli chemotactic motion in the presence of a chemonutrient field can be studied using wet laboratory experiments or macroscale-level partial differential equations (PDEs) (among others). Bridging experimental measurements and chemotactic Partial Differential Equations requires knowledge of the evolution of all underlying fields, initial and boundary conditions, and often necessitates strong assumptions. In this work, we propose machine learning approaches, along with ideas from the Whitney and Takens embedding theorems, to circumvent these challenges., Results: Machine learning approaches for identifying underlying PDEs were (a) validated through the use of simulation data from established continuum models and (b) used to infer chemotactic PDEs from experimental data. Such data-driven models were surrogates either for the entire chemotactic PDE right-hand-side (black box models), or, in a more targeted fashion, just for the chemotactic term (gray box models). Furthermore, it was demonstrated that a short history of bacterial density may compensate for the missing measurements of the field of chemonutrient concentration. In fact, given reasonable conditions, such a short history of bacterial density measurements could even be used to infer chemonutrient concentration., Conclusion: Data-driven PDEs are an important modeling tool when studying Chemotaxis at the macroscale, as they can learn bacterial motility from various data sources, fidelities (here, computational models, experiments) or coordinate systems. The resulting data-driven PDEs can then be simulated to reproduce/predict computational or experimental bacterial density profile data independent of the coordinate system, approximate meaningful parameters or functional terms, and even possibly estimate the underlying (unmeasured) chemonutrient field evolution., (© 2024. The Author(s).)

Transformations are a key tool in the qualitative study of dynamical systems: transformations to a normal form, for example, underpin the study of instabilities and bifurcations. In this work, we test, and when possible establish, an equivalence between two different artificial neural networks by attempting to construct a data-driven transformation between them, using diffusion maps with a Mahalanobis-like metric. If the construction succeeds, the two networks can be thought of as belonging to the same equivalence class. We first discuss transformation functions between only the outputs of the two networks; we then also consider transformations that take into account outputs (activations) of a number of internal neurons from each network. Whitney's theorem dictates the number of (generic) measurements from one of the networks required to reconstruct each and every feature of the second network. The construction of the transformation function relies on a consistent, intrinsic representation of the network input space. We illustrate our algorithm by matching neural network pairs trained to learn (a) observations of scalar functions, (b) observations of two-dimensional vector fields, and (c) representations of images of a moving three-dimensional object (a rotating horse). We also demonstrate reconstruction of a network's input (and output) from minimal partial observations of intermediate neuron activations. The construction of equivalences across different network instantiations clearly relates to transfer learning and will also be valuable in establishing equivalence between different machine learning-based tools., (© 2024 Author(s). Published under an exclusive license by AIP Publishing.)

Deriving closed-form analytical expressions for reduced-order models, and judiciously choosing the closures leading to them, has long been the strategy of choice for studying phase- and noise-induced transitions for agent-based models (ABMs). In this paper, we propose a data-driven framework that pinpoints phase transitions for an ABM-the Desai-Zwanzig model-in its mean-field limit, using a smaller number of variables than traditional closed-form models. To this end, we use the manifold learning algorithm Diffusion Maps to identify a parsimonious set of data-driven latent variables, and we show that they are in one-to-one correspondence with the expected theoretical order parameter of the ABM. We then utilize a deep learning framework to obtain a conformal reparametrization of the data-driven coordinates that facilitates, in our example, the identification of a single parameter-dependent ordinary differential equation (ODE) in these coordinates. We identify this ODE through a residual neural network inspired by a numerical integration scheme (forward Euler). We then use the identified ODE-enabled through an odd symmetry transformation-to construct the bifurcation diagram exhibiting the phase transition.

We study the tipping point collective dynamics of an adaptive susceptible-infected-susceptible (SIS) epidemiological network in a data-driven, machine learning-assisted manner. We identify a parameter-dependent effective stochastic differential equation (eSDE) in terms of physically meaningful coarse mean-field variables through a deep-learning ResNet architecture inspired by numerical stochastic integrators. We construct an approximate effective bifurcation diagram based on the identified drift term of the eSDE and contrast it with the mean-field SIS model bifurcation diagram. We observe a subcritical Hopf bifurcation in the evolving network's effective SIS dynamics that causes the tipping point behavior; this takes the form of large amplitude collective oscillations that spontaneously-yet rarely-arise from the neighborhood of a (noisy) stationary state. We study the statistics of these rare events both through repeated brute force simulations and by using established mathematical/computational tools exploiting the right-hand side of the identified SDE. We demonstrate that such a collective SDE can also be identified (and the rare event computations also performed) in terms of data-driven coarse observables, obtained here via manifold learning techniques, in particular, Diffusion Maps. The workflow of our study is straightforwardly applicable to other complex dynamic problems exhibiting tipping point dynamics., (© 2024 Author(s). Published under an exclusive license by AIP Publishing.)

We present a machine learning framework bridging manifold learning, neural networks, Gaussian processes, and Equation-Free multiscale approach, for the construction of different types of effective reduced order models from detailed agent-based simulators and the systematic multiscale numerical analysis of their emergent dynamics. The specific tasks of interest here include the detection of tipping points, and the uncertainty quantification of rare events near them. Our illustrative examples are an event-driven, stochastic financial market model describing the mimetic behavior of traders, and a compartmental stochastic epidemic model on an Erdös-Rényi network. We contrast the pros and cons of the different types of surrogate models and the effort involved in learning them. Importantly, the proposed framework reveals that, around the tipping points, the emergent dynamics of both benchmark examples can be effectively described by a one-dimensional stochastic differential equation, thus revealing the intrinsic dimensionality of the normal form of the specific type of the tipping point. This allows a significant reduction in the computational cost of the tasks of interest., (© 2024. The Author(s).)

Finding the dynamical law of observable quantities lies at the core of physics. Within the particular field of statistical mechanics, the generalized Langevin equation (GLE) comprises a general model for the evolution of observables covering a great deal of physical systems with many degrees of freedom and an inherently stochastic nature. Although formally exact, GLE brings its own great challenges. It depends on the complete history of the observables under scrutiny, as well as the microscopic degrees of freedom, all of which are often inaccessible. We show that these drawbacks can be overcome by adopting elements of machine learning from empirical data, in particular coupling a multilayer perceptron (MLP) with the formal structure of GLE and calibrating the MLP with the data. This yields a powerful computational tool capable of describing noisy complex systems beyond the realms of statistical mechanics. It is exemplified with a number of representative examples from different fields: from a single colloidal particle and particle chains in a thermal bath to climatology and finance, showing in all cases excellent agreement with the actual observable dynamics. The new framework offers an alternative perspective for the study of nonequilibrium processes opening also a new route for stochastic modeling.

A valuable step in the modeling of multiscale dynamical systems in fields such as computational chemistry, biology, and materials science is the representative sampling of the phase space over long time scales of interest; this task is not, however, without challenges. For example, the long term behavior of a system with many degrees of freedom often cannot be efficiently computationally explored by direct dynamical simulation; such systems can often become trapped in local free energy minima. In the study of physics-based multi-time-scale dynamical systems, techniques have been developed for enhancing sampling in order to accelerate exploration beyond free energy barriers. On the other hand, in the field of machine learning (ML), a generic goal of generative models is to sample from a target density, after training on empirical samples from this density. Score-based generative models (SGMs) have demonstrated state-of-the-art capabilities in generating plausible data from target training distributions. Conditional implementations of such generative models have been shown to exhibit significant parallels with long-established-and physics-based-solutions to enhanced sampling. These physics-based methods can then be enhanced through coupling with the ML generative models, complementing the strengths and mitigating the weaknesses of each technique. In this work, we show that SGMs can be used in such a coupling framework to improve sampling in multiscale dynamical systems., (© 2024 Author(s). Published under an exclusive license by AIP Publishing.)

We propose the Poisson neural networks (PNNs) to learn Poisson systems and trajectories of autonomous systems from data. Based on the Darboux-Lie theorem, the phase flow of a Poisson system can be written as the composition of: 1) a coordinate transformation; 2) an extended symplectic map; and 3) the inverse of the transformation. In this work, we extend this result to the unknotted trajectories of autonomous systems. We employ structured neural networks with physical priors to approximate the three aforementioned maps. We demonstrate through several simulations that PNNs are capable of handling very accurately several challenging tasks, including the motion of a particle in the electromagnetic potential, the nonlinear Schrödinger equation, and pixel observations of the two-body problem.

We propose a machine learning framework for the data-driven discovery of macroscopic chemotactic Partial Differential Equations (PDEs)-and the closures that lead to them- from high-fidelity, individual-based stochastic simulations of Escherichia coli bacterial motility. The fine scale, chemomechanical, hybrid (continuum-Monte Carlo) simulation model embodies the underlying biophysics, and its parameters are informed from experimental observations of individual cells. Using a parsimonious set of collective observables, we learn effective, coarse-grained "Keller-Segel class" chemotactic PDEs using machine learning regressors: (a) (shallow) feedforward neural networks and (b) Gaussian Processes. The learned laws can be black-box (when no prior knowledge about the PDE law structure is assumed) or gray-box when parts of the equation (e.g. the pure diffusion part) is known and "hardwired" in the regression process. More importantly, we discuss data-driven corrections (both additive and functional), to analytically known, approximate closures., (© 2023. The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature.)

Finding saddle points of dynamical systems is an important problem in practical applications, such as the study of rare events of molecular systems. Gentlest ascent dynamics (GAD) (10.1088/0951-7715/24/6/008) is one of a number of algorithms in existence that attempt to find saddle points. It works by deriving a new dynamical system in which saddle points of the original system become stable equilibria. GAD has been recently generalized to the study of dynamical systems on manifolds (differential algebraic equations) described by equality constraints (10.1007/s10915-022-01838-3) and given in an extrinsic formulation. In this paper, we present an extension of GAD to manifolds defined by point-clouds, formulated by using an intrinsic viewpoint. These point-clouds are adaptively sampled during an iterative process that drives the system from the initial conformation (typically in the neighborhood of a stable equilibrium) to a saddle point. Our method requires the reactant (initial conformation), does not require the explicit constraint equations to be specified, and is purely data-driven.

It is shown that Machine Learning (ML) algorithms can usefully capture the effect of crystallization composition and conditions (inputs) on key microstructural characteristics (outputs) of faujasite type zeolites (structure types FAU, EMT, and their intergrowths), which are widely used zeolite catalysts and adsorbents. The utility of ML (in particular, Geometric Harmonics) toward learning input-output relationships of interest is demonstrated, and a comparison with Neural Networks and Gaussian Process Regression, as alternative approaches, is provided. Through ML, synthesis conditions were identified to enhance the Si/Al ratio of high purity FAU zeolite to the hitherto highest level (i.e., Si/Al = 3.5) achieved via direct (not seeded), and organic structure-directing-agent-free synthesis from sodium aluminosilicate sols. The analysis of the ML algorithms' results offers the insight that reduced Na 2 O content is key to formulating FAU materials with high Si/Al ratio. An acid catalyst prepared by partial ion exchange of the high-Si/Al-ratio FAU (Si/Al = 3.5) exhibits improved proton reactivity (as well as specific activity, per unit mass of catalyst) in propane cracking and dehydrogenation compared to the catalyst prepared from the previously reported highest Si/Al ratio (Si/Al = 2.8)., (© 2023. The Author(s).)

Depolymerization is a promising strategy for recycling waste plastic into constituent monomers for subsequent repolymerization 1 . However, many commodity plastics cannot be selectively depolymerized using conventional thermochemical approaches, as it is difficult to control the reaction progress and pathway. Although catalysts can improve the selectivity, they are susceptible to performance degradation 2 . Here we present a catalyst-free, far-from-equilibrium thermochemical depolymerization method that can generate monomers from commodity plastics (polypropylene (PP) and poly(ethylene terephthalate) (PET)) by means of pyrolysis. This selective depolymerization process is realized by two features: (1) a spatial temperature gradient and (2) a temporal heating profile. The spatial temperature gradient is achieved using a bilayer structure of porous carbon felt, in which the top electrically heated layer generates and conducts heat down to the underlying reactor layer and plastic. The resulting temperature gradient promotes continuous melting, wicking, vaporization and reaction of the plastic as it encounters the increasing temperature traversing the bilayer, enabling a high degree of depolymerization. Meanwhile, pulsing the electrical current through the top heater layer generates a temporal heating profile that features periodic high peak temperatures (for example, about 600 °C) to enable depolymerization, yet the transient heating duration (for example, 0.11 s) can suppress unwanted side reactions. Using this approach, we depolymerized PP and PET to their monomers with yields of about 36% and about 43%, respectively. Overall, this electrified spatiotemporal heating (STH) approach potentially offers a solution to the global plastic waste problem., (© 2023. The Author(s), under exclusive licence to Springer Nature Limited.)

Radiation exposure of healthy cells can halt cell cycle temporarily or permanently. In this work, we analyze the time evolution of p21 and p53 from two single cell datasets of retinal pigment epithelial cells exposed to several levels of radiation, and in particular, the effect of radiation on cell cycle arrest. Employing various quantification methods from signal processing, we show how p21 levels, and to a lesser extent p53 levels, dictate whether the cells are arrested in their cell cycle and how frequently these mitosis events are likely to occur. We observed that single cells exposed to the same dose of DNA damage exhibit heterogeneity in cellular outcomes and that the frequency of cell division is a more accurate monitor of cell damage rather than just radiation level. Finally, we show how heterogeneity in DNA damage signaling is manifested early in the response to radiation exposure level and has potential to predict long-term fate., (© 2022. The Author(s), under exclusive licence to ADMC Associazione Differenziamento e Morte Cellulare.)

We identify effective stochastic differential equations (SDEs) for coarse observables of fine-grained particle- or agent-based simulations; these SDEs then provide useful coarse surrogate models of the fine scale dynamics. We approximate the drift and diffusivity functions in these effective SDEs through neural networks, which can be thought of as effective stochastic ResNets. The loss function is inspired by, and embodies, the structure of established stochastic numerical integrators (here, Euler-Maruyama and Milstein); our approximations can thus benefit from backward error analysis of these underlying numerical schemes. They also lend themselves naturally to "physics-informed" gray-box identification when approximate coarse models, such as mean field equations, are available. Existing numerical integration schemes for Langevin-type equations and for stochastic partial differential equations can also be used for training; we demonstrate this on a stochastically forced oscillator and the stochastic wave equation. Our approach does not require long trajectories, works on scattered snapshot data, and is designed to naturally handle different time steps per snapshot. We consider both the case where the coarse collective observables are known in advance, as well as the case where they must be found in a data-driven manner.

We present a data-driven approach to learning surrogate models for amplitude equations and illustrate its application to interfacial dynamics of phase field systems. In particular, we demonstrate learning effective partial differential equations describing the evolution of phase field interfaces from full phase field data. We illustrate this on a model phase field system, where analytical approximate equations for the dynamics of the phase field interface (a higher-order eikonal equation and its approximation, the Kardar-Parisi-Zhang equation) are known. For this system, we discuss data-driven approaches for the identification of equations that accurately describe the front interface dynamics. When the analytical approximate models mentioned above become inaccurate, as we move beyond the region of validity of the underlying assumptions, the data-driven equations outperform them. In these regimes, going beyond black box identification, we explore different approaches to learning data-driven corrections to the analytically approximate models, leading to effective gray box partial differential equations.

Circadian rhythmicity lies at the center of various important physiological and behavioral processes in mammals, such as sleep, metabolism, homeostasis, mood changes, and more. Misalignment of intrinsic neuronal oscillations with the external day-night cycle can disrupt such processes and lead to numerous disorders. In this work, we computationally determine the limits of circadian synchronization to external light signals of different frequency, duty cycle, and simulated amplitude. Instead of modeling circadian dynamics with generic oscillator models (e.g., Kuramoto-type), we use a detailed computational neuroscience model, which integrates biomolecular dynamics, neuronal electrophysiology, and network effects. This allows us to investigate the effect of small drug molecules, such as Longdaysin, and connect our results with experimental findings. To combat the high dimensionality of such a detailed model, we employ a matrix-free approach, while our entire algorithmic pipeline enables numerical continuation and construction of bifurcation diagrams using only direct simulation. We, thus, computationally explore the effect of heterogeneity in the circadian neuronal network, as well as the effect of the corrective therapeutic intervention of Longdaysin. Last, we employ unsupervised learning to construct a data-driven embedding space for representing neuronal heterogeneity.

We present a data-driven approach to characterizing nonidentifiability of a model's parameters and illustrate it through dynamic as well as steady kinetic models. By employing Diffusion Maps and their extensions, we discover the minimal combinations of parameters required to characterize the output behavior of a chemical system: a set of effective parameters for the model. Furthermore, we introduce and use a Conformal Autoencoder Neural Network technique, as well as a kernel-based Jointly Smooth Function technique, to disentangle the redundant parameter combinations that do not affect the output behavior from the ones that do. We discuss the interpretability of our data-driven effective parameters, and demonstrate the utility of the approach both for behavior prediction and parameter estimation. In the latter task, it becomes important to describe level sets in parameter space that are consistent with a particular output behavior. We validate our approach on a model of multisite phosphorylation, where a reduced set of effective parameters (nonlinear combinations of the physical ones) has previously been established analytically., (© The Author(s) 2022. Published by Oxford University Press on behalf of National Academy of Sciences.)

We address a three-tier numerical framework based on nonlinear manifold learning for the forecasting of high-dimensional time series, relaxing the "curse of dimensionality" related to the training phase of surrogate/machine learning models. At the first step, we embed the high-dimensional time series into a reduced low-dimensional space using nonlinear manifold learning (local linear embedding and parsimonious diffusion maps). Then, we construct reduced-order surrogate models on the manifold (here, for our illustrations, we used multivariate autoregressive and Gaussian process regression models) to forecast the embedded dynamics. Finally, we solve the pre-image problem, thus lifting the embedded time series back to the original high-dimensional space using radial basis function interpolation and geometric harmonics. The proposed numerical data-driven scheme can also be applied as a reduced-order model procedure for the numerical solution/propagation of the (transient) dynamics of partial differential equations (PDEs). We assess the performance of the proposed scheme via three different families of problems: (a) the forecasting of synthetic time series generated by three simplistic linear and weakly nonlinear stochastic models resembling electroencephalography signals, (b) the prediction/propagation of the solution profiles of a linear parabolic PDE and the Brusselator model (a set of two nonlinear parabolic PDEs), and (c) the forecasting of a real-world data set containing daily time series of ten key foreign exchange rates spanning the time period 3 September 2001-29 October 2020.

We propose an approach to learn effective evolution equations for large systems of interacting agents. This is demonstrated on two examples, a well-studied system of coupled normal form oscillators and a biologically motivated example of coupled Hodgkin-Huxley-like neurons. For such types of systems there is no obvious space coordinate in which to learn effective evolution laws in the form of partial differential equations. In our approach, we accomplish this by learning embedding coordinates from the time series data of the system using manifold learning as a first step. In these emergent coordinates, we then show how one can learn effective partial differential equations, using neural networks, that do not only reproduce the dynamics of the oscillator ensemble, but also capture the collective bifurcations when system parameters vary. The proposed approach thus integrates the automatic, data-driven extraction of emergent space coordinates parametrizing the agent dynamics, with machine-learning assisted identification of an emergent PDE description of the dynamics in this parametrization., (© 2022. The Author(s).)

Conventional thermochemical syntheses by continuous heating under near-equilibrium conditions face critical challenges in improving the synthesis rate, selectivity, catalyst stability and energy efficiency, owing to the lack of temporal control over the reaction temperature and time, and thus the reaction pathways 1-3 . As an alternative, we present a non-equilibrium, continuous synthesis technique that uses pulsed heating and quenching (for example, 0.02 s on, 1.08 s off) using a programmable electric current to rapidly switch the reaction between high (for example, up to 2,400 K) and low temperatures. The rapid quenching ensures high selectivity and good catalyst stability, as well as lowers the average temperature to reduce the energy cost. Using CH 4 pyrolysis as a model reaction, our programmable heating and quenching technique leads to high selectivity to value-added C 2 products (>75% versus <35% by the conventional non-catalytic method and versus <60% by most conventional methods using optimized catalysts). Our technique can be extended to a range of thermochemical reactions, such as NH 3 synthesis, for which we achieve a stable and high synthesis rate of about 6,000 μmol g Fe -1  h -1 at ambient pressure for >100 h using a non-optimized catalyst. This study establishes a new model towards highly efficient non-equilibrium thermochemical synthesis., (© 2022. The Author(s), under exclusive licence to Springer Nature Limited.)

High-entropy nanoparticles have become a rapidly growing area of research in recent years. Because of their multielemental compositions and unique high-entropy mixing states (i.e., solid-solution) that can lead to tunable activity and enhanced stability, these nanoparticles have received notable attention for catalyst design and exploration. However, this strong potential is also accompanied by grand challenges originating from their vast compositional space and complex atomic structure, which hinder comprehensive exploration and fundamental understanding. Through a multidisciplinary view of synthesis, characterization, catalytic applications, high-throughput screening, and data-driven materials discovery, this review is dedicated to discussing the important progress of high-entropy nanoparticles and unveiling the critical needs for their future development for catalysis, energy, and sustainability applications.

In this work, we propose a method to learn multivariate probability distributions using sample path data from stochastic differential equations. Specifically, we consider temporally evolving probability distributions (e.g., those produced by integrating local or nonlocal Fokker-Planck equations). We analyze this evolution through machine learning assisted construction of a time-dependent mapping that takes a reference distribution (say, a Gaussian) to each and every instance of our evolving distribution. If the reference distribution is the initial condition of a Fokker-Planck equation, what we learn is the time-T map of the corresponding solution. Specifically, the learned map is a multivariate normalizing flow that deforms the support of the reference density to the support of each and every density snapshot in time. We demonstrate that this approach can approximate probability density function evolutions in time from observed sampled data for systems driven by both Brownian and Lévy noise. We present examples with two- and three-dimensional, uni- and multimodal distributions to validate the method.

Landscapes play an important role in many areas of biology, in which biological lives are deeply entangled. Here we discuss a form of landscape in evolutionary biology which takes into account (1) initial growth rates, (2) mutation rates, (3) resource consumption by organisms, and (4) cyclic changes in the resources with time. The long-term equilibrium number of surviving organisms as a function of these four parameters forms what we call a success landscape, a landscape we would claim is qualitatively different from fitness landscapes which commonly do not include mutations or resource consumption/changes in mapping genomes to the final number of survivors. Although our analysis is purely theoretical, we believe the results have possibly strong connections to how we might treat diseases such as cancer in the future with a deeper understanding of the interplay between resource degradation, mutation, and uncontrolled cell growth., (© 2021. The Author(s), under exclusive licence to Springer Nature B.V.)

The study of nonlinear waves that collapse in finite time is a theme of universal interest, e.g., within optical, atomic, plasma physics, and nonlinear dynamics. Here we revisit the quintessential example of the nonlinear Schrödinger equation and systematically derive a normal form for the emergence of radially symmetric blowup solutions from stationary ones. While this is an extensively studied problem, such a normal form, based on the methodology of asymptotics beyond all algebraic orders, applies to both the dimension-dependent and power-law-dependent bifurcations previously studied. It yields excellent agreement with numerics in both leading and higher-order effects, it is applicable to both infinite and finite domains, and it is valid in both critical and supercritical regimes.

We present an approach, based on learning an intrinsic data manifold, for the initialization of the internal state values of long short-term memory (LSTM) recurrent neural networks, ensuring consistency with the initial observed input data. Exploiting the generalized synchronization concept, we argue that the converged, "mature" internal states constitute a function on this learned manifold. The dimension of this manifold then dictates the length of observed input time series data required for consistent initialization. We illustrate our approach through a partially observed chemical model system, where initializing the internal LSTM states in this fashion yields visibly improved performance. Finally, we show that learning this data manifold enables the transformation of partially observed dynamics into fully observed ones, facilitating alternative identification paths for nonlinear dynamical systems.

Large collections of coupled, heterogeneous agents can manifest complex dynamical behavior presenting difficulties for simulation and analysis. However, if the collective dynamics lie on a low-dimensional manifold, then the original agent-based model may be approximated with a simplified surrogate model on and near the low-dimensional space where the dynamics live. Analytically identifying such simplified models can be challenging or impossible, but here we present a data-driven coarse-graining methodology for discovering such reduced models. We consider two types of reduced models: globally based models that use global information and predict dynamics using information from the whole ensemble and locally based models that use local information, that is, information from just a subset of agents close (close in heterogeneity space, not physical space) to an agent, to predict the dynamics of an agent. For both approaches, we are able to learn laws governing the behavior of the reduced system on the low-dimensional manifold directly from time series of states from the agent-based system. These laws take the form of either a system of ordinary differential equations (ODEs), for the globally based approach, or a partial differential equation (PDE) in the locally based case. For each technique, we employ a specialized artificial neural network integrator that has been templated on an Euler time stepper (i.e., a ResNet) to learn the laws of the reduced model. As part of our methodology, we utilize the proper orthogonal decomposition (POD) to identify the low-dimensional space of the dynamics. Our globally based technique uses the resulting POD basis to define a set of coordinates for the agent states in this space and then seeks to learn the time evolution of these coordinates as a system of ODEs. For the locally based technique, we propose a methodology for learning a partial differential equation representation of the agents; the PDE law depends on the state variables and partial derivatives of the state variables with respect to model heterogeneities. We require that the state variables are smooth with respect to model heterogeneities, which permit us to cast the discrete agent-based problem as a continuous one in heterogeneity space. The agents in such a representation bear similarity to the discretization points used in typical finite element/volume methods. As an illustration of the efficacy of our techniques, we consider a simplified coupled neuron model for rhythmic oscillations in the pre-Bötzinger complex and demonstrate how our data-driven surrogate models are able to produce dynamics comparable to the dynamics of the full system. A nontrivial conclusion is that the dynamics can be equally well reproduced by an all-to-all coupled and by a locally coupled model of the same agents.

Equations governing physico-chemical processes are usually known at microscopic spatial scales, yet one suspects that there exist equations, e.g., in the form of partial differential equations (PDEs), that can explain the system evolution at much coarser, meso-, or macroscopic length scales. Discovering those coarse-grained effective PDEs can lead to considerable savings in computation-intensive tasks like prediction or control. We propose a framework combining artificial neural networks with multiscale computation, in the form of equation-free numerics, for the efficient discovery of such macro-scale PDEs directly from microscopic simulations. Gathering sufficient microscopic data for training neural networks can be computationally prohibitive; equation-free numerics enable a more parsimonious collection of training data by only operating in a sparse subset of the space-time domain. We also propose using a data-driven approach, based on manifold learning (including one using the notion of unnormalized optimal transport of distributions and one based on moment-based description of the distributions), to identify macro-scale dependent variable(s) suitable for the data-driven discovery of said PDEs. This approach can corroborate physically motivated candidate variables or introduce new data-driven variables, in terms of which the coarse-grained effective PDE can be formulated. We illustrate our approach by extracting coarse-grained evolution equations from particle-based simulations with a priori unknown macro-scale variable(s) while significantly reducing the requisite data collection computational effort.

Dynamic control of engineered microbes using light via optogenetics has been demonstrated as an effective strategy for improving the yield of biofuels, chemicals, and other products. An advantage of using light to manipulate microbial metabolism is the relative simplicity of interfacing biological and computer systems, thereby enabling in silico control of the microbe. Using this strategy for control and optimization of product yield requires an understanding of how the microbe responds in real-time to the light inputs. Toward this end, we present mechanistic models of a set of yeast optogenetic circuits. We show how these models can predict short- and long-time response to varying light inputs and how they are amenable to use with model predictive control (the industry standard among advanced control algorithms). These models reveal dynamics characterized by time-scale separation of different circuit components that affect the steady and transient levels of the protein under control of the circuit. Ultimately, this work will help enable real-time control and optimization tools for improving yield and consistency in the production of biofuels and chemicals using microbial fermentations.

The use of optogenetics in metabolic engineering for light-controlled microbial chemical production raises the prospect of utilizing control and optimization techniques routinely deployed in traditional chemical manufacturing. However, such mechanisms require well-characterized, customizable tools that respond fast enough to be used as real-time inputs during fermentations. Here, we present OptoINVRT7, a new rapid optogenetic inverter circuit to control gene expression in Saccharomyces cerevisiae . The circuit induces gene expression in only 0.6 h after switching cells from light to darkness, which is at least 6 times faster than previous OptoINVRT optogenetic circuits used for chemical production. In addition, we introduce an engineered inducible GAL1 promoter (P GAL1-S ), which is stronger than any constitutive or inducible promoter commonly used in yeast. Combining OptoINVRT7 with P GAL1-S achieves strong and light-tunable levels of gene expression with as much as 132.9 ± 22.6-fold induction in darkness. The high performance of this new optogenetic circuit in controlling metabolic enzymes boosts production of lactic acid and isobutanol by more than 50% and 15%, respectively. The strength and controllability of OptoINVRT7 and P GAL1-S open the door to applying process control tools to engineered metabolisms to improve robustness and yields in microbial fermentations for chemical production.

We propose a local conformal autoencoder (LOCA) for standardized data coordinates. LOCA is a deep learning-based method for obtaining standardized data coordinates from scientific measurements. Data observations are modeled as samples from an unknown, nonlinear deformation of an underlying Riemannian manifold, which is parametrized by a few normalized, latent variables. We assume a repeated measurement sampling strategy, common in scientific measurements, and present a method for learning an embedding in [Formula: see text] that is isometric to the latent variables of the manifold. The coordinates recovered by our method are invariant to diffeomorphisms of the manifold, making it possible to match between different instrumental observations of the same phenomenon. Our embedding is obtained using LOCA, which is an algorithm that learns to rectify deformations by using a local z-scoring procedure, while preserving relevant geometric information. We demonstrate the isometric embedding properties of LOCA in various model settings and observe that it exhibits promising interpolation and extrapolation capabilities, superior to the current state of the art. Finally, we demonstrate LOCA's efficacy in single-site Wi-Fi localization data and for the reconstruction of three-dimensional curved surfaces from two-dimensional projections., Competing Interests: The authors declare no competing interest., (Copyright © 2020 the Author(s). Published by PNAS.)

Systems of coupled dynamical units (e.g., oscillators or neurons) are known to exhibit complex, emergent behaviors that may be simplified through coarse-graining: a process in which one discovers coarse variables and derives equations for their evolution. Such coarse-graining procedures often require extensive experience and/or a deep understanding of the system dynamics. In this paper we present a systematic, data-driven approach to discovering "bespoke" coarse variables based on manifold learning algorithms. We illustrate this methodology with the classic Kuramoto phase oscillator model, and demonstrate how our manifold learning technique can successfully identify a coarse variable that is one-to-one with the established Kuramoto order parameter. We then introduce an extension of our coarse-graining methodology which enables us to learn evolution equations for the discovered coarse variables via an artificial neural network architecture templated on numerical time integrators (initial value solvers). This approach allows us to learn accurate approximations of time derivatives of state variables from sparse flow data, and hence discover useful approximate differential equation descriptions of their dynamic behavior. We demonstrate this capability by learning ODEs that agree with the known analytical expression for the Kuramoto order parameter dynamics at the continuum limit. We then show how this approach can also be used to learn the dynamics of coarse variables discovered through our manifold learning methodology. In both of these examples, we compare the results of our neural network based method to typical finite differences complemented with geometric harmonics. Finally, we present a series of computational examples illustrating how a variation of our manifold learning methodology can be used to discover sets of "effective" parameters, reduced parameter combinations, for multi-parameter models with complex coupling. We conclude with a discussion of possible extensions of this approach, including the possibility of obtaining data-driven effective partial differential equations for coarse-grained neuronal network behavior, as illustrated by the synchronization dynamics of Hodgkin-Huxley type neurons with a Chung-Lu network. Thus, we build an integrated suite of tools for obtaining data-driven coarse variables, data-driven effective parameters, and data-driven coarse-grained equations from detailed observations of networks of oscillators., (Copyright © 2020 Thiem, Kooshkbaghi, Bertalan, Laing and Kevrekidis.)

Data mining is routinely used to organize ensembles of short temporal observations so as to reconstruct useful, low-dimensional realizations of an underlying dynamical system. In this paper, we use manifold learning to organize unstructured ensembles of observations ("trials") of a system's response surface. We have no control over where every trial starts, and during each trial, operating conditions are varied by turning "agnostic" knobs, which change system parameters in a systematic, but unknown way. As one (or more) knobs "turn," we record (possibly partial) observations of the system response. We demonstrate how such partial and disorganized observation ensembles can be integrated into coherent response surfaces whose dimension and parametrization can be systematically recovered in a data-driven fashion. The approach can be justified through the Whitney and Takens embedding theorems, allowing reconstruction of manifolds/attractors through different types of observations. We demonstrate our approach by organizing unstructured observations of response surfaces, including the reconstruction of a cusp bifurcation surface for hydrogen combustion in a continuous stirred tank reactor. Finally, we demonstrate how this observation-based reconstruction naturally leads to informative transport maps between the input parameter space and output/state variable spaces.

Complex spatiotemporal dynamics of physicochemical processes are often modeled at a microscopic level (through, e.g., atomistic, agent-based, or lattice models) based on first principles. Some of these processes can also be successfully modeled at the macroscopic level using, e.g., partial differential equations (PDEs) describing the evolution of the right few macroscopic observables (e.g., concentration and momentum fields). Deriving good macroscopic descriptions (the so-called "closure problem") is often a time-consuming process requiring deep understanding/intuition about the system of interest. Recent developments in data science provide alternative ways to effectively extract/learn accurate macroscopic descriptions approximating the underlying microscopic observations. In this paper, we introduce a data-driven framework for the identification of unavailable coarse-scale PDEs from microscopic observations via machine-learning algorithms. Specifically, using Gaussian processes, artificial neural networks, and/or diffusion maps, the proposed framework uncovers the relation between the relevant macroscopic space fields and their time evolution (the right-hand side of the explicitly unavailable macroscopic PDE). Interestingly, several choices equally representative of the data can be discovered. The framework will be illustrated through the data-driven discovery of macroscopic, concentration-level PDEs resulting from a fine-scale, lattice Boltzmann level model of a reaction/transport process. Once the coarse evolution law is identified, it can be simulated to produce long-term macroscopic predictions. Different features (pros as well as cons) of alternative machine-learning algorithms for performing this task (Gaussian processes and artificial neural networks) are presented and discussed.

Concise, accurate descriptions of physical systems through their conserved quantities abound in the natural sciences. In data science, however, current research often focuses on regression problems, without routinely incorporating additional assumptions about the system that generated the data. Here, we propose to explore a particular type of underlying structure in the data: Hamiltonian systems, where an "energy" is conserved. Given a collection of observations of such a Hamiltonian system over time, we extract phase space coordinates and a Hamiltonian function of them that acts as the generator of the system dynamics. The approach employs an autoencoder neural network component to estimate the transformation from observations to the phase space of a Hamiltonian system. An additional neural network component is used to approximate the Hamiltonian function on this constructed space, and the two components are trained jointly. As an alternative approach, we also demonstrate the use of Gaussian processes for the estimation of such a Hamiltonian. After two illustrative examples, we extract an underlying phase space as well as the generating Hamiltonian from a collection of movies of a pendulum. The approach is fully data-driven and does not assume a particular form of the Hamiltonian function.

Large scale dynamical systems (e.g. many nonlinear coupled differential equations) can often be summarized in terms of only a few state variables (a few equations), a trait that reduces complexity and facilitates exploration of behavioral aspects of otherwise intractable models. High model dimensionality and complexity makes symbolic, pen-and-paper model reduction tedious and impractical, a difficulty addressed by recently developed frameworks that computerize reduction. Symbolic work has the benefit, however, of identifying both reduced state variables and parameter combinations that matter most ( effective parameters , "inputs"); whereas current computational reduction schemes leave the parameter reduction aspect mostly unaddressed. As the interest in mapping out and optimizing complex input-output relations keeps growing, it becomes clear that combating the curse of dimensionality also requires efficient schemes for input space exploration and reduction. Here, we explore systematic, data-driven parameter reduction by means of effective parameter identification , starting from current nonlinear manifoldlearning techniques enabling state space reduction. Our approach aspires to extend the data-driven determination of effective state variables with the data-driven discovery of effective model parameters , and thus to accelerate the exploration of high-dimensional parameter spaces associated with complex models.

In statistical modelling with Gaussian process regression, it has been shown that combining (few) high-fidelity data with (many) low-fidelity data can enhance prediction accuracy, compared to prediction based on the few high-fidelity data only. Such information fusion techniques for multi-fidelity data commonly approach the high-fidelity model f h ( t ) as a function of two variables ( t , s ), and then use f l ( t ) as the s data. More generally, the high-fidelity model can be written as a function of several variables ( t , s 1 , s 2 ….); the low-fidelity model f l and, say, some of its derivatives can then be substituted for these variables. In this paper, we will explore mathematical algorithms for multi-fidelity information fusion that use such an approach towards improving the representation of the high-fidelity function with only a few training data points. Given that f h may not be a simple function-and sometimes not even a function-of f l , we demonstrate that using additional functions of t , such as derivatives or shifts of f l , can drastically improve the approximation of f h through Gaussian processes. We also point out a connection with 'embedology' techniques from topology and dynamical systems. Our illustrative examples range from instructive caricatures to computational biology models, such as Hodgkin-Huxley neural oscillations., Competing Interests: We declare we have no competing interests.

When studying observations of chemical reaction dynamics, closed form equations based on a putative mechanism may not be available. Yet when sufficient data from experimental observations can be obtained, even without knowing what exactly the physical meaning of the parameter settings or recorded variables are, data-driven methods can be used to construct minimal (and in a sense, robust) realizations of the system. The approach attempts, in a sense, to circumvent physical understanding, by building intrinsic "information geometries" of the observed data, and thus enabling prediction without physical/chemical knowledge. Here we use such an approach to obtain evolution equations for a data-driven realization of the original system - in effect, allowing prediction based on the informed interrogation of the agnostically organized observation database. We illustrate the approach on observations of (a) the normal form for the cusp singularity, (b) a cusp singularity for the nonisothermal CSTR, and (c) a random invertible transformation of the nonisothermal CSTR, showing that one can predict even when the observables are not "simply explainable" physical quantities. We discuss current limitations and possible extensions of the procedure.

The macroscopic properties of molecular materials can be drastically influenced by their solid-state packing arrangements, of which there can be many (e.g., polymorphism). Strategies to controllably and predictively access select polymorphs are thus highly desired, but computationally predicting the conditions necessary to access a given polymorph is challenging with the current state of the art. Using derivatives of contorted hexabenzocoronene, cHBC, we employed data mining, rather than first-principles approaches, to find relationships between the crystallizing molecule, postdeposition solvent-vapor annealing conditions that induce polymorphic transformation, and the resulting polymorphs. This analysis yields a correlative function that can be used to successfully predict the appearance of either one of two polymorphs in thin films of cHBC derivatives. Within the postdeposition processing phase space of cHBC derivatives, we have demonstrated an approach to generate guidelines to select crystallization conditions to bias polymorph access. We believe this approach can be applied more broadly to accelerate the predictions of processing conditions to access desired molecular polymorphs, making progress toward one of the grand challenges identified by the Materials Genome Initiative.

Recent studies have demonstrated that enzyme-instructed self-assembly (EISA) in extra- or intracellular environments can serve as a multistep process for controlling cell fate. There is little knowledge, however, about the kinetics of EISA in the complex environments in or around cells. Here, we design and synthesize three dipeptidic precursors (ld-1-SO 3 , dl-1-SO 3 , dd-1-SO 3 ), consisting of diphenylalanine (l-Phe-d-Phe, d-Phe-l-Phe, d-Phe-d-Phe, respectively) as the backbone, which are capped by 2-(naphthalen-2-yl)acetic acid at the N-terminal and by 2-(4-(2-aminoethoxy)-4-oxobutanamido)ethane-1-sulfonic acid at the C-terminal. On hydrolysis by carboxylesterases (CES), these precursors result in hydrogelators, which self-assemble in water at different rates. Whereas all three precursors selectively kill cancer cells, especially high-grade serous ovarian carcinoma cells, by undergoing intracellular EISA, dl-1-SO 3 and dd-1-SO 3 exhibit the lowest and the highest activities, respectively, against the cancer cells. This trend inversely correlates with the rates of converting the precursors to the hydrogelators in phosphate-buffered saline. Because CES exists both extra- and intracellularly, we use kinetic modeling to analyze the kinetics of EISA inside cells and to calculate the cytotoxicity of each precursor for killing cancer cells. Our results indicate that (i) the stereochemistry of the precursors affects the morphology of the nanostructures formed by the hydrogelators, as well as the rate of enzymatic conversion; (ii) decreased extracellular hydrolysis of precursors favors intracellular EISA inside the cells; (iii) the inherent features ( e.g., self-assembling ability and morphology) of the EISA molecules largely dictate the cytotoxicity of intracellular EISA. As the kinetic analysis of intracellular EISA, this work elucidates how the stereochemistry modulates EISA in the complex extra- and/or intracellular environment for developing anticancer molecular processes. Moreover, it provides insights for understanding the kinetics and cytotoxicity of aggregates of aberrant proteins or peptides formed inside and outside cells.