Search

Phytoplankton blooms exhibit varying patterns in timing and number of peaks within ecosystems. These differences in blooming patterns are partly explained by phytoplankton:nutrient interactions and external factors such as temperature, salinity and light availability. Understanding these interactions and drivers is essential for effective bloom management and modelling as driving factors potentially differ or are shared across ecosystems on regional scales. Here, we used a 22-year data set (19 years training and 3 years validation data) containing chlorophyll, nutrients (dissolved and total), and external drivers (temperature, salinity, light) of the southern Baltic Sea coast, a European brackish shelf sea, which constituted six different phytoplankton blooming patterns. We employed generalized additive mixed models to characterize similar blooming patterns and trained an artificial neural network within the Universal Differential Equation framework to learn a differential equation representation of these pattern. Applying Sparse Identification of Nonlinear Dynamics uncovered algebraic relationships in phytoplankton:nutrient:external driver interactions. Nutrients availability was driving factor for blooms in enclosed coastal waters; nutrients and temperature in more open regions. We found evidence of hydrodynamical export of phytoplankton, natural mortality or external grazing not explicitly measured in the data. This data-driven workflow allows new insight into driver-differences in region specific blooming dynamics. [ABSTRACT FROM AUTHOR]

In the context of hybrid twins, a data-driven enrichment is added to the physics-based solution to represent with higher accuracy the reference solution assumed to be known at different points in the physical domain. Such an approach enables better predictions. However, the data-driven enrichment is usually represented by a regression, whose main drawbacks are (i) the difficulty of understanding the subjacent physics and (ii) the risks induced by the data-driven model extrapolation. This paper proposes a procedure enabling the extraction of a differential operator associated with the enrichment provided by the data-driven regression. For that purpose, a sparse Singular Value Decomposition, SVD, is introduced. It is then employed, first, in a full operator representation regularized optimization problem, where sparsity is promoted, leading to a linear programming problem, and then in a tensor decomposition of the operator's identification procedure. The results show the ability of the method to identify the exact missing operators from the model. The regularized optimization problem was also able to identify the weights of the missing terms with a relative error of about 10% on average, depending on the selected use case. [ABSTRACT FROM AUTHOR]

We develop a novel methodology for data-driven simulation of coupled multi-physics systems. The result of the method is a learned numerical integrator of the coupled system dynamics. In order to preserve the fundamental physics of the coupled systems, and thus preserve the geometrical properties of the governing equations—even if they may be completely unknown—we impose a port-metriplectic structure on the system evolution, i.e., a combination of a symplectic evolution for the system energy with a gradient flow for the entropy of each system, which can be exchanged through predefined ports. The resulting method guarantees by construction the satisfaction of the laws of thermodynamics for open systems, leading to accurate predictions of the future states of their dynamics. Examples are given for systems of varying complexity, based on synthetic as well as experimental data. [ABSTRACT FROM AUTHOR]

Gradient-based meta-learning methods have primarily been applied to classical machine learning tasks such as image classification. Recently, PDE-solving deep learning methods, such as neural operators, are starting to make an important impact on learning and predicting the response of a complex physical system directly from observational data. Taking the material modeling problems for example, the neural operator approach learns a surrogate mapping from the loading field to the corresponding material response field, which can be seen as learning the solution operator of a hidden PDE. The microstructure and mechanical parameters of each material specimen correspond to the (possibly heterogeneous) parameter field in this hidden PDE. Due to the limitation on experimental measurement techniques, the data acquisition for each material specimen is commonly challenging and costly. This fact calls for the utilization and transfer of existing knowledge to new and unseen material specimens, which corresponds to sampling efficient learning of the solution operator of a hidden PDE with a different parameter field. Herein, we propose a novel meta-learning approach for neural operators, which can be seen as transferring the knowledge of solution operators between governing (unknown) PDEs with varying parameter fields. Our approach is a provably universal solution operator for multiple PDE solving tasks, with a key theoretical observation that underlying parameter fields can be captured in the first layer of neural operator models, in contrast to typical final-layer transfer in existing meta-learning methods. As applications, we demonstrate the efficacy of our proposed approach on PDE-based datasets and a real-world material modeling problem, illustrating that our method can handle complex and nonlinear physical response learning tasks while greatly improving the sampling efficiency in unseen tasks.

Abstract This research introduces an accelerated training approach for Vanilla Physics-Informed Neural Networks (PINNs) that addresses three factors affecting the loss function: the initial weight state of the neural network, the ratio of domain to boundary points, and the loss weighting factor. The proposed method involves two phases. In the initial phase, a unique loss function is created using a subset of boundary conditions and partial differential equation terms. Furthermore, we introduce preprocessing procedures that aim to decrease the variance during initialization and choose domain points according to the initial weight state of various neural networks. The second phase resembles Vanilla-PINN training, but a portion of the random weights are substituted with weights from the first phase. This implies that the neural network’s structure is designed to prioritize the boundary conditions, subsequently affecting the overall convergence. The study evaluates the method using three benchmarks: two-dimensional flow over a cylinder, an inverse problem of inlet velocity determination, and the Burger equation. Incorporating weights generated in the first training phase neutralizes imbalance effects. Notably, the proposed approach outperforms Vanilla-PINN in terms of speed, convergence likelihood and eliminates the need for hyperparameter tuning to balance the loss function.

Abstract Learning operators with deep neural networks is an emerging paradigm for scientific computing. Deep Operator Network (DeepONet) is a modular operator learning framework that allows for flexibility in choosing the kind of neural network to be used in the trunk and/or branch of the DeepONet. This is beneficial as it has been shown many times that different types of problems require different kinds of network architectures for effective learning. In this work, we design an efficient neural operator based on the DeepONet architecture. We introduce U-Net enhanced DeepONet (U-DeepONet) for learning the solution operator of highly complex CO2-water two-phase flow in heterogeneous porous media. The U-DeepONet is more accurate in predicting gas saturation and pressure buildup than the state-of-the-art U-Net based Fourier Neural Operator (U-FNO) and the Fourier-enhanced Multiple-Input Operator (Fourier-MIONet) trained on the same dataset. Moreover, our U-DeepONet is significantly more efficient in training times than both the U-FNO (more than 18 times faster) and the Fourier-MIONet (more than 5 times faster), while consuming less computational resources. We also show that the U-DeepONet is more data efficient and better at generalization than both the U-FNO and the Fourier-MIONet.

This research introduces an accelerated training approach for Vanilla Physics-Informed Neural Networks (PINNs) that addresses three factors affecting the loss function: the initial weight state of the neural network, the ratio of domain to boundary points, and the loss weighting factor. The proposed method involves two phases. In the initial phase, a unique loss function is created using a subset of boundary conditions and partial differential equation terms. Furthermore, we introduce preprocessing procedures that aim to decrease the variance during initialization and choose domain points according to the initial weight state of various neural networks. The second phase resembles Vanilla-PINN training, but a portion of the random weights are substituted with weights from the first phase. This implies that the neural network's structure is designed to prioritize the boundary conditions, subsequently affecting the overall convergence. The study evaluates the method using three benchmarks: two-dimensional flow over a cylinder, an inverse problem of inlet velocity determination, and the Burger equation. Incorporating weights generated in the first training phase neutralizes imbalance effects. Notably, the proposed approach outperforms Vanilla-PINN in terms of speed, convergence likelihood and eliminates the need for hyperparameter tuning to balance the loss function. [ABSTRACT FROM AUTHOR]

Learning operators with deep neural networks is an emerging paradigm for scientific computing. Deep Operator Network (DeepONet) is a modular operator learning framework that allows for flexibility in choosing the kind of neural network to be used in the trunk and/or branch of the DeepONet. This is beneficial as it has been shown many times that different types of problems require different kinds of network architectures for effective learning. In this work, we design an efficient neural operator based on the DeepONet architecture. We introduce U-Net enhanced DeepONet (U-DeepONet) for learning the solution operator of highly complex CO2-water two-phase flow in heterogeneous porous media. The U-DeepONet is more accurate in predicting gas saturation and pressure buildup than the state-of-the-art U-Net based Fourier Neural Operator (U-FNO) and the Fourier-enhanced Multiple-Input Operator (Fourier-MIONet) trained on the same dataset. Moreover, our U-DeepONet is significantly more efficient in training times than both the U-FNO (more than 18 times faster) and the Fourier-MIONet (more than 5 times faster), while consuming less computational resources. We also show that the U-DeepONet is more data efficient and better at generalization than both the U-FNO and the Fourier-MIONet. [ABSTRACT FROM AUTHOR]

Learning controllers from data for stabilizing dynamical systems typically follows a two-step process of first identifying a model and then constructing a controller based on the identified model. However, learning models means identifying generic descriptions of the dynamics of systems, which can require large amounts of data and extracting information that are unnecessary for the specific task of stabilization. The contribution of this work is to show that if a linear dynamical system has dimension (McMillan degree) n , then there always exist n states from which a stabilizing feedback controller can be constructed, independent of the dimension of the representation of the observed states and the number of inputs. By building on previous work, this finding implies that any linear dynamical system can be stabilized from fewer observed states than the minimal number of states required for learning a model of the dynamics. The theoretical findings are demonstrated with numerical experiments that show the stabilization of the flow behind a cylinder from less data than necessary for learning a model. [ABSTRACT FROM AUTHOR]

This work introduces Pontryagin Neural Networks (PoNNs), a specialised subset of Physics-Informed Neural Networks (PINNs) that aim to learn optimal control actions for optimal control problems (OCPs) characterised by integral quadratic cost functions. PoNNs employ the Pontryagin Minimum Principle (PMP) to establish necessary conditions for optimality, resulting in a two-point boundary value problem (TPBVP) that involves both state and costate variables within a system of ordinary differential equations (ODEs). By modelling the unknown solutions of the TPBVP with neural networks, PoNNs effectively learn the optimal control strategies. We also derive upper bounds on the generalisation error of PoNNs in solving these OCPs, taking into account the selection and quantity of training points along with the training error. To validate our theoretical analysis, we perform numerical experiments on benchmark linear and nonlinear OCPs. The results indicate that PoNNs can successfully learn open-loop control actions for the considered class of OCPs, outperforming the commercial software GPOPS-II in terms of both accuracy and computational efficiency. The reduced computational time suggests that PoNNs hold promise for real-time applications.

We introduce a machine-learning framework named statistics-informed neural network (SINN) for learning stochastic dynamics from data. This new architecture was theoretically inspired by a universal approximation theorem for stochastic systems, which we introduce in this paper, and the projection-operator formalism for stochastic modeling. We devise mechanisms for training the neural network model to reproduce the correct statistical behavior of a target stochastic process. Numerical simulation results demonstrate that a well-trained SINN can reliably approximate both Markovian and non-Markovian stochastic dynamics. We demonstrate the applicability of SINN to coarse-graining problems and the modeling of transition dynamics. Furthermore, we show that the obtained reduced-order model can be trained on temporally coarse-grained data and hence is well suited for rare-event simulations.

Julia is a general purpose programming language that was designed for simplifying and accelerating numerical analysis and computational science. In particular the Scientific Machine Learning (SciML) ecosystem of Julia packages includes frameworks for high-performance symbolic-numeric computations. It allows users to automatically enhance high-level descriptions of their models with symbolic preprocessing and automatic sparsification and parallelization of computations. This enables performant solution of differential equations, efficient parameter estimation and methodologies for automated model discovery with neural differential equations and sparse identification of nonlinear dynamics. To give the systems biology community easy access to SciML, we developed SBMLToolkit.jl. SBMLToolkit.jl imports dynamic SBML models into the SciML ecosystem to accelerate model simulation and fitting of kinetic parameters. By providing computational systems biologists with easy access to the open-source Julia ecosystevnm, we hope to catalyze the development of further Julia tools in this domain and the growth of the Julia bioscience community. SBMLToolkit.jl is freely available under the MIT license. The source code is available at https://github.com/SciML/SBMLToolkit.jl.

Molecular dynamics (MD) simulations are used across many fields from chemical science to engineering. In recent years, Scientific Machine Learning (Sci-ML) in MD attracted significant attention and has become a new direction of scientific research. However, effectively integrating Sci-ML with MD simulations remains challenging. Compliance with the physical principles, comparable performance to a numerical method, and integration of start-of-the-art ML architectures are top-concerned examples of those gaps. This work addresses these challenges by introducing, for the first time, the neuromorphic physics-informed spiking neural network (NP-SNN) architecture to solve Newton’s equations of motion for MD systems. Unlike conventional Sci-ML methods that heavily rely on prior training data, NP-SNN performs without needing pre-existing data by embedding MD fundamentals directly into its learning process. It also leverages the enhanced representation of real biological neural systems through spiking neural network integration with molecular dynamic physical principles, offering greater efficiency compared to conventional AI algorithms. NP-SNN integrates three core components: (1) embedding MD principles directly into the training, (2) employing best practices for training physics-informed ML systems, and (3) utilizing a highly advanced and efficient SNN architecture. By integrating these core components, this proposed architecture proves its efficacy through testing across various molecular dynamics systems. In contrast to traditional MD numerical methods, NP-SNN is trained and deployed within a continuous time framework, effectively mitigating common issues related to time step stability. The results indicate that NP-SNN provides a robust Sci-ML framework that can make accurate predictions across diverse scientific molecular applications. This architecture accelerates and enhances molecular simulations, facilitating deeper insights into interactions and system dynamics at the molecular level. The proposed NP-SNN paves the way for foundational advancements across various domains of chemical and material sciences especially in energy, environment, and sustainability fields.

The recently proposed Chemical Reaction Neural Network (CRNN) discovers chemical reaction pathways from time resolved species concentration data in a deterministic manner. Since the weights and biases of a CRNN are physically interpretable, the CRNN acts as a digital twin of a classical chemical reaction network. In this study, we employ a Bayesian inference analysis coupled with neural ordinary differential equations (ODEs) on this digital twin to discover chemical reaction pathways in a probabilistic manner. This allows for estimation of the uncertainty surrounding the learned reaction network. To achieve this, we propose an algorithm which combines neural ODEs with a preconditioned stochastic gradient langevin descent (pSGLD) Bayesian framework, and ultimately performs posterior sampling on the neural network weights. We demonstrate the successful implementation of this algorithm on several reaction systems by not only recovering the chemical reaction pathways but also estimating the uncertainty in our predictions. We compare the results of the pSGLD with that of the standard SGLD and show that this optimizer more efficiently and accurately estimates the posterior of the reaction network parameters. Additionally, we demonstrate how the embedding of scientific knowledge improves extrapolation accuracy by comparing results to purely data-driven machine learning methods. Together, this provides a new framework for robust, autonomous Bayesian inference on unknown or complex chemical and biological reaction systems. [ABSTRACT FROM AUTHOR]

This study presents an innovative approach that utilizes scientific machine learning and two types of enhanced neural networks for modeling a parametric guidance algorithm within the framework of ordinary differential equations to optimize the landing phase of reusable rockets. Our approach addresses various challenges, such as reducing prediction uncertainty, minimizing the need for extensive training data, improving convergence speed, decreasing computational complexity, and enhancing prediction accuracy for unseen data. We developed two distinct enhanced neural network architectures to achieve these objectives: Adaptive (AQResNet) and Rowdy Adaptive (RAQResNet) Quadratic Residual Neural Networks. These architectures exhibited outstanding performance in our simulations. Notably, the RAQResNet model achieved a validation loss approximately 300 times lower than the standard architecture with an equal number of trainable parameters and 50 times lower than the standard architecture with twice the number of trainable parameters. Furthermore, these models require significantly less computational power, enabling real-time computation on modern flight hardware. The inference times of our proposed models were measured in approximately microseconds on a single-board computer. Additionally, we conducted an extensive Monte Carlo analysis that considers a wide range of factors, extending beyond aerodynamic uncertainty, to assess the robustness of our models. The results demonstrate the impressive adaptability of our proposed guidance policy to new conditions and distributions outside the training domain. Overall, this study makes a substantial contribution to the field of reusable rocket landing guidance and establishes a foundation for future advancements.

Adverse Drug Reactions(ADRs) due to drug-drug interactions present a public health problem worldwide that deserves attention due to its impact on mortality, morbidity, and healthcare costs. There have been major challenges in healthcare with the ever-increasing complexity of therapeutics and an aging population in many regions. At present, no standard method to detect such adverse drug reactions exists until otherwise reported by patients after the drug is released to the market. Further, several studies show that it is extremely challenging to detect these rare cases during clinical trials held before the drug is released. Therefore, a reliable and efficient technique to predict such side effects before the release of the drug to the market is the need of the hour. Through the power of Graph Neural Networks and the knowledge representation abilities of self-supervised learning, we designed an effective framework to model drug-drug interactions by leveraging the spatial and physical properties of drugs by representing them as molecular graphs. Through this approach, we developed a technique that resembles the dynamics of a chemical interaction. On training and testing this approach on the TwoSIDES Polypharmacy Dataset by Therapeutic Data Commons(TDC), we achieve a state of the art results by obtaining a precision of 75% and an accuracy of 90% on the test dataset. Further, we also perform a case study on the DrugBank dataset and compare our results on the interaction type prediction task to validate our approach on the drug-drug interaction domain and achieve excellent results with precision, F1, and accuracy of 99%. Our study and experimental approaches lay the groundwork for further research on side-effect prediction through drug-drug interaction and the use of Graph Neural Networks in the field of Molecular Biology.

Differential equations play a significant role in modeling of real world dynamical problems. A large amount of prior physical information in the form of differential equations are inherited in the dynamical systems. However, the black box machine learning models fail to express insightful scientific information from the data. Physics-informed neural networks (PINNs) can bridge the gap between scientific computing and black box models. This paper exploits a new application of PINNs for approximating the displacement of an object in mass-spring-damper systems. In this regard, we present solutions of two realistic application problems using PINNs. The accuracy of the predicted displacements of objects is established through results from literature.

Scientific machine learning and physics-informed neural networks are novel conceptual approaches that integrate scientific knowledge with methods from data science and deep learning. This emerging field has garnered increasing interest due to its unique features and potential applications. However, key topics and subject-specific applications remain under explored. To address this gap, we conducted a scientometric analysis using large-scale bibliographic and citation data from Scopus. Our study provides a global overview of publication and citation trends, explores the co-occurrence of subject areas to uncover multidisciplinary relationships, and identifies the most collaborative fields. Additionally, we employ a Latent Dirichlet Allocation (LDA) model for topic modeling, introducing a novel information-theoretic approach to determine the optimal number of topics. Furthermore, we conduct a trend analysis based on the convergence behavior of entropy and an interpretation grounded in statistical physics, revealing a consolidation process of research directions indicated as an equilibrium state. Lastly, by analyzing Dirichlet distributions from LDA, we estimate the dimension of the transdisciplinarity of scientific machine learning, offering insights into the primary coherent research areas within this field.

Developing a well-predictive machine learning model that also offers improved interpretability is a key challenge to widen the application of artificial intelligence in various application domains. In this work, we present a Data Information integrated Neural Network (DINN) algorithm that incorporates the correlation information present in the dataset for the model development. The predictive performance of DINN is also compared with a standard artificial neural network (ANN) model. The DINN algorithm is applied on two case studies of energy systems namely energy efficiency cooling (ENC) & energy efficiency heating (ENH) of the buildings, and power generation from a 365 MW capacity industrial gas turbine. For ENC, DINN presents lower mean RMSE for testing datasets (RMSE_test = 1.23 %) in comparison with the ANN model (RMSE_test = 1.41 %). Similarly, DINN models have presented better predictive performance to model the output variables of the two case studies. The input perturbation analysis following the Gaussian distribution for noise generation reveals the order of significance of the variables, as made by DINN, can be better explained by the domain knowledge of the power generation operation of the gas turbine. This research work demonstrates the potential advantage to integrate the information present in the data for the well-predictive model development complemented with improved interpretation performance thereby opening avenues for industry-wide inclusion and other potential applications of machine learning.

This review discusses Operator Inference, a nonintrusive reduced modeling approach that incorporates physical governing equations by defining a structured polynomial form for the reduced model, and then learns the corresponding reduced operators from simulated training data. The polynomial model form of Operator Inference is sufficiently expressive to cover a wide range of nonlinear dynamics found in fluid mechanics and other fields of science and engineering, while still providing efficient reduced model computations. The learning steps of Operator Inference are rooted in classical projection-based model reduction; thus, some of the rich theory of model reduction can be applied to models learned with Operator Inference. This connection to projection-based model reduction theory offers a pathway toward deriving error estimates and gaining insights to improve predictions. Furthermore, through formulations of Operator Inference that preserve Hamiltonian and other structures, important physical properties such as energy conservation can be guaranteed in the predictions of the reduced model beyond the training horizon. This review illustrates key computational steps of Operator Inference through a large-scale combustion example. [ABSTRACT FROM AUTHOR]

We propose a physics-constrained convolutional neural network (PC-CNN) to solve two types of inverse problems in partial differential equations (PDEs), which are nonlinear and vary both in space and time. In the first inverse problem, we are given data that is offset by spatially varying systematic error (i.e., the bias, also known as the epistemic uncertainty). The task is to uncover the true state, which is the solution of the PDE, from the biased data. In the second inverse problem, we are given sparse information on the solution of a PDE. The task is to reconstruct the solution in space with high resolution. First, we present the PC-CNN, which constrains the PDE with a time-windowing scheme to handle sequential data. Second, we analyze the performance of the PC-CNN to uncover solutions from biased data. We analyze both linear and nonlinear convection-diffusion equations, and the Navier–Stokes equations, which govern the spatiotemporally chaotic dynamics of turbulent flows. We find that the PC-CNN correctly recovers the true solution for a variety of biases, which are parameterized as non-convex functions. Third, we analyze the performance of the PC-CNN for reconstructing solutions from sparse information for the turbulent flow. We reconstruct the spatiotemporal chaotic solution on a high-resolution grid from only 1% of the information contained in it. For both tasks, we further analyze the Navier–Stokes solutions. We find that the inferred solutions have a physical spectral energy content, whereas traditional methods, such as interpolation, do not. This work opens opportunities for solving inverse problems with partial differential equations.

Many physical systems exhibit limit-cycle oscillations that can typically be modeled as stochastically driven self-oscillators. In this work, we focus on a self-oscillator model where the nonlinearity is on the damping term. In various applications, it is crucial to determine the nonlinear damping term and the noise intensity of the driving force. This article presents a novel approach that employs a deep operator network (DeepONet) for parameter identification of self-oscillators. We build our work upon a system identification methodology based on the adjoint Fokker–Planck formulation, which is robust to the finite sampling interval effects. We employ DeepONet as a surrogate model for the operator that maps the first Kramers–Moyal (KM) coefficient to the first and second finite-time KM coefficients. The proposed approach can directly predict the finite-time KM coefficients, eliminating the intermediate computation of the solution field of the adjoint Fokker–Planck equation. Additionally, the differentiability of the neural network readily facilitates the use of gradient-based optimizers, further accelerating the identification process. The numerical experiments demonstrate that the proposed methodology can recover desired parameters with a significant reduction in time while maintaining an accuracy comparable to that of the classical finite-difference approach. The low computational time of the forward path enables Bayesian inference of the parameters. Metropolis-adjusted Langevin algorithm is employed to obtain the posterior distribution of the parameters. The proposed method is validated against numerical simulations and experimental data obtained from a linearly unstable turbulent combustor.

Deep learning (DL) shows notable success in biomedical studies. However, most DL algorithms work as black boxes, exclude biomedical experts, and need extensive data. This is especially problematic for fundamental research in the laboratory, where often only small and sparse data are available and the objective is knowledge discovery rather than automation. Furthermore, basic research is usually hypothesis‐driven and extensive prior knowledge (priors) exists. To address this, the Self‐Enhancing Multi‐Photon Artificial Intelligence (SEMPAI) that is designed for multiphoton microscopy (MPM)‐based laboratory research is presented. It utilizes meta‐learning to optimize prior (and hypothesis) integration, data representation, and neural network architecture simultaneously. By this, the method allows hypothesis testing with DL and provides interpretable feedback about the origin of biological information in 3D images. SEMPAI performs multi‐task learning of several related tasks to enable prediction for small datasets. SEMPAI is applied on an extensive MPM database of single muscle fibers from a decade of experiments, resulting in the largest joint analysis of pathologies and function for single muscle fibers to date. It outperforms state‐of‐the‐art biomarkers in six of seven prediction tasks, including those with scarce data. SEMPAI's DL models with integrated priors are superior to those without priors and to prior‐only approaches. [ABSTRACT FROM AUTHOR]

We develop inductive biases for the machine learning of complex physical systems based on the port-Hamiltonian formalism. To satisfy by construction the principles of thermodynamics in the learned physics (conservation of energy, non-negative entropy production), we modify accordingly the port-Hamiltonian formalism so as to achieve a port-metriplectic one. We show that the constructed networks are able to learn the physics of complex systems by parts, thus alleviating the burden associated to the experimental characterization and posterior learning process of this kind of systems. Predictions can be done, however, at the scale of the complete system. Examples are shown on the performance of the proposed technique. [ABSTRACT FROM AUTHOR]

Data-driven modeling has become a key building block in computational science and engineering. However, data that are available in science and engineering are typically scarce, often polluted with noise and affected by measurement errors and other perturbations, which makes learning the dynamics of systems challenging. In this work, we propose to combine data-driven modeling via operator inference with the dynamic training via roll outs of neural ordinary differential equations. Operator inference with roll outs inherits interpretability, scalability, and structure preservation of traditional operator inference while leveraging the dynamic training via roll outs over multiple time steps to increase stability and robustness for learning from low-quality and noisy data. Numerical experiments with data describing shallow water waves and surface quasi-geostrophic dynamics demonstrate that operator inference with roll outs provides predictive models from training trajectories even if data are sampled sparsely in time and polluted with noise of up to 10%. [ABSTRACT FROM AUTHOR]

In recent years, Scientific Machine Learning (SciML) methods for solving partial differential equations (PDEs) have gained wide popularity. Within such a paradigm, Physics-Informed Neural Networks (PINNs) are novel deep learning frameworks for solving forward and inverse problems with non-linear PDEs. Recently, PINNs have shown promising results in different application domains. In this paper, we approach the groundwater flow equations numerically by searching for the unknown hydraulic head. Since singular terms in differential equations are very challenging from a numerical point of view, we approximate the Dirac distribution by different regularization terms. Furthermore, from a computational point of view, this study investigate how a PINN can solve higher-dimensional flow equations. In particular, we analyze the approximation error for one and two-dimensional cases in a statistical learning framework. The numerical experiments discussed include one and two-dimensional cases of a single or multiple pumping well in an infinite aquifer, demonstrating the effectiveness of this approach in the hydrology application domain. Lastly, we compare our results with the Finite Difference Method (FDM) to emphasize the several advantages of PINNs in solving PDEs without the need for discretization. • Our proposal is to use SciML to solve groundwater flow equations. • We recommend a regularization technique for the Dirac delta function in sink terms. • We explore the use of a PINN to handle multidimensional groundwater flow equations. • We investigate the patterns of errors in a case study involving multiple dimensions. [ABSTRACT FROM AUTHOR]

A neural-network-based machine learning interface has been developed for the Multiphysics Object-Oriented Simulation Environment (MOOSE). The interface relies on Libtorch, the C++ front-end of PyTorch, and enables an online interaction between modern machine learning algorithms and all the existing simulation, modeling, and analysis processes available in MOOSE. New capabilities in MOOSE include the native generation and training of artificial neural networks together with options to load pretrained neural networks in TorchScript format. Furthermore, the MOOSE stochastic tools module (MOOSE-STM) has been enhanced with neural network-based surrogate and reduced-order model generation options for efficient stochastic analyses. Lastly, a reinforcement learning capability has been added to MOOSE-STM for the interactive control and optimization of complex multiphysics problems.

Flavor is the focal point in the flavor industry, which follows social tendencies and behaviors. The research and development of new flavoring agents and molecules are essential in this field. However, the development of natural flavors plays a critical role in modern society. Considering this, the present work proposes a novel framework based on scientific machine learning to undertake an emerging problem in flavor engineering and industry. It proposes a combining system composed of generative and reinforcement learning models. Therefore, this work brings an innovative methodology to design new flavor molecules. The molecules were evaluated regarding synthetic accessibility, the number of atoms, and the likeness to a natural or pseudo-natural product. This work brings as contributions the implementation of a web scraper code to sample a flavors database and the integration of two scientific machine learning techniques in a complex system as a framework. The implementation of the complex system instead of the generative model by itself obtained 10% more molecules within the optimal results. The designed molecules obtained as an output of the reinforcement learning model's generation were assessed regarding their existence or not in the market and whether they are already used in the flavor industry or not. Thus, we corroborated the potentiality of the framework presented for the search of molecules to be used in the development of flavor-based products. [ABSTRACT FROM AUTHOR]

A digital twin (DT) is an automation strategy that combines a physical plant with an adaptive real-time simulation environment, where both are connected by bidirectional communication. In process engineering, DTs promise real-time monitoring, prediction of future conditions, predictive maintenance, process optimization, and control. However, the full implementation of DTs often fails in reality. To address this issue, we first examine various definitions of DTs and its core components, followed by a review of its current applications in process engineering. We then turn to the computational and numerical challenges for building the simulation environments necessary for successful DTs implementation • Explore the potential and realization of digital twins in process engineering. • Discuss computational tools for digital twins. • Link computational tools to engineering tasks. [ABSTRACT FROM AUTHOR]

The deep operator network (DeepONet) has shown remarkable potential in solving partial differential equations (PDEs) by mapping between infinite-dimensional function spaces using labeled datasets. However, in scenarios lacking labeled data, the physics-informed DeepONet (PI-DeepONet) approach, which utilizes the residual loss of the governing PDE to optimize the network parameters, faces significant computational challenges, particularly due to the curse of dimensionality. This limitation has hindered its application to high-dimensional problems, making even standard 3D spatial with 1D temporal problems computationally prohibitive. Additionally, the computational requirement increases exponentially with the discretization density of the domain. To address these challenges and enhance scalability for high-dimensional PDEs, we introduce the Separable physics-informed DeepONet (Sep-PI-DeepONet). This framework employs a factorization technique, utilizing sub-networks for individual one-dimensional coordinates, thereby reducing the number of forward passes and the size of the Jacobian matrix required for gradient computations. By incorporating forward-mode automatic differentiation (AD), we further optimize computational efficiency, achieving linear scaling of computational cost with discretization density and dimensionality, making our approach highly suitable for high-dimensional PDEs. We demonstrate the effectiveness of Sep-PI-DeepONet through three benchmark PDE models: the viscous Burgers' equation, Biot's consolidation theory, and a parameterized heat equation. Our framework maintains accuracy comparable to the conventional PI-DeepONet while reducing training time by two orders of magnitude. Notably, for the heat equation solved as a 4D problem, the conventional PI-DeepONet was computationally infeasible (estimated 289.35 h), while the Sep-PI-DeepONet completed training in just 2.5 h. These results underscore the potential of Sep-PI-DeepONet in efficiently solving complex, high-dimensional PDEs, marking a significant advancement in physics-informed machine learning. [ABSTRACT FROM AUTHOR]

Model Order Reduction (MOR) is a core technology for the creation of comprehensive executable Digital Twins, since it efficiently reduces the computational burden of high-fidelity models. When dealing with nonlinear structural Finite Element analyses, several Hyper-Reduction (HR) approaches have been developed to reduce the computational cost. Nonetheless, HR approaches are typically intrusive in nature, posing challenges when it comes to integration into existing (commercial) software. Recently, data driven Non-Intrusive MOR methodologies have been proposed. However, these techniques often suffer from overfitting and violate key physics properties, leading to unstable behavior. This work proposes to use Scientific Machine Learning to reintegrate critical stability-preserving physics properties. It introduces a data-driven, physics-augmented, parametric approach that combines Proper Orthogonal Decomposition (POD) with a Partially Input Convex Neural Network (PICNN) architecture. The proposed method effectively reduces the computational burden associated with parametric static nonlinear elastic structural problems while retaining material consistency, hyper-elasticity, and material stability properties in the Reduced Order Model. Numerical validation on several structural models subjected to geometrical and material nonlinearities under static loading conditions demonstrates the effectiveness of the POD-PICNN approach. Additionally, three different sampling strategies have been compared to assess their impact on the method performance. The results emphasize that physics-augmentation is required, as it inherently embeds essential physical constraints into the neural network architecture, ensuring stable and consistent behavior, while highlighting its potential for dynamic and multiphysics applications. • Full Order Models are numerically expensive; Model Order Reduction can mitigate. • Hyper-Reduction techniques are efficient but intrusive and typically non-parametric. • Non-Intrusive techniques are limited in extrapolation and may violate physics. • Partial Input Convex Neural Networks combine strengths of both approaches. [ABSTRACT FROM AUTHOR]

Simulating spatiotemporal turbulence with high fidelity remains a cornerstone challenge in computational fluid dynamics (CFD) due to its intricate multiscale nature and prohibitive computational demands. Traditional approaches typically employ closure models, which attempt to represent small-scale features in an unresolved manner. However, these methods often sacrifice accuracy and lose high-frequency/wavenumber information, especially in scenarios involving complex flow physics. In this paper, we introduce an innovative neural differentiable modeling framework designed to enhance the predictability and efficiency of spatiotemporal turbulence simulations. Our approach features differentiable hybrid modeling techniques that seamlessly integrate deep neural networks with numerical PDE solvers within a differentiable programming framework, synergizing deep learning with physics-based CFD modeling. Specifically, a hybrid differentiable neural solver is constructed on a coarser grid to capture large-scale turbulent phenomena, followed by the application of a Bayesian conditional diffusion model that generates small-scale turbulence conditioned on large-scale flow predictions. Two innovative hybrid architecture designs are studied, and their performance is evaluated through comparative analysis against conventional large eddy simulation techniques with physics-based subgrid-scale closures and purely data-driven neural solvers. The findings underscore the potential of the neural differentiable modeling framework to significantly enhance the accuracy and computational efficiency of turbulence simulations. This study not only demonstrates the efficacy of merging deep learning with physics-based numerical solvers but also sets a new precedent for advanced CFD modeling techniques, highlighting the transformative impact of differentiable programming in scientific computing. [ABSTRACT FROM AUTHOR]

Forecasting high-dimensional dynamical systems is a fundamental challenge in various fields, such as geosciences and engineering. Neural Ordinary Differential Equations (NODEs), which combine the power of neural networks and numerical solvers, have emerged as a promising algorithm for forecasting complex nonlinear dynamical systems. However, classical techniques used for NODE training are ineffective for learning chaotic dynamical systems. In this work, we propose a novel NODE-training approach that allows for robust learning of chaotic dynamical systems. Our method addresses the challenges of non-convexity and exploding gradients associated with underlying chaotic dynamics. Training data trajectories from such systems are split into multiple, non-overlapping time windows. In addition to the deviation from the training data, the optimization loss term further penalizes the discontinuities of the predicted trajectory between the time windows. The window size is selected based on the fastest Lyapunov time scale of the system. Multi-step penalty(MP) method is first demonstrated on Lorenz equation, to illustrate how it improves the loss landscape and thereby accelerates the optimization convergence. MP method can optimize chaotic systems in a manner similar to least-squares shadowing with significantly lower computational costs. Our proposed algorithm, denoted the Multistep Penalty NODE, is applied to chaotic systems such as the Kuramoto–Sivashinsky equation, the two-dimensional Kolmogorov flow, and ERA5 reanalysis data for the atmosphere. It is observed that MP-NODE provide viable performance for such chaotic systems, not only for short-term trajectory predictions but also for invariant statistics that are hallmarks of the chaotic nature of these dynamics. [ABSTRACT FROM AUTHOR]

Finding the ground-state structure with minimum energy is paramount to designing any material. In ABO _3 -type perovskite oxides with Pnma symmetry, the lowest energy phase is driven by an inherent trilinear coupling between the two primary order parameters such as rotation and tilt with antiferroelectric displacement of the A-site cations as established via hybrid improper ferroelectric mechanism. Conventionally, finding the relevant mode coupling driving phase transition requires performing first-principles calculations which is computationally time-consuming as well as expensive. It involves following an intuitive iterative hit and trial method of (a) adding two or multiple mode vectors, followed by (b) evaluating which combination would lead to the ground-state energy. In this study, we show how a hypothesis-driven active learning framework can identify suitable mode couplings within the Landau free energy expansion with minimal information on amplitudes of modes for a series of double perovskite oxides with A-site layered, columnar and rocksalt ordering. This scheme is expected to be applicable universally for understanding atomistic mechanisms derived from various structural mode couplings behind functionalities, for e.g. polarization, magnetization and metal–insulator transitions.