Search

Phytoplankton blooms exhibit varying patterns in timing and number of peaks within ecosystems. These differences in blooming patterns are partly explained by phytoplankton:nutrient interactions and external factors such as temperature, salinity and light availability. Understanding these interactions and drivers is essential for effective bloom management and modelling as driving factors potentially differ or are shared across ecosystems on regional scales. Here, we used a 22-year data set (19 years training and 3 years validation data) containing chlorophyll, nutrients (dissolved and total), and external drivers (temperature, salinity, light) of the southern Baltic Sea coast, a European brackish shelf sea, which constituted six different phytoplankton blooming patterns. We employed generalized additive mixed models to characterize similar blooming patterns and trained an artificial neural network within the Universal Differential Equation framework to learn a differential equation representation of these pattern. Applying Sparse Identification of Nonlinear Dynamics uncovered algebraic relationships in phytoplankton:nutrient:external driver interactions. Nutrients availability was driving factor for blooms in enclosed coastal waters; nutrients and temperature in more open regions. We found evidence of hydrodynamical export of phytoplankton, natural mortality or external grazing not explicitly measured in the data. This data-driven workflow allows new insight into driver-differences in region specific blooming dynamics. [ABSTRACT FROM AUTHOR]

In the context of hybrid twins, a data-driven enrichment is added to the physics-based solution to represent with higher accuracy the reference solution assumed to be known at different points in the physical domain. Such an approach enables better predictions. However, the data-driven enrichment is usually represented by a regression, whose main drawbacks are (i) the difficulty of understanding the subjacent physics and (ii) the risks induced by the data-driven model extrapolation. This paper proposes a procedure enabling the extraction of a differential operator associated with the enrichment provided by the data-driven regression. For that purpose, a sparse Singular Value Decomposition, SVD, is introduced. It is then employed, first, in a full operator representation regularized optimization problem, where sparsity is promoted, leading to a linear programming problem, and then in a tensor decomposition of the operator's identification procedure. The results show the ability of the method to identify the exact missing operators from the model. The regularized optimization problem was also able to identify the weights of the missing terms with a relative error of about 10% on average, depending on the selected use case. [ABSTRACT FROM AUTHOR]

We develop a novel methodology for data-driven simulation of coupled multi-physics systems. The result of the method is a learned numerical integrator of the coupled system dynamics. In order to preserve the fundamental physics of the coupled systems, and thus preserve the geometrical properties of the governing equations—even if they may be completely unknown—we impose a port-metriplectic structure on the system evolution, i.e., a combination of a symplectic evolution for the system energy with a gradient flow for the entropy of each system, which can be exchanged through predefined ports. The resulting method guarantees by construction the satisfaction of the laws of thermodynamics for open systems, leading to accurate predictions of the future states of their dynamics. Examples are given for systems of varying complexity, based on synthetic as well as experimental data. [ABSTRACT FROM AUTHOR]

Generalization is a key property of machine learning models to perform accurately on unseen data. Conversely, in the field of scientific machine learning (SciML), generalization entails not only predictive accuracy but also the capacity of the model to encapsulate underlying physical principles. In this paper, we delve into the concept of generalization for Physics-informed neural networks (PINNs) by investigating the consistency of the predictions of a PINN outside of its training domain. Through the lenses of a novel metric and statistical analysis, we study the scenarios in which a PINN can provide consistent predictions outside the region considered for training and hereinafter assess whether the algorithmic setup of the model can influence its potential for generalizing. Our results highlight why overparametrization is not a crucial component in SciML while encouraging overfitting on the training data. Despite being counterintuitive, the outcome of our analysis serves as a guideline for training PINNs for engineering applications. [ABSTRACT FROM AUTHOR]

This research introduces an accelerated training approach for Vanilla Physics-Informed Neural Networks (PINNs) that addresses three factors affecting the loss function: the initial weight state of the neural network, the ratio of domain to boundary points, and the loss weighting factor. The proposed method involves two phases. In the initial phase, a unique loss function is created using a subset of boundary conditions and partial differential equation terms. Furthermore, we introduce preprocessing procedures that aim to decrease the variance during initialization and choose domain points according to the initial weight state of various neural networks. The second phase resembles Vanilla-PINN training, but a portion of the random weights are substituted with weights from the first phase. This implies that the neural network's structure is designed to prioritize the boundary conditions, subsequently affecting the overall convergence. The study evaluates the method using three benchmarks: two-dimensional flow over a cylinder, an inverse problem of inlet velocity determination, and the Burger equation. Incorporating weights generated in the first training phase neutralizes imbalance effects. Notably, the proposed approach outperforms Vanilla-PINN in terms of speed, convergence likelihood and eliminates the need for hyperparameter tuning to balance the loss function. [ABSTRACT FROM AUTHOR]

Learning operators with deep neural networks is an emerging paradigm for scientific computing. Deep Operator Network (DeepONet) is a modular operator learning framework that allows for flexibility in choosing the kind of neural network to be used in the trunk and/or branch of the DeepONet. This is beneficial as it has been shown many times that different types of problems require different kinds of network architectures for effective learning. In this work, we design an efficient neural operator based on the DeepONet architecture. We introduce U-Net enhanced DeepONet (U-DeepONet) for learning the solution operator of highly complex CO2-water two-phase flow in heterogeneous porous media. The U-DeepONet is more accurate in predicting gas saturation and pressure buildup than the state-of-the-art U-Net based Fourier Neural Operator (U-FNO) and the Fourier-enhanced Multiple-Input Operator (Fourier-MIONet) trained on the same dataset. Moreover, our U-DeepONet is significantly more efficient in training times than both the U-FNO (more than 18 times faster) and the Fourier-MIONet (more than 5 times faster), while consuming less computational resources. We also show that the U-DeepONet is more data efficient and better at generalization than both the U-FNO and the Fourier-MIONet. [ABSTRACT FROM AUTHOR]

We propose to enhance the training of physics-informed neural networks. To this aim, we introduce nonlinear additive and multiplicative preconditioning strategies for the widely used L-BFGS optimizer. The nonlinear preconditioners are constructed by utilizing the Schwarz domain decomposition framework, where the parameters of the network are decomposed in a layer-wise manner. Through a series of numerical experiments, we demonstrate that both additive and multiplicative preconditioners significantly improve the convergence of the standard L-BFGS optimizer while providing more accurate solutions of the underlying PDEs. Moreover, the additive preconditioner is inherently parallel, thus giving rise to a novel approach to model parallelism. [ABSTRACT FROM AUTHOR]

Deep neural networks (DNNs) and, in particular, convolutional neural networks (CNNs) have brought significant advances in a wide range of modern computer application problems. However, the increasing availability of large numbers of datasets and the increasing available computational power of modern computers have led to steady growth in the complexity and size of DNN and CNN models, respectively, and thus, to longer training times. Hence, various methods and attempts have been developed to accelerate and parallelize the training of complex network architectures. In this work, a novel CNN-DNN architecture is proposed that naturally supports a model parallel training strategy and that is loosely inspired by two-level domain decomposition methods (DDMs). First, local CNN models, that is, subnetworks, are defined that operate on overlapping or nonoverlapping parts of the input data, for example, subimages. The subnetworks can be trained completely in parallel and independently of each other. Each subnetwork then outputs a local decision for the given machine learning problem which is exclusively based on the respective local input data. Subsequently, in a second step, an additional DNN model is trained which evaluates the local decisions of the local subnetworks and generates a final, global decision. With respect to the analogy to DDMs, the DNN models can be loosely interpreted as a coarse problem and hence, the new approach can be interpreted as a two-level domain decomposition. In this paper, we apply the proposed architecture to image classification problems using CNNs. Experimental results for different two-dimensional image classification problems are provided, as well as a face recognition problem and a classification problem for three-dimensional computed tomography (CT) scans. Therefore, classical Residual Network (ResNet) and VGG architectures are considered. More modern architectures, such as, e.g., MobileNet2, are left for future work. The results show that the proposed approach can significantly accelerate the required training time compared to the global model and, additionally, can also help to improve the accuracy of the underlying classification problem. [ABSTRACT FROM AUTHOR]

By learning the mappings between infinite function spaces using carefully designed neural networks, the operator learning methodology has exhibited significantly more efficiency than traditional methods in solving differential equations, but faces concerns about their accuracy and reliability. To overcome these limitations through robustly enforcing boundary conditions (BCs), a general neural architecture named spectral operator learning is introduced by combining with the structures of the spectral numerical method. One variant called the orthogonal polynomial neural operator (OPNO) is proposed later, aiming at PDEs with Dirichlet, Neumann, and Robin BCs. The strict BC satisfaction properties and the universal approximation capacity of the OPNO are theoretically proven. A variety of numerical experiments with physical backgrounds demonstrate that the OPNO outperforms other existing deep learning methodologies, showcasing potential of comparable accuracy with the traditional second-order finite difference method that employs a considerably fine mesh (with relative errors on the order of 10–6 ), and is up to almost 5 magnitudes faster over the traditional method. [ABSTRACT FROM AUTHOR]

Supervised operator learning centers on the use of training data, in the form of inputoutput pairs, to estimate maps between infinite-dimensional spaces. It is emerging as a powerful tool to complement traditional scientific computing, which may often be framed in terms of operators mapping between spaces of functions. Building on the classical random features methodology for scalar regression, this paper introduces the function-valued random features method. This leads to a supervised operator learning architecture that is practical for nonlinear problems yet is structured enough to facilitate efficient training through the optimization of a convex, quadratic cost. Due to the quadratic structure, the trained model is equipped with convergence guarantees and error and complexity bounds, properties that are not readily available for most other operator learning architectures. At its core, the proposed approach builds a linear combination of random operators. This turns out to be a low-rank approximation of an operator-valued kernel ridge regression algorithm, and hence the method also has strong connections to Gaussian process regression. The paper designs function-valued random features that are tailored to the structure of two nonlinear operator learning benchmark problems arising from parametric partial differential equations. Numerical results demonstrate the scalability, discretization invariance, and transferability of the function-valued random features method. [ABSTRACT FROM AUTHOR]

Learning controllers from data for stabilizing dynamical systems typically follows a two-step process of first identifying a model and then constructing a controller based on the identified model. However, learning models means identifying generic descriptions of the dynamics of systems, which can require large amounts of data and extracting information that are unnecessary for the specific task of stabilization. The contribution of this work is to show that if a linear dynamical system has dimension (McMillan degree) n , then there always exist n states from which a stabilizing feedback controller can be constructed, independent of the dimension of the representation of the observed states and the number of inputs. By building on previous work, this finding implies that any linear dynamical system can be stabilized from fewer observed states than the minimal number of states required for learning a model of the dynamics. The theoretical findings are demonstrated with numerical experiments that show the stabilization of the flow behind a cylinder from less data than necessary for learning a model. [ABSTRACT FROM AUTHOR]

The recently proposed Chemical Reaction Neural Network (CRNN) discovers chemical reaction pathways from time resolved species concentration data in a deterministic manner. Since the weights and biases of a CRNN are physically interpretable, the CRNN acts as a digital twin of a classical chemical reaction network. In this study, we employ a Bayesian inference analysis coupled with neural ordinary differential equations (ODEs) on this digital twin to discover chemical reaction pathways in a probabilistic manner. This allows for estimation of the uncertainty surrounding the learned reaction network. To achieve this, we propose an algorithm which combines neural ODEs with a preconditioned stochastic gradient langevin descent (pSGLD) Bayesian framework, and ultimately performs posterior sampling on the neural network weights. We demonstrate the successful implementation of this algorithm on several reaction systems by not only recovering the chemical reaction pathways but also estimating the uncertainty in our predictions. We compare the results of the pSGLD with that of the standard SGLD and show that this optimizer more efficiently and accurately estimates the posterior of the reaction network parameters. Additionally, we demonstrate how the embedding of scientific knowledge improves extrapolation accuracy by comparing results to purely data-driven machine learning methods. Together, this provides a new framework for robust, autonomous Bayesian inference on unknown or complex chemical and biological reaction systems. [ABSTRACT FROM AUTHOR]

Uncertainty quantification (UQ) in machine learning is currently drawing increasing research interest, driven by the rapid deployment of deep neural networks across different fields, such as computer vision and natural language processing, and by the need for reliable tools in risk-sensitive applications. Recently, various machine learning models have also been developed to tackle problems in the field of scientific computing with applications to computational science and engineering (CSE). Physics-informed neural networks and deep operator networks are two such models for solving partial differential equations (PDEs) and learning operator mappings, respectively. In this regard, a comprehensive study of UQ methods tailored specifically for scientific machine learning (SciML) models has been provided in [A. F. Psaros et al., J. Comput. Phys., 477 (2023), art. 111902]. Nevertheless, and despite their theoretical merit, implementations of these methods are not straightforward, especially in large-scale CSE applications, hindering their broad adoption in both research and industry settings. In this paper, we present an open-source Python library (https://github.com/Crunch-UQ4MI), termed NeuralUQ and accompanied by an educational tutorial, for employing UQ methods for SciML in a convenient and structured manner. The library, designed for both educational and research purposes, supports multiple modern UQ methods and SciML models. It is based on a succinct workflow and facilitates flexible employment and easy extensions by the users. We first present a tutorial of NeuralUQ and subsequently demonstrate its applicability and efficiency in four diverse examples, involving dynamical systems and high-dimensional parametric and time-dependent PDEs. [ABSTRACT FROM AUTHOR]

This review discusses Operator Inference, a nonintrusive reduced modeling approach that incorporates physical governing equations by defining a structured polynomial form for the reduced model, and then learns the corresponding reduced operators from simulated training data. The polynomial model form of Operator Inference is sufficiently expressive to cover a wide range of nonlinear dynamics found in fluid mechanics and other fields of science and engineering, while still providing efficient reduced model computations. The learning steps of Operator Inference are rooted in classical projection-based model reduction; thus, some of the rich theory of model reduction can be applied to models learned with Operator Inference. This connection to projection-based model reduction theory offers a pathway toward deriving error estimates and gaining insights to improve predictions. Furthermore, through formulations of Operator Inference that preserve Hamiltonian and other structures, important physical properties such as energy conservation can be guaranteed in the predictions of the reduced model beyond the training horizon. This review illustrates key computational steps of Operator Inference through a large-scale combustion example. [ABSTRACT FROM AUTHOR]

Deep learning (DL) shows notable success in biomedical studies. However, most DL algorithms work as black boxes, exclude biomedical experts, and need extensive data. This is especially problematic for fundamental research in the laboratory, where often only small and sparse data are available and the objective is knowledge discovery rather than automation. Furthermore, basic research is usually hypothesis‐driven and extensive prior knowledge (priors) exists. To address this, the Self‐Enhancing Multi‐Photon Artificial Intelligence (SEMPAI) that is designed for multiphoton microscopy (MPM)‐based laboratory research is presented. It utilizes meta‐learning to optimize prior (and hypothesis) integration, data representation, and neural network architecture simultaneously. By this, the method allows hypothesis testing with DL and provides interpretable feedback about the origin of biological information in 3D images. SEMPAI performs multi‐task learning of several related tasks to enable prediction for small datasets. SEMPAI is applied on an extensive MPM database of single muscle fibers from a decade of experiments, resulting in the largest joint analysis of pathologies and function for single muscle fibers to date. It outperforms state‐of‐the‐art biomarkers in six of seven prediction tasks, including those with scarce data. SEMPAI's DL models with integrated priors are superior to those without priors and to prior‐only approaches. [ABSTRACT FROM AUTHOR]

We develop inductive biases for the machine learning of complex physical systems based on the port-Hamiltonian formalism. To satisfy by construction the principles of thermodynamics in the learned physics (conservation of energy, non-negative entropy production), we modify accordingly the port-Hamiltonian formalism so as to achieve a port-metriplectic one. We show that the constructed networks are able to learn the physics of complex systems by parts, thus alleviating the burden associated to the experimental characterization and posterior learning process of this kind of systems. Predictions can be done, however, at the scale of the complete system. Examples are shown on the performance of the proposed technique. [ABSTRACT FROM AUTHOR]

Data-driven modeling has become a key building block in computational science and engineering. However, data that are available in science and engineering are typically scarce, often polluted with noise and affected by measurement errors and other perturbations, which makes learning the dynamics of systems challenging. In this work, we propose to combine data-driven modeling via operator inference with the dynamic training via roll outs of neural ordinary differential equations. Operator inference with roll outs inherits interpretability, scalability, and structure preservation of traditional operator inference while leveraging the dynamic training via roll outs over multiple time steps to increase stability and robustness for learning from low-quality and noisy data. Numerical experiments with data describing shallow water waves and surface quasi-geostrophic dynamics demonstrate that operator inference with roll outs provides predictive models from training trajectories even if data are sampled sparsely in time and polluted with noise of up to 10%. [ABSTRACT FROM AUTHOR]

In recent years, Scientific Machine Learning (SciML) methods for solving partial differential equations (PDEs) have gained wide popularity. Within such a paradigm, Physics-Informed Neural Networks (PINNs) are novel deep learning frameworks for solving forward and inverse problems with non-linear PDEs. Recently, PINNs have shown promising results in different application domains. In this paper, we approach the groundwater flow equations numerically by searching for the unknown hydraulic head. Since singular terms in differential equations are very challenging from a numerical point of view, we approximate the Dirac distribution by different regularization terms. Furthermore, from a computational point of view, this study investigate how a PINN can solve higher-dimensional flow equations. In particular, we analyze the approximation error for one and two-dimensional cases in a statistical learning framework. The numerical experiments discussed include one and two-dimensional cases of a single or multiple pumping well in an infinite aquifer, demonstrating the effectiveness of this approach in the hydrology application domain. Lastly, we compare our results with the Finite Difference Method (FDM) to emphasize the several advantages of PINNs in solving PDEs without the need for discretization. • Our proposal is to use SciML to solve groundwater flow equations. • We recommend a regularization technique for the Dirac delta function in sink terms. • We explore the use of a PINN to handle multidimensional groundwater flow equations. • We investigate the patterns of errors in a case study involving multiple dimensions. [ABSTRACT FROM AUTHOR]

Noise poses a challenge for learning dynamical-system models because already small variations can distort the dynamics described by trajectory data. This work builds on operator inference from scientific machine learning to infer low-dimensional models from high-dimensional state trajectories polluted with noise. The presented analysis shows that, under certain conditions, the inferred operators are unbiased estimators of the well-studied projection-based reduced operators from traditional model reduction. Furthermore, the connection between operator inference and projection-based model reduction enables bounding the mean-squared errors of predictions made with the learned models with respect to traditional reduced models. The analysis also motivates an active operator inference approach that judiciously samples high-dimensional trajectories with the aim of achieving a low mean-squared error by reducing the effect of noise. Numerical experiments with high-dimensional linear and nonlinear state dynamics demonstrate that predictions obtained with active operator inference have orders of magnitude lower mean-squared errors than operator inference with traditional, equidistantly sampled trajectory data. [ABSTRACT FROM AUTHOR]

This work formulates a new approach to reduced modeling of parameterized, timedependent partial differential equations (PDEs). The method employs Operator Inference, a scientific machine learning framework combining data-driven learning and physics-based modeling. The parametric structure of the governing equations is embedded directly into the reduced-order model, and parameterized reduced-order operators are learned via a data-driven linear regression problem. The result is a reduced-order model that can be solved rapidly to map parameter values to approximate PDE solutions. Such parameterized reduced-order models may be used as physics-based surrogates for uncertainty quantification and inverse problems that require many forward solves of parametric PDEs. Numerical issues such as well-posedness and the need for appropriate regularization in the learning problem are considered, and an algorithm for hyperparameter selection is presented. The method is illustrated for a parametric heat equation and demonstrated for the FitzHugh-Nagumo neuron model. [ABSTRACT FROM AUTHOR]

This study help investigates different Scientific Machine Learning (SciML) approaches for the analysis of functionally graded (FG) porous beams and compares them under a new framework. The beam material properties are assumed to vary as an arbitrary continuous function. The methods consider the output of a neural network/operator as an approximation to the displacement fields and derive the equations governing beam behavior based on the continuum formulation. They are implemented in the framework and formulated by three approaches: (a) the vector approach leads to a Physics-Informed Neural Network (PINN), (b) the energy approach brings about the Deep Energy Method (DEM), and (c) the data-driven approach, which results in a class of Neural Operator methods. Finally, a neural operator has been trained to predict the response of the porous beam with functionally graded material under any porosity distribution pattern and any arbitrary traction condition. The results are validated with analytical and numerical reference solutions. The data and code accompanying this manuscript will be publicly available at https://github.com/eshaghi-ms/DeepNetBeam. • We develop a framework based on machine learning for the study of functionally graded (FG) porous beams. • Several formulations are considered: residual-based approach, energy approach, and the data-driven approach. • The models can predict the displacements of FG beams under any porosity distribution and traction conditions. • The framework speeds-up the analysis of FG beams with fewer assumptions than traditional beam models. • Validation examples show good accuracy when the model outputs are compared to analytical and numerical solutions. [ABSTRACT FROM AUTHOR]

Data-driven modelling and scientific machine learning have been responsible for significant advances in determining suitable models to describe data. Within dynamical systems, neural ordinary differential equations (ODEs), where the system equations are set to be governed by a neural network, have become a popular tool for this challenge in recent years. However, less emphasis has been placed on systems that are only partially-observed. In this work, we employ a hybrid neural ODE structure, where the system equations are governed by a combination of a neural network and domain-specific knowledge, together with symbolic regression (SR), to learn governing equations of partially-observed dynamical systems. We test this approach on two case studies: A 3-dimensional model of the Lotka–Volterra system and a 5-dimensional model of the Lorenz system. We demonstrate that the method is capable of successfully learning the true underlying governing equations of unobserved states within these systems, with robustness to measurement noise. • A hybrid neural ODE structure together with symbolic regression is used to recover governing equations of unobserved state variables. • Ensembles of the framework are employed to increase robustness to measurement noise. • Applicability is demonstrated in ecological (Lotka–Volterra) and chaotic (Lorenz) systems. • The framework is adapted to handle additional system uncertainty and to quantify confidence in the learned models. [ABSTRACT FROM AUTHOR]

This study employs scientific machine learning to identify transient time series of dynamical systems near a fold bifurcation of periodic solutions. The unique aspect of this work is that a convolutional neural network (CNN) is trained with a relatively small amount of data and on a single, very simple system, yet it is tested on much more complicated systems. This task requires strong generalization capabilities, which are achieved by incorporating physics-based information. This information is provided through a specific pre-processing of the input data, which includes transformation into polar coordinates, normalization, transformation into the logarithmic scale, and filtering through a moving mean. The results demonstrate that such data pre-processing enables the CNN to grasp the important features related to transient time-series near a fold bifurcation, namely, the trend of the oscillation amplitude, and disregard other characteristics that are not particularly relevant, such as the vibration frequency. The developed CNN was able to correctly classify transient trajectories near a fold for a mass-on-moving-belt system, a van der Pol-Duffing oscillator with an attached tuned mass damper, and a pitch-and-plunge wing profile. The results contribute to the progress towards the development of similar CNNs effective in real-life applications such as safety monitoring of dynamical systems. [Display omitted] • A CNN for identifying trajectories close to a fold bifurcation is developed. • Physics-based data pre-processing enables CNN to capture key features. • Trained on a simple system and tested on complex ones with strong generalization. • Successfully classifies transient trajectories in various mechanical systems. • The CNN is robust to noise. [ABSTRACT FROM AUTHOR]

A digital twin (DT) is an automation strategy that combines a physical plant with an adaptive real-time simulation environment, where both are connected by bidirectional communication. In process engineering, DTs promise real-time monitoring, prediction of future conditions, predictive maintenance, process optimization, and control. However, the full implementation of DTs often fails in reality. To address this issue, we first examine various definitions of DTs and its core components, followed by a review of its current applications in process engineering. We then turn to the computational and numerical challenges for building the simulation environments necessary for successful DTs implementation • Explore the potential and realization of digital twins in process engineering. • Discuss computational tools for digital twins. • Link computational tools to engineering tasks. [ABSTRACT FROM AUTHOR]

The deep operator network (DeepONet) has shown remarkable potential in solving partial differential equations (PDEs) by mapping between infinite-dimensional function spaces using labeled datasets. However, in scenarios lacking labeled data, the physics-informed DeepONet (PI-DeepONet) approach, which utilizes the residual loss of the governing PDE to optimize the network parameters, faces significant computational challenges, particularly due to the curse of dimensionality. This limitation has hindered its application to high-dimensional problems, making even standard 3D spatial with 1D temporal problems computationally prohibitive. Additionally, the computational requirement increases exponentially with the discretization density of the domain. To address these challenges and enhance scalability for high-dimensional PDEs, we introduce the Separable physics-informed DeepONet (Sep-PI-DeepONet). This framework employs a factorization technique, utilizing sub-networks for individual one-dimensional coordinates, thereby reducing the number of forward passes and the size of the Jacobian matrix required for gradient computations. By incorporating forward-mode automatic differentiation (AD), we further optimize computational efficiency, achieving linear scaling of computational cost with discretization density and dimensionality, making our approach highly suitable for high-dimensional PDEs. We demonstrate the effectiveness of Sep-PI-DeepONet through three benchmark PDE models: the viscous Burgers' equation, Biot's consolidation theory, and a parameterized heat equation. Our framework maintains accuracy comparable to the conventional PI-DeepONet while reducing training time by two orders of magnitude. Notably, for the heat equation solved as a 4D problem, the conventional PI-DeepONet was computationally infeasible (estimated 289.35 h), while the Sep-PI-DeepONet completed training in just 2.5 h. These results underscore the potential of Sep-PI-DeepONet in efficiently solving complex, high-dimensional PDEs, marking a significant advancement in physics-informed machine learning. [ABSTRACT FROM AUTHOR]

Model Order Reduction (MOR) is a core technology for the creation of comprehensive executable Digital Twins, since it efficiently reduces the computational burden of high-fidelity models. When dealing with nonlinear structural Finite Element analyses, several Hyper-Reduction (HR) approaches have been developed to reduce the computational cost. Nonetheless, HR approaches are typically intrusive in nature, posing challenges when it comes to integration into existing (commercial) software. Recently, data driven Non-Intrusive MOR methodologies have been proposed. However, these techniques often suffer from overfitting and violate key physics properties, leading to unstable behavior. This work proposes to use Scientific Machine Learning to reintegrate critical stability-preserving physics properties. It introduces a data-driven, physics-augmented, parametric approach that combines Proper Orthogonal Decomposition (POD) with a Partially Input Convex Neural Network (PICNN) architecture. The proposed method effectively reduces the computational burden associated with parametric static nonlinear elastic structural problems while retaining material consistency, hyper-elasticity, and material stability properties in the Reduced Order Model. Numerical validation on several structural models subjected to geometrical and material nonlinearities under static loading conditions demonstrates the effectiveness of the POD-PICNN approach. Additionally, three different sampling strategies have been compared to assess their impact on the method performance. The results emphasize that physics-augmentation is required, as it inherently embeds essential physical constraints into the neural network architecture, ensuring stable and consistent behavior, while highlighting its potential for dynamic and multiphysics applications. • Full Order Models are numerically expensive; Model Order Reduction can mitigate. • Hyper-Reduction techniques are efficient but intrusive and typically non-parametric. • Non-Intrusive techniques are limited in extrapolation and may violate physics. • Partial Input Convex Neural Networks combine strengths of both approaches. [ABSTRACT FROM AUTHOR]

In this paper we present the Physics-Aware Deep-NMF (PAD-NMF) algorithm and we apply it to hit detection in mechanical systems from audio mixtures. The algorithm accurately extracts the acoustic emission made by the physical source to be monitored, to be properly used in engineering analysis, like e.g. the hit detection here considered. This is mainly a source separation problem where sources have a precise physical meaning, that should be retained by the processing algorithm. For this reason we call this algorithm a Physics-aware soft-sensor. We give a detailed description of the algorithm and show its results on a general application with critical signal-to-noise ratios, where the noise is a mixture of random and deterministic acoustic sources. [ABSTRACT FROM AUTHOR]

When solving Hamiltonian systems using numerical integrators, preserving the symplectic structure may be crucial for many problems. At the same time, solving chaotic or stiff problems requires integrators to approximate the trajectories with extreme precision. So, integrating Hamilton's equations to a level of scientific reliability such that the answer can be used for scientific interpretation, may be computationally expensive. However, a neural network can be a viable alternative to numerical integrators, offering high-fidelity solutions orders of magnitudes faster. To understand whether it is also important to preserve the symplecticity when neural networks are used, we analyze three well-known neural network architectures that are including the symplectic structure inside the neural network's topology. Between these neural network architectures many similarities can be found. This allows us to formulate a new, generalized framework for these architectures. In the generalized framework Symplectic Recurrent Neural Networks, SympNets and HénonNets are included as special cases. Additionally, this new framework enables us to find novel neural network topologies by transitioning between the established ones. We compare new Generalized Hamiltonian Neural Networks (GHNNs) against the already established SympNets, HénonNets and physics-unaware multilayer perceptrons. This comparison is performed with data for a pendulum, a double pendulum and a gravitational 3-body problem. In order to achieve a fair comparison, the hyperparameters of the different neural networks are chosen such that the prediction speeds of all four architectures are the same during inference. A special focus lies on the capability of the neural networks to generalize outside the training data. The GHNNs outperform all other neural network architectures for the problems considered. • Three different neural network architectures for Hamiltonian systems can be united. • Comparison of neural networks with similar computational costs is important. • A newly introduced neural network architecture can outperform all others. • Not only the number of trainable parameters is important, but also their structure. • Structure-preservation greatly improves the generalization outside of training data. [ABSTRACT FROM AUTHOR]

Simulating spatiotemporal turbulence with high fidelity remains a cornerstone challenge in computational fluid dynamics (CFD) due to its intricate multiscale nature and prohibitive computational demands. Traditional approaches typically employ closure models, which attempt to represent small-scale features in an unresolved manner. However, these methods often sacrifice accuracy and lose high-frequency/wavenumber information, especially in scenarios involving complex flow physics. In this paper, we introduce an innovative neural differentiable modeling framework designed to enhance the predictability and efficiency of spatiotemporal turbulence simulations. Our approach features differentiable hybrid modeling techniques that seamlessly integrate deep neural networks with numerical PDE solvers within a differentiable programming framework, synergizing deep learning with physics-based CFD modeling. Specifically, a hybrid differentiable neural solver is constructed on a coarser grid to capture large-scale turbulent phenomena, followed by the application of a Bayesian conditional diffusion model that generates small-scale turbulence conditioned on large-scale flow predictions. Two innovative hybrid architecture designs are studied, and their performance is evaluated through comparative analysis against conventional large eddy simulation techniques with physics-based subgrid-scale closures and purely data-driven neural solvers. The findings underscore the potential of the neural differentiable modeling framework to significantly enhance the accuracy and computational efficiency of turbulence simulations. This study not only demonstrates the efficacy of merging deep learning with physics-based numerical solvers but also sets a new precedent for advanced CFD modeling techniques, highlighting the transformative impact of differentiable programming in scientific computing. [ABSTRACT FROM AUTHOR]

We propose physics-informed holomorphic neural networks (PIHNNs) as a method to solve boundary value problems where the solution can be represented via holomorphic functions. Specifically, we consider the case of plane linear elasticity and, by leveraging the Kolosov–Muskhelishvili representation of the solution in terms of holomorphic potentials, we train a complex-valued neural network to fulfill stress and displacement boundary conditions while automatically satisfying the governing equations. This is achieved by designing the network to return only approximations that inherently satisfy the Cauchy-Riemann conditions through specific choices of layers and activation functions. To ensure generality, we provide a universal approximation theorem guaranteeing that, under basic assumptions, the proposed holomorphic neural networks can approximate any holomorphic function. Furthermore, we suggest a new tailored weight initialization technique to mitigate the issue of vanishing/exploding gradients. Compared to the standard PINN approach, noteworthy benefits of the proposed method for the linear elasticity problem include a more efficient training, as evaluations are needed solely on the boundary of the domain, lower memory requirements, due to the reduced number of training points, and C ∞ regularity of the learned solution. Several benchmark examples are used to verify the correctness of the obtained PIHNN approximations, the substantial benefits over traditional PINNs, and the possibility to deal with non-trivial, multiply-connected geometries via a domain-decomposition strategy. [ABSTRACT FROM AUTHOR]

Recently, incremental learning (IL) has been widely used in intelligent fault diagnosis of electronic machinery. Most of the typical IL methods have adopted the exemplar-replay strategy to retain the learned diagnostic knowledge. However, it is almost impossible to have infinite storage space to retain fault samples in practical industrial scenarios, which brings a significant challenge for actual industrial applications. To solve this issue, a novel Relationship-Aware Calibrated Prototypical Network (RACPN) is proposed for incremental fault diagnosis of electric motors, which retains learned diagnostic knowledge without requiring the storage of any fault samples from previous sessions. Firstly, a fault prototype calibration (FPC) method is employed to learn new diagnostic knowledge from new sessions. Secondly, a task-relationship representation (TRR), which stands for a method to represent the relationships between tasks, is utilized to enhance the maintenance of diagnostic knowledge across different sessions. Finally, a Gaussian Bayes classifier with Mahalanobis metric is adopted to enhance the inference reliability for classifying fault categories. Experiments conducted on two electrical motor datasets demonstrate the superiority and effectiveness of the proposed RACPN. The results validate that current signals as model input can achieve satisfactory diagnostic performance. The proposed RACPN is a promising tool for incremental fault diagnosis in electric motors. [ABSTRACT FROM AUTHOR]

Approximating nonlinear differential equations using a neural network provides a robust and efficient tool for various scientific computing tasks, including real-time predictions, inverse problems, optimal controls, and surrogate modeling. Previous works have focused on embedding dynamical systems into networks through two approaches: learning a single operator (i.e., the mapping from input parameterised functions to solutions) or learning the governing system of equations (i.e., the constitutive model relative to the state variables). Both of these approaches yield different representations for the same underlying data or function. Observing that families of differential equations often share key characteristics, we seek one network representation across a wide range of equations. Our multimodality approach, called Pr edicting Multiple O perators and S ymbolic E xpressions (PROSE), is capable of constructing multi-operators and governing equations simultaneously through a novel fusion structure. In particular, PROSE solves differential equations, predicts future states, and generates the underlying equations of motion by incorporating symbolic "words" through a language model. Experiments with 25600 distinct equations show that PROSE benefits from its multimodal nature, resulting in robust generalization (e.g. noisy observations, equation misspecification, and data imbalance) supported by comparison and ablation studies. PROSE provides a new operator learning framework that incorporates multimodal input/output and language models for solving forward and inverse problems related to differential equations. • PROSE is the first multimodal scientific foundation model for zero-shot prediction. • PROSE solves multi-operator learning challenges and learns the equations simultaneously. • PROSE adopts trainable symbols to encode the operator. • We show that PROSE can extrapolate to equations unseen during training. [ABSTRACT FROM AUTHOR]

Forecasting high-dimensional dynamical systems is a fundamental challenge in various fields, such as geosciences and engineering. Neural Ordinary Differential Equations (NODEs), which combine the power of neural networks and numerical solvers, have emerged as a promising algorithm for forecasting complex nonlinear dynamical systems. However, classical techniques used for NODE training are ineffective for learning chaotic dynamical systems. In this work, we propose a novel NODE-training approach that allows for robust learning of chaotic dynamical systems. Our method addresses the challenges of non-convexity and exploding gradients associated with underlying chaotic dynamics. Training data trajectories from such systems are split into multiple, non-overlapping time windows. In addition to the deviation from the training data, the optimization loss term further penalizes the discontinuities of the predicted trajectory between the time windows. The window size is selected based on the fastest Lyapunov time scale of the system. Multi-step penalty(MP) method is first demonstrated on Lorenz equation, to illustrate how it improves the loss landscape and thereby accelerates the optimization convergence. MP method can optimize chaotic systems in a manner similar to least-squares shadowing with significantly lower computational costs. Our proposed algorithm, denoted the Multistep Penalty NODE, is applied to chaotic systems such as the Kuramoto–Sivashinsky equation, the two-dimensional Kolmogorov flow, and ERA5 reanalysis data for the atmosphere. It is observed that MP-NODE provide viable performance for such chaotic systems, not only for short-term trajectory predictions but also for invariant statistics that are hallmarks of the chaotic nature of these dynamics. [ABSTRACT FROM AUTHOR]

Physics-informed neural networks (PINN) have gradually attracted attention in the field of geotechnical engineering. This paper proposes a novel PINN-based framework for the three-dimensional (3D) stability analysis of soil slopes. Based on the fundamental theorem of plasticity and limit analysis, the partial differential equations (PDE) with regard to slope collapse are derived and integrated into the physics-guided loss function. A kinematically admissible failure mechanism that rigorously satisfies the Mohr-Coulomb associated flow rule is obtained by minimizing the loss function, thereby circumventing complex mathematical calculations. The discrete points generated by PINN are selected and refined through a series of procedures to represent the failure block of slopes using a two-dimensional matrix. The entire training process of the PINN-based framework is conducted without the need for any labeled data. The resulting discretized failure mechanism is meshfree and capable of accommodating spatially discrete data. A validation exercise is performed to verify the proposed framework by comparing it with the classical 3D rotational failure mechanism. To further consider the impact of external excitation on slope stability, a hybrid PINN framework is developed to assess the stability of slopes subjected to complex external environments. In addition to the PINN to generate a failure mechanism, a parallel PINN is employed to acquire the corresponding spatially discrete data of specified external excitations. The hybrid PINN framework for seismic stability assessment of slopes is demonstrated by way of example, indicating favorable feasibility and applicability of the developed approach. The proposed PINN-based framework provides innovative and promising avenues for 3D slope stability analysis. [ABSTRACT FROM AUTHOR]

Robust learning is an important issue in Scientific Machine Learning (SciML). There are several works in the literature addressing this topic. However, there is an increasing demand for methods that can simultaneously consider all the different uncertainty components involved in SciML model identification. Hence, this work proposes a comprehensive methodology for uncertainty evaluation of the SciML that also considers several possible sources of uncertainties involved in the identification process. The uncertainties considered in the proposed method are the absence of a theory, causal models, sensitivity to data corruption or imperfection, and computational effort. Therefore, it is possible to provide an overall strategy for uncertainty-aware models in the SciML field. The methodology is validated through a case study developing a soft sensor for a polymerization reactor. The first step is to build the nonlinear model parameter probability distribution (PDF) by Bayesian inference. The second step is to obtain the machine learning model uncertainty by Monte Carlo simulations. In the first step, a PDF with 30,000 samples is built. In the second step, the uncertainty of the machine learning model is evaluated by sampling 10,000 values through Monte Carlo simulation. The results demonstrate that the identified soft sensors are robust to uncertainties, corroborating the consistency of the proposed approach. [ABSTRACT FROM AUTHOR]

• A PINN methodology for a Bernoulli free boundary variational problem is proposed. • Quasi Monte-Carlo integration methods are used in the learning process of the PINN. • A redistribution strategy of domain points based on solution's Laplacian is developed. • Analysis on convex and non-convex domains assess the PINN reliability. Bernoulli free boundary problems (BFBP) govern many real applications, from chemistry to fluid dynamics. BFBP are overdetermined differential models in which the boundary of the solution's domain appears as an unknown. This work provides a variational formulation for BFBP, and a computational method focused on the novel methodology of Physics Informed Neural Networks (PINNs) is proposed. It consists in training a neural network to approximate the solution of the differential problem, minimizing a suitable cost function built, taking into account the physics constraint given by the model. In particular, since physics laws are injected through a loss function containing integral terms in our case, an approach based on quasi Monte-Carlo integration methods has been implemented. Moreover, an adaptive strategy to increase the model accuracy, exploiting topological information related to the shape of the solution, has been developed. Finally, when available, comparisons with the analytical solutions and studies in the case of non-convex domains assess the reliability of the PINN approach in the examined context. [ABSTRACT FROM AUTHOR]

As an emerging paradigm in scientific machine learning, neural operators aim to learn operators, via neural networks, that map between infinite-dimensional function spaces. Several neural operators have been recently developed. However, all the existing neural operators are only designed to learn operators defined on a single Banach space; i.e., the input of the operator is a single function. Here, for the first time, we study the operator regression via neural networks for multiple input operators defined on the product of Banach spaces. We first prove a universal approximation theorem of continuous multiple-input operators. We also provide a detailed theoretical analysis including the approximation error, which provides guidance for the design of the network architecture. Based on our theory and a low-rank approximation, we propose a novel neural operator, MIONet, to learn multiple-input operators. MIONet consists of several branch nets for encoding the input functions and a trunk net for encoding the domain of the output function. We demonstrate that MIONet can learn solution operators involving systems governed by ordinary and partial differential equations. In our computational examples, we also show that we can endow MIONet with prior knowledge of the underlying system, such as linearity and periodicity, to further improve accuracy. [ABSTRACT FROM AUTHOR]

• A symplectic machine learning framework is proposed for multibody dynamic analysis. • Symplectic properties of the proposed machine learning framework are proofed. • High accuracy and efficiency of the proposed strategy are validated. Due to the complexity and nonlinearity of mechanical system, traditional dynamic analysis methods are often struggle to balance computational efficiency and accuracy. In order to overcome the challenges existing in the current numerical methods, this paper proposes a novel mechanism-data hybrid-driven method for dynamic analysis of mechanical system via symplectic mathematical theory, machine learning theory and multibody system theory. The proposed method introduces the variational principle into neural network to establish a symplectic machine learning framework, which leverages the high precision advantages of symplectic theory and the efficient and strong generalization ability of neural network, thereby achieving efficient and high precision dynamic prediction of constrained mechanical system. And the characteristic of symplectic conservation of the proposed strategy is proved both in mathematical and numerical perspectives. In addition, three numerical examples are studied, the comparison results indicate that the proposed strategy can perform outstanding advantages in terms of numerical accuracy and computational efficiency, and the proposed method can also predict high precision results without the need for retraining when certain parameters change of the mechanical system. [ABSTRACT FROM AUTHOR]

Multi-task learning (MTL) is an inductive transfer mechanism designed to leverage useful information from multiple tasks to improve generalization performance compared to single-task learning. It has been extensively explored in traditional machine learning to address issues such as data sparsity and overfitting in neural networks. In this work, we apply MTL to problems in science and engineering governed by partial differential equations (PDEs). However, implementing MTL in this context is complex, as it requires task-specific modifications to accommodate various scenarios representing different physical processes. To this end, we present a multi-task deep operator network (MT-DeepONet) to learn solutions across various functional forms of source terms in a PDE and multiple geometries in a single concurrent training session. We introduce modifications in the branch network of the vanilla DeepONet to account for various functional forms of a parameterized coefficient in a PDE. Additionally, we handle parameterized geometries by introducing a binary mask in the branch network and incorporating it into the loss term to improve convergence and generalization to new geometry tasks. Our approach is demonstrated on three benchmark problems: (1) learning different functional forms of the source term in the Fisher equation; (2) learning multiple geometries in a 2D Darcy Flow problem and showcasing better transfer learning capabilities to new geometries; and (3) learning 3D parameterized geometries for a heat transfer problem and demonstrate the ability to predict on new but similar geometries. Our MT-DeepONet framework offers a novel approach to solving PDE problems in engineering and science under a unified umbrella based on synergistic learning that reduces the overall training cost for neural operators. [ABSTRACT FROM AUTHOR]

Thermodynamics-informed neural networks employ inductive biases for the enforcement of the first and second principles of thermodynamics. To construct these biases, a metriplectic evolution of the physical system under study is assumed. This provides excellent results, when compared to uninformed, black box networks. While the degree of accuracy can be increased in one or two orders of magnitude, in the case of graph networks, this requires assembling global Poisson and dissipation matrices, which breaks the local structure of such networks. In order to avoid this drawback, a local version of the metriplectic biases has been developed in this work, which avoids the aforementioned matrix assembly, thus preserving the node-by-node structure of the graph networks. We apply this framework for examples in the fields of solid and fluid mechanics. Our approach demonstrates significant computational efficiency and strong generalization capabilities, accurately making inferences on examples significantly different from those encountered during training. [ABSTRACT FROM AUTHOR]

The accuracy and reliability of physics-based lithium-ion battery degradation models are limited by incomplete understanding of degradation mechanisms. This article presents Universal Differential Equations (UDE) based degradation modelling, which integrates neural networks into physics-based model differential equations to learn partially understood degradation mechanisms. Therefore, this approach combines the function approximation capabilities of machine learning with the interpretability of physics-based models. However, the widespread adoption of this methodology is hindered by the high cost of training neural networks placed within a physics-based degradation model. To address this, we propose a cost-effective parameterisation method that exploits the large difference between electrochemical and degradation time scales, to speed up the gradient calculation using the continuous adjoint sensitivity analysis. Additionally, efficient scaling of this method to multiple ageing datasets is ensured through mini-batching. Finally, we demonstrate this approach by developing a novel UDE calendar ageing model and validating it against in-house experimental data covering 39 storage conditions (13 states of charge at 0 °C, 25 °C, and 45 °C). The predictions on full cell capacity and loss of active material (LAM) at negative electrode align well with experimental observations with an average RMSE of 0.66% and 1.11% respectively, which was a significant improvement over the baseline physics-based model. • Universal Differential Equations Framework was used for degradation modelling. • Neural networks are embedded within continuum physics-based ageing models. • UDE learns optimal constitutive relations for poorly modelled degradation mechanisms. • Reduced UDE parameterisation cost via timescale separation and minibatching. • UDE calendar aging model outperformed physics-based models in accuracy. [ABSTRACT FROM AUTHOR]

A slow decaying Kolmogorov n-width of the solution manifold of a parametric partial differential equation precludes the realization of efficient linear projection-based reduced-order models. This is due to the high dimensionality of the reduced space needed to approximate with sufficient accuracy the solution manifold. To solve this problem, neural networks, in the form of different architectures, have been employed to perform accurate nonlinear regressions of the solution manifolds. However, the majority of the implementations are non-intrusive black-box surrogate models and only a part of them perform dimension reduction from the number of degrees of freedom of the discretized parametric models to a latent dimension. We present a new intrusive and explicable methodology for reduced-order modeling that employs neural networks for the solution manifold approximation but that does not discard the physical and numerical models underneath in the predictive/online stage. We will focus on autoencoders used to compress further the dimensionality of linear approximants of solution manifolds, achieving in the end a nonlinear dimension reduction. After having obtained an accurate nonlinear approximant, we seek for the solutions on the latent manifold with the residual-based nonlinear least-squares Petrov-Galerkin method, opportunely hyper-reduced in order to be independent of the number of degrees of freedom. New adaptive hyper-reduction strategies are developed along with the employment of local nonlinear approximants. We test our methodology on two nonlinear time dependent parametric benchmarks involving a supersonic flow past a NACA airfoil with changing Mach number and an incompressible turbulent flow around the Ahmed body with changing slant angle. [ABSTRACT FROM AUTHOR]

Effective verification and validation techniques for modern scientific machine learning workflows are challenging to devise. Statistical methods are abundant and easily deployed, but often rely on speculative assumptions about the data and methods involved. Error bounds for classical interpolation techniques can provide mathematically rigorous estimates of accuracy, but often are difficult or impractical to determine computationally. In this work, we present a best-of-both-worlds approach to verifiable scientific machine learning by demonstrating that (1) multiple standard interpolation techniques have informative error bounds that can be computed or estimated efficiently; (2) comparative performance among distinct interpolants can aid in validation goals; (3) deploying interpolation methods on latent spaces generated by deep learning techniques enables some interpretability for black-box models. We present a detailed case study of our approach for predicting lift-drag ratios from airfoil images. Code developed for this work is available in a public Github repository. • Interpolation methods can be leveraged to inform scientific machine learning pipelines. • Error bounds for interpolation methods are useful diagnostic tools for understanding performance of scientific machine learning workflows. • Practical application of interpolation-informed analysis on a real-world noisy machine learning problem is demonstrated. [ABSTRACT FROM AUTHOR]

We present a generalized version of the discretization-invariant neural operator in Zhang et al. (2022) and prove that the network is a universal approximation in the operator sense. Moreover, by incorporating additional terms in the architecture, we establish a connection between this discretization-invariant neural operator network and those discussed in Chen and Chen (1995) and Lu et al. (2021). The discretization-invariance property of the operator network implies that different input functions can be sampled using various sensor locations within the same training and testing phases. Additionally, since the network learns a "basis" for the input and output function spaces, our approach enables the evaluation of input functions on different discretizations. To evaluate the performance of the proposed discretization-invariant neural operator, we focus on challenging examples from multiscale partial differential equations. Our experimental results indicate that the method achieves lower prediction errors compared to previous networks and benefits from its discretization-invariant property. [ABSTRACT FROM AUTHOR]

Several techniques have been proposed to find analytical and numerical solutions to (partial differential equations) PDEs and fractional PDEs. However, exact analytical solutions to fractional PDEs are rarely available, numerical methods are so far applicable. Therefore, accurate methods for finding numerical solutions to fractional PDEs are still under investigation and are currently an active area of research. One of the accurate techniques for finding approximate numerical solutions for PDEs is a physics-informed neural network (PINN) through scientific machine learning. In this paper, we introduce an effective homotopy-preserving physics-informed neural network (HP-PINN) that adopts smooth trainable embedding functional parameters through a neural network using the homotopy technique to solve forward and inverse problems governed by conformable space–time fractional PDEs. A more general homotopy perturbation technique is introduced to obtain the analytical solutions for the debated problems. A numerical comparison shows the effectiveness and accuracy of the proposed HP-PINN. [ABSTRACT FROM AUTHOR]

The recent rise of deep learning has led to numerous applications, including solving partial differential equations using Physics-Informed Neural Networks. This approach has proven highly effective in several academic cases. However, their lack of physical invariances, coupled with other significant weaknesses, such as an inability to handle complex geometries or their lack of generalization capabilities, make them unable to compete with classical numerical solvers in industrial settings. In this work, a limitation regarding the use of automatic differentiation in the context of physics-informed learning is highlighted. A hybrid approach combining physics-informed graph neural networks with numerical kernels from finite elements is introduced. After studying the theoretical properties of our model, we apply it to complex geometries, in two and three dimensions. Our choices are supported by an ablation study, and we evaluate the generalization capacity of the proposed approach. • Limitations of auto-differentiation hinder its accuracy to compute physical gradients. • Foreign numerical operators can be used to compute the physical gradients. • A model enriched with physical invariances knowledge can manage complex geometries. • Variational approaches to include physical laws allow better generalization. [ABSTRACT FROM AUTHOR]

The development of inductive biases has been shown to be a very effective way to increase the accuracy and robustness of neural networks, particularly when they are used to predict physical phenomena. These biases significantly increase the certainty of predictions, decrease the error made and allow considerably smaller datasets to be used. There are a multitude of methods in the literature to develop these biases. One of the most effective ways, when dealing with physical phenomena, is to introduce physical principles of recognised validity into the network architecture. The problem becomes more complex without knowledge of the physical principles governing the phenomena under study. A very interesting possibility then is to turn to the principles of thermodynamics, which are universally valid, regardless of the level of abstraction of the description sought for the phenomenon under study. To ensure compliance with the principles of thermodynamics, there are formulations that have a long tradition in many branches of science. In the field of rheology, for example, two main types of formalisms are used to ensure compliance with these principles: one-generator and two-generator formalisms. In this paper we study the advantages and disadvantages of each, using classical problems with known solutions and synthetic data. [ABSTRACT FROM AUTHOR]