Search

Incorporating unstructured data into physical models is a challenging problem that is emerging in data assimilation. Traditional approaches focus on well-defined observation operators whose functional forms are typically assumed to be known. This prevents these methods from achieving a consistent model-data synthesis in configurations where the mapping from data-space to model-space is unknown. To address these shortcomings, in this paper we develop a physics-informed dynamical variational autoencoder (Φ-DVAE) to embed diverse data streams into time-evolving physical systems described by differential equations. Our approach combines a standard, possibly nonlinear, filter for the latent state-space model and a VAE, to assimilate the unstructured data into the latent dynamical system. Unstructured data, in our example systems, comes in the form of video data and velocity field measurements, however the methodology is suitably generic to allow for arbitrary unknown observation operators. A variational Bayesian framework is used for the joint estimation of the encoding, latent states, and unknown system parameters. To demonstrate the method, we provide case studies with the Lorenz-63 ordinary differential equation, and the advection and Korteweg-de Vries partial differential equations. Our results, with synthetic data, show that Φ-DVAE provides a data efficient dynamics encoding methodology which is competitive with standard approaches. Unknown parameters are recovered with uncertainty quantification, and unseen data are accurately predicted. • Bayesian inference methodology for unstructured data assimilation. • Variational autoencoder embeds data to observations of latent differential equation. • Statistical FEM construction and parameter estimation account for misspecification. • Demonstrated on Lorenz-63, advection and Korteweg-de Vries differential equations. • Embedding physical prior knowledge produces data-efficient learning. [ABSTRACT FROM AUTHOR]

• A novel physics-informed convolutional neural network f-PICNN for PDEs without any labelled data. • Nonlinear convolutional units (NCUs). • Memory mechanism (numerical results show it can considerably speed up the convergence). • Finite discretization schemes (e.g., Euler, Crank Nicolson and fourth-order Runge-Kutta). • Auto-regressive framework. The physics and interdisciplinary problems in science and engineering are mainly described as partial differential equations (PDEs). Recently, a novel method using physics-informed neural networks (PINNs) to solve PDEs by employing deep neural networks with physical constraints as data-driven models has been pioneered for surrogate modelling and inverse problems. However, the original PINNs based on fully connected neural networks pose intrinsic limitations and poor performance for the PDEs with nonlinearity, drastic gradients, multiscale characteristics or high dimensionality in which the complex features are hard to capture. This leads to difficulties in convergence to correct solutions and high computational costs. To address the above problems, in this paper, a novel physics-informed convolutional neural network framework based on finite discretization schemes with a stack of a series of nonlinear convolutional units (NCUs) for solving PDEs in the space-time domain without any labelled data (f-PICNN) is proposed, in which the memory mechanism can considerably speed up the convergence. Specifically, the initial conditions (ICs) are hard-encoded into the network as the first time-step solution and used to extrapolate the next time-step solution. The Dirichlet boundary conditions (BCs) are constrained by soft BC enforcement while the Neumann BCs are hard enforced. Furthermore, the loss function is designed as a set of discretized PDE residuals and optimized to conform to physics laws. Finally, the proposed auto-regressive model has been proven to be effective in a wide range of 1D and 2D nonlinear PDEs in both space and time under different finite discretization schemes (e.g., Euler, Crank Nicolson and fourth-order Runge-Kutta). The numerical results demonstrate that the proposed framework not only shows the ability to learn the PDEs efficiently but also provides an opportunity for greater conceptual simplicity, and potential for extrapolation from learning the PDEs using a limited dataset. [ABSTRACT FROM AUTHOR]

Inverse problems, i.e., estimating parameters of physical models from experimental data, are ubiquitous in science and engineering. The Bayesian formulation is the gold standard because it alleviates ill-posedness issues and quantifies epistemic uncertainty. Since analytical posteriors are not typically available, one resorts to Markov chain Monte Carlo sampling or approximate variational inference. However, inference needs to be rerun from scratch for each new set of data. This drawback limits the applicability of the Bayesian formulation to real-time settings, e.g., health monitoring of engineered systems, and medical diagnosis. The objective of this paper is to develop a methodology that enables real-time inference by learning the Bayesian inverse map, i.e., the map from data to posteriors. Our approach is as follows. We parameterize the posterior distribution as a function of data. This work outlines two distinct approaches to do this. The first method involves parameterizing the posterior using an amortized full-rank Gaussian guide, implemented through neural networks. The second method utilizes a Conditional Normalizing Flow guide, employing conditional invertible neural networks for cases where the target posterior is arbitrarily complex. In both approaches, we learn the network parameters by amortized variational inference which involves maximizing the expectation of evidence lower bound over all possible datasets compatible with the model. We demonstrate our approach by solving a set of benchmark problems from science and engineering. Our results show that the posterior estimates of our approach are in agreement with the corresponding ground truth obtained by Markov chain Monte Carlo. Once trained, our approach provides the posterior distribution for a given observation just at the cost of a forward pass of the neural network. • An approach eliminating the need to re-run posterior inference from scratch for each new data. • Posterior parameterization using amortized posteriors represented by neural nets. • Formulation of an amortized loss function and training the network using it. • Application to solving benchmark problems in science and engineering. • Posteriors are estimated just at the cost of forward pass of the network once trained. [ABSTRACT FROM AUTHOR]

Bayesian Physics Informed Neural Networks (B-PINNs) have gained significant attention for inferring physical parameters and learning the forward solutions for problems based on partial differential equations. However, the overparameterized nature of neural networks poses a computational challenge for high-dimensional posterior inference. Existing inference approaches, such as particle-based or variance inference methods, are either computationally expensive for high-dimensional posterior inference or provide unsatisfactory uncertainty estimates. In this paper, we present a new efficient inference algorithm for B-PINNs that uses Ensemble Kalman Inversion (EKI) for high-dimensional inference tasks. By reframing the setup of B-PINNs as a traditional Bayesian inverse problem, we can take advantage of EKI's key features: (1) gradient-free, (2) computational complexity scales linearly with the dimension of the parameter spaces, and (3) rapid convergence with typically O (100) iterations. We demonstrate the applicability and performance of the proposed method through various types of numerical examples. We find that our proposed method can achieve inference results with informative uncertainty estimates comparable to Hamiltonian Monte Carlo (HMC)-based B-PINNs with a much reduced computational cost. These findings suggest that our proposed approach has great potential for uncertainty quantification in physics-informed machine learning for practical applications. • Proposed a gradient-free inference algorithm for Bayesian physics informed neural networks (BPINNs). • Reframed BPINNs as a classic inverse problem to utilize Ensemble Kalman Inversion (EKI). • Demonstrated our method achieves informative uncertainty estimates at lower cost. [ABSTRACT FROM AUTHOR]

This paper presents a Gaussian Process (GP) framework, a non-parametric technique widely acknowledged for regression and classification tasks, to address inverse problems in mean field games (MFGs). By leveraging GPs, we aim to recover agents' strategic actions and the environment's configurations from partial and noisy observations of the population of agents and the setup of the environment. Our method is a probabilistic tool to infer the behaviors of agents in MFGs from data in scenarios where the comprehensive dataset is either inaccessible or contaminated by noises. [ABSTRACT FROM AUTHOR]

In many Bayesian inverse problems the goal is to recover a spatially varying random field. Such problems are often computationally challenging especially when the forward model is governed by complex partial differential equations (PDEs). The challenge is particularly severe when the spatial domain is large and the unknown random field needs to be represented by a high-dimensional parameter. In this paper, we present a domain-decomposed method to attack the dimensionality issue and the method decomposes the spatial domain and the parameter domain simultaneously. On each subdomain, a local Karhunen-Loève (KL) expansion is constructed, and a local inversion problem is solved independently in a parallel manner, and more importantly, in a lower-dimensional space. After local posterior samples are generated through conducting Markov chain Monte Carlo (MCMC) simulations on subdomains, a novel projection procedure is developed to effectively reconstruct the global field. In addition, the domain decomposition interface conditions are dealt with an adaptive Gaussian process-based fitting strategy. Numerical examples are provided to demonstrate the performance of the proposed method. [ABSTRACT FROM AUTHOR]

This paper investigates an inverse scattering problem of time-harmonic elastic waves incident on a rigid random periodic structure. A novel and efficient numerical method is proposed to quantify the randomness, i.e., to reconstruct key statistical properties of random structures from boundary measurements of the scattering data. This method consists of two ingredients, the Monte Carlo technique for sampling the probability space and a continuation method with respect to the wavenumber. Unlike existing methods, this approach does not require a priori knowledge of the randomness and can handle a wide range of random structures, including non-Gaussian processes with extremely high levels of complexity. To illustrate the accuracy and effectiveness of the proposed method, numerical examples involving Gaussian and non-Gaussian processes with various covariances are provided. [ABSTRACT FROM AUTHOR]

• We proposed variable linear transformation improved physics-informed neural networks (VLT-PINNs) to solve thin-layer flow forward and inverse problems. • Three principles for determining the VLT parameters were identified for three types of variables in PINNs respectively. • The VLT-PINNs with the principle-suggested parameters show excellent performance over the ten test cases. • Only under the constraints of the VLT principles can we obtain satisfactory results, while traditional linear transformation may yield totally wrong results. • VLT method is an attempt to combine the three advantages of accuracy, universality and simplicity. Physics-informed neural networks (PINNs) have attracted wide attention due to their ability to seamlessly embed the learning process with physical laws and their considerable success in solving forward and inverse differential equation (DE) problems. While most studies are improving the learning process and network architecture of PINNs, less attention has been paid to the modification of the DE system, which may play an important role in addressing some limitations of PINNs. One of the simplest modifications that can be implemented to all DE systems is the variable linear transformation (VLT). Therefore, in this work, we propose the VLT-PINNs that solve the DE systems of the linear-transformed variables instead of the original ones. To clearly illustrate the importance of prior knowledge in determining the VLT parameters, we choose the thin-layer flow problems as our focus. Ten related cases were tested, including the jet flows, wake flows, mixing layers, boundary layers and Kovasznay flows. Based on the principle of normalization and for a better match of the DE system to the preference of NNs, we identify three principles for determining the VLT parameters: magnitude normalization for dependent variables (principle 1), local normalization for independent variables (principle 2), and appropriate scaling for physics-related parameters in inverse problems (principle 3). The VLT-PINNs with the VLT parameters suggested by the proposed principles show excellent performance over all the test cases, while the results are quite poor with the VLT parameters suggested by traditional linear transformations, such as nondimensionalization and global normalization. Comparison studies also show that only under the constraints of the VLT principles can we obtain satisfactory results. Besides, we find tanh is more appropriate as the activation function than sin for thin-layer flow problems, from both posteriori results and priori analyses with physical intuition. We highlight that our VLT method is an attempt to combine the three advantages of accuracy, universality and simplicity, and hope that it can provide new insights into the better integration of prior knowledge, physical intuition and the nature of NNs. The code for this paper is available on https://github.com/CAME-THU/VLT-PINN. [ABSTRACT FROM AUTHOR]

There are many physical processes that have inherent discontinuities in their mathematical formulations. This paper is motivated by the specific case of collisions between two rigid or deformable bodies and the intrinsic nature of that discontinuity. The impulse response to a collision is discontinuous with the lack of any response when no collision occurs, which causes difficulties for numerical approaches that require differentiability which are typical in machine learning, inverse problems, and control. We theoretically and numerically demonstrate that the derivative of the collision time with respect to the parameters becomes infinite as one approaches the barrier separating colliding from not colliding, and use lifting to complexify the solution space so that solutions on the other side of the barrier are directly attainable as precise values. Subsequently, we mollify the barrier posed by the unbounded derivatives, so that one can tunnel back and forth in a smooth and reliable fashion facilitating the use of standard numerical approaches. Moreover, we illustrate that standard approaches fail in numerous ways mostly due to a lack of understanding of the mathematical nature of the problem (e.g. typical backpropagation utilizes many rules of differentiation, but ignores L'Hopital's rule). • Novel root-finding methods address collision barriers in rigid and deformable bodies. • Demonstrated unbounded derivatives at collision barriers. • Used complexification and mollification for smooth transitions. • Overcame limitations of standard numerical methods. • Highlighted overlooked principles, like L'Hopital's rule, in backpropagation algorithms. [ABSTRACT FROM AUTHOR]

We are concerned with a novel Bayesian statistical framework for the characterization of natural subsurface formations, a very challenging task. Because of the large dimension of the stochastic space of the prior distribution in the framework, typically a dimensional reduction method, such as a Karhunen-Leove expansion (KLE), needs to be applied to the prior distribution to make the characterization computationally tractable. Due to the large variability of properties of subsurface formations (such as permeability and porosity) it may be of value to localize the sampling strategy so that it can better adapt to large local variability of rock properties. In this paper, we introduce the concept of multiscale sampling to localize the search in the stochastic space. We combine the simplicity of a preconditioned Markov Chain Monte Carlo method with a new algorithm to decompose the stochastic space into orthogonal subspaces, through a one-to-one mapping of the subspaces to subdomains of a non-overlapping domain decomposition of the region of interest. The localization of the search is performed by a multiscale blocking strategy within Gibbs sampling: we apply a KL expansion locally, at the subdomain level. Within each subdomain, blocking is applied again, for the sampling of the KLE random coefficients. The effectiveness of the proposed framework is tested in the solution of inverse problems related to elliptic partial differential equations arising in porous media flows. We use multi-chain studies in a multi-GPU cluster to show that the new algorithm clearly improves the convergence rate of the preconditioned MCMC method. Moreover, we illustrate the importance of a few conditioning points to further improve the convergence of the proposed method. [ABSTRACT FROM AUTHOR]