Search

Due to the complex behavior arising from non-uniqueness, symmetry, and bifurcations in the solution space, solving inverse problems of nonlinear differential equations (DEs) with multiple solutions is a challenging task. To address this, we propose homotopy physics-informed neural networks (HomPINNs), a novel framework that leverages homotopy continuation and neural networks (NNs) to solve inverse problems. The proposed framework begins with the use of NNs to simultaneously approximate unlabeled observations across diverse solutions while adhering to DE constraints. Through homotopy continuation, the proposed method solves the inverse problem by tracing the observations and identifying multiple solutions. The experiments involve testing the performance of the proposed method on one-dimensional DEs and applying it to solve a two-dimensional Gray-Scott simulation. Our findings demonstrate that the proposed method is scalable and adaptable, providing an effective solution for solving DEs with multiple solutions and unknown parameters. Moreover, it has significant potential for various applications in scientific computing, such as modeling complex systems and solving inverse problems in physics, chemistry, biology, etc. • Physics-informed neural networks are used to estimate unknown parameters with physical constraints. • HomPINNs helps to identify multiple solutions and solve inverse problems. • Numerical examples demonstrate its scalability and adaptability. • HomPINNs facilitates the discovery of previously unknown solutions. [ABSTRACT FROM AUTHOR]

Deep neural networks have been shown to provide accurate function approximations in high dimensions. However, fitting network parameters requires informative training data that are often challenging to collect in science and engineering applications. This work proposes Neural Galerkin schemes based on deep learning that generate training data with active learning for numerically solving high-dimensional partial differential equations. Neural Galerkin schemes build on the Dirac-Frenkel variational principle to train networks by minimizing the residual sequentially over time, which enables adaptively collecting new training data in a self-informed manner that is guided by the dynamics described by the partial differential equations. This is in contrast to other machine learning methods that aim to fit network parameters globally in time without taking into account training data acquisition. Our finding is that the active form of gathering training data of the proposed Neural Galerkin schemes is key for numerically realizing the expressive power of networks in high dimensions. Numerical experiments demonstrate that Neural Galerkin schemes have the potential to enable simulating phenomena and processes with many variables for which traditional and other deep-learning-based solvers fail, especially when features of the solutions evolve locally such as in high-dimensional wave propagation problems and interacting particle systems described by Fokker-Planck and kinetic equations. • Coupling training of deep networks with adaptive data acquisition to solve high-dimensional evolution equations. • Enables predicting local features in high dimensions for which training with classical, uniform data acquisition fails. • Sequentially collects data over time and so avoids requirement of having rich initial data set over the time-space domain. • Suited for transport problems, for which collocation-based deep-network methods with uniform data sampling are inefficient. [ABSTRACT FROM AUTHOR]

Machine learning has been successfully applied to various fields of scientific computing in recent years. For problems with multiscale features such as flows in porous media and mechanical properties of composite materials, however, it remains difficult for machine learning methods to resolve such multiscale features, especially when the small scale information is present. In this work, we propose a sparse radial basis function neural network method to solve elliptic partial differential equations (PDEs) with multiscale coefficients. Inspired by the deep mixed residual method, we rewrite the second-order problem into a first-order system and employ multiple radial basis function neural networks (RBFNNs) to approximate unknown functions in the system. To avoid the overfitting and improve the performance of RBFNN, an additional regularization is introduced in the loss function. Thus the loss function contains two parts: the L 2 loss for the residual of the first-order system and boundary conditions, and the ℓ 1 regularization term for the weights of radial basis functions (RBFs). An algorithm for optimizing the specific loss function is introduced to accelerate the training process. The accuracy and effectiveness of the proposed method are demonstrated through a collection of multiscale problems with scale separation, discontinuity and multiple scales from one to three dimensions. Notably, the ℓ 1 regularization can achieve the goal of representing the solution by fewer RBFs. As a consequence, the total number of RBFs scales like O (ε − n τ) , where ε is the smallest scale, n is the dimensionality, and τ is typically smaller than 1. It is worth mentioning that the proposed method not only has the numerical convergence and thus provides a reliable numerical solution in three dimensions when a classical method is typically not affordable, but also outperforms most other available machine learning methods in terms of accuracy and robustness. Executable files are available at https://github.com/wangzhiwensuda/SRBFNN-for-multiscale-elliptic-problems. The program was implemented in PyTorch and was run on Linux. • A sparse radial basis function neural network (SRBFNN) is proposed to solve multiscale elliptic problems. • SRBFNN achieves higher accuracy in derivatives and solution with fewer basis functions. • The number of basis functions scales as O (ε − τ n) , where ε is the smallest scale, n is the dimensionality, and τ < 1. [ABSTRACT FROM AUTHOR]

We consider the inverse problem of estimating parameters of a driven diffusion (e.g., the underlying fluid flow, diffusion coefficient, or source terms) from point measurements of a passive scalar (e.g., the concentration of a pollutant). We present two particle methods that leverage the structure of the inverse problem to enable efficient computation of the forward map, one for time evolution problems and one for Dirichlet boundary-value problems. The methods scale in a natural fashion to modern computational architectures, enabling substantial speedup for applications involving sparse observations and high-dimensional unknowns. Numerical examples of applications to Bayesian inference and numerical optimization are provided. • Particle algorithms are presented for PDE-constrained inverse problems. • The methods leverage problem structure to enable fast computation of the forward map. • The methods scale in a natural fashion to modern computational architectures. • Computational complexity for these and traditional two-step algorithms is provided. • Applications to Bayesian inference and numerical optimization are provided. [ABSTRACT FROM AUTHOR]

While deep learning algorithms demonstrate a great potential in scientific computing, its application to multi-scale problems remains to be a big challenge. This is manifested by the "frequency principle" that neural networks tend to learn low frequency components first. Novel architectures such as multi-scale deep neural network (MscaleDNN) were proposed to alleviate this problem to some extent. In this paper, we construct a subspace decomposition based DNN (dubbed SD2NN) architecture for a class of multi-scale problems by combining MscaleDNN algorithms with traditional numerical analysis ideas. The proposed architecture includes one low frequency normal DNN submodule, and one (or a few) high frequency MscaleDNN submodule(s), which are designed to capture the smooth part and the oscillatory part of the multi-scale solutions simultaneously. We demonstrate that the SD2NN outperforms existing models such as MscaleDNN, through several benchmark multi-scale problems in regular or perforated domains. • We propose SD2NN, a new architecture, combining subspace decomposition from numerical analysis and multiscale DNN solvers. • SD2NN has low/high frequency submodules, simultaneously capturing smooth/oscillatory parts of multi-scale solutions. • The new method outperforms state-of-the-art DNN solvers for 1D-3D multiscale PDEs on regular and perforated domains. [ABSTRACT FROM AUTHOR]

We propose an anomaly detection method for multi-variate scientific data based on analysis of high-order joint moments. Using kurtosis as a reliable measure of outliers, we suggest that principal kurtosis vectors, by analogy to principal component analysis (PCA) vectors, signify the principal directions along which outliers appear. The inception of an anomaly, then, manifests as a change in the principal values and vectors of kurtosis. Obtaining the principal kurtosis vectors requires decomposing a fourth order joint cumulant tensor for which we use a simple, computationally less expensive approach that involves performing a singular value decomposition (SVD) over the matricized tensor. We demonstrate the efficacy of this approach on synthetic data, and develop an algorithm to identify the occurrence of a spatial and/or temporal anomalous event in scientific phenomena. The algorithm decomposes the data into several spatial sub-domains and time steps to identify regions with such events. Feature moment metrics, based on the alignments of the principal kurtosis vectors, are computed at each sub-domain and time step for all features to quantify their relative importance towards the overall kurtosis in the data. Accordingly, spatial and temporal anomaly metrics for each sub-domain are proposed using the Hellinger distance of the feature moment metric distribution from a suitable nominal distribution. We apply the algorithm to two turbulent auto-ignition combustion cases and demonstrate that the anomaly metrics reliably capture the occurrence of auto-ignition in relevant spatial sub-domains at the right time steps. • A novel unsupervised anomaly detection method for multi-scale, multi-variate data. • Hypothesis: anomalous events have a signature in the higher order joint moments. • Extends the concept of PCA to higher order moment, specifically kurtosis. • The method identifies principal vectors of kurtosis as directions of anomaly. • The algorithm is designed for distributed large-scale HPC simulations and data. [ABSTRACT FROM AUTHOR]

We introduce a data-driven approach to building reduced dynamical models through manifold learning; the reduced latent space is discovered using Diffusion Maps (a manifold learning technique) on time series data. A second round of Diffusion Maps on those latent coordinates allows the approximation of the reduced dynamical models. This second round enables mapping the latent space coordinates back to the full ambient space (what is called lifting); it also enables the approximation of full state functions of interest in terms of the reduced coordinates. In our work, we develop and test three different reduced numerical simulation methodologies, either through pre-tabulation in the latent space and integration on the fly or by going back and forth between the ambient space and the latent space. The data-driven latent space simulation results, based on the three different approaches, are validated through (a) the latent space observation of the full simulation through the Nyström Extension formula, or through (b) lifting the reduced trajectory back to the full ambient space, via Latent Harmonics. Latent space modeling often involves additional regularization to favor certain properties of the space over others, and the mapping back to the ambient space is then constructed mostly independently from these properties; here, we use the same data-driven approach to construct the latent space and then map back to the ambient space. • Construction of reduced data-driven dynamical models. • Latent Space modeling through manifold learning. • Latent Harmonics to approximate functions defined in Latent Space (here Diffusion Maps) coordinates. • Latent-Space assisted Scientific Computing. [ABSTRACT FROM AUTHOR]

This paper proves that shadowing solutions can be almost surely nonphysical. This finding invalidates the argument that small perturbations in a chaotic system can only have a small impact on its statistical behavior. This theoretical finding has implications for many applications in which chaotic dynamics plays an important role. It suggests, for example, that we can control the climate through subtle perturbations. It also suggests that numerical simulations of chaotic dynamics, such as turbulent flows and global atmosphere and ocean circulation, may fail to predict the true long-term or statistical behavior. • Shadowing orbits may not be typical orbits or physical solutions. • This atypicality implies shadowing-based sensitivity can be biased. • Numerical solutions of chaotic dynamical systems can be statistically inconsistent. • Minor parameter changes can significantly alter the statistics of a chaotic system. [ABSTRACT FROM AUTHOR]

• This numerical scheme is devised by the primal-dual weak Galerkin(PDWG) framework. • The PDWG method can be interpreted as a constraint optimization process. • The PDWG scheme conserves mass locally on each element. • Optimal order error estimates are derived for the PDWG solution. This article devises a new numerical method for first-order transport problems by using the primal-dual weak Galerkin (PD-WG) finite element method recently developed in scientific computing. The PD-WG method is based on a variational formulation of the modeling equation for which the differential operator is applied to the test function so that low regularity for the exact solution of the original equation is sufficient for computation. The PD-WG finite element method indeed yields a symmetric system involving both the original equation for the primal variable and its dual for the dual variable (also known as Lagrangian multiplier). For the linear transport problem, it is shown that the PD-WG method offers numerical solutions that conserve mass locally on each element. Optimal order error estimates in various norms are derived for the numerical solutions arising from the PD-WG method with weak regularity assumptions on the modelling equations. A variety of numerical results are presented to demonstrate the accuracy and stability of the new method. [ABSTRACT FROM AUTHOR]