Search

Deep neural networks have been shown to provide accurate function approximations in high dimensions. However, fitting network parameters requires informative training data that are often challenging to collect in science and engineering applications. This work proposes Neural Galerkin schemes based on deep learning that generate training data with active learning for numerically solving high-dimensional partial differential equations. Neural Galerkin schemes build on the Dirac-Frenkel variational principle to train networks by minimizing the residual sequentially over time, which enables adaptively collecting new training data in a self-informed manner that is guided by the dynamics described by the partial differential equations. This is in contrast to other machine learning methods that aim to fit network parameters globally in time without taking into account training data acquisition. Our finding is that the active form of gathering training data of the proposed Neural Galerkin schemes is key for numerically realizing the expressive power of networks in high dimensions. Numerical experiments demonstrate that Neural Galerkin schemes have the potential to enable simulating phenomena and processes with many variables for which traditional and other deep-learning-based solvers fail, especially when features of the solutions evolve locally such as in high-dimensional wave propagation problems and interacting particle systems described by Fokker-Planck and kinetic equations. • Coupling training of deep networks with adaptive data acquisition to solve high-dimensional evolution equations. • Enables predicting local features in high dimensions for which training with classical, uniform data acquisition fails. • Sequentially collects data over time and so avoids requirement of having rich initial data set over the time-space domain. • Suited for transport problems, for which collocation-based deep-network methods with uniform data sampling are inefficient. [ABSTRACT FROM AUTHOR]

Abstract We introduce physics-informed neural networks – neural networks that are trained to solve supervised learning tasks while respecting any given laws of physics described by general nonlinear partial differential equations. In this work, we present our developments in the context of solving two main classes of problems: data-driven solution and data-driven discovery of partial differential equations. Depending on the nature and arrangement of the available data, we devise two distinct types of algorithms, namely continuous time and discrete time models. The first type of models forms a new family of data-efficient spatio-temporal function approximators, while the latter type allows the use of arbitrarily accurate implicit Runge–Kutta time stepping schemes with unlimited number of stages. The effectiveness of the proposed framework is demonstrated through a collection of classical problems in fluids, quantum mechanics, reaction–diffusion systems, and the propagation of nonlinear shallow-water waves. Highlights • We put forth a deep learning framework that enables the synergistic combination of mathematical models and data. • We introduce an effective mechanism for regularizing the training of deep neural networks in small data regimes. • The proposed methods enable scientific prediction and discovery from incomplete models and incomplete data. [ABSTRACT FROM AUTHOR]

While deep learning algorithms demonstrate a great potential in scientific computing, its application to multi-scale problems remains to be a big challenge. This is manifested by the "frequency principle" that neural networks tend to learn low frequency components first. Novel architectures such as multi-scale deep neural network (MscaleDNN) were proposed to alleviate this problem to some extent. In this paper, we construct a subspace decomposition based DNN (dubbed SD2NN) architecture for a class of multi-scale problems by combining MscaleDNN algorithms with traditional numerical analysis ideas. The proposed architecture includes one low frequency normal DNN submodule, and one (or a few) high frequency MscaleDNN submodule(s), which are designed to capture the smooth part and the oscillatory part of the multi-scale solutions simultaneously. We demonstrate that the SD2NN outperforms existing models such as MscaleDNN, through several benchmark multi-scale problems in regular or perforated domains. • We propose SD2NN, a new architecture, combining subspace decomposition from numerical analysis and multiscale DNN solvers. • SD2NN has low/high frequency submodules, simultaneously capturing smooth/oscillatory parts of multi-scale solutions. • The new method outperforms state-of-the-art DNN solvers for 1D-3D multiscale PDEs on regular and perforated domains. [ABSTRACT FROM AUTHOR]