Search

Due to the complex behavior arising from non-uniqueness, symmetry, and bifurcations in the solution space, solving inverse problems of nonlinear differential equations (DEs) with multiple solutions is a challenging task. To address this, we propose homotopy physics-informed neural networks (HomPINNs), a novel framework that leverages homotopy continuation and neural networks (NNs) to solve inverse problems. The proposed framework begins with the use of NNs to simultaneously approximate unlabeled observations across diverse solutions while adhering to DE constraints. Through homotopy continuation, the proposed method solves the inverse problem by tracing the observations and identifying multiple solutions. The experiments involve testing the performance of the proposed method on one-dimensional DEs and applying it to solve a two-dimensional Gray-Scott simulation. Our findings demonstrate that the proposed method is scalable and adaptable, providing an effective solution for solving DEs with multiple solutions and unknown parameters. Moreover, it has significant potential for various applications in scientific computing, such as modeling complex systems and solving inverse problems in physics, chemistry, biology, etc. • Physics-informed neural networks are used to estimate unknown parameters with physical constraints. • HomPINNs helps to identify multiple solutions and solve inverse problems. • Numerical examples demonstrate its scalability and adaptability. • HomPINNs facilitates the discovery of previously unknown solutions. [ABSTRACT FROM AUTHOR]

Deep neural networks have been shown to provide accurate function approximations in high dimensions. However, fitting network parameters requires informative training data that are often challenging to collect in science and engineering applications. This work proposes Neural Galerkin schemes based on deep learning that generate training data with active learning for numerically solving high-dimensional partial differential equations. Neural Galerkin schemes build on the Dirac-Frenkel variational principle to train networks by minimizing the residual sequentially over time, which enables adaptively collecting new training data in a self-informed manner that is guided by the dynamics described by the partial differential equations. This is in contrast to other machine learning methods that aim to fit network parameters globally in time without taking into account training data acquisition. Our finding is that the active form of gathering training data of the proposed Neural Galerkin schemes is key for numerically realizing the expressive power of networks in high dimensions. Numerical experiments demonstrate that Neural Galerkin schemes have the potential to enable simulating phenomena and processes with many variables for which traditional and other deep-learning-based solvers fail, especially when features of the solutions evolve locally such as in high-dimensional wave propagation problems and interacting particle systems described by Fokker-Planck and kinetic equations. • Coupling training of deep networks with adaptive data acquisition to solve high-dimensional evolution equations. • Enables predicting local features in high dimensions for which training with classical, uniform data acquisition fails. • Sequentially collects data over time and so avoids requirement of having rich initial data set over the time-space domain. • Suited for transport problems, for which collocation-based deep-network methods with uniform data sampling are inefficient. [ABSTRACT FROM AUTHOR]

Machine learning has been successfully applied to various fields of scientific computing in recent years. For problems with multiscale features such as flows in porous media and mechanical properties of composite materials, however, it remains difficult for machine learning methods to resolve such multiscale features, especially when the small scale information is present. In this work, we propose a sparse radial basis function neural network method to solve elliptic partial differential equations (PDEs) with multiscale coefficients. Inspired by the deep mixed residual method, we rewrite the second-order problem into a first-order system and employ multiple radial basis function neural networks (RBFNNs) to approximate unknown functions in the system. To avoid the overfitting and improve the performance of RBFNN, an additional regularization is introduced in the loss function. Thus the loss function contains two parts: the L 2 loss for the residual of the first-order system and boundary conditions, and the ℓ 1 regularization term for the weights of radial basis functions (RBFs). An algorithm for optimizing the specific loss function is introduced to accelerate the training process. The accuracy and effectiveness of the proposed method are demonstrated through a collection of multiscale problems with scale separation, discontinuity and multiple scales from one to three dimensions. Notably, the ℓ 1 regularization can achieve the goal of representing the solution by fewer RBFs. As a consequence, the total number of RBFs scales like O (ε − n τ) , where ε is the smallest scale, n is the dimensionality, and τ is typically smaller than 1. It is worth mentioning that the proposed method not only has the numerical convergence and thus provides a reliable numerical solution in three dimensions when a classical method is typically not affordable, but also outperforms most other available machine learning methods in terms of accuracy and robustness. Executable files are available at https://github.com/wangzhiwensuda/SRBFNN-for-multiscale-elliptic-problems. The program was implemented in PyTorch and was run on Linux. • A sparse radial basis function neural network (SRBFNN) is proposed to solve multiscale elliptic problems. • SRBFNN achieves higher accuracy in derivatives and solution with fewer basis functions. • The number of basis functions scales as O (ε − τ n) , where ε is the smallest scale, n is the dimensionality, and τ < 1. [ABSTRACT FROM AUTHOR]

While deep learning algorithms demonstrate a great potential in scientific computing, its application to multi-scale problems remains to be a big challenge. This is manifested by the "frequency principle" that neural networks tend to learn low frequency components first. Novel architectures such as multi-scale deep neural network (MscaleDNN) were proposed to alleviate this problem to some extent. In this paper, we construct a subspace decomposition based DNN (dubbed SD2NN) architecture for a class of multi-scale problems by combining MscaleDNN algorithms with traditional numerical analysis ideas. The proposed architecture includes one low frequency normal DNN submodule, and one (or a few) high frequency MscaleDNN submodule(s), which are designed to capture the smooth part and the oscillatory part of the multi-scale solutions simultaneously. We demonstrate that the SD2NN outperforms existing models such as MscaleDNN, through several benchmark multi-scale problems in regular or perforated domains. • We propose SD2NN, a new architecture, combining subspace decomposition from numerical analysis and multiscale DNN solvers. • SD2NN has low/high frequency submodules, simultaneously capturing smooth/oscillatory parts of multi-scale solutions. • The new method outperforms state-of-the-art DNN solvers for 1D-3D multiscale PDEs on regular and perforated domains. [ABSTRACT FROM AUTHOR]

We introduce a data-driven approach to building reduced dynamical models through manifold learning; the reduced latent space is discovered using Diffusion Maps (a manifold learning technique) on time series data. A second round of Diffusion Maps on those latent coordinates allows the approximation of the reduced dynamical models. This second round enables mapping the latent space coordinates back to the full ambient space (what is called lifting); it also enables the approximation of full state functions of interest in terms of the reduced coordinates. In our work, we develop and test three different reduced numerical simulation methodologies, either through pre-tabulation in the latent space and integration on the fly or by going back and forth between the ambient space and the latent space. The data-driven latent space simulation results, based on the three different approaches, are validated through (a) the latent space observation of the full simulation through the Nyström Extension formula, or through (b) lifting the reduced trajectory back to the full ambient space, via Latent Harmonics. Latent space modeling often involves additional regularization to favor certain properties of the space over others, and the mapping back to the ambient space is then constructed mostly independently from these properties; here, we use the same data-driven approach to construct the latent space and then map back to the ambient space. • Construction of reduced data-driven dynamical models. • Latent Space modeling through manifold learning. • Latent Harmonics to approximate functions defined in Latent Space (here Diffusion Maps) coordinates. • Latent-Space assisted Scientific Computing. [ABSTRACT FROM AUTHOR]

This paper proves that shadowing solutions can be almost surely nonphysical. This finding invalidates the argument that small perturbations in a chaotic system can only have a small impact on its statistical behavior. This theoretical finding has implications for many applications in which chaotic dynamics plays an important role. It suggests, for example, that we can control the climate through subtle perturbations. It also suggests that numerical simulations of chaotic dynamics, such as turbulent flows and global atmosphere and ocean circulation, may fail to predict the true long-term or statistical behavior. • Shadowing orbits may not be typical orbits or physical solutions. • This atypicality implies shadowing-based sensitivity can be biased. • Numerical solutions of chaotic dynamical systems can be statistically inconsistent. • Minor parameter changes can significantly alter the statistics of a chaotic system. [ABSTRACT FROM AUTHOR]