Search

In recent years, the advancement of deep learning has led to the utilization of related technologies to enhance the efficiency and accuracy of scientific computing. Physics-Informed Neural Networks (PINNs) are a type of deep learning method applied to scientific computing, widely used to solve various partial differential equations (PDEs), demonstrating tremendous potential. This study improved upon original PINNs and applied them to forward and inverse problems in the nonlinear science field. In addition to incorporating the constraints of PDEs, the improved PINNs added constraints on gradient information, which further enhanced the physical constraints. Moreover, an adaptive learning method was used to update the weight coefficients of the loss function and dynamically adjust the weight proportion of each constraint term. In the experiment, the improved PINNs were used to numerically simulate localized waves and two-dimensional lid-driven cavity flow described by partial differential equations. Meanwhile, we critically evaluate the accuracy of the prediction results. Furthermore, the improved PINNs were utilized to solve the inverse problems of nonlinear PDEs, where the results showed that even with noisy data, the unknown parameters could be discovered satisfactorily. The study results indicated that the improved PINNs were significantly superior to original PINNs, with shorter training time, increased accuracy in prediction results, and greater potential for application. [ABSTRACT FROM AUTHOR]

This paper presents NSGA-PINN, a multi-objective optimization framework for the effective training of physics-informed neural networks (PINNs). The proposed framework uses the non-dominated sorting genetic algorithm (NSGA-II) to enable traditional stochastic gradient optimization algorithms (e.g., ADAM) to escape local minima effectively. Additionally, the NSGA-II algorithm enables satisfying the initial and boundary conditions encoded into the loss function during physics-informed training precisely. We demonstrate the effectiveness of our framework by applying NSGA-PINN to several ordinary and partial differential equation problems. In particular, we show that the proposed framework can handle challenging inverse problems with noisy data. [ABSTRACT FROM AUTHOR]

Physics-informed neural networks (PINNs) have been widely used to solve various scientific computing problems. However, large training costs limit PINNs for some real-time applications. Although some works have been proposed to improve the training efficiency of PINNs, few consider the influence of initialization. To this end, we propose a New Reptile initialization-based Physics-Informed Neural Network (NRPINN). The original Reptile algorithm is a meta-learning initialization method based on labeled data. PINNs can be trained with less labeled data or even without any labeled data by adding partial differential equations (PDEs) as a penalty term into the loss function. Inspired by this idea, we propose the new Reptile initialization to sample more tasks from the parameterized PDEs and adapt the penalty term of the loss. The new Reptile initialization can acquire initialization parameters from related tasks by supervised, unsupervised, and semi-supervised learning. Then, PINNs with initialization parameters can efficiently solve PDEs. Besides, the new Reptile initialization can also be used for the variants of PINNs. Finally, we demonstrate and verify the NRPINN considering both forward problems, including solving Poisson, Burgers, and Schrödinger equations, as well as inverse problems, where unknown parameters in the PDEs are estimated. Experimental results show that the NRPINN training is much faster and achieves higher accuracy than PINNs with other initialization methods. [ABSTRACT FROM AUTHOR]

Deep learning (DL) has had unprecedented success and is now entering scientific computing with full force. However, current DL methods typically suffer from instability, even when universal approximation properties guarantee the existence of stable neural networks (NNs). We address this paradox by demonstrating basic well-conditioned problems in scientific computing where one can prove the existence of NNs with great approximation qualities; however, there does not exist any algorithm, even randomized, that can train (or compute) such a NN. For any positive integers K > 2 and L, there are cases where simultaneously 1) no randomized training algorithm can compute a NN correct to K digits with probability greater than 1/2; 2) there exists a deterministic training algorithm that computes a NN with K − 1 correct digits, but any such (even randomized) algorithm needs arbitrarily many training data; and 3) there exists a deterministic training algorithm that computes a NN with K − 2 correct digits using no more than L training samples. These results imply a classification theory describing conditions under which (stable) NNs with a given accuracy can be computed by an algorithm. We begin this theory by establishing sufficient conditions for the existence of algorithms that compute stable NNs in inverse problems. We introduce fast iterative restarted networks (FIRENETs), which we both prove and numerically verify are stable. Moreover, we prove that only O(| log(ϵ)|) layers are needed for an -accurate solution to the inverse problem. [ABSTRACT FROM AUTHOR]

Due to the complex behavior arising from non-uniqueness, symmetry, and bifurcations in the solution space, solving inverse problems of nonlinear differential equations (DEs) with multiple solutions is a challenging task. To address this, we propose homotopy physics-informed neural networks (HomPINNs), a novel framework that leverages homotopy continuation and neural networks (NNs) to solve inverse problems. The proposed framework begins with the use of NNs to simultaneously approximate unlabeled observations across diverse solutions while adhering to DE constraints. Through homotopy continuation, the proposed method solves the inverse problem by tracing the observations and identifying multiple solutions. The experiments involve testing the performance of the proposed method on one-dimensional DEs and applying it to solve a two-dimensional Gray-Scott simulation. Our findings demonstrate that the proposed method is scalable and adaptable, providing an effective solution for solving DEs with multiple solutions and unknown parameters. Moreover, it has significant potential for various applications in scientific computing, such as modeling complex systems and solving inverse problems in physics, chemistry, biology, etc. • Physics-informed neural networks are used to estimate unknown parameters with physical constraints. • HomPINNs helps to identify multiple solutions and solve inverse problems. • Numerical examples demonstrate its scalability and adaptability. • HomPINNs facilitates the discovery of previously unknown solutions. [ABSTRACT FROM AUTHOR]

Least squares form one of the most prominent classes of optimization problems with numerous applications in scientific computing and data fitting. When such formulations aim at modeling complex systems, the optimization process must account for nonlinear dynamics by incorporating constraints. In addition, these systems often incorporate a large number of variables, which increases the difficulty of the problem and motivates the need for efficient algorithms amenable to large-scale implementations. In this paper, we propose and analyze a Levenberg--Marquardt algorithm for nonlinear least squares subject to nonlinear equality constraints. Our algorithm is based on inexact solves of linear least-squares problems that only require Jacobian-vector products. Global convergence is guaranteed by the combination of a composite step approach and a nonmonotone step acceptance rule. We illustrate the performance of our method on several test cases from data assimilation and inverse problems; our algorithm is able to reach the vicinity of a solution from an arbitrary starting point and can outperform the most natural alternatives for these classes of problems. [ABSTRACT FROM AUTHOR]

• We propose a PINN for solving conservative-form transport equations (PINNforCTE). • PINNforCTE facilitates the treatment of Neumann and Robin boundary conditions. • PINNforCTE reduces the training time by ∼50% compared to the traditional PINN. • PINNforCTE improves the prediction accuracy compared to the traditional PINN. • PINNforCTE solves the inverse problems with high accuracy and strong robustness. Physics-informed neural network (PINN) has recently gained significant attention in the field of scientific computing. This paper proposes a PINN framework for solving conservative-form transport equations (PINNforCTE). The PINNforCTE features an accelerated back-propagation in the neural network (NN) training due to low-order derivatives in the governing equations. We demonstrate the application of PINNforCTE in simulating a classical flow and heat transfer problem, i.e., the lid-driven cavity flow and heat transfer under three types of boundary conditions, and the simulation results are compared with those obtained from the traditional PINN and the finite element CFD method. The comparisons indicate that PINNforCTE can significantly accelerate the NN training and potentially shorten the solution time. For all cases considered, the training time of PINNforCTE is reduced by approximately ∼50% compared with the conventional PINN. Furthermore, PINNforCTE can accelerate the Adam optimization algorithm (L-BFGS-B optimization algorithm) by ∼100% (∼80%) compared to the traditional PINN. Additionally, PINNforCTE shows greater convenience in applying the Neumann and Robin boundary conditions directly and higher calculation accuracy than the conventional PINN. Finally, the ability of PINNforCTE to solve the inverse problems is confirmed with an example of 2D steady lid-driven cavity flow and heat transfer. PINNforCTE is an efficient NN solver for conservative-form transport equations (CTEs) that are ubiquitous in science and engineering. [ABSTRACT FROM AUTHOR]

We consider the inverse problem of estimating parameters of a driven diffusion (e.g., the underlying fluid flow, diffusion coefficient, or source terms) from point measurements of a passive scalar (e.g., the concentration of a pollutant). We present two particle methods that leverage the structure of the inverse problem to enable efficient computation of the forward map, one for time evolution problems and one for Dirichlet boundary-value problems. The methods scale in a natural fashion to modern computational architectures, enabling substantial speedup for applications involving sparse observations and high-dimensional unknowns. Numerical examples of applications to Bayesian inference and numerical optimization are provided. • Particle algorithms are presented for PDE-constrained inverse problems. • The methods leverage problem structure to enable fast computation of the forward map. • The methods scale in a natural fashion to modern computational architectures. • Computational complexity for these and traditional two-step algorithms is provided. • Applications to Bayesian inference and numerical optimization are provided. [ABSTRACT FROM AUTHOR]

Regularization techniques are widely employed in optimization-based approaches for solving ill-posed inverse problems in data analysis and scientific computing. These methods are based on augmenting the objective with a penalty function, which is specified based on prior domain-specific expertise to induce a desired structure in the solution. We consider the problem of learning suitable regularization functions from data in settings in which precise domain knowledge is not directly available. Previous work under the title of 'dictionary learning' or 'sparse coding' may be viewed as learning a regularization function that can be computed via linear programming. We describe generalizations of these methods to learn regularizers that can be computed and optimized via semidefinite programming. Our framework for learning such semidefinite regularizers is based on obtaining structured factorizations of data matrices, and our algorithmic approach for computing these factorizations combines recent techniques for rank minimization problems along with an operator analog of Sinkhorn scaling. Under suitable conditions on the input data, our algorithm provides a locally linearly convergent method for identifying the correct regularizer that promotes the type of structure contained in the data. Our analysis is based on the stability properties of Operator Sinkhorn scaling and their relation to geometric aspects of determinantal varieties (in particular tangent spaces with respect to these varieties). The regularizers obtained using our framework can be employed effectively in semidefinite programming relaxations for solving inverse problems. [ABSTRACT FROM AUTHOR]

Abstract We introduce physics-informed neural networks – neural networks that are trained to solve supervised learning tasks while respecting any given laws of physics described by general nonlinear partial differential equations. In this work, we present our developments in the context of solving two main classes of problems: data-driven solution and data-driven discovery of partial differential equations. Depending on the nature and arrangement of the available data, we devise two distinct types of algorithms, namely continuous time and discrete time models. The first type of models forms a new family of data-efficient spatio-temporal function approximators, while the latter type allows the use of arbitrarily accurate implicit Runge–Kutta time stepping schemes with unlimited number of stages. The effectiveness of the proposed framework is demonstrated through a collection of classical problems in fluids, quantum mechanics, reaction–diffusion systems, and the propagation of nonlinear shallow-water waves. Highlights • We put forth a deep learning framework that enables the synergistic combination of mathematical models and data. • We introduce an effective mechanism for regularizing the training of deep neural networks in small data regimes. • The proposed methods enable scientific prediction and discovery from incomplete models and incomplete data. [ABSTRACT FROM AUTHOR]

Recently, a class of machine learning methods called physics-informed neural networks (PINNs) has been proposed and gained prevalence in solving various scientific computing problems. This approach enables the solution of partial differential equations (PDEs) via embedding physical laws into the loss function. Many inverse problems can be tackled by simply combining the data from real life scenarios with existing PINN algorithms. In this paper, we present a multi-task learning method using uncertainty weighting to improve the training efficiency and accuracy of PINNs for inverse problems in linear elasticity and hyperelasticity. Furthermore, we demonstrate an application of PINNs to a practical inverse problem in structural analysis: prediction of external loads of diverse engineering structures based on limited displacement monitoring points. To this end, we first determine a simplified loading scenario at the offline stage. By setting unknown boundary conditions as learnable parameters, PINNs can predict the external loads with the support of measured data. When it comes to the online stage in real engineering projects, transfer learning is employed to fine-tune the pre-trained model from offline stage. Our results show that, even with noisy gappy data, satisfactory results can still be obtained from the PINN model due to the dual regularization of physics laws and prior knowledge, which exhibits better robustness compared to traditional analysis methods. Our approach is capable of bridging the gap between various structures with geometric scaling and under different loading scenarios, and the convergence of training is also greatly accelerated through not only the layer freezing but also the multi-task weight inheritance from pre-trained models, thus making it possible to be applied as surrogate models in actual engineering projects. • Non-dimensionalization and multi-task learning improves the performance of PINNs. • This method can infer the unknown boundary conditions based on limited noisy data. • The presented method can be applied to structures under different loading scenarios. • Transfer learning accelerates the training convergence of PINNs. [ABSTRACT FROM AUTHOR]