1. Comparison of neural closure models for discretised PDEs
- Author
-
Hugo Melchers, Daan Crommelin, Barry Koren, Vlado Menkovski, Benjamin Sanderse, Centrum Wiskunde & Informatica, Amsterdam (CWI), The Netherlands, Scientific Computing, EIRES, EAISI High Tech Systems, ICMS Affiliated, EAISI Health, and Data Mining
- Subjects
FOS: Computer and information sciences ,Computer Science - Machine Learning ,I.2.6 ,G.1.7 ,G.1.8 ,Closure model ,Numerical Analysis (math.NA) ,Neural ODE ,Partial differential equations ,Machine Learning (cs.LG) ,68T07 (Primary), 65M22 (Secondary) ,Computational Mathematics ,Multiscale modelling ,Computational Theory and Mathematics ,Modeling and Simulation ,FOS: Mathematics ,Mathematics - Numerical Analysis ,Neural networks ,Ordinary differential equations - Abstract
Neural closure models have recently been proposed as a method for efficiently approximating small scales in multiscale systems with neural networks. The choice of loss function and associated training procedure has a large effect on the accuracy and stability of the resulting neural closure model. In this work, we systematically compare three distinct procedures: "derivative fitting", "trajectory fitting" with discretise-then-optimise, and "trajectory fitting" with optimise-then-discretise. Derivative fitting is conceptually the simplest and computationally the most efficient approach and is found to perform reasonably well on one of the test problems (Kuramoto-Sivashinsky) but poorly on the other (Burgers). Trajectory fitting is computationally more expensive but is more robust and is therefore the preferred approach. Of the two trajectory fitting procedures, the discretise-then-optimise approach produces more accurate models than the optimise-then-discretise approach. While the optimise-then-discretise approach can still produce accurate models, care must be taken in choosing the length of the trajectories used for training, in order to train the models on long-term behaviour while still producing reasonably accurate gradients during training. Two existing theorems are interpreted in a novel way that gives insight into the long-term accuracy of a neural closure model based on how accurate it is in the short term., Comment: 24 pages and 9 figures. Submitted to Computers and Mathematics with Applications. For associated code, see https://github.com/HugoMelchers/neural-closure-models
- Published
- 2023