Search

We present a novel algorithmic approach and an error analysis leveraging Quasi-Monte Carlo (QMC) points for training deep neural network (DNN) surrogates of holomorphic Data-to-Observable (DtO) maps in engineering design. Our analysis reveals higher-order consistent, deterministic choices of training points in the input parameter space for both deep and shallow neural networks with holomorphic activation functions such as tanh. We prove that higher-order QMC training points facilitate higher-order decay (in terms of the number of training samples) of the underlying generalization error, with consistency error bounds that are free from the curse of dimensionality in terms of the number of input parameters, provided that DNN weights in hidden layers satisfy certain summability conditions. We present numerical experiments for DtO maps from elliptic and parabolic PDEs with uncertain inputs that confirm the theoretical analysis. [ABSTRACT FROM AUTHOR]

We propose a multi-level method to increase the accuracy of machine learning algorithms for approximating observables in scientific computing, particularly those that arise in systems modelled by differential equations. The algorithm relies on judiciously combining a large number of computationally cheap training data on coarse resolutions with a few expensive training samples on fine grid resolutions. Theoretical arguments for lowering the generalisation error, based on reducing the variance of the underlying maps, are provided and numerical evidence, indicating significant gains over underlying single-level machine learning algorithms, are presented. Moreover, we also apply the multi-level algorithm in the context of forward uncertainty quantification and observe a considerable speedup over competing algorithms. [ABSTRACT FROM AUTHOR]

We propose a supervised deep learning algorithm based on low-discrepancy sequences as the training set. By a combination of theoretical arguments and extensive numerical experiments we demonstrate that the proposed algorithm significantly outperforms standard deep learning algorithms that are based on randomly chosen training data for problems in moderately high dimensions. The proposed algorithm provides an efficient method for building inexpensive surrogates for many underlying maps in the context of scientific computing. [ABSTRACT FROM AUTHOR]

The difference between observed and modelled precession/nutation reveals unmodelled signals commonly referred to as Celestial Pole Offsets (CPO), denoted by dX and dY. CPO are currently observed only by Very Long Baseline Interferometry (VLBI), but there is nearly 4 weeks of latency by which the data centers provide the most accurate, final CPO series. This latency problem necessitates predicting CPO for high-accuracy, real-time applications that require information regarding Earth rotation, such as spacecraft navigation. Even though the International Earth Rotation and Reference Systems Service (IERS) provides so-called rapid CPO, they are usually less accurate and therefore, may not satisfy the requirements of the mentioned applications. To enhance the quality of CPO predictions, we present a new methodology based on Neural Additive Models (NAMs), a class of interpretable machine learning algorithms. We formulate the problem based on long short-term memory neural networks and derive simple analytical relations for the quantification of prediction uncertainty and feature importance, thereby enhancing the intelligibility of predictions made by machine learning. We then focus on the short-term prediction of CPO with a forecasting horizon of 30 days. We develop an operational framework that consistently provides CPO predictions. Using the CPO series of Jet Propulsion Laboratory as the input to the algorithm, we show that NAMs predictions improve the IERS rapid products on average by 57% for dX and 25% for dY under fully operational conditions. Our predictions are both accurate and overcome the latency issue of final CPO series and thus, can be used in real-time applications. [ABSTRACT FROM AUTHOR]

• Novel machine learning algorithm based on physics informed neural networks (PINNs) for approximating solutions of forward and inverse problems for radiative transfer. • Fast, robust and accurate approach, independent of the equation dimensionality. • Estimates on the generalization error of PINNs for the unsteady and steady forward problem. • Extensive numerical experiments demonstrating the accuracy and efficiency of the proposed algorithm. We propose a novel machine learning algorithm for simulating radiative transfer. Our algorithm is based on physics informed neural networks (PINNs), which are trained by minimizing the residual of the underlying radiative transfer equations. We present extensive experiments and theoretical error estimates to demonstrate that PINNs provide a very easy to implement, fast, robust and accurate method for simulating radiative transfer. We also present a PINN based algorithm for simulating inverse problems for radiative transfer efficiently. [ABSTRACT FROM AUTHOR]

We present a novel active learning algorithm, termed as iterative surrogate model optimization (ISMO), for robust and efficient numerical approximation of PDE constrained optimization problems. This algorithm is based on deep neural networks and its key feature is the iterative selection of training data through a feedback loop between deep neural networks and any underlying standard optimization algorithm. Numerical examples for optimal control, parameter identification and shape optimization problems for PDEs are provided to demonstrate that ISMO significantly outperforms a standard deep neural network based surrogate optimization algorithm as well as standard optimization algorithms. [ABSTRACT FROM AUTHOR]

Many large scale problems in computational fluid dynamics such as uncertainty quantification, Bayesian inversion, data assimilation and PDE constrained optimization are considered very challenging computationally as they require a large number of expensive (forward) numerical solutions of the corresponding PDEs. We propose a machine learning algorithm, based on deep artificial neural networks, that predicts the underlying input parameters to observable map from a few training samples (computed realizations of this map). By a judicious combination of theoretical arguments and empirical observations, we find suitable network architectures and training hyperparameters that result in robust and efficient neural network approximations of the parameters to observable map. Numerical experiments are presented to demonstrate low prediction errors for the trained network networks, even when the network has been trained with a few samples, at a computational cost which is several orders of magnitude lower than the underlying PDE solver. Moreover, we combine the proposed deep learning algorithm with Monte Carlo (MC) and Quasi-Monte Carlo (QMC) methods to efficiently compute uncertainty propagation for nonlinear PDEs. Under the assumption that the underlying neural networks generalize well, we prove that the deep learning MC and QMC algorithms are guaranteed to be faster than the baseline (quasi-) Monte Carlo methods. Numerical experiments demonstrating one to two orders of magnitude speed up over baseline QMC and MC algorithms, for the intricate problem of computing probability distributions of the observable, are also presented. • The parameters to observable map in CFD can be efficiently approximated by deep neural networks. • Very low prediction errors with neural networks, trained on a few samples. • Proposed deep learning algorithm for UQ provides speedup over (Quasi)-Monte Carlo. • Training on Sobol points led to greater accuracy than on Random points. [ABSTRACT FROM AUTHOR]

Celestial Pole Offsets (CPO), denoted by dX and dY, describe the differences in the observed position of the pole in the celestial frame with respect to a certain precession-nutation model. Precession and nutation components are part of the transformation matrix between terrestrial and celestial systems. Therefore, various applications in geodetic science such as high-precision spacecraft navigation require information regrading precession and nutation. For this purpose, CPO can be added to the precession-nutation model to precisely describe the motion of the celestial pole. However, as Very Long Baseline Interferometry (VLBI)—currently the only technique providing CPO—requires long data processing times resulting in several weeks of latency, predictions of CPO become necessary. Here we present a new methodology named Deep Ensemble Geophysics-Informed Neural Networks (DEGINNs) to provide accurate CPO predictions. The methodology has three main elements: (1) deep ensemble learning to provide the prediction uncertainty; (2) broad-band Liouville equation as a geophysical constraint connecting the rotational dynamics of CPO to the atmospheric and oceanic Effective Angular Momentum (EAM) functions and (3) coupled oscillatory recurrent neural networks to model the sequential characteristics of CPO time-series, also capable of handling irregularly sampled time-series. To test the methodology, we use the newest version of the final CPO time-series of International Earth Rotation and Reference Systems Service (IERS), namely IERS 20 C04. We focus on a forecasting horizon of 90 days, the practical forecasting horizon needed in space-geodetic applications. Furthermore, for validation purposes we generate an independent global VLBI solution for CPO since 1984 up to the end of 2022 and analyse the series. We draw the following conclusions. First, the prediction performance of DEGINNs demonstrates up to 25 and 33 percent improvement, respectively, for dX and dY, with respect to the rapid data provided by IERS. Secondly, predictions made with the help of EAM are more accurate compared to those without EAM, thus providing a clue to the role of atmosphere and ocean on the excitation of CPO. Finally, free core nutation period shows temporal variations with a dominant periodicity of around one year, partially excited by EAM. [ABSTRACT FROM AUTHOR]

Rapid provision of Earth orientation parameters (EOPs, here polar motion and dUT1) is indispensable in many geodetic applications and also for spacecraft navigation. There are, however, discrepancies between the rapid EOPs and the final EOPs that have a higher latency but the highest accuracy. To reduce these discrepancies, we focus on a data‐driven approach, present a novel method named ResLearner, and use it in the context of deep ensemble learning. Furthermore, we introduce a geophysically constrained approach for ResLearner. We show that the most important geophysical information to improve the rapid EOPs is the effective angular momentum functions of atmosphere, ocean, land hydrology, and sea level. In addition, semidiurnal, diurnal, and long‐period tides coupled with prograde and retrograde tidal excitations are important features. The influence of some climatic indices on the prediction accuracy of dUT1 is discussed, and El Niño Southern Oscillation is found to be influential. We developed an operational framework, providing the improved EOPs on a daily basis with a prediction window of 63 days to fully cover the latency of final EOPs. We show that under the operational conditions and using the rapid EOPs of the International Earth Rotation and Reference Systems Service (IERS), we achieve improvements as high as 60%, thus significantly reducing the differences between rapid and final EOPs. Furthermore, we discuss how the new final series IERS 20 C04 is preferred over 14 C04. Finally, we compare against EOP hindcast experiments of the European Space Agency, on which ResLearner presents comparable improvements. Plain Language Summary: The International Earth Rotation and Reference Systems Service (IERS) provides rapid Earth orientation parameters (EOPs) using different space‐geodetic techniques to bridge the latency of the final, most accurate EOPs solution. However, these rapid EOPs are not in full agreement with the final EOPs. In order to reduce the differences between the rapid and final EOPs, we focus on the application of machine learning and present a novel method named ResLearner, which is based on geodetic data and geophysical constraints. We present the method in the context of deep ensemble learning, focusing on a prediction window of 63 days. We also attempt to link informative geophysical effects to these discrepancies. We show that they are linked to a mixture of atmospheric, oceanic, hydrological, and sea level effective angular momentum functions, dominance of the Global Navigation Satellite Systems‐derived polar motion, and various short‐ and long‐term tidal excitations. El Niño Southern Oscillation is also relevant for dUT1 prediction. The methodology can provide significant improvements of up to 60% in operational settings with respect to rapid EOPs provided by IERS. Additional validation is done by using the data of Jet Propulsion Laboratory final EOP series and also EOP series provided by the European Space Agency. Key Points: We introduce a novel machine learning algorithm named ResLearner to improve the accuracy of rapid and predicted Earth orientation parameters (EOPs)We also present geophysically constrained ResLearner, using Earth's effective angular momentum functions, tides, and climatic indicesBesides prediction, ResLearner is also able to effectively correct deficits in rapidly processed EOPs with respect to final EOPs [ABSTRACT FROM AUTHOR]