Search

Since model bias and associated initialization shock are serious shortcomings that reduce prediction skills in state-of-the-art decadal climate prediction efforts, we pursue a complementary machine-learning-based approach to climate prediction. The example problem setting we consider consists of predicting natural variability of the North Atlantic sea surface temperature on the interannual timescale in the pre-industrial control simulation of the Community Earth System Model (CESM2). While previous works have considered the use of recurrent networks such as convolutional LSTMs and reservoir computing networks in this and other similar problem settings, we currently focus on the use of feedforward convolutional networks. In particular, we find that a feedforward convolutional network with a Densenet architecture is able to outperform a convolutional LSTM in terms of predictive skill. Next, we go on to consider a probabilistic formulation of the same network based on Stein variational gradient descent and find that in addition to providing useful measures of predictive uncertainty, the probabilistic (Bayesian) version improves on its deterministic counterpart in terms of predictive skill. Finally, we characterize the reliability of the ensemble of ML models obtained in the probabilistic setting by using analysis tools developed in the context of ensemble numerical weather prediction., Comment: 32 pages, 12 figures

In this paper we demonstrate that reservoir computing can be used to learn the dynamics of the shallow-water equations. In particular, while most previous applications of reservoir computing have required training on a particular trajectory to further predict the evolution along that trajectory alone, we show the capability of reservoir computing to predict trajectories of the shallow-water equations with initial conditions not seen in the training process. However, in this setting, we find that the performance of the network deteriorates for initial conditions with ambient conditions (such as total water height and average velocity) that are different from those in the training dataset. To circumvent this deficiency, we introduce a transfer learning approach wherein a small additional training step with the relevant ambient conditions is used to improve the predictions.

Radiography is often used to probe complex, evolving density fields in dynamic systems and in so doing gain insight into the underlying physics. This technique has been used in numerous fields including materials science, shock physics, inertial confinement fusion, and other national security applications. In many of these applications, however, complications resulting from noise, scatter, complex beam dynamics, etc. prevent the reconstruction of density from being accurate enough to identify the underlying physics with sufficient confidence. As such, density reconstruction from static/dynamic radiography has typically been limited to identifying discontinuous features such as cracks and voids in a number of these applications. In this work, we propose a fundamentally new approach to reconstructing density from a temporal sequence of radiographic images. Using only the robust features identifiable in radiographs, we combine them with the underlying hydrodynamic equations of motion using a machine learning approach, namely, conditional generative adversarial networks (cGAN), to determine the density fields from a dynamic sequence of radiographs. Next, we seek to further enhance the hydrodynamic consistency of the ML-based density reconstruction through a process of parameter estimation and projection onto a hydrodynamic manifold. In this context, we note that the distance from the hydrodynamic manifold given by the training data to the test data in the parameter space considered both serves as a diagnostic of the robustness of the predictions and serves to augment the training database, with the expectation that the latter will further reduce future density reconstruction errors. Finally, we demonstrate the ability of this method to outperform a traditional radiographic reconstruction in capturing allowable hydrodynamic paths even when relatively small amounts of scatter are present., Comment: Submitted to Optics Express

We present a study using a class of post-hoc local explanation methods i.e., feature importance methods for "understanding" a deep learning (DL) emulator of climate. Specifically, we consider a multiple-input-single-output emulator that uses a DenseNet encoder-decoder architecture and is trained to predict interannual variations of sea surface temperature (SST) at 1, 6, and 9 month lead times using the preceding 36 months of (appropriately filtered) SST data. First, feature importance methods are employed for individual predictions to spatio-temporally identify input features that are important for model prediction at chosen geographical regions and chosen prediction lead times. In a second step, we also examine the behavior of feature importance in a generalized sense by considering an aggregation of the importance heatmaps over training samples. We find that: 1) the climate emulator's prediction at any geographical location depends dominantly on a small neighborhood around it; 2) the longer the prediction lead time, the further back the "importance" extends; and 3) to leading order, the temporal decay of "importance" is independent of geographical location. An ablation experiment is adopted to verify the findings. From the perspective of climate dynamics, these findings suggest a dominant role for local processes and a negligible role for remote teleconnections at the spatial and temporal scales we consider. From the perspective of network architecture, the spatio-temporal relations between the inputs and outputs we find suggest potential model refinements. We discuss further extensions of our methods, some of which we are considering in ongoing work.

We show that in addition to providing effective and competitive closures, when analysed in terms of dynamics and physically-relevant diagnostics, artificial neural networks (ANNs) can be both interpretable and provide useful insights in the on-going task of developing and improving turbulence closures. In the context of large-eddy simulations (LES) of a passive scalar in homogeneous isotropic turbulence, exact subfilter fluxes obtained by filtering direct numerical simulations (DNS) are used both to train deep ANN models as a function of filtered variables, and to optimise the coefficients of a turbulent Prandtl number LES closure. \textit{A-priori} analysis of the subfilter scalar variance transfer rate demonstrates that learnt ANN models out-perform optimised turbulent Prandtl number closures and Clark-type gradient models. Next, \textit{a-posteriori} solutions are obtained with each model over several integral timescales. These experiments reveal, with single- and multi-point diagnostics, that ANN models temporally track exact resolved scalar variance with greater accuracy compared to other subfilter flux models for a given filter length scale. Finally, we interpret the artificial neural networks statistically with differential sensitivity analysis to show that the ANN models feature dynamics reminiscent of so-called "mixed models", where mixed models are understood as comprising both a structural and functional component. Besides enabling enhanced-accuracy LES of passive scalars henceforth, we anticipate this work to contribute to utilising neural network models as a tool in interpretability, robustness and model discovery.

Fluid turbulence is characterized by strong coupling across a broad range of scales. Furthermore, besides the usual local cascades, such coupling may extend to interactions that are non-local in scale-space. As such the computational demands associated with explicitly resolving the full set of scales and their interactions, as in the Direct Numerical Simulation (DNS) of the Navier-Stokes equations, in most problems of practical interest are so high that reduced modeling of scales and interactions is required before further progress can be made. While popular reduced models are typically based on phenomenological modeling of relevant turbulent processes, recent advances in machine learning techniques have energized efforts to further improve the accuracy of such reduced models. In contrast to such efforts that seek to improve an existing turbulence model, we propose a machine learning(ML) methodology that captures, de novo, underlying turbulence phenomenology without a pre-specified model form. To illustrate the approach, we consider transient modeling of the dissipation of turbulent kinetic energy, a fundamental turbulent process that is central to a wide range of turbulence models using a Neural ODE approach. After presenting details of the methodology, we show that this approach outperforms state-of-the-art approaches.

We focus on improving the accuracy of an approximate model of a multiscale dynamical system that uses a set of parameter-dependent terms to account for the effects of unresolved or neglected dynamics on resolved scales. We start by considering various methods of calibrating and analyzing such a model given a few well-resolved simulations. After presenting results for various point estimates and discussing some of their shortcomings, we demonstrate (a) the potential of hierarchical Bayesian analysis to uncover previously unanticipated physical dependencies in the approximate model, and (b) how such insights can then be used to improve the model. In effect parametric dependencies found from the Bayesian analysis are used to improve structural aspects of the model. While we choose to illustrate the procedure in the context of a closure model for buoyancy-driven, variable-density turbulence, the statistical nature of the approach makes it more generally applicable. Towards addressing issues of increased computational cost associated with the procedure, we demonstrate the use of a neural network based surrogate in accelerating the posterior sampling process and point to recent developments in variational inference as an alternative methodology for greatly mitigating such costs. We conclude by suggesting that modern validation and uncertainty quantification techniques such as the ones we consider have a valuable role to play in the development and improvement of approximate models., Comment: 22 pages with 7 figures

Anomaly-diffusing energy balance models (AD-EBM) are routinely employed to analyze and emulate the warming response of both observed and simulated Earth systems. We demonstrate a deficiency in common multi-layer as well as continuous-diffusion AD-EBM variants: They are unable to, simultaneously, properly represent surface warming and the vertical distribution of heat uptake. We show that this inability is due to the diffusion approximation. On the other hand, it is well understood that transport of water from the surface mixed layer into the ocean interior is achieved, in large part, by the process of ventilation---a process associated with outcropping isopycnals. We, therefore, start from a configuration of outcropping isopycnals and demonstrate how an AD-EBM can be modified to include the effect of ventilation on ocean uptake of anomalous radiative forcing. The resulting EBM is able to successfully represent both surface warming and the vertical distribution of heat uptake, and indeed, a simple four layer model suffices. The simplicity of the models notwithstanding, the analysis presented and the necessity of the modification highlight the role played by processes related to the down-welling branch of global ocean circulation in shaping the vertical distribution of ocean heat uptake., Comment: 26 pages with 17 figures. Climatic Change (2018)

This paper focuses on the problem of quantifying the effects of model-structure uncertainty in the context of time-evolving dynamical systems. This is motivated by multi-model uncertainty in computer physics simulations: developers often make different modeling choices in numerical approximations and process simplifications, leading to different numerical codes that ostensibly represent the same underlying dynamics. We consider model-structure inference as a two-step methodology: the first step is to perform system identification on numerical codes for which it is possible to observe the full state; the second step is structural uncertainty quantification (UQ), in which the goal is to search candidate models "close" to the numerical code surrogates for those that best match a quantity-of-interest (QOI) from some empirical dataset. Specifically, we: (1) define a discrete, local representation of the structure of a partial differential equation, which we refer to as the "local dynamical operator" (LDO); (2) identify model structure non-intrusively from numerical code output; (3) non-intrusively construct a reduced order model (ROM) of the numerical model through POD-DEIM-Galerkin projection; (4) perturb the ROM dynamics to approximate the behavior of alternate model structures; and (5) apply Bayesian inference and energy conservation laws to calibrate a LDO to a given QOI. We demonstrate these techniques using the two-dimensional rotating shallow water (RSW) equations as an example system., Comment: 30 pages, 14 figures

We consider the nonlinear evolution of an unstable baroclinic wave in a regime of rotating stratified flow that is of relevance to interior circulation in the oceans and in the atmosphere---a regime characterized by small large-scale Rossby and Froude numbers, a small vertical to horizontal aspect ratio, and no bounding horizontal surfaces. Using high-resolution simulations of the non-hydrostatic Boussinesq equations and companion integrations of the balanced quasi-geostrophic equations, we present evidence for a local route to dissipation of balanced energy directly through interior turbulent cascades. Analysis of simulations presented in this study suggest that a developing baroclinic instability can lead to secondary instabilities that can cascade a small fraction of the energy forward to unbalanced scales. Mesoscale shear and strain resulting from the hydrostatic geostrophic baroclinic instability drive frontogenesis. The fronts in turn support ageostrophic secondary circulation and instabilities. These two processes acting together lead to a quick rise in dissipation rate which then reaches a peak and begins to fall slowly when frontogenesis slows down; eventually balanced and imbalanced modes decouple. Dissipation of balanced energy by imbalanced processes scales exponentially with Rossby number of the base flow. We expect that this scaling will hold more generally than for the specific setup we consider given the fundamental nature of the dynamics involved. A break is seen in the total energy spectrum at small scales: While a steep $k^{-3}$ geostrophic scaling (where $k$ is the three-dimensional wavenumber) is seen at intermediate scales, the smaller scales display a shallower $k^{-5/3}$ scaling, reminiscent of the atmospheric spectra of Nastrom & Gage. At higher Ro, the vertical shear spectrum has a minimum, like in some relevant obervations

In the Large Eddy Simulation (LES) framework for modeling a turbulent flow, when the large scale velocity field is defined by low-pass filtering the full velocity field, a Taylor series expansion of the full velocity field in terms of the large scale velocity field leads (at the leading order) to the nonlinear gradient model for the subfilter stresses. Motivated by the fact that while the nonlinear gradient model shows excellent a priori agreement in resolved simulations, the use of this model by itself is problematic, we consider two models that are related, but better behaved: The Rational LES model that uses a sub-diagonal Pade approximation instead of a Taylor series expansion and the Lagrangian Averaged Navier-Stokes-{\alpha} model that uses a regulariza- tion approach to modeling turbulence. In this article, we show that these two latter models are identical in two dimensions.

Recently, stochastic, as opposed to deterministic, parameterizations are being investigated to model the effects of unresolved subgrid scales (SGS) in large eddy simulations (LES) of geophysical flows. We analyse such a stochastic approach in the barotropic vorticity equation to show that (i) if the stochastic parameterization approximates the actual SGS stresses, then the solution of the stochastic LES approximates the "true" solution at appropriate scale sizes; and that (ii) when the filter scale size approaches zero, the solution of the stochastic LES approaches the true solution., Comment: 3 figures, 1 class

An inviscid two-dimensional fluid model with nonlinear dispersion that arises simultaneously in coarse-grained descriptions of the dynamics of the Euler equation and in the description of non-Newtonian fluids of second grade is considered. The scaling of the equilibrium states of this model for conserved energy and enstrophy retains the corresponding scaling for the Euler equations on the large scales and at the same time greatly deemphasizes the importance of small scales. This is the first clear demonstration of the beneficial effect of nonlinear dispersion in the model, and should highlight its utility as a subgrid model in more realistic situations., Comment: 6 pages, 3 figures