Search

Inference for spatial extremal dependence models can be computationally burdensome in moderate-to-high dimensions due to their reliance on intractable and/or censored likelihoods. Exploiting recent advances in likelihood-free inference with neural Bayes estimators (that is, neural estimators that target Bayes estimators), we develop a novel approach to construct highly efficient estimators for censored peaks-over-threshold models by encoding censoring information in the neural network architecture. Our new method provides a paradigm shift that challenges traditional censored likelihood-based inference for spatial extremes. Our simulation studies highlight significant gains in both computational and statistical efficiency, relative to competing likelihood-based approaches, when applying our novel estimators for inference of popular extremal dependence models, such as max-stable, $r$-Pareto, and random scale mixture processes. We also illustrate that it is possible to train a single estimator for a general censoring level, obviating the need to retrain when the censoring level is changed. We illustrate the efficacy of our estimators by making fast inference on hundreds-of-thousands of high-dimensional spatial extremal dependence models to assess particulate matter 2.5 microns or less in diameter (PM2.5) concentration over the whole of Saudi Arabia.

Accurate accounting of emissions and removals of CO2 is critical for the planning and verification of emission reduction targets in support of the Paris Agreement. Here, we present a pilot dataset of country-specific net carbon exchange (NCE; fossil plus terrestrial ecosystem fluxes) and terrestrial carbon stock changes aimed at informing countries’ carbon budgets. These estimates are based on “top-down” NCE outputs from the v10 Orbiting Carbon Observatory (OCO-2) modeling intercomparison project (MIP), wherein an ensemble of inverse modeling groups conducted standardized experiments assimilating OCO-2 column-averaged dry-air mole fraction (XCO2 ) retrievals (ACOS v10), in situ CO2 measurements or combinations of these data. The v10 OCO-2 MIP NCE estimates are combined with “bottom-up” estimates of fossil fuel emissions and lateral carbon fluxes to estimate changes in terrestrial carbon stocks, which are impacted by anthropogenic and natural drivers. These flux and stock change estimates are reported annually (2015–2020) as both a global 1◦ × 1 ◦ gridded dataset and a country-level dataset and are available for download from the Committee on Earth Observation Satellites’ (CEOS) website: https://doi.org/10.48588/npf6-sw92 (Byrne et al., 2022). Across the v10 OCO-2 MIP experiments, we obtain increases in the ensemble median terrestrial carbon stocks of 3.29–4.58 PgCO2 yr−1 (0.90–1.25 PgC yr−1 ). This is a result of broad increases in terrestrial carbon stocks across the northern extratropics, while the tropics generally have stock losses but with considerable regional variability and differences between v10 OCO-2 MIP experiments. We discuss the state of the science for tracking emissions and removals using top-down methods, including current limitations and future developments towards top-down monitoring and verification systems. This research has been supported by the European Commission, Horizon 2020 Framework Programme (CoCO2 (grant no. 958927 856612/EMME-CARE)) and Copernicus Atmosphere Monitoring Service (grant no. CAMS73), the Australian Research Council (grant nos. DP190100180, DE180100203, DP160100598, LE0668470, DP140101552, DP110103118, DP0879468 and FT180100327), the Environmental Restoration and Conservation Agency (grant no. JPMEERF21S20800), the Korea Meteorological Administration (grant no. KMA2018-00320), the National Aeronautics and Space Administration (grant nos. 20-OCOST20-0004, 80NSSC18K0908, 80NSSC18K0976, 80NSSC20K0006, 80NSSC21K1068, 80NSSC21K1073, 80NSSC21K1077, 80NSSC21K1080, 80HQTR21T0069, NAG512247, NNG05GD07G, NNH17ZDA001N-OCO2 and NNX15AG93G), and the National Oceanic and Atmospheric Administration (grant no. NA18OAR4310266).

Normalizing flows are objects used for modeling complicated probability density functions, and have attracted considerable interest in recent years. Many flexible families of normalizing flows have been developed. However, the focus to date has largely been on normalizing flows on Euclidean domains; while normalizing flows have been developed for spherical and other non-Euclidean domains, these are generally less flexible than their Euclidean counterparts. To address this shortcoming, in this work we introduce a mixture-of-normalizing-flows model to construct complicated probability density functions on the sphere. This model provides a flexible alternative to existing parametric, semiparametric, and nonparametric, finite mixture models. Model estimation is performed using the expectation maximization algorithm and a variant thereof. The model is applied to simulated data, where the benefit over the conventional (single component) normalizing flow is verified. The model is then applied to two real-world data sets of events occurring on the surface of Earth; the first relating to earthquakes, and the second to terrorist activity. In both cases, we see that the mixture-of-normalizing-flows model yields a good representation of the density of event occurrence.

Models with random effects, such as generalised linear mixed models (GLMMs), are often used for analysing clustered data. Parameter inference with these models is difficult because of the presence of cluster-specific random effects, which must be integrated out when evaluating the likelihood function. Here, we propose a sequential variational Bayes algorithm, called Recursive Variational Gaussian Approximation for Latent variable models (R-VGAL), for estimating parameters in GLMMs. The R-VGAL algorithm operates on the data sequentially, requires only a single pass through the data, and can provide parameter updates as new data are collected without the need of re-processing the previous data. At each update, the R-VGAL algorithm requires the gradient and Hessian of a "partial" log-likelihood function evaluated at the new observation, which are generally not available in closed form for GLMMs. To circumvent this issue, we propose using an importance-sampling-based approach for estimating the gradient and Hessian via Fisher's and Louis' identities. We find that R-VGAL can be unstable when traversing the first few data points, but that this issue can be mitigated by using a variant of variational tempering in the initial steps of the algorithm. Through illustrations on both simulated and real datasets, we show that R-VGAL provides good approximations to the exact posterior distributions, that it can be made robust through tempering, and that it is computationally efficient.

Neural point estimators are neural networks that map data to parameter point estimates. They are fast, likelihood free and, due to their amortised nature, amenable to fast bootstrap-based uncertainty quantification. In this paper, we aim to increase the awareness of statisticians to this relatively new inferential tool, and to facilitate its adoption by providing user-friendly open-source software. We also give attention to the ubiquitous problem of making inference from replicated data, which we address in the neural setting using permutation-invariant neural networks. Through extensive simulation studies we show that these neural point estimators can quickly and optimally (in a Bayes sense) estimate parameters in weakly-identified and highly-parameterised models with relative ease. We demonstrate their applicability through an analysis of extreme sea-surface temperature in the Red Sea where, after training, we obtain parameter estimates and bootstrap-based confidence intervals from hundreds of spatial fields in a fraction of a second.

Deep neural network models have become ubiquitous in recent years, and have been applied to nearly all areas of science, engineering, and industry. These models are particularly useful for data that have strong dependencies in space (e.g., images) and time (e.g., sequences). Indeed, deep models have also been extensively used by the statistical community to model spatial and spatio-temporal data through, for example, the use of multi-level Bayesian hierarchical models and deep Gaussian processes. In this review, we first present an overview of traditional statistical and machine learning perspectives for modeling spatial and spatio-temporal data, and then focus on a variety of hybrid models that have recently been developed for latent process, data, and parameter specifications. These hybrid models integrate statistical modeling ideas with deep neural network models in order to take advantage of the strengths of each modeling paradigm. We conclude by giving an overview of computational technologies that have proven useful for these hybrid models, and with a brief discussion on future research directions., Comment: 27 pages, 1 figure

Flux inversion is the process by which sources and sinks of a gas are identified from observations of gas mole fraction. The inversion often involves running a Lagrangian particle dispersion model (LPDM) to generate sensitivities between observations and fluxes over a spatial domain of interest. The LPDM must be run backward in time for every gas measurement, and this can be computationally prohibitive. To address this problem, here we develop a novel spatio-temporal emulator for LPDM sensitivities that is built using a convolutional variational autoencoder (CVAE). With the encoder segment of the CVAE, we obtain approximate (variational) posterior distributions over latent variables in a low-dimensional space. We then use a spatio-temporal Gaussian process emulator on the low-dimensional space to emulate new variables at prediction locations and time points. Emulated variables are then passed through the decoder segment of the CVAE to yield emulated sensitivities. We show that our CVAE-based emulator outperforms the more traditional emulator built using empirical orthogonal functions and that it can be used with different LPDMs. We conclude that our emulation-based approach can be used to reliably reduce the computing time needed to generate LPDM outputs for use in high-resolution flux inversions., 25 pages, 8 figures, 2 tables, data & code available

Non-Gaussian spatial and spatio-temporal data are becoming increasingly prevalent, and their analysis is needed in a variety of disciplines. FRK is an R package for spatial/spatio-temporal modelling and prediction with very large data sets that, to date, has only supported linear process models and Gaussian data models. In this paper, we describe a major upgrade to FRK that allows for non-Gaussian data to be analysed in a generalised linear mixed model framework. These vastly more general spatial and spatio-temporal models are fitted using the Laplace approximation via the software TMB. The existing functionality of FRK is retained with this advance into non-Gaussian models; in particular, it allows for automatic basis-function construction, it can handle both point-referenced and areal data simultaneously, and it can predict process values at any spatial support from these data. This new version of FRK also allows for the use of a large number of basis functions when modelling the spatial process, and is thus often able to achieve more accurate predictions than previous versions of the package in a Gaussian setting. We demonstrate innovative features in this new version of FRK, highlight its ease of use, and compare it to alternative packages using both simulated and real data sets., 38 pages, 15 figures

Markov chain Monte Carlo methods for exponential family models with intractable normalizing constant, such as the exchange algorithm, require simulations of the sufficient statistics at every iteration of the Markov chain, which often result in expensive computations. Surrogate models for the likelihood function have been developed to accelerate inference algorithms in this context. However, these surrogate models tend to be relatively inflexible, and often provide a poor approximation to the true likelihood function. In this article, we propose the use of a warped, gradient-enhanced, Gaussian process surrogate model for the likelihood function, which jointly models the sample means and variances of the sufficient statistics, and uses warping functions to capture covariance nonstationarity in the input parameter space. We show that both the consideration of nonstationarity and the inclusion of gradient information can be leveraged to obtain a surrogate model that outperforms the conventional stationary Gaussian process surrogate model when making inference, particularly in regions where the likelihood function exhibits a phase transition. We also show that the proposed surrogate model can be used to improve the effective sample size per unit time when embedded in exact inferential algorithms. The utility of our approach in speeding up inferential algorithms is demonstrated on simulated and real-world data.

WOMBAT (the WOllongong Methodology for Bayesian Assimilation of Trace-gases) is a fully Bayesian hierarchical statistical framework for flux inversion of trace gases from flask, in situ, and remotely sensed data. WOMBAT extends the conventional Bayesian-synthesis framework through the consideration of a correlated error term, the capacity for online bias correction, and the provision of uncertainty quantification on all unknowns that appear in the Bayesian statistical model. We show, in an observing system simulation experiment (OSSE), that these extensions are crucial when the data are indeed biased and have errors that are correlated. Using the GEOS-Chem atmospheric transport model, we show that WOMBAT is able to obtain posterior means and uncertainties on non-fossil-fuel CO$_2$ fluxes from Orbiting Carbon Observatory-2 (OCO-2) data that are comparable to those from the Model Intercomparison Project (MIP) reported in Crowell et al. (2019, Atmos. Chem. Phys., vol. 19). We also find that our predictions of out-of-sample retrievals from the Total Column Carbon Observing Network are, for the most part, more accurate than those made by the MIP participants. Subsequent versions of the OCO-2 datasets will be ingested into WOMBAT as they become available., 46 pages, 13 figures

Bayesian methods for modelling and inference are being increasingly used in the cryospheric sciences, and glaciology in particular. Here, we present a review of recent works in glaciology that adopt a Bayesian approach when conducting an analysis. We organise the chapter into three categories: i) Gaussian-Gaussian models, ii) Bayesian hierarchical models, and iii) Bayesian calibration approaches. In addition, we present two detailed case studies that involve the application of Bayesian hierarchical models in glaciology. The first case study is on the spatial prediction of surface mass balance across the Icelandic mountain glacier Langj\"okull, and the second is on the prediction of sea-level rise contributions from the Antactcic ice sheet. This chapter is presented in such a way that it is accessible to both statisticians as well as earth scientists., Comment: This work will appear as a chapter of the forthcoming book titled Statistical Modelling Using Latent Gaussian Models - with Applications in Geophysics and Environmental Sciences, expected to be published by Springer in 2022

Non-homogeneous Poisson processes are used in a wide range of scientific disciplines, ranging from the environmental sciences to the health sciences. Often, the central object of interest in a point process is the underlying intensity function. Here, we present a general model for the intensity function of a non-homogeneous Poisson process using measure transport. The model is built from a flexible bijective mapping that maps from the underlying intensity function of interest to a simpler reference intensity function. We enforce bijectivity by modeling the map as a composition of multiple simple bijective maps, and show that the model exhibits an important approximation property. Estimation of the flexible mapping is accomplished within an optimization framework, wherein computations are efficiently done using recent technological advances in deep learning and a graphics processing unit. Although we find that intensity function estimates obtained with our method are not necessarily superior to those obtained using conventional methods, the modeling representation brings with it other advantages such as facilitated point process simulation and uncertainty quantification. Modeling point processes in higher dimensions is also facilitated using our approach. We illustrate the use of our model on both simulated data, and a real data set containing the locations of seismic events near Fiji since 1964.

Poster atEGU 2019, Vienna, Austria, 7-12 April 2019

Presentation atIGS British Branch Meeting 2018, Exeter, 4 September 2018

Poster atEGU 2018, Vienna, Austria, 8-13 April 2018

Presentation atEGU 2018, Vienna, Austria, 8-13 April 2018

Poster presented at the Regional Sea Level Changes and Coastal Impacts conference, New-York, 10-14 July 2017

The Gaussian process is an indispensable tool for spatial data analysts. The onset of the "big data" era, however, has lead to the traditional Gaussian process being computationally infeasible for modern spatial data. As such, various alternatives to the full Gaussian process that are more amenable to handling big spatial data have been proposed. These modern methods often exploit low rank structures and/or multi-core and multi-threaded computing environments to facilitate computation. This study provides, first, an introductory overview of several methods for analyzing large spatial data. Second, this study describes the results of a predictive competition among the described methods as implemented by different groups with strong expertise in the methodology. Specifically, each research group was provided with two training datasets (one simulated and one observed) along with a set of prediction locations. Each group then wrote their own implementation of their method to produce predictions at the given location and each which was subsequently run on a common computing environment. The methods were then compared in terms of various predictive diagnostics. Supplementary materials regarding implementation details of the methods and code are available for this article online.

Presentation at PALSEA2 2016 Workshop: Sea-level budgets at decadal to millennial time scales to bridge the paleo and instrumental records, Oregon, 19-21 September 2016

Antarctica is the world's largest fresh-water reservoir, with the potential to raise sea levels by about 60 m. An ice sheet contributes to sea-level rise (SLR) when its rate of ice discharge and/or surface melting exceeds accumulation through snowfall. Constraining the contribution of the ice sheets to present-day SLR is vital both for coastal development and planning, and climate projections. Information on various ice sheet processes is available from several remote sensing data sets, as well as in situ data such as global positioning system data. These data have differing coverage, spatial support, temporal sampling and sensing characteristics, and thus, it is advantageous to combine them all in a single framework for estimation of the SLR contribution and the assessment of processes controlling mass exchange with the ocean. In this paper, we predict the rate of height change due to salient geophysical processes in Antarctica and use these to provide estimates of SLR contribution with associated uncertainties. We employ a multivariate spatio-temporal model, approximated as a Gaussian Markov random field, to take advantage of differing spatio-temporal properties of the processes to separate the causes of the observed change. The process parameters are estimated from geophysical models, while the remaining parameters are estimated using a Markov chain Monte Carlo scheme, designed to operate in a high-performance computing environment across multiple nodes. We validate our methods against a separate data set and compare the results to those from studies that invariably employ numerical model outputs directly. We conclude that it is possible, and insightful, to assess Antarctica's contribution without explicit use of numerical models. Further, the results obtained here can be used to test the geophysical numerical models for which in situ data are hard to obtain. © 2015 The Authors. Environmetrics published by John Wiley & Sons Ltd.

Multivariate geostatistics is based on modelling all covariances between all possible combinations of two or more variables at any sets of locations in a continuously indexed domain. Multivariate spatial covariance models need to be built with care, since any covariance matrix that is derived from such a model must be nonnegative-definite. In this article, we develop a conditional approach for spatial-model construction whose validity conditions are easy to check. We start with bivariate spatial covariance models and go on to demonstrate the approach's connection to multivariate models defined by networks of spatial variables. In some circumstances, such as modelling respiratory illness conditional on air pollution, the direction of conditional dependence is clear. When it is not, the two directional models can be compared. More generally, the graph structure of the network reduces the number of possible models to compare. Model selection then amounts to finding possible causative links in the network. We demonstrate our conditional approach on surface temperature and pressure data, where the role of the two variables is seen to be asymmetric., Comment: 22 pages, 3 figures

Dr Andrew Zammit Mangion writes about his career: from CERN to winning awards for research based on data released by WikiLeaks. http://www.um.edu.mt/think/exploring-the-world-of-big-data/, N/A

Analysis of spatio-temporal point patterns plays an important role in several disciplines, yet inference in these systems remains computationally challenging due to the high resolution modelling generally required by large data sets and the analytically intractable likelihood function. Here, we exploit the sparsity structure of a fully-discretised log-Gaussian Cox process model by using expectation constrained approximate inference. The resulting family of expectation propagation algorithms scale well with the state dimension and the length of the temporal horizon with moderate loss in distributional accuracy. They hence provide a flexible and faster alternative to both the filtering-smoothing type algorithms and the approaches which implement the Laplace method or expectation propagation on (block) sparse latent Gaussian models. We demonstrate the use of the proposed method in the reconstruction of conflict intensity levels in Afghanistan from a WikiLeaks data set.

Haptics refers to a widespread area of research that focuses on the interaction between humans and machine interfaces as applied to the sense of touch. A haptic interface is designed to increase the realism of tactile and kinesthetic sensations in applications such as virtual reality, teleoperation, and other scenarios where situational awareness is considered important, if not vital. This paper investigates the use of electric actuators and non-linear algorithms to provide force feedback to an input command device for providing haptics to the human operator. In particular, this work involves the study and implementation of a special case of feedback linearization known as inverse dynamics control and several outer loop impedance control topologies. It also investigates the issues concerned with force sensing and the application of model based controller functions in order to vary the desired inertia and the desired mass matrix. Results of the controllers’ abilities to display any desired impedance and provide the required kinesthetic constraint of virtual environments are shown on two experimental test rigs designed for this purpose., peer-reviewed