Search

Statistical regression methods are widely used in remote sensing applications but tend to lack physical interpretability. In this article, we introduce a methodological framework to improve model emulation and its understanding with machine learning feature selection. Our wrapper-forward feature selection method seamlessly integrates physics knowledge into model emulation, improving the tradeoff between accuracy and interpretability. We illustrate our methodology by applying it to atmospheric radiative transfer models (RTMs) in the context of global sensitivity analysis (GSA) and emulation. Our approach consistently aligns with variance-based GSA, pinpointing the critical features of aerosol properties, solar zenith angle, and water vapor. While our physically based emulators yield only a modest accuracy improvement of 0.2% over conventional Gaussian process (GP) emulators, its introduction signifies a step forward to physics-aware machine learning-based emulation. The emulator performance remains steadfast, unaffected by substantial changes, further underscoring the reliability of our approach.

Feature selection (FS) is essential in various fields of science and engineering, from remote sensing to computer vision. Reducing data dimensionality by removing redundant features and selecting the most informative ones improves machine learning algorithms’ performance, especially in supervised classification tasks, while lowering storage needs. Graph-embedding (GE) techniques have recently been found efficient for FS since they preserve the geometric structure of the original feature space while embedding data into a low-dimensional subspace. However, the main drawback is the high computational cost of solving an eigenvalue decomposition problem, especially for large-scale problems. This article addresses this issue by combining the GE framework and representation theory for a novel FS method. Inspired by the high-dimensional model representation (HDMR), the feature transformation is assumed to be a linear combination of a set of univariate orthogonal functions carried out in the GE framework. As a result, an explicit embedding function is created, which can be utilized to embed out-of-samples into low-dimensional space and provide a feature relevance score. The significant contribution of the proposed method is to divide an $n$ -dimensional generalized eigenvalue problem into $n$ small-sized eigenvalue problems. With this property, the computational complexity (CC) of the GE is significantly reduced, resulting in a scalable FS method, which could be easily parallelized too. The performance of the proposed method is compared favorably to its counterparts in high-dimensional hyperspectral image (HSI) processing in terms of classification accuracy, feature stability, and computational time.

Clouds play a key role in regulating climate change but are difficult to simulate within Earth system models (ESMs). Improving the representation of clouds is one of the key tasks toward more robust climate change projections. This study introduces a new machine-learning-based framework relying on satellite observations to improve understanding of the representation of clouds and their relevant processes in climate models. The proposed method is capable of assigning distributions of established cloud types to coarse data. It facilitates a more objective evaluation of clouds in ESMs and improves the consistency of cloud process analysis. The method is built on satellite data from the Moderate Resolution Imaging Spectroradiometer (MODIS) instrument labeled by deep neural networks with cloud types defined by the World Meteorological Organization (WMO), using cloud-type labels from CloudSat as ground truth. The method is applicable to datasets with information about physical cloud variables comparable to MODIS satellite data and at sufficiently high temporal resolution. We apply the method to alternative satellite data from the Cloud_cci project (ESA Climate Change Initiative), coarse-grained to typical resolutions of climate models. The resulting cloud-type distributions are physically consistent and the horizontal resolutions typical of ESMs are sufficient to apply our method. We recommend outputting crucial variables required by our method for future ESM data evaluation. This will enable the use of labeled satellite data for a more systematic evaluation of clouds in climate models.

Atmospheric radiative transfer models (RTMs) are widely used in satellite data processing to correct for the scattering and absorption effects caused by aerosols and gas molecules in the Earth’s atmosphere. As the complexity of RTMs grows and the requirements for future Earth Observation missions become more demanding, the conventional lookup-table (LUT) interpolation approach faces important challenges. Emulators have been suggested as an alternative to LUT interpolation, but they are still too slow for operational satellite data processing. Our research introduces a solution that harnesses the power of multifidelity methods to improve the accuracy and runtime of Gaussian process (GP) emulators. We investigate the impact of the number of fidelity layers, dimensionality reduction, and training dataset size on the performance of multifidelity GP emulators. We find that an optimal multifidelity emulator can achieve relative errors in surface reflectance below 0.5% and performs atmospheric correction of hyperspectral PRISMA satellite data (one million pixels) in a few minutes. Additionally, we provide a suite of functions and tools for automating the creation and generation of atmospheric RTM emulators.

Interannual variations in ecosystem primary productivity are dominated by water availability. Until recently, characterizing the photosynthetic response of different ecosystems to soil moisture anomalies was hampered by observational limitations. Here, we use a number of satellite‐based proxies for productivity, including spectral indices, sun‐induced chlorophyll fluorescence, and data‐driven estimates of gross primary production, to reevaluate the relationship between terrestrial photosynthesis and water. In contrast to nonwoody vegetation, we find a resilience of forested ecosystems to reduced soil moisture. Sun‐induced chlorophyll fluorescence and data‐driven gross primary production indicate an increase in photosynthesis as a result of the accompanying higher amounts of light and temperature despite lowered light‐use‐efficiency. Conversely, remote sensing indicators of greenness reach their detection limit and largely remain stable. Our study thus highlights the differential responses of ecosystems along a tree cover gradient and illustrates the importance of differentiating photosynthesis indicators from those of greenness for the monitoring and understanding of ecosystems. The capacity of vegetation to thrive and to sequester carbon depends on how much water they can have access to. In this work, we evaluate how different types of satellite observations can describe the response of vegetation to changes in soil moisture over the entire planet. The first source of observation measures only the greenness of the land surface, the second measures light that is emitted by pigments in plants which are photosynthetically active (chlorophyll fluorescence), and the third are simulations of gross carbon uptake derived from machine learning techniques. For periods of water shortage all three indicate a reduction of growth in ecosystems with few trees. However, in cold boreal forests, when soil moisture is particularly low, we still detect an increase in photosynthesis due to higher light and temperature conditions, but this is not reflected in the greenness indicator. This work illustrates how lack of water is not necessarily harmful for catching carbon through photosynthesis, but to monitor this effect, we need remote sensing indicators that measure more than just how green the plants are, and fluorescence is likely a good candidate. Satellite‐based plant greenness, chlorophyll fluorescence, and gross photosynthesis are evaluated with respect to soil moisture fluctuationsEcosystem responses to water availability change sign along a tree cover gradientWe show spaceborne observations of light‐ and temperature‐enhanced forest photosynthesis in times of reduced soil moisture at constant greenness

Machine learning approaches are increasingly used to extract patterns and insights from the ever-increasing stream of geospatial data, but current approaches may not be optimal when system behaviour is dominated by spatial or temporal context. Here, rather than amending classical machine learning, we argue that these contextual cues should be used as part of deep learning (an approach that is able to extract spatio-temporal features automatically) to gain further process understanding of Earth system science problems, improving the predictive ability of seasonal forecasting and modelling of long-range spatial connections across multiple timescales, for example. The next step will be a hybrid modelling approach, coupling physical process models with the versatility of data-driven machine learning.

This brief addresses two main issues of the standard kernel entropy component analysis (KECA) algorithm: the optimization of the kernel decomposition and the optimization of the Gaussian kernel parameter. KECA roughly reduces to a sorting of the importance of kernel eigenvectors by entropy instead of variance, as in the kernel principal components analysis. In this brief, we propose an extension of the KECA method, named optimized KECA (OKECA), that directly extracts the optimal features retaining most of the data entropy by means of compacting the information in very few features (often in just one or two). The proposed method produces features which have higher expressive power. In particular, it is based on the independent component analysis framework, and introduces an extra rotation to the eigen decomposition, which is optimized via gradient-ascent search. This maximum entropy preservation suggests that OKECA features are more efficient than KECA features for density estimation. In addition, a critical issue in both the methods is the selection of the kernel parameter, since it critically affects the resulting performance. Here, we analyze the most common kernel length-scale selection criteria. The results of both the methods are illustrated in different synthetic and real problems. Results show that OKECA returns projections with more expressive power than KECA, the most successful rule for estimating the kernel parameter is based on maximum likelihood, and OKECA is more robust to the selection of the length-scale parameter in kernel density estimation. [ABSTRACT FROM PUBLISHER]

Groundwater is an integral component of the water cycle, and it also influences the carbon cycle by supplying moisture to ecosystems. However, the extent and determinants of groundwater-vegetation interactions are poorly understood at the global scale. Using several high-resolution data products, we show that the spatial patterns of ecosystem gross primary productivity and groundwater table depth are correlated during at least one season in more than two thirds of the global vegetated area. Positive relationships, i.e., larger productivity under shallower groundwater table, predominate in moisture-limited dry to mesic conditions with herbaceous and shrub vegetation. Negative relationships, i.e., larger productivity under deeper groundwater, predominate in humid climates with forests, possibly indicating a drawdown of groundwater table due to substantial ecosystem water use. Interestingly, these opposite groundwater-vegetation interactions are primarily associated with differences in vegetation than with climate and surface characteristics. These findings put forth the first evidence, and a need for better representation, of extensive and non-negligible groundwater-vegetation interactions at the global scale. Local-scale groundwater-vegetation spatial covariations are prevalent globallyBoth positive and negative relationships are equally widespreadThere is a stronger association of the sign of relationship with vegetation than with climate and land surface characteristics

Large interannual variations in the measured growth rate of atmospheric carbon dioxide (CO2) originate primarily from fluctuations in carbon uptake by land ecosystems. It remains uncertain, however, to what extent temperature and water availability control the carbon balance of land ecosystems across spatial and temporal scales. Here we use empirical models based on eddy covariance data and process-based models to investigate the effect of changes in temperature and water availability on gross primary productivity (GPP), terrestrial ecosystem respiration (TER) and net ecosystem exchange (NEE) at local and global scales. We find that water availability is the dominant driver of the local interannual variability in GPP and TER. To a lesser extent this is true also for NEE at the local scale, but when integrated globally, temporal NEE variability is mostly driven by temperature fluctuations. We suggest that this apparent paradox can be explained by two compensatory water effects. Temporal water-driven GPP and TER variations compensate locally, dampening water-driven NEE variability. Spatial water availability anomalies also compensate, leaving a dominant temperature signal in the year-to-year fluctuations of the land carbon sink. These findings help to reconcile seemingly contradictory reports regarding the importance of temperature and water in controlling the interannual variability of the terrestrial carbon balance. Our study indicates that spatial climate covariation drives the global carbon cycle response.

Climate change exacerbates the occurence of extreme droughts and heatwaves, increasing the frequency and intensity of large wildfires across the globe. Forecasting wildfire danger and uncovering the drivers behind fire events become central for understanding relevant climate‐land surface feedback and aiding wildfire management. In this work, we leverage Deep Learning (DL) to predict the next day's wildfire danger in a fire‐prone part of the Eastern Mediterranean and explainable Artificial Intelligence (xAI) to diagnose model attributions. We implement DL models that capture the temporal and spatio‐temporal context, generalize well for extreme wildfires, and demonstrate improved performance over the traditional Fire Weather Index. Leveraging xAI, we identify the substantial contribution of wetness‐related variables and unveil the temporal focus of the models. The variability of the contribution of the input variables across wildfire events hints into different wildfire mechanisms. The presented methodology paves the way to more robust, accurate, and trustworthy data‐driven anticipation of wildfires. Climate change drives the aggravation of wildfires globally. In this context, it is crucial to better predict and understand wildfires. Our work proposes methods to accurately forecast wildfire danger and hints into the mechanisms that drive it. We use Machine Learning (ML) to predict next‐day wildfire danger, using meteorological, vegetation, and human‐related data, improving the results of the widely used Fire Weather Index, which is based only on meteorological forecasts. We then look to understand what the models have learned and how the different conditioning factors contribute to the final prediction. We find that ML models focus more on soil moisture and relative humidity for their predictions. They also take into account the cumulative temperature and precipitation, as well as the last day's wind speed and the relative humidity in the 2 days preceding fire ignition. These associations lead to new, data‐driven ways to anticipate wildfires. When forecasting fires that lead to large burned areas, Deep Learning models indicate higher predictive skill than the Fire Weather IndexSoil moisture, Normalized Difference Vegetation Index, and weather are the most important predictors; changes in their importance across events reveal diverse wildfire typesModels' explainability uncovers physically‐consistent associations and temporal dynamics of the fire drivers When forecasting fires that lead to large burned areas, Deep Learning models indicate higher predictive skill than the Fire Weather Index Soil moisture, Normalized Difference Vegetation Index, and weather are the most important predictors; changes in their importance across events reveal diverse wildfire types Models' explainability uncovers physically‐consistent associations and temporal dynamics of the fire drivers

The terrestrial biosphere is currently slowing down global warming by absorbing about 30% of human emissions of carbon dioxide (CO2). The largest flux of the terrestrial carbon uptake is gross primary production (GPP) defined as the production of carbohydrates by photosynthesis. Elevated atmospheric CO2concentration is expected to increase GPP (“CO2fertilization effect”). However, Earth system models (ESMs) exhibit a large range in simulated GPP projections. In this study, we combine an existing emergent constraint on CO2fertilization with a machine learning approach to constrain the spatial variations of multimodel GPP projections. In a first step, we use observed changes in the CO2seasonal cycle at Cape Kumukahi to constrain the global mean GPP at the end of the 21st century (2091–2100) in Representative Concentration Pathway 8.5 simulations with ESMs participating in the Coupled Model Intercomparison Project Phase 5 (CMIP5) to 171 ± 12 Gt C yr−1, compared to the unconstrained model range of 156–247 Gt C yr−1. In a second step, we use a machine learning model to constrain gridded future absolute GPP and gridded fractional GPP change in two independent approaches. For this, observational data are fed into the machine learning algorithm that has been trained on CMIP5 data to learn relationships between present‐day physically relevant diagnostics and the target variable. In a leave‐one‐model‐out cross‐validation approach, the machine learning model shows superior performance to the CMIP5 ensemble mean. Our approach predicts an increased GPP change in northern high latitudes compared to regions closer to the equator. About a quarter of human emissions of carbon dioxide (CO2) is absorbed by vegetation and soil on the Earth's surface and hence does not contribute to global warming caused by CO2in the atmosphere. Thus, in order to better define the remaining carbon budgets left to meet particular warming targets like the 1.5°C of the Paris Agreement, it is important to accurately quantify the carbon uptake by plants in the future. Currently, this is modeled by Earth system models yet with great uncertainties. In this work, we present an alternative machine learning approach to reduce spatial uncertainties of vegetation carbon uptake in future climate projections using observations of today's conditions. An emergent constraint on CO2seasonal cycle amplitude changes reduces uncertainties in global mean gross primary production projectionsA machine learning model with multiple predictors can further constrain the spatial distribution of gross primary productionHigh‐latitude ecosystems show higher gross primary production increase over the 21st century compared to regions closer to the equator

Abstract Bayesian methods and their implementations by means of sophisticated Monte Carlo techniques have become very popular in signal processing over the last years. Importance Sampling (IS) is a well-known Monte Carlo technique that approximates integrals involving a posterior distribution by means of weighted samples. In this work, we study the assignation of a single weighted sample which compresses the information contained in a population of weighted samples. Part of the theory that we present as Group Importance Sampling (GIS) has been employed implicitly in different works in the literature. The provided analysis yields several theoretical and practical consequences. For instance, we discuss the application of GIS into the Sequential Importance Resampling framework and show that Independent Multiple Try Metropolis schemes can be interpreted as a standard Metropolis–Hastings algorithm, following the GIS approach. We also introduce two novel Markov Chain Monte Carlo (MCMC) techniques based on GIS. The first one, named Group Metropolis Sampling method, produces a Markov chain of sets of weighted samples. All these sets are then employed for obtaining a unique global estimator. The second one is the Distributed Particle Metropolis–Hastings technique, where different parallel particle filters are jointly used to drive an MCMC algorithm. Different resampled trajectories are compared and then tested with a proper acceptance probability. The novel schemes are tested in different numerical experiments such as learning the hyperparameters of Gaussian Processes, two localization problems in a wireless sensor network (with synthetic and real data) and the tracking of vegetation parameters given satellite observations, where they are compared with several benchmark Monte Carlo techniques. Three illustrative Matlab demos are also provided. [ABSTRACT FROM AUTHOR]

Monte Carlo methods are essential tools for Bayesian inference. Gibbs sampling is a well-known Markov chain Monte Carlo (MCMC) algorithm, extensively used in signal processing, machine learning, and statistics, employed to draw samples from complicated high-dimensional posterior distributions. The key point for the successful application of the Gibbs sampler is the ability to draw efficiently samples from the full-conditional probability density functions. Since in the general case this is not possible, in order to speed up the convergence of the chain, it is required to generate auxiliary samples whose information is eventually disregarded. In this work, we show that these auxiliary samples can be recycled within the Gibbs estimators, improving their efficiency with no extra cost. This novel scheme arises naturally after pointing out the relationship between the standard Gibbs sampler and the chain rule used for sampling purposes. Numerical simulations involving simple and real inference problems confirm the excellent performance of the proposed scheme in terms of accuracy and computational efficiency. In particular we give empirical evidence of performance in a toy example, inference of Gaussian processes hyperparameters, and learning dependence graphs through regression. [ABSTRACT FROM AUTHOR]