Search

Physics is a field of science that has traditionally used the scientific method to answer questions about why natural phenomena occur and to make testable models that explain the phenomena. Discovering equations, laws and principles that are invariant, robust and causal explanations of the world has been fundamental in physical sciences throughout the centuries. Discoveries emerge from observing the world and, when possible, performing interventional studies in the system under study. With the advent of big data and the use of data-driven methods, causal and equation discovery fields have grown and made progress in computer science, physics, statistics, philosophy, and many applied fields. All these domains are intertwined and can be used to discover causal relations, physical laws, and equations from observational data. This paper reviews the concepts, methods, and relevant works on causal and equation discovery in the broad field of Physics and outlines the most important challenges and promising future lines of research. We also provide a taxonomy for observational causal and equation discovery, point out connections, and showcase a complete set of case studies in Earth and climate sciences, fluid dynamics and mechanics, and the neurosciences. This review demonstrates that discovering fundamental laws and causal relations by observing natural phenomena is being revolutionised with the efficient exploitation of observational data, modern machine learning algorithms and the interaction with domain knowledge. Exciting times are ahead with many challenges and opportunities to improve our understanding of complex systems., 137 pages

Cirrus clouds are key modulators of Earth's climate. Their dependencies on meteorological and aerosol conditions are among the largest uncertainties in global climate models. This work uses three years of satellite and reanalysis data to study the link between cirrus drivers and cloud properties. We use a gradient-boosted machine learning model and a Long Short-Term Memory (LSTM) network with an attention layer to predict the ice water content and ice crystal number concentration. The models show that meteorological and aerosol conditions can predict cirrus properties with $R^2 = 0.49$. Feature attributions are calculated with SHapley Additive exPlanations (SHAP) to quantify the link between meteorological and aerosol conditions and cirrus properties. For instance, the minimum concentration of supermicron-sized dust particles required to cause a decrease in ice crystal number concentration predictions is $2 \times 10^{-4}$ mg m\textsuperscript{-3}. The last 15 hours before the observation predict all cirrus properties., Comment: Presented at Climate Informatics 2023 in Cambridge; Submitted to Environmental Data Science Journal Updates Version: New version of dataset is linked. Please use that version: https://zenodo.org/record/7965381

Wildfires are increasingly exacerbated as a result of climate change, necessitating advanced proactive measures for effective mitigation. It is important to forecast wildfires weeks and months in advance to plan forest fuel management, resource procurement and allocation. To achieve such accurate long-term forecasts at a global scale, it is crucial to employ models that account for the Earth system's inherent spatio-temporal interactions, such as memory effects and teleconnections. We propose a teleconnection-driven vision transformer (TeleViT), capable of treating the Earth as one interconnected system, integrating fine-grained local-scale inputs with global-scale inputs, such as climate indices and coarse-grained global variables. Through comprehensive experimentation, we demonstrate the superiority of TeleViT in accurately predicting global burned area patterns for various forecasting windows, up to four months in advance. The gain is especially pronounced in larger forecasting windows, demonstrating the improved ability of deep learning models that exploit teleconnections to capture Earth system dynamics. Code available at https://github.com/Orion-Ai-Lab/TeleViT.

Earth observation (EO) is a prime instrument for monitoring land and ocean processes, studying the dynamics at work, and taking the pulse of our planet. This article gives a bird's eye view of the essential scientific tools and approaches informing and supporting the transition from raw EO data to usable EO-based information. The promises, as well as the current challenges of these developments, are highlighted under dedicated sections. Specifically, we cover the impact of (i) Computer vision; (ii) Machine learning; (iii) Advanced processing and computing; (iv) Knowledge-based AI; (v) Explainable AI and causal inference; (vi) Physics-aware models; (vii) User-centric approaches; and (viii) the much-needed discussion of ethical and societal issues related to the massive use of ML technologies in EO.

Granger causality (GC) is often considered not an actual form of causality. Still, it is arguably the most widely used method to assess the predictability of a time series from another one. Granger causality has been widely used in many applied disciplines, from neuroscience and econometrics to Earth sciences. We revisit GC under a graphical perspective of state-space models. For that, we use GraphEM, a recently presented expectation-maximisation algorithm for estimating the linear matrix operator in the state equation of a linear-Gaussian state-space model. Lasso regularisation is included in the M-step, which is solved using a proximal splitting Douglas-Rachford algorithm. Experiments in toy examples and challenging climate problems illustrate the benefits of the proposed model and inference technique over standard Granger causality methods., Comment: 4 pages, 2 figures, 3 tables, CausalStats23: When Causal Inference meets Statistical Analysis, April 17-21, 2023, Paris, France

We introduce Mesogeos, a large-scale multi-purpose dataset for wildfire modeling in the Mediterranean. Mesogeos integrates variables representing wildfire drivers (meteorology, vegetation, human activity) and historical records of wildfire ignitions and burned areas for 17 years (2006-2022). It is designed as a cloud-friendly spatio-temporal dataset, namely a datacube, harmonizing all variables in a grid of 1km x 1km x 1-day resolution. The datacube structure offers opportunities to assess machine learning (ML) usage in various wildfire modeling tasks. We extract two ML-ready datasets that establish distinct tracks to demonstrate this potential: (1) short-term wildfire danger forecasting and (2) final burned area estimation given the point of ignition. We define appropriate metrics and baselines to evaluate the performance of models in each track. By publishing the datacube, along with the code to create the ML datasets and models, we encourage the community to foster the implementation of additional tracks for mitigating the increasing threat of wildfires in the Mediterranean.

The synergistic combination of deep learning models and Earth observation promises significant advances to support the sustainable development goals (SDGs). New developments and a plethora of applications are already changing the way humanity will face the living planet challenges. This paper reviews current deep learning approaches for Earth observation data, along with their application towards monitoring and achieving the SDGs most impacted by the rapid development of deep learning in Earth observation. We systematically review case studies to 1) achieve zero hunger, 2) sustainable cities, 3) deliver tenure security, 4) mitigate and adapt to climate change, and 5) preserve biodiversity. Important societal, economic and environmental implications are concerned. Exciting times ahead are coming where algorithms and Earth data can help in our endeavor to address the climate crisis and support more sustainable development.

Cloud detection is a crucial step in the optical satellite image processing pipeline for Earth observation. Clouds in optical remote sensing images seriously affect the visibility of the background and greatly reduce the usability of images for land applications. Traditional methods based on thresholding, multi-temporal or multi-spectral information are often specific to a particular satellite sensor. Convolutional Neural Networks for cloud detection often require labeled cloud masks for training that are very time-consuming and expensive to obtain. To overcome these challenges, this paper presents a hybrid cloud detection method based on the synergistic combination of generative adversarial networks (GAN) and a physics-based cloud distortion model (CDM). The proposed weakly-supervised GAN-CDM method (available online https://github.com/Neooolee/GANCDM) only requires patch-level labels for training, and can produce cloud masks at pixel-level in both training and testing stages. GAN-CDM is trained on a new globally distributed Landsat 8 dataset (WHUL8-CDb, available online doi:https://doi.org/10.5281/zenodo.6420027) including image blocks and corresponding block-level labels. Experimental results show that the proposed GAN-CDM method trained on Landsat 8 image blocks achieves much higher cloud detection accuracy than baseline deep learning-based methods, not only in Landsat 8 images (L8 Biome dataset, 90.20% versus 72.09%) but also in Sentinel-2 images (“S2 Cloud Mask Catalogue” dataset, 92.54% versus 77.00%). This suggests that the proposed method provides accurate cloud detection in Landsat images, has good transferability to Sentinel-2 images, and can quickly be adapted for different optical satellite sensors.

Wildfire forecasting is of paramount importance for disaster risk reduction and environmental sustainability. We approach daily fire danger prediction as a machine learning task, using historical Earth observation data from the last decade to predict next-day's fire danger. To that end, we collect, pre-process and harmonize an open-access datacube, featuring a set of covariates that jointly affect the fire occurrence and spread, such as weather conditions, satellite-derived products, topography features and variables related to human activity. We implement a variety of Deep Learning (DL) models to capture the spatial, temporal or spatio-temporal context and compare them against a Random Forest (RF) baseline. We find that either spatial or temporal context is enough to surpass the RF, while a ConvLSTM that exploits the spatio-temporal context performs best with a test Area Under the Receiver Operating Characteristic of 0.926. Our DL-based proof-of-concept provides national-scale daily fire danger maps at a much higher spatial resolution than existing operational solutions., Comment: Accepted to the workshop on Artificial Intelligence for Humanitarian Assistance and Disaster Response at the 35th Conference on Neural Information Processing Systems (NeurIPS 2021)

Atmospheric correction of Earth Observation data is one of the most critical steps in the data processing chain of a satellite mission for successful remote sensing applications. Atmospheric Radiative Transfer Models (RTM) inversion methods are typically preferred due to their high accuracy. However, the execution of RTMs on a pixel-per-pixel basis is impractical due to their high computation time, thus large multi-dimensional look-up tables (LUTs) are precomputed for their later interpolation. To further reduce the RTM computation burden and the error in LUT interpolation, we have developed a method to automatically select the minimum and optimal set of nodes to be included in a LUT. We present the gradient-based automatic LUT generator algorithm (GALGA) which relies on the notion of an acquisition function that incorporates (a) the Jacobian evaluation of an RTM, and (b) information about the multivariate distribution of the current nodes. We illustrate the capabilities of GALGA in the automatic construction and optimization of MODerate resolution atmospheric TRANsmission (MODTRAN) LUTs for several input dimensions. Our results indicate that, when compared to a pseudo-random homogeneous distribution of the LUT nodes, GALGA reduces (1) the LUT size by $\sim$75\% and (2) the maximum interpolation relative errors by 0.5\% It is concluded that automatic LUT design might benefit from the methodology proposed in GALGA to reduce computation time and interpolation errors.

Density destructors are differentiable and invertible transforms that map multivariate PDFs of arbitrary structure (low entropy) into non-structured PDFs (maximum entropy). Multivariate Gaussianization and multivariate equalization are specific examples of this family, which break down the complexity of the original PDF through a set of elementary transforms that progressively remove the structure of the data. We demonstrate how this property of density destructive flows is connected to classical information theory, and how density destructors can be used to get more accurate estimates of information theoretic quantities. Experiments with total correlation and mutual information inmultivariate sets illustrate the ability of density destructors compared to competing methods. These results suggest that information theoretic measures may be an alternative optimization criteria when learning density destructive flows., Accepted at the Workshop on Invertible Neural Nets and Normalizing Flows, ICML 2019

In the current context of climate change, extreme heatwaves, droughts, and floods are not only impacting the biosphere and atmosphere but the anthroposphere too. Human populations are forcibly displaced, which are now referred to as climate-induced migrants. In this work, we investigate which climate and structural factors forced major human displacements in the presence of floods and storms during the years 2017-2019. We built, curated, and harmonized a database of meteorological and remote sensing indicators along with structural factors of 27 developing countries worldwide. We show how we can use Gaussian Processes to learn what variables can explain the impact of floods and storms in the context of forced displacements and to develop models that reproduce migration flows. Our results at regional, global, and disaster-specific scales show the importance of structural factors in the determination of the magnitude of displacements. The study may have both societal, political, and economical implications., Presented at NeurIPS 2020 Workshop on Machine Learning for the Developing World

Most problems in Earth sciences aim to do inferences about the system, where accurate predictions are just a tiny part of the whole problem. Inferences mean understanding variables relations, deriving models that are physically interpretable, that are simple parsimonious, and mathematically tractable. Machine learning models alone are excellent approximators, but very often do not respect the most elementary laws of physics, like mass or energy conservation, so consistency and confidence are compromised. In this paper, we describe the main challenges ahead in the field, and introduce several ways to live in the Physics and machine learning interplay: to encode differential equations from data, constrain data-driven models with physics-priors and dependence constraints, improve parameterizations, emulate physical models, and blend data-driven and process-based models. This is a collective long-term AI agenda towards developing and applying algorithms capable of discovering knowledge in the Earth system., 24 pages, 10 figures, 3 tables, expanded AAAI PGAI 2020 Symposium

Information theory is an outstanding framework to measure uncertainty, dependence and relevance in data and systems. It has several desirable properties for real world applications: it naturally deals with multivariate data, it can handle heterogeneous data types, and the measures can be interpreted in physical units. However, it has not been adopted by a wider audience because obtaining information from multidimensional data is a challenging problem due to the curse of dimensionality. Here we propose an indirect way of computing information based on a multivariate Gaussianization transform. Our proposal mitigates the difficulty of multivariate density estimation by reducing it to a composition of tractable (marginal) operations and simple linear transformations, which can be interpreted as a particular deep neural network. We introduce specific Gaussianization-based methodologies to estimate total correlation, entropy, mutual information and Kullback-Leibler divergence. We compare them to recent estimators showing the accuracy on synthetic data generated from different multivariate distributions. We made the tools and datasets publicly available to provide a test-bed to analyze future methodologies. Results show that our proposal is superior to previous estimators particularly in high-dimensional scenarios; and that it leads to interesting insights in neuroscience, geoscience, computer vision, and machine learning., This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible

The Earth is a complex dynamic network system. Modelling and understanding the system is at the core of scientific endeavour. We approach these problems with machine learning algorithms. I will review several ML approaches we have developed in the last years: 1) advanced Gaussian processes models for bio-geo-physical parameter estimation, which can incorporate physical laws, blend multisensor data while providing credible confidence intervals for the estimates and improved interpretability, 2) nonlinear dimensionality reduction methods to decompose Earth data cubes in spatially-explicit and temporally-resolved modes of variability that summarize the information content of the data and allow for identifying relations with physical processes, and 3) advances in causal inference that can uncover cause and effect relations from purely observational data.

Understanding the complex interdependencies of processes in our climate system has become one of the most critical challenges for society with our main current tools being cli- mate modeling and observational data analysis, in particular observational causal discovery. Causal discovery is still in its infancy in Earth sciences and a major issue is that current methods are not well adapted to climate data challenges. We here present an overview of a NeurIPS 2019 competition on causal discovery for climate time series. The Causality 4 Climate (C4C) competition was hosted on the benchmark platform www.causeme.net. C4C offers an extensive number of climate model-based time series datasets with known causal ground truth that incorporate the main challenges of causal discovery in climate research. We give an overview over the benchmark platform, the challenges modeled, how datasets were generated, and implementation details. The goal of C4C is to spur more focused methodological research on causal discovery for understanding our climate system.

Global warming is leading to unprecedented changes in our planet, with great societal, economical and environmental implications, especially with the growing demand of biofuels and food. Assessing the impact of climate on vegetation is of pressing need. We approached the attribution problem with a novel nonlinear Granger causal (GC) methodology and used a large data archive of remote sensing satellite products, environmental and climatic variables spatio-temporally gridded over more than 30 years. We generalize kernel Granger causality by considering the variables cross-relations explicitly in Hilbert spaces, and use the covariance in Gaussian processes. The method generalizes the linear and kernel GC methods, and comes with tighter bounds of performance based on Rademacher complexity. Spatially-explicit global Granger footprints of precipitation and soil moisture on vegetation greenness are identified more sharply than previous GC methods., Comment: 7 pages, 3 figures, AI for Earth Sciences Workshop at NeurIPS 2020. arXiv admin note: text overlap with arXiv:2011.14444

Monitoring spatio-temporal changes in terrestrial gross primary productivity (GPP) of crops is key for estimating, understanding and predicting global carbon fluxes. Satellite remote sensing has been widely applied in the last decades to monitor agricultural resources, and the amount and quality of remote sensing data continuously increase. Since recently, and partly due the European Copernicus Programme, an unprecedented amount of open access data suitable for agriculture observations is now available. Benefiting from recent developments in satellite remote sensing technology, great advances in machine learning and advancements in our understanding of photosynthetic processes leading to increasingly complex and detailed photosynthesis models, we developed a hybrid approach to model GPP using satellite reflectance data by combining radiative transfer modeling and machine learning (ML). We have combined process-based model SCOPE with ML algorithms to estimate GPP of C3 crops using a variety of satellite data (Sentinel-2, Landsat and MODIS) and ancillary meteorological information. We link reflectance and meteorological data directly with crop GPP, bypassing the need of retrieving the set of input vegetation parameters needed to represent photosynthesis in an intermediate step, while still accounting for the complex processes of the original model. Several ML models, trained with the simulated data, were tested and validated using flux tower data. First, we tested our approach using Sentinel-2 data, which provide high frequency of observation, high spatial resolution of 20 m and multiple bands including red edge. Our final neural network model was able to estimate GPP at the tested flux towers with r2 of 0.92 and RMSE of 1.38 gC m$^{-2}$ d-1. Our model successfully estimated GPP across a variety of C3 crop types and environmental conditions, including periods of no vegetation, even tough it did not use any additional local information from the site. Since our learning approach is fast and efficient in the test phase and, at the same time, is based on a process-based model (and not on local empirical relationships), it can be applied globally. Furthermore, the simulated training dataset can be easily adapted to band settings of different instruments, assuring thus consistency among many sensors. However, such a global application requires high computational power and therefore we applied our approach to Landsat and MODIS data using Google Earth Engine (GEE) platform that provides cloud computing resources for processing large geospatial datasets. The results were validated using the FLUXNET2015 Dataset.

med 2018, 11-12 December 2018, Frascati, Rome, Italy, Dimensional reduction methods are key to Earth system sciences, since they allow dealing with extensive spatio-temporal climatic data sets in terms of individual variables as time series and spatial distributions. Principal Component Analysis (PCA), also called Empirical Orthogonal Functions (EOF) in geophysics, is a simple yet very powerful method to uncover the main geographical patterns at distinct temporal scales present in the data. A variety of PCA extensions have been proposed to deal with specific aspects of the input data, as the complex PCA (CEOF), the rotated PCA (REOF) or the nonlinear or Kernel PCA. The rotated complex kernel PCA (ROCK PCA) was recently proposed to combine these three methods and allow making a physically interpretable spatio-temporal decomposition of extensive climatic variables. In this work, we apply the ROCK-PCA method to the 8 years of ESA's SMOS Soil Moisture (SM) and Ocean Salinity (OS) data available over the Mediterranean region. Results show the distribution of the total SM and OS variance among its different components, and indicate the dominant modes of temporal variability of these two variables. The connection of these modes to human-induced changes or natural fluctuations (e.g. El Niño Southern Oscillation) have been explored and will be shown at the workshop, Research funded by the European Research Council (ERC) under the ERC-CoG-2014 SEDAL project (grant agreement 647423) and TEC2016-81900-REDT. M. Piles is supported by a Ramón y Cajal contract (MINECO)

This report summarizes the program and main outcomes of the Dagstuhl Seminar 17091 entitled ``Computer Science Meets Ecolog''. Ecology is a discipline that poses many challenging problems involving big data collection, provenance and integration, as well as difficulties in data analysis, prediction and understanding. All these issues are precisely the arena where computer science is concerned. The seminar motivation was rooted in the belief that ecology could largely benefit from modern computer science. The seminar attracted scientists from both fields who discussed important topics in ecology (e.g. botany, animal science, biogeochemistry) and how to approach them with machine learning, computer vision, pattern recognition and data mining. A set of specific problems and techniques were treated, and the main building blocks were set up. The important topics of education, outreach, data and models accessibility were also touched upon. The seminar proposed a distinctive perspective by promoting cross-fertilization in a unique environment and a unique set of individuals.

New social and economic activities massively exploit big data and machine learning algorithms to do inference on people's lives. Applications include automatic curricula evaluation, wage determination, and risk assessment for credits and loans. Recently, many governments and institutions have raised concerns about the lack of fairness, equity and ethics in machine learning to treat these problems. It has been shown that not including sensitive features that bias fairness, such as gender or race, is not enough to mitigate the discrimination when other related features are included. Instead, including fairness in the objective function has been shown to be more efficient. We present novel fair regression and dimensionality reduction methods built on a previously proposed fair classification framework. Both methods rely on using the Hilbert Schmidt independence criterion as the fairness term. Unlike previous approaches, this allows us to simplify the problem and to use multiple sensitive variables simultaneously. Replacing the linear formulation by kernel functions allows the methods to deal with nonlinear problems. For both linear and nonlinear formulations the solution reduces to solving simple matrix inversions or generalized eigenvalue problems. This simplifies the evaluation of the solutions for different trade-off values between the predictive error and fairness terms. We illustrate the usefulness of the proposed methods in toy examples, and evaluate their performance on real world datasets to predict income using gender and/or race discrimination as sensitive variables, and contraceptive method prediction under demographic and socio-economic sensitive descriptors., Comment: Work published on ECML'17, http://ecmlpkdd2017.ijs.si/papers/paperID275.pdf

Kernel dependence measures yield accurate estimates of nonlinear relations between random variables, and they are also endorsed with solid theoretical properties and convergence rates. Besides, the empirical estimates are easy to compute in closed form just involving linear algebra operations. However, they are hampered by two important problems: the high computational cost involved, as two kernel matrices of the sample size have to be computed and stored, and the interpretability of the measure, which remains hidden behind the implicit feature map. We here address these two issues. We introduce the Sensitivity Maps (SMs) for the Hilbert-Schmidt independence criterion (HSIC). Sensitivity maps allow us to explicitly analyze and visualize the relative relevance of both examples and features on the dependence measure. We also present the randomized HSIC (RHSIC) and its corresponding sensitivity maps to cope with large scale problems. We build upon the framework of random features and the Bochner's theorem to approximate the involved kernels in the canonical HSIC. The power of the RHSIC measure scales favourably with the number of samples, and it approximates HSIC and the sensitivity maps efficiently. Convergence bounds of both the measure and the sensitivity map are also provided. Our proposal is illustrated in synthetic examples, and challenging real problems of dependence estimation, feature selection, and causal inference from empirical data.

Gaussian processes (GPs) have experienced tremendous success in biogeophysical parameter retrieval in the last few years. GPs constitute a solid Bayesian framework to consistently formulate many function approximation problems. This article reviews the main theoretical GP developments in the field, considering new algorithms that respect signal and noise characteristics, extract knowledge via automatic relevance kernels to yield feature rankings automatically, and allow applicability of associated uncertainty intervals to transport GP models in space and time that can be used to uncover causal relations between variables and can encode physically meaningful prior knowledge via radiative transfer model (RTM) emulation. The important issue of computational efficiency will also be addressed. These developments are illustrated in the field of geosciences and remote sensing at local and global scales through a set of illustrative examples. In particular, important problems for land, ocean, and atmosphere monitoring are considered, from accurately estimating oceanic chlorophyll content and pigments to retrieving vegetation properties from multi- and hyperspectral sensors as well as estimating atmospheric parameters (e.g., temperature, moisture, and ozone) from infrared sounders.

The wealth of sensory data coming from different modalities has opened numerous opportu- nities for data analysis. The data are of increasing volume, complexity and dimensionality, thus calling for new methodological innovations towards multimodal data processing. How- ever, multimodal architectures must rely on models able to adapt to changes in the data dis- tribution. Differences in the density functions can be due to changes in acquisition conditions (pose, illumination), sensors characteristics (number of channels, resolution) or different views (e.g. street level vs. aerial views of a same building). We call these different acquisition modes domains, and refer to the adaptation problem as domain adaptation. In this paper, instead of adapting the trained models themselves, we alternatively focus on finding mappings of the data sources into a common, semantically meaningful, representa- tion domain. This field of manifold alignment extends traditional techniques in statistics such as canonical correlation analysis (CCA) to deal with nonlinear adaptation and possibly non-corresponding data pairs between the domains. We introduce a kernel method for man- ifold alignment (KEMA) that can match an arbitrary number of data sources without needing corresponding pairs, just few labeled examples in all domains. KEMA has interesting prop- erties: 1) it generalizes other manifold alignment methods, 2) it can align manifolds of very different complexities, performing a discriminative alignment preserving each manifold inner structure, 3) it can define a domain-specific metric to cope with multimodal specifici- ties, 4) it can align data spaces of different dimensionality, 5) it is robust to strong nonlinear feature deformations, and 6) it is closed-form invertible, which allows transfer across- domains and data synthesis. To authors’ knowledge this is the first method addressing all these important issues at once. We also present a reduced-rank version of KEMA for computational efficiency, and discuss the generalization performance of KEMA under Rademacher principles of stability. Aligning multimodal data with KEMA reports outstanding benefits when used as a data pre-conditioner step in the standard data analysis processing chain. KEMA exhibits very good performance over competing methods in synthetic con- trolled examples, visual object recognition and recognition of facial expressions tasks. KEMA is especially well-suited to deal with high-dimensional problems, such as images and videos, and under complicated distortions, twists and warpings of the data manifolds. A fully functional toolbox is available at https://github.com/dtuia/KEMA.git.

We introduce a kernel method for manifold alignment (KEMA) and domain adaptation that can match an arbitrary number of data sources without needing corresponding pairs, just few labeled examples in all domains. KEMA has interesting properties: 1) it generalizes other manifold alignment methods, 2) it can align manifolds of very different complexities, performing a sort of manifold unfolding plus alignment, 3) it can define a domain-specific metric to cope with multimodal specificities, 4) it can align data spaces of different dimensionality, 5) it is robust to strong nonlinear feature deformations, and 6) it is closed-form invertible which allows transfer across-domains and data synthesis. We also present a reduced-rank version for computational efficiency and discuss the generalization performance of KEMA under Rademacher principles of stability. KEMA exhibits very good performance over competing methods in synthetic examples, visual object recognition and recognition of facial expressions tasks.

Deep learning is a fundamental technique in modern Artificial Intelligence and is being applied to disciplines across the scientific spectrum; earth science is no exception. Yet, the link between deep learning and Earth sciences has only recently entered academic curricula and thus has not yet proliferated. Deep Learning for the Earth Sciences delivers a unique perspective and treatment of the concepts, skills, and practices necessary to quickly become familiar with the application of deep learning techniques to the Earth sciences. The book prepares readers to be ready to use the technologies and principles described in their own research.

Spatio-temporal fields of land–atmosphere fluxes derived from data-driven models can complement simulations by process-based land surface models. While a number of strategies for empirical models with eddy-covariance flux data have been applied, a systematic intercomparison of these methods has been missing so far. In this study, we performed a cross-validation experiment for predicting carbon dioxide, latent heat, sensible heat and net radiation fluxes across different ecosystem types with 11 machine learning (ML) methods from four different classes (kernel methods, neural networks, tree methods, and regression splines). We applied two complementary setups: (1) 8-day average fluxes based on remotely sensed data and (2) daily mean fluxes based on meteorological data and a mean seasonal cycle of remotely sensed variables. The patterns of predictions from different ML and experimental setups were highly consistent. There were systematic differences in performance among the fluxes, with the following ascending order: net ecosystem exchange (R20.6), gross primary production (R2>0.7), latent heat (R2>0.7), sensible heat (R2>0.7), and net radiation (R2>0.8). The ML methods predicted the across-site variability and the mean seasonal cycle of the observed fluxes very well (R2>0.7), while the 8-day deviations from the mean seasonal cycle were not well predicted (R2, Biogeosciences, 13 (14), ISSN:1726-4170

Spatio-temporal fields of land–atmosphere fluxes derived from data-driven models can complement simulations by process-based land surface models. While a number of strategies for empirical models with eddy-covariance flux data have been applied, a systematic intercomparison of these methods has been missing so far. In this study, we performed a cross-validation experiment for predicting carbon dioxide, latent heat, sensible heat and net radiation fluxes across different ecosystem types with 11 machine learning (ML) methods from four different classes (kernel methods, neural networks, tree methods, and regression splines). We applied two complementary setups: (1) 8-day average fluxes based on remotely sensed data and (2) daily mean fluxes based on meteorological data and a mean seasonal cycle of remotely sensed variables. The patterns of predictions from different ML and experimental setups were highly consistent. There were systematic differences in performance among the fluxes, with the following ascending order: net ecosystem exchange (R20.6), gross primary production (R2>0.7), latent heat (R2>0.7), sensible heat (R2>0.7), and net radiation (R2>0.8). The ML methods predicted the across-site variability and the mean seasonal cycle of the observed fluxes very well (R2>0.7), while the 8-day deviations from the mean seasonal cycle were not well predicted (R2, Biogeosciences Discussions, ISSN:1810-6277, ISSN:1810-6285

The terrestrial biosphere is currently slowing down global warming by absorbing about 30% of human emissions of carbon dioxide (CO2). The largest flux of the terrestrial carbon uptake is gross primary production (GPP) defined as the production of carbohydrates by photosynthesis. Elevated atmospheric CO2 concentration is expected to increase GPP (“CO2 fertilization effect”). However, Earth system models (ESMs) exhibit a large range in simulated GPP projections. In this study, we combine an existing emergent constraint on CO2 fertilization with a machine learning approach to constrain the spatial variations of multimodel GPP projections. In a first step, we use observed changes in the CO2 seasonal cycle at Cape Kumukahi to constrain the global mean GPP at the end of the 21st century (2091–2100) in Representative Concentration Pathway 8.5 simulations with ESMs participating in the Coupled Model Intercomparison Project Phase 5 (CMIP5) to 171 ± 12 Gt C yr−1, compared to the unconstrained model range of 156–247 Gt C yr−1. In a second step, we use a machine learning model to constrain gridded future absolute GPP and gridded fractional GPP change in two independent approaches. For this, observational data are fed into the machine learning algorithm that has been trained on CMIP5 data to learn relationships between present‐day physically relevant diagnostics and the target variable. In a leave‐one‐model‐out cross‐validation approach, the machine learning model shows superior performance to the CMIP5 ensemble mean. Our approach predicts an increased GPP change in northern high latitudes compared to regions closer to the equator., Plain Language Summary: About a quarter of human emissions of carbon dioxide (CO2) is absorbed by vegetation and soil on the Earth's surface and hence does not contribute to global warming caused by CO2 in the atmosphere. Thus, in order to better define the remaining carbon budgets left to meet particular warming targets like the 1.5°C of the Paris Agreement, it is important to accurately quantify the carbon uptake by plants in the future. Currently, this is modeled by Earth system models yet with great uncertainties. In this work, we present an alternative machine learning approach to reduce spatial uncertainties of vegetation carbon uptake in future climate projections using observations of today's conditions., Key Points: An emergent constraint on CO2 seasonal cycle amplitude changes reduces uncertainties in global mean gross primary production projections. A machine learning model with multiple predictors can further constrain the spatial distribution of gross primary production. High‐latitude ecosystems show higher gross primary production increase over the 21st century compared to regions closer to the equator., EC | Horizon 2020 Framework Programme 4C, EC | Horizon 2020 Framework Programme CRESCENDO, ERC Consolidator Grant SEDAL, ERC Synergy Grant USMILE

Contextual bandits are a rich model for sequential decision making given side information, with important applications, e.g., in recommender systems. We propose novel algorithms for contextual bandits harnessing neural networks to approximate the unknown reward function. We resolve the open problem of proving sublinear regret bounds in this setting for general context sequences, considering both fully-connected and convolutional networks. To this end, we first analyze NTK-UCB, a kernelized bandit optimization algorithm employing the Neural Tangent Kernel (NTK), and bound its regret in terms of the NTK maximum information gain $\gamma_T$, a complexity parameter capturing the difficulty of learning. Our bounds on $\gamma_T$ for the NTK may be of independent interest. We then introduce our neural network based algorithm NN-UCB, and show that its regret closely tracks that of NTK-UCB. Under broad non-parametric assumptions about the reward function, our approach converges to the optimal policy at a $\tilde{\mathcal{O}}(T^{-1/2d})$ rate, where $d$ is the dimension of the context., Comment: 37 pages, 6 figures

We propose a new approach to model the collective dynamics of a population of particles evolving with time. As is often the case in challenging scientific applications, notably single-cell genomics, measuring features for these particles requires destroying them. As a result, the population can only be monitored with periodic snapshots, obtained by sampling a few particles that are sacrificed in exchange for measurements. Given only access to these snapshots, can we reconstruct likely individual trajectories for all other particles? We propose to model these trajectories as collective realizations of a causal Jordan-Kinderlehrer-Otto (JKO) flow of measures: The JKO scheme posits that the new configuration taken by a population at time t + 1 is one that trades off an improvement, in the sense that it decreases an energy, while remaining close (in Wasserstein distance) to the previous configuration observed at t. In order to learn such an energy using only snapshots, we propose JKONET, a neural architecture that computes (in end-to-end differentiable fashion) the JKO flow given a parametric energy and initial configuration of points. We demonstrate the good performance and robustness of the JKONET fitting procedure, compared to a more direct forward method. ISSN:2640-3498

Simulation models of complex dynamics in the natural and social sciences commonly lack a tractable likelihood function, rendering traditional likelihood-based statistical inference impossible. Recent advances in machine learning have introduced novel algorithms for estimating otherwise intractable likelihood functions using a likelihood ratio trick based on binary classifiers. Consequently, efficient likelihood approximations can be obtained whenever good probabilistic classifiers can be constructed. We propose a kernel classifier for sequential data using path signatures based on the recently introduced signature kernel. We demonstrate that the representative power of signatures yields a highly performant classifier, even in the crucially important case where sample numbers are low. In such scenarios, our approach can outperform sophisticated neural networks for common posterior inference tasks., Comment: Accepted for publication at AISTATS 2022

Earth observation through satellite sensors, models and in situ measurements provides a way to monitor our planet with unprecedented spatial and temporal resolution. The amount and diversity of the data which is recorded and made available is ever-increasing. This data allows us to perform crop yield prediction, track land-use change such as deforestation, monitor and respond to natural disasters and predict and mitigate climate change. The last two decades have seen a large increase in the application of machine learning algorithms in Earth observation in order to make efficient use of the growing data-stream. Machine learning algorithms, however, are typically model agnostic and too flexible and so end up not respecting fundamental laws of physics. On the other hand there has, in recent years, been an increase in research attempting to embed physics knowledge in machine learning algorithms in order to obtain interpretable and physically meaningful solutions. The main objective of this thesis is to explore different ways of encoding physical knowledge to provide machine learning methods tailored for specific problems in remote sensing.Ways of expressing expert knowledge about the relevant physical systems in remote sensing abound, ranging from simple relations between reflectance indices and biophysical parameters to complex models that compute the radiative transfer of electromagnetic radiation through our atmosphere, and differential equations that explain the dynamics of key parameters. This thesis focuses on inversion problems, emulation of radiative transfer models, and incorporation of the above-mentioned domain knowledge in machine learning algorithms for remote sensing applications. We explore new methods that can optimally model simulated and in-situ data jointly, incorporate differential equations in machine learning algorithms, handle more complex inversion problems and large-scale data, obtain accurate and computationally efficient emulators that are consistent with physical models, and that efficiently perform approximate Bayesian inversion over radiative transfer models.

In Bayesian Optimization (BO) we study black-box function optimization with noisy point evaluations and Bayesian priors. Convergence of BO can be greatly sped up by batching, where multiple evaluations of the black-box function are performed in a single round. The main difficulty in this setting is to propose at the same time diverse and informative batches of evaluation points. In this work, we introduce DPP-Batch Bayesian Optimization (DPP-BBO), a universal framework for inducing batch diversity in sampling based BO by leveraging the repulsive properties of Determinantal Point Processes (DPP) to naturally diversify the batch sampling procedure. We illustrate this framework by formulating DPP-Thompson Sampling (DPP-TS) as a variant of the popular Thompson Sampling (TS) algorithm and introducing a Markov Chain Monte Carlo procedure to sample from it. We then prove novel Bayesian simple regret bounds for both classical batched TS as well as our counterpart DPP-TS, with the latter bound being tighter. Our real-world, as well as synthetic, experiments demonstrate improved performance of DPP-BBO over classical batching methods with Gaussian process and Cox process models., Comment: To be published in AISTATS 2022

Semantic segmentation consists of the generation of a categorical map, given an image in which each pixel of the image is automatically assigned a class. Deep learning allows the influence of the pixel's context to be learned by capturing the non-linear relationships between surrounding image features at multiple scales, leading to large improvements in performance and opening up the door to new applications. This chapter explores the use of deep learning-based semantic segmentation in Earth observation imagery and presents in detail three approaches specifically aimed at Earth observation applications.

We study adaptive sensing of Cox point processes, a widely used model from spatial statistics. We introduce three tasks: maximization of captured events, search for the maximum of the intensity function and learning level sets of the intensity function. We model the intensity function as a sample from a truncated Gaussian process, represented in a specially constructed positive basis. In this basis, the positivity constraint on the intensity function has a simple form. We show how the minimal description positive basis can be adapted to the covariance kernel, to non-stationarity and make connections to common positive bases from prior works. Our adaptive sensing algorithms use Langevin dynamics and are based on posterior sampling (Cox-TnomPsoN) and top-two posterior sampling (ToP2) principles. With latter, the difference between samples serves as a surrogate to the uncertainty. We demonstrate the approach using examples from environmental monitoring and crime rate modeling, and compare it to the classical Bayesian experimental design approach. ISSN:2640-3498

El aprendizaje automático ha hecho grandes avances en la ciencia e ingeniería actuales en general y en las ciencias de la Tierra en particular. Sin embargo, los datos de la Tierra plantean problemas particularmente difíciles para el aprendizaje automático debido no sólo al volumen de datos implicado, sino también por la presencia de correlaciones no lineales tanto espaciales como temporales, por una gran diversidad de fuentes de ruido y de incertidumbre, así como por la heterogeneidad de las fuentes de información involucradas. Más datos no implica necesariamente más información. Por lo tanto, extraer conocimiento y contenido informativo mediante el análisis y el modelado de datos resulta crucial, especialmente ahora donde el volumen y la heterogeneidad de los datos aumentan constantemente. Este hecho requiere avances en métodos que puedan cuantificar la información y caracterizar las distribuciones e incertidumbres con precisión. Cuantificar el contenido informativo a los datos y los modelos de nuestro sistema son problemas no resueltos en estadística y el aprendizaje automático. Esta tesis introduce nuevos modelos de aprendizaje automático para extraer conocimiento e información a partir de datos de observación de la Tierra. Proponemos métodos núcleo ('kernel methods'), procesos gaussianos y gaussianización multivariada para tratar la incertidumbre y la cuantificación de la información, y aplicamos estos métodos a una amplia gama de problemas científicos del sistema terrestre. Estos conllevan muchos tipos de problemas de aprendizaje, incluida la clasificación, regresión, estimación de densidad, síntesis, propagación de errores y estimación de medidas teóricas de la información. También demostramos cómo funcionan estos métodos con diferentes fuentes de datos, provenientes de distintos sensores (radar, multiespectrales, hiperespectrales), productos de datos (observaciones, reanálisis y simulaciones de modelos) y cubos de datos (agregados de varias fuentes de datos espacial-temporales ). Las metodologías presentadas nos permiten cuantificar y visualizar cuáles son las características relevantes que gobiernan distintos métodos núcleo, tales como clasificadores, métodos de regresión o incluso las medidas de independencia estadística, como propagar mejor los errores y las distorsiones de los datos de entrada con procesos gaussianos, así como dónde y cuándo se puede encontrar más información en cubos arbitrarios espacio-temporales. Las técnicas presentadas abren una amplia gama de posibles casos de uso y de aplicaciones, con las que prevemos un uso más extenso y robusto de algoritmos estadísticos en las ciencias de la Tierra y el clima. Machine learning has made great strides in today's Science and engineering in general and Earth Sciences in particular. However, Earth data poses particularly challenging problems for machine learning due to not only the volume of data, but also the spatial-temporal nonlinear correlations, noise and uncertainty sources, and heterogeneous sources of information. More data does not necessarily imply more information. Therefore, extracting knowledge and information content using data analysis and modeling is important and is especially prevalent in an era where data volume and heterogeneity is steadily increasing. This calls for advances in methods that can quantify information and characterize distributions accurately. Quantifying information content within our system's data and models are still unresolved problems in statistics and machine learning. This thesis introduces new machine learning models to extract knowledge and information from Earth data. We propose kernel methods, Gaussian processes and multivariate Gaussianization to handle uncertainty and information quantification and we apply these methods to a wide range of Earth system science problems. These involve many types of learning problems including classification, regression, density estimation, synthesis, error propagation and information-theoretic measures estimation. We also demonstrate how these methods perform with different data sources including sensory data (radar, multispectral, hyperspectral, infrared sounders), data products (observations, reanalysis and model simulations) and data cubes (aggregates of various spatial-temporal data sources). The presented methodologies allow us to quantify and visualize what are the salient features driving kernel classifiers, regressors or dependence measures, how to better propagate errors and distortions of input data with Gaussian processes, and where and when more information can be found in arbitrary spatial-temporal data cubes. The presented techniques open a wide range of possible use cases and applications and we anticipate a wider adoption in the Earth sciences.

In times of global change, we must closely monitor the state of our planet in order to understand gradual or abrupt changes early on. In fact, each of the Earth's subsystems-i.e. the biosphere, atmosphere, hydrosphere, cryosphere, and anthroposphere-can be analyzed from a multitude of data streams. However, since it is very hard to jointly interpret multiple monitoring data streams in parallel, one often aims for some summarizing indicator. Climate indices, for example, summarize the state of atmospheric circulation in a region, e.g. the Multivariate ENSO (El Ñino-Southern Oscillation) Index. Indicator approaches have been used extensively to describe socioeconomic data too, and a range of indices have been proposed to synthesize and interpret this information. For instance the "Human Development Index" (HDI) by the United Nations Development Programme was designed to capture specific aspects of development. "Dimensionality reduction" (DR) is a widely used approach to find low dimensional and interpretable representations of data that are natively embedded in high-dimensional spaces. Here, we propose a robust method to create indicators using dimensionality reduction to better represent the terrestrial biosphere and the global socioeconomic system. We aim to explore the performance of the approach and to interpret the resulting indicators. For biosphere indicators, the concept was tested using 12 explanatory variables representing the biophysical states of ecosystems and land-atmosphere water, energy, and carbon fluxes. We find that two indicators account for 73% of the variance of the state of the biosphere in space and time. While the first indicator summarizes productivity patterns, the second indicator summarizes variables representing water and energy availability. Anomalies in the indicators clearly identify extreme events, such as the Amazon droughts (2005 and 2010) and the Russian heatwave (2010), they also allow us to interpret the impacts of these events. The indicators also reveal changes in the seasonal cycle, e.g. increasing seasonal amplitudes of productivity in agricultural areas and in arctic regions. We also apply the method on the "World Development Indicators", a database with more than 1500 variables, to track the socioeconomic development at a country level. The aim was to extract the core dimensions of development in a highly efficient way, using a method of nonlinear dimensionality reduction. We find that over 90% of variance in the WDIs can be represented by five uncorrelated and nonlinear dimensions. The first dimension (explaining 74%) represents the state of education, health, income, infrastructure, trade, population, and pollution. The second dimension (explaining 10%) differentiates countries by gender ratios, labor market, and energy production patterns. Overall, we find that the data contained in the WDIs are highly nonlinear therefore requiring nonlinear methods to extract the main patterns of development. Globally, most countries show rather consistent temporal trends towards wealthier and aging societies. Deviations from the long-term trajectories are detected with our approach during warfare, environmental disasters, or fundamental political changes. In general we find that the indicator approach is able to extract general patterns from complex databases and that it can be applied to databases of varying characteristics. We also find that indicators are can different kinds of changes occurring in the system, such as extreme events, permanent changes or trends. Therefore it is a useful tool for general monitoring and exploratory data analysis. The approach is flexible and can be applied to complex datasets, such as large data, nonlinear data, as well as data with many missing values. In times of global change, we must closely monitor the state of our planet in order to understand gradual or abrupt changes early on. In fact, each of the Earth's subsystems-i.e. the biosphere, atmosphere, hydrosphere, cryosphere, and anthroposphere-can be analyzed from a multitude of data streams. However, since it is very hard to jointly interpret multiple monitoring data streams in parallel, one often aims for some summarizing indicator. Climate indices, for example, summarize the state of atmospheric circulation in a region, e.g. the Multivariate ENSO (El Ñino-Southern Oscillation) Index. Indicator approaches have been used extensively to describe socioeconomic data too, and a range of indices have been proposed to synthesize and interpret this information. For instance the "Human Development Index" (HDI) by the United Nations Development Programme was designed to capture specific aspects of development. "Dimensionality reduction" (DR) is a widely used approach to find low dimensional and interpretable representations of data that are natively embedded in high-dimensional spaces. Here, we propose a robust method to create indicators using dimensionality reduction to better represent the terrestrial biosphere and the global socioeconomic system. We aim to explore the performance of the approach and to interpret the resulting indicators. For biosphere indicators, the concept was tested using 12 explanatory variables representing the biophysical states of ecosystems and land-atmosphere water, energy, and carbon fluxes. We find that two indicators account for 73% of the variance of the state of the biosphere in space and time. While the first indicator summarizes productivity patterns, the second indicator summarizes variables representing water and energy availability. Anomalies in the indicators clearly identify extreme events, such as the Amazon droughts (2005 and 2010) and the Russian heatwave (2010), they also allow us to interpret the impacts of these events. The indicators also reveal changes in the seasonal cycle, e.g. increasing seasonal amplitudes of productivity in agricultural areas and in arctic regions. We also apply the method on the "World Development Indicators", a database with more than 1500 variables, to track the socioeconomic development at a country level. The aim was to extract the core dimensions of development in a highly efficient way, using a method of nonlinear dimensionality reduction. We find that over 90% of variance in the WDIs can be represented by five uncorrelated and nonlinear dimensions. The first dimension (explaining 74%) represents the state of education, health, income, infrastructure, trade, population, and pollution. The second dimension (explaining 10%) differentiates countries by gender ratios, labor market, and energy production patterns. Overall, we find that the data contained in the WDIs are highly nonlinear therefore requiring nonlinear methods to extract the main patterns of development. Globally, most countries show rather consistent temporal trends towards wealthier and aging societies. Deviations from the long-term trajectories are detected with our approach during warfare, environmental disasters, or fundamental political changes. In general we find that the indicator approach is able to extract general patterns from complex databases and that it can be applied to databases of varying characteristics. We also find that indicators are can different kinds of changes occurring in the system, such as extreme events, permanent changes or trends. Therefore it is a useful tool for general monitoring and exploratory data analysis. The approach is flexible and can be applied to complex datasets, such as large data, nonlinear data, as well as data with many missing values.

The terrestrial biosphere is currently slowing down global warming by absorbing about 30% of human emissions of carbon dioxide (CO2). The largest flux of the terrestrial carbon uptake is gross primary production (GPP) defined as the production of carbohydrates by photosynthesis. Elevated atmospheric CO2 concentration is expected to increase GPP (“CO2 fertilization effect”). However, Earth system models (ESMs) exhibit a large range in simulated GPP projections. In this study, we combine an existing emergent constraint on CO2 fertilization with a machine learning approach to constrain the spatial variations of multimodel GPP projections. In a first step, we use observed changes in the CO2 seasonal cycle at Cape Kumukahi to constrain the global mean GPP at the end of the 21st century (2091–2100) in Representative Concentration Pathway 8.5 simulations with ESMs participating in the Coupled Model Intercomparison Project Phase 5 (CMIP5) to 171 ± 12 Gt C yr−1, compared to the unconstrained model range of 156–247 Gt C yr−1. In a second step, we use a machine learning model to constrain gridded future absolute GPP and gridded fractional GPP change in two independent approaches. For this, observational data are fed into the machine learning algorithm that has been trained on CMIP5 data to learn relationships between present‐day physically relevant diagnostics and the target variable. In a leave‐one‐model‐out cross‐validation approach, the machine learning model shows superior performance to the CMIP5 ensemble mean. Our approach predicts an increased GPP change in northern high latitudes compared to regions closer to the equator., Plain Language Summary: About a quarter of human emissions of carbon dioxide (CO2) is absorbed by vegetation and soil on the Earth's surface and hence does not contribute to global warming caused by CO2 in the atmosphere. Thus, in order to better define the remaining carbon budgets left to meet particular warming targets like the 1.5°C of the Paris Agreement, it is important to accurately quantify the carbon uptake by plants in the future. Currently, this is modeled by Earth system models yet with great uncertainties. In this work, we present an alternative machine learning approach to reduce spatial uncertainties of vegetation carbon uptake in future climate projections using observations of today's conditions., Key Points: An emergent constraint on CO2 seasonal cycle amplitude changes reduces uncertainties in global mean gross primary production projections. A machine learning model with multiple predictors can further constrain the spatial distribution of gross primary production. High‐latitude ecosystems show higher gross primary production increase over the 21st century compared to regions closer to the equator., EC | Horizon 2020 Framework Programme 4C, EC | Horizon 2020 Framework Programme CRESCENDO, ERC Consolidator Grant SEDAL, ERC Synergy Grant USMILE

L’observació de la Terra a partir de les dades proporcionades per sensors abord de satèl·lits, així com les proporcionades per models de transferència radiativa o climàtics, juntament amb les mesures in situ proporcionen una manera sense precedents de monitorar el nostre planeta amb millors resolucions espacials i temporals. La riquesa, quantitat i diversitat de les dades adquirides i posades a disposició també augmenta molt ràpidament. Aquestes dades ens permeten predir el rendiment dels cultius, fer un seguiment del canvi d’ús del sòl com ara la desforestació, supervisar i respondre als desastres naturals, i predir i mitigar el canvi climàtic. Per tal de fer front a tots aquests reptes, les dues darreres dècades han evidenciat un gran augment en l'aplicació d'algorismes d'aprenentatge automàtic en l'observació de la Terra. Amb l'anomenat `machine learning' es pot fer un ús eficient del flux de dades creixent en quantitat i diversitat. Els algorismes d'aprenentatge màquina, però, solen ser models agnòstics i massa flexibles i, per tant, acaben per no respectar les lleis fonamentals de la física. D’altra banda, en els darrers anys s’ha produït un augment de la investigació que intenta integrar el coneixement de física en algorismes d’aprenentatge, amb la finalitat d’obtenir solucions interpretables i que tinguin sentit físic. L’objectiu principal d’aquesta tesi és dissenyar diferents maneres de codificar el coneixement físic per proporcionar mètodes d’aprenentatge automàtic adaptats a problemes específics en teledetecció. Introduïm nous mètodes que poden fusionar de manera òptima fonts de dades heterogènies, explotar les regularitats de dades, incorporar equacions diferencials, obtenir models precisos que emulen, i per tant són coherents amb models físics, i models que aprenen parametrizacions del sistema combinant models i simulacions. Earth observation through satellite sensors, models and in situ measurements provides a way to monitor our planet with unprecedented spatial and temporal resolution. The amount and diversity of the data which is recorded and made available is ever-increasing. This data allows us to perform crop yield prediction, track land-use change such as deforestation, monitor and respond to natural disasters and predict and mitigate climate change. The last two decades have seen a large increase in the application of machine learning algorithms in Earth observation in order to make efficient use of the growing data-stream. Machine learning algorithms, however, are typically model agnostic and too flexible and so end up not respecting fundamental laws of physics. On the other hand there has, in recent years, been an increase in research attempting to embed physics knowledge in machine learning algorithms in order to obtain interpretable and physically meaningful solutions. The main objective of this thesis is to explore different ways of encoding physical knowledge to provide machine learning methods tailored for specific problems in remote sensing. Ways of expressing expert knowledge about the relevant physical systems in remote sensing abound, ranging from simple relations between reflectance indices and biophysical parameters to complex models that compute the radiative transfer of electromagnetic radiation through our atmosphere, and differential equations that explain the dynamics of key parameters. This thesis focuses on inversion problems, emulation of radiative transfer models, and incorporation of the abovementioned domain knowledge in machine learning algorithms for remote sensing applications. We explore new methods that can optimally model simulated and in-situ data jointly, incorporate differential equations in machine learning algorithms, handle more complex inversion problems and large-scale data, obtain accurate and computationally efficient emulators that are consistent with physical models, and that efficiently perform approximate Bayesian inversion over radiative transfer models.

[ES] Reseña de tesis doctoral defendida el 17 de Julio de 2017. Lugar: Facultat de Física, Universitat de València., The research leading to these results was conducted within the ERMES FP7 project (http://www. ermes-fp7space.eu/) which received funding from the European Union Seventh Framework Program (FP7/2007-2013) under grant agreement 606983. This work was also supported by the European Research Council (ERC) funding under the ERC Consolidator Grant 2014 SEDAL (Statistical Learning for Earth Observation Data Analysis) project under Grant Agreement 647423 and by the European Organization for the Exploitation of Meteorological Satellites (EUMETSAT) through the LSA SAF (Land Surface Analysis Satellite Applications Facility) project.

Technological advances in the last decades have improved our capabilities of collecting and storing high data volumes. However, this makes that in some fields, such as remote sensing several problems are generated in the data processing due to the peculiar characteristics of their data. High data volume, high dimensionality, heterogeneity and their nonlinearity, make that the analysis and extraction of relevant information from these images could be a bottleneck for many real applications. The research applying image processing and machine learning techniques along with feature extraction, allows the reduction of the data dimensionality while keeps the maximum information. Therefore, developments and applications of feature extraction methodologies using these techniques have increased exponentially in remote sensing. This improves the data visualization and the knowledge discovery. Several feature extraction methods have been addressed in the literature depending on the data availability, which can be classified in supervised, semisupervised and unsupervised. In particular, feature extraction can use in combination with kernel methods (nonlinear). The process for obtaining a space that keeps greater information content is facilitated by this combination. One of the most important properties of the combination is that can be directly used for general tasks including classification, regression, clustering, ranking, compression, or data visualization. In this Thesis, we address the problems of different nonlinear feature extraction approaches based on kernel methods for remote sensing data analysis. Several improvements to the current feature extraction methods are proposed to transform the data in order to make high dimensional data tasks easier, such as classification or biophysical parameter estimation. This Thesis focus on three main objectives to reach these improvements in the current feature extraction methods: The first objective is to include invariances into supervised kernel feature extraction methods. Throughout these invariances it is possible to generate virtual samples that help to mitigate the problem of the reduced number of samples in supervised methods. The proposed algorithm is a simple method that essentially generates new (synthetic) training samples from available labeled samples. These samples along with original samples should be used in feature extraction methods obtaining more independent features between them that without virtual samples. The introduction of prior knowledge by means of the virtual samples could obtain classification and biophysical parameter estimation methods more robust than without them. The second objective is to use the generative kernels, i.e. probabilistic kernels, that directly learn by means of clustering techniques from original data by finding local-to-global similarities along the manifold. The proposed kernel is useful for general feature extraction purposes. Furthermore, the kernel attempts to improve the current methods because the kernel not only contains labeled data information but also uses the unlabeled information of the manifold. Moreover, the proposed kernel is parameter free in contrast with the parameterized functions such as, the radial basis function (RBF). Using probabilistic kernels is sought to obtain new unsupervised and semisupervised methods in order to reduce the number and cost of labeled data in remote sensing. Third objective is to develop new kernel feature extraction methods for improving the features obtained by the current methods. Optimizing the functional could obtain improvements in new algorithm. For instance, the Optimized Kernel Entropy Component Analysis (OKECA) method. The method is based on the Independent Component Analysis (ICA) framework resulting more efficient than the standard Kernel Entropy Component Analysis (KECA) method in terms of dimensionality reduction. In this Thesis, the methods are focused on remote sensing data analysis. Nevertheless, feature extraction methods are used to analyze data of several research fields whereas data are multidimensional. For these reasons, the results are illustrated into experimental sequence. First, the projections are analyzed by means of Toy examples. The algorithms are tested through standard databases with supervised information to proceed to the last step, the analysis of remote sensing images by the proposed methods.

This thesis summarizes different works developed in the framework of analyzing the relation between image processing, statistics and neuroscience. These relations are analyzed from the efficient coding hypothesis point of view (H. Barlow [1961] and Attneave [1954]). This hypothesis suggests that the human visual system has been adapted during the ages in order to process the visual information in an efficient way, i.e. taking advantage of the statistical regularities of the visual world. Under this classical idea different works in different directions are developed. One direction is analyzing the statistical properties of a revisited, extended and fitted classical model of the human visual system. No statistical information is used in the model. Results show that this model obtains a representation with good statistical properties, which is a new evidence in favor of the efficient coding hypothesis. From the statistical point of view, different methods are proposed and optimized using natural images. The models obtained using these statistical methods show similar behavior to the human visual system, both in the spatial and color dimensions, which are also new evidences of the efficient coding hypothesis. Applications in image processing are an important part of the Thesis. Statistical and neuroscience based methods are employed to develop a wide set of image processing algorithms. Results of these methods in denoising, classification, synthesis and quality assessment are comparable to some of the most successful current methods.

La citricultura constituye actualmente una parte importantísima de la economía en la Comunidad Valenciana. Uno de los principales problemas presentes en la industria citrícola son las podredumbres producidas por los hongos Penicillium digitatum y Penicillium italicum. La detección temprana del daño producido por los hongos en los cítricos es especialmente importante para la industria distribuidora de cítricos, ya que un reducido número de frutas infectadas puede diseminar la infección a toda una partida de cítricos, produciendo grandes pérdidas económicas. Actualmente en la industria citrícola la inspección y eliminación de cítricos dañados por podredumbres se realiza de manera manual por operarios, ya que en etapas tempranas el daño producido por el hongo es difícilmente detectable por sistemas de inspección automáticos. En esta Tesis Doctoral se va a desarrollar un sistema de visión hiperespectral basado en filtros de cristal líquido sintonizables y se van a estudiar técnicas para procesar las imágenes proporcionadas por el mismo de forma automática. La finalidad de dichas técnicas será obtener un grupo reducido y lo suficientemente informativo de bandas espectrales mediante métodos de selección de características que permita detectar los frutos afectados por podredumbres tempranas mediante algoritmos automáticos de clasificación., The citrus industry is now an important part of the economy in Spain. One of the main problems present in the citrus industry are the rottenness produced by Penicillium digitatum and Penicillium italicum fungi. Early detection of damage produced by fungi in citrus is especially important for the distribution of citrus industry, because a small number of infected fruit can spread infection to a batch of citrus, causing great economic losses. Nowadays the inspection and removal of damaged fruit is performed manually by operators since the damage caused by the fungi is hardly detectable by automatic inspection systems. In this PhD thesis, a hyperspectral vision system based on liquid crystal tunable filters will be developed and techniques for processing the images provided by it automatically will be studied. The purpose of these techniques will get a small and informative spectral bands using feature selection methods. Rottenness early detection will be performed using linear and non linear classification algorithms.

Earth Observation systems monitor our Planet by measuring, at different wavelengths, the electromagnetic radiation that is reflected by the surface, crosses the atmosphere, and reaches the sensor at the satellite platform. In this process, clouds are one of the most important components of the Earth's atmosphere affecting the quality of the measured electromagnetic signal and, consequently, the properties retrieved from these signals. This Thesis faces the challenging problem of cloud screening in multispectral and hyperspectral images acquired by space-borne sensors working in the visible and near-infrared range of the electromagnetic spectrum. The main objective is to provide new operational cloud screening tools for the derivation of cloud location maps from these sensors' data. Moreover, the method must provide cloud abundance maps --instead of a binary classification-- to better describe clouds (abundance, type, height, subpixel coverage), thus allowing the retrieval of surface biophysical parameters from satellite data acquired over land and ocean. In this context, this Thesis is intended to support the growing interest of the scientific community in two multispectral sensors on board two satellites of the European Space Agency (ESA). The first one is the MEdium Resolution Imaging Spectrometer (MERIS), placed on board the biggest environmental satellite ever launched, ENVISAT. The second one is the Compact High Resolution Imaging Spectrometer (CHRIS) hyperspectral instrument, mounted on board the technology demonstration mission PROBA (Project for On-Board Autonomy). The proposed cloud screening algorithm takes advantage of the high spectral and radiometric resolution of MERIS, and of the high number of spectral bands of CHRIS, as well as the specific location of some bands (e.g., oxygen and water vapor absorption bands) to increase the cloud detection accuracy. To attain this objective, advanced pattern recognition and machine learning techniques to detect clouds are specifically developed in the frame of this Thesis. First, a feature extraction based on meaningful physical facts is carried out in order to provide informative inputs to the algorithms. Then, the cloud screening algorithm is conceived trying to make use of the wealth of unlabeled samples in Earth Observation images, and thus unsupervised and semi-supervised learning methods are explored. Results show that applying unsupervised clustering methods over the whole image allows us to take advantage of the wealth of information and the high degree of spatial and spectral correlation of the image pixels, while semi-supervised learning methods offer the opportunity of exploiting also the available labeled samples., Los sistemas de observación de la Tierra observan nuestro planeta midiendo, en diferentes longitudes de onda, la radiación electromagnética que es reflejada por la superficie, atraviesa la atmósfera, y llega al sensor en el satélite. En este proceso, las nubes son uno de los componentes más importantes ya que afectan a la señal electromagnética medida. Esta Tesis aborda el problema de la detección de nubes en imágenes multiespectrales e hiperespectrales adquiridas por sensores de satélite que trabajan en el rango visible e infrarrojo cercano del espectro electromagnético. El objetivo principal es desarrollar herramientas operativas que proporcionen mapas con la ubicación de las nubes a partir de los datos de satélite adquiridos. Además, el método debe proporcionar mapas de la abundancia de nubes (en lugar de una clasificación binaria) para describir mejor las nubes (abundancia, tipo, altura, cobertura subpixel). En este contexto, esta Tesis analiza dos sensores multiespectrales a bordo de dos satélites de la Agencia Espacial Europea (ESA). El primero de ellos es el MEdium Resolution Imaging Spectrometer (MERIS), a bordo del satélite medioambiental Envisat. El segundo es el instrumento hiperespectral Compact High Resolution Imaging Spectrometer (CHRIS), a bordo del satélite de demostración tecnológica PROBA. El algoritmo de detección de nubes propuesto aprovecha la alta resolución espectral y radiométrica de MERIS, y el número de bandas espectrales de CHRIS, así como la ubicación específica de algunas bandas (absorción del oxígeno y vapor de agua) para aumentar la precisión de la detección de las nubes. Para alcanzar este objetivo, técnicas avanzadas de reconocimiento de patrones y aprendizaje máquina han sido específicamente desarrolladas en el marco de esta. En primer lugar, se lleva a cabo la extracción de características basadas en fundamentos físicos con el fin de proporcionar información relevante a los algoritmos empleados. Después, el algoritmo de detección de nubes se ha concebido tratando de hacer uso de la riqueza de las muestras sin etiquetar de las imágenes, por lo que se exploran métodos de aprendizaje no supervisados y semisupervisados. Los resultados muestran que los algoritmos de agrupamientos no supervisados sobre la imagen completa nos permite aprovechar la riqueza de la información y el alto grado de correlación espacial y espectral de los píxeles de la imagen, mientras que los métodos de aprendizaje semisupervisado ofrecen también la oportunidad de explotar las muestras etiquetadas disponibles.