Search

Bivariate causal discovery amounts to inferring the causal association between two random variables, usually from observational data. This task is the simplest and most fundamental causal discovery problem from which more complex discovery methods can be envisioned and developed. Classical bivariate causal discovery methods exploit a combination of specific sets of assumptions and data to obtain identifiability of the causal direction. Data-driven supervised approaches train machine learning models over large sets of causally-labeled bivariate datasets to learn the task of inferring the causal relationship from data. In this work, an ensemble algorithm based on support measure machines is proposed with the aim of combining the strength of different classical approaches (base methods) with data-driven decisions. In particular, support measure machine classifiers are trained to estimate the performance of each base method. Their decision functions are then used as data-dependent weights of a weighted voting scheme to estimate the causal direction in a bivariate causal discovery problem. This work demonstrates that the proposed algorithm, denoted as Causal Ensemble Measure Machine, performs equal to or better than state-of-the-art methods on a wide range of synthetic and real-world bivariate problems. Perhaps more importantly, this method enables a closer examination of the assumption dependence of existing algorithms on observational data. • New methodology for pairwise causal discovery based on kernel mean embeddings. • Optimized ensemble method combining base causal discovery approaches. • State-of-the-art performance and interpretable results in many bench- mark datasets. [ABSTRACT FROM AUTHOR]

Current anomaly detection (AD) algorithms are typically challenged by either accuracy or efficiency. More accurate nonlinear detectors are typically slow and not scalable. In this letter, we propose two families of techniques to improve the efficiency of the standard kernel Reed–Xiaoli (KRX) method for AD by approximating the kernel function with either the data-independent random Fourier features or the data-dependent basis with the Nyström approach. We compare all methods for both real multi- and hyperspectral images. We show that the proposed efficient methods have a lower computational cost, and they perform similar to (or outperform) the standard KRX algorithm thanks to their implicit regularization effect. Last but not least, the Nyström approach has an improved power of detection. [ABSTRACT FROM AUTHOR]

Remote sensing observations, products, and simulations are fundamental sources of information to monitor our planet and its climate variability. Uncovering the main modes of spatial and temporal variability in Earth data is essential to analyze and understand the underlying physical dynamics and processes driving the Earth System. Dimensionality reduction methods can work with spatio-temporal data sets and decompose the information efficiently. Principal component analysis (PCA), also known as empirical orthogonal functions (EOFs) in geophysics, has been traditionally used to analyze climatic data. However, when nonlinear feature relations are present, PCA/EOF fails. In this article, we propose a nonlinear PCA method to deal with spatio-temporal Earth system data. The proposed method, called rotated complex kernel PCA (ROCK-PCA for short), works in reproducing kernel Hilbert spaces to account for nonlinear processes, operates in the complex kernel domain to account for both space and time features, and adds an extra rotation for improved flexibility. The result is an explicitly resolved spatio-temporal decomposition of the Earth data cube. The method is unsupervised and computationally very efficient. We illustrate its ability to uncover spatio-temporal patterns using synthetic experiments and real data. Results of the decomposition of three essential climate variables are shown: satellite-based global gross primary productivity (GPP), soil moisture (SM), and reanalysis sea surface temperature (SST) data. The ROCK-PCA method allows identifying their annual and seasonal oscillations, as well as their nonseasonal trends and spatial variability patterns. The main modes of variability of GPP and SM match expected distributions of land-cover and eco-hydrological zones, respectively; the interannual component of SM is shown to be highly correlated with El Niño Southern Oscillation (ENSO) phenomenon; and the SST annual oscillation is perfectly uncoupled in magnitude and phase from the global warming trend and ENSO anomalies, as well as from their mutual interactions. We provide the working source code of the presented method for the interested reader in https://github.com/DiegoBueso/ROCK-PCA. [ABSTRACT FROM AUTHOR]

In many remote sensing applications, one wants to estimate variables or parameters of interest from observations. When the target variable is available at a resolution that matches the remote sensing observations, standard algorithms, such as neural networks, random forests, or the Gaussian processes, are readily available to relate the two. However, we often encounter situations where the target variable is only available at the group level, i.e., collectively associated with a number of remotely sensed observations. This problem setting is known in statistics and machine learning as multiple instance learning (MIL) or distribution regression (DR). This article introduces a nonlinear (kernel-based) method for DR that solves the previous problems without making any assumption on the statistics of the grouped data. The presented formulation considers distribution embeddings in reproducing kernel Hilbert spaces and performs standard least squares regression with the empirical means therein. A flexible version to deal with multisource data of different dimensionality and sample sizes is also presented and evaluated. It allows working with the native spatial resolution of each sensor, avoiding the need for matchup procedures. Noting the large computational cost of the approach, we introduce an efficient version via random Fourier features to cope with millions of points and groups. Real experiments involve the Soil Moisture Active Passive (SMAP) vegetation optical depth (VOD) data for the estimation of crop production in the U.S. Corn Belt and the Moderate Resolution Imaging Spectroradiometer (MODIS) and Multi-angle Imaging SpectroRadiometer (MISR) reflectances for the estimation of aerosol optical depth (AOD). An exhaustive empirical evaluation of the method is done against naive (linear and nonlinear) approaches based on input-space means as well as previously presented methods for MIL. We provide source code of our methods in http://isp.uv.es/code/dr.html. [ABSTRACT FROM AUTHOR]

Anomalous change detection (ACD) is an important problem in remote sensing image processing. Detecting not only pervasive but also anomalous or extreme changes has many applications for which methodologies are available. This paper introduces a nonlinear extension of a full family of anomalous change detectors. In particular, we focus on algorithms that utilize Gaussian and elliptically contoured (EC) distribution and extend them to their nonlinear counterparts based on the theory of reproducing kernels’ Hilbert space. We illustrate the performance of the kernel methods introduced in both pervasive and ACD problems with real and simulated changes in multispectral and hyperspectral imagery with different resolutions (AVIRIS, Sentinel-2, WorldView-2, and Quickbird). A wide range of situations is studied in real examples, including droughts, wildfires, and urbanization. Excellent performance in terms of detection accuracy compared to linear formulations is achieved, resulting in improved detection accuracy and reduced false-alarm rates. Results also reveal that the EC assumption may be still valid in Hilbert spaces. We provide an implementation of the algorithms as well as a database of natural anomalous changes in real scenarios http://isp.uv.es/kacd.html. [ABSTRACT FROM AUTHOR]

In this paper, we analyze the effect of spatial and spectral compression on the performance of statistically based retrieval. Although the quality of the information is not completely preserved during the coding process, experiments reveal that a certain amount of compression may yield a positive impact on the accuracy of retrievals. We unveil two strategies, both with interesting benefits: either to apply a very high compression, which still maintains the same retrieval performance as that obtained for uncompressed data; or to apply a moderate to high compression, which improves the performance. As a second contribution of this paper, we focus on the origins of these benefits. On the one hand, we show that a certain amount of noise is removed during the compression stage, which benefits the retrievals performance. On the other hand, we analyze the effect of compression on spectral/spatial regularization (smoothing). We quantify the amount of information shared among the spatial neighbors for the different methods and compression ratios. We also propose a simple strategy to specifically exploit spectral and spatial relations and find that, when these relations are taken into account beforehand, the benefits of compression are reduced. These experiments suggest that compression can be understood as an indirect way to regularize the data and exploit spatial neighbors information, which improves the performance of pixelwise statistics-based retrieval algorithms. [ABSTRACT FROM AUTHOR]

Earth observation from satellite sensory data poses challenging problems, where machine learning is currently a key player. In recent years, Gaussian Process (GP) regression has excelled in biophysical parameter estimation tasks from airborne and satellite observations. GP regression is based on solid Bayesian statistics, and generally yields efficient and accurate parameter estimates. However, GPs are typically used for inverse modeling based on concurrent observations and in situ measurements only. Very often a forward model encoding the well-understood physical relations between the state vector and the radiance observations is available though and could be useful to improve predictions and understanding. In this work, we review three GP models that respect and learn the physics of the underlying processes in the context of both forward and inverse modeling . After reviewing the traditional application of GPs for parameter retrieval, we introduce a Joint GP (JGP) model that combines in situ measurements and simulated data in a single GP model. Then, we present a latent force model (LFM) for GP modeling that encodes ordinary differential equations to blend data-driven modeling and physical constraints of the system governing equations. The LFM performs multi-output regression, adapts to the signal characteristics, is able to cope with missing data in the time series, and provides explicit latent functions that allow system analysis and evaluation. Finally, we present an Automatic Gaussian Process Emulator (AGAPE) that approximates the forward physical model using concepts from Bayesian optimization and at the same time builds an optimally compact look-up-table for inversion. We give empirical evidence of the performance of these models through illustrative examples of vegetation monitoring and atmospheric modeling. [ABSTRACT FROM AUTHOR]

Solving inverse problems is central in geosciences and remote sensing. The radiative transfer models (RTMs) represent mathematically the physical laws that rule the phenomena in remote sensing applications (forward models). The numerical inversion of the RTM equations is a challenging and computationally demanding problem. For this reason, often the application of a simpler statistical regression is preferred. In general, the regression models predict the biophysical parameter of interest from the corresponding received radiance, learning a mapping from in situ data. However, this approach does not employ the physical information encoded in the RTMs. An alternative strategy, which attempts to include the physical knowledge, consists in learning a regression model trained using simulated data by an RTM code. In this paper, we introduce a nonlinear nonparametric regression model that combines the benefits of the two aforementioned approaches. The inversion is performed considering jointly both real observations and RTMsimulated data. The proposed joint Gaussian process (JGP) provides a solid framework for exploiting the regularities between the two types of data, in order to perform inverse modeling. The JGP automatically detects the relative quality of the simulated and real data, and combines them properly. This occurs by learning an additional hyperparameter with respect to a standard Gaussian process model, so that the novel scheme is at the same time simple and robust, i.e., capable of adapting to different scenarios. The advantages of the JGP method compared with benchmark strategies are shown considering synthetic and real data in different experiments. Specifically, we consider leaf area index retrieval from Landsat data combined with simulated data generated by the PROSAIL model. [ABSTRACT FROM AUTHOR]

Automatic land cover classification from satellite image time series is of paramount relevance to assess vegetation and crop status, with important implications in agriculture, biofuels, and food. However, due to the high cost and human resources needed to characterize and classify land cover through field campaigns, a recurrent limiting factor is the lack of available labeled data. On top of this, the biophysical–geophysical variables exhibit particular temporal structures that need to be exploited. Land cover classification based on image time series is very complex because of the data manifold distortions through time. We propose the use of the kernel manifold alignment (KEMA) method for domain adaptation of remote sensing time series before classification. KEMA is nonlinear and semisupervised and reduces to solve a simple generalized eigenproblem. We give empirical evidence of performance through classification of biophysical (leaf area index, fraction of absorbed photosynthetically active radiation, fractional vegetation cover, and normalized difference vegetation index) time series on a global scale. [ABSTRACT FROM PUBLISHER]

The infrared atmospheric sounding interferometer (IASI) is flying on board of the Metop satellite series, which is part of the EUMETSAT Polar System. Products obtained from IASI data represent a significant improvement in the accuracy and quality of the measurements used for meteorological models. Notably, the IASI collects rich spectral information to derive temperature and moisture profiles, among other relevant trace gases, essential for atmospheric forecasts and for the understanding of weather. Here, we investigate the impact of near-lossless and lossy compression on IASI L1C data when statistical retrieval algorithms are later applied. We search for those compression ratios that yield a positive impact on the accuracy of the statistical retrievals. The compression techniques help reduce certain amount of noise on the original data and, at the same time, incorporate spatial–spectral feature relations in an indirect way without increasing the computational complexity. We observed that compressing images, at relatively low bit rates, improves results in predicting temperature and dew point temperature, and we advocate that some amount of compression prior to model inversion is beneficial. This research can benefit the development of current and upcoming retrieval chains in infrared sounding and hyperspectral sensors. [ABSTRACT FROM PUBLISHER]

Gaussian process regression (GPR) has experienced tremendous success in biophysical parameter retrieval in the past years. The GPR provides a full posterior predictive distribution so one can derive mean and variance predictive estimates, i.e., point-wise predictions and associated confidence intervals. GPR typically uses translation invariant covariances that make the prediction function very flexible and nonlinear. This, however, makes the relative relevance of the input features hardly accessible, unlike in linear prediction models. In this paper, we introduce the sensitivity analysis of the GPR predictive mean and variance functions to derive feature rankings and spectral spacings, respectively. The methodology can be used to uncover knowledge in any kernel-based regression method, it is fast to compute, and it is expressed in closed-form. The methodology is evaluated on GPR for global ocean chlorophyll prediction, revealing the most important spectral bands and their spectral spacings. We illustrate the (successful) methodology in several datasets and sensors. [ABSTRACT FROM PUBLISHER]

In this paper we present an approach to perform relative spectral alignment between optical cross-sensor acquisitions. The proposed method aims at projecting the images from two different and possibly disjoint input spaces into a common latent space, in which standard change detection algorithms can be applied. The system relies on the regularized kernel canonical correlation analysis transformation (kCCA), which can accommodate nonlinear dependencies between pixels by means of kernel functions. To learn the projections, the method employs a subset of samples belonging to the unchanged areas or to uninteresting radiometric differences. Since the availability of ground truth information to perform model selection is limited, we propose a completely automatic strategy to select the hyperparameters of the system as well as the dimensionality of the transformed (latent) space. The proposed scheme is fully automatic and allows the use of any change detection algorithm in the transformed latent space. A synthetic problem built from real images and a case study involving a real cross-sensor change detection problem illustrate the capabilities of the proposed method. Results show that the proposed system outperforms the linear baseline and provides accuracies close the ones obtained with a fully supervised strategy. We provide a MATLAB implementation of the proposed method as well as the real cross-sensor data we prepared and employed at https://sites.google.com/site/michelevolpiresearch/codes/cross-sensor . [ABSTRACT FROM AUTHOR]

Highlights • Kernel signal-to-noise ratio presented for some of the most relevant tasks in machine learning. • kSNR seeks to maximize the signal variance while minimizing the estimated noise variance explicitly in a rkHs. • kSNR provides a regularizer that successfully deals with non-linear signal-to-noise relations. • Noise estimation is evaluated for both implicit and explicit formulations. • Theoretical properties are analyzed to reduce the computational cost and to ensure the stability. Abstract This paper introduces the kernel signal-to-noise ratio (kSNR) for different machine learning and signal processing applications. The kSNR seeks to maximize the signal variance while minimizing the estimated noise variance explicitly in a reproducing kernel Hilbert space (rkHs). The kSNR gives rise to considering complex signal-to-noise relations beyond additive noise models, and can be seen as a useful regularizer for feature extraction and dimensionality reduction. We show that the kSNR generalizes kernel PCA (and other spectral dimensionality reduction methods), least squares SVM, and kernel ridge regression to deal with cases where signal and noise cannot be assumed independent. We give computationally efficient alternatives based on reduced-rank Nyström and projection on random Fourier features approximations, and analyze the bounds of performance and its stability. We illustrate the method through different examples, including nonlinear regression, nonlinear classification in channel equalization, nonlinear feature extraction from high-dimensional spectral satellite images, and bivariate causal inference. Experimental results show that the proposed kSNR yields more accurate solutions and extracts more noise-free features when compared to standard approaches. [ABSTRACT FROM AUTHOR]

Remote sensing image classification exploiting multiple sensors is a very challenging problem: data from different modalities are affected by spectral distortions and mis-alignments of all kinds, and this hampers re-using models built for one image to be used successfully in other scenes. In order to adapt and transfer models across image acquisitions, one must be able to cope with datasets that are not co-registered, acquired under different illumination and atmospheric conditions, by different sensors, and with scarce ground references. Traditionally, methods based on histogram matching have been used. However, they fail when densities have very different shapes or when there is no corresponding band to be matched between the images. An alternative builds upon manifold alignment . Manifold alignment performs a multidimensional relative normalization of the data prior to product generation that can cope with data of different dimensionality (e.g. different number of bands) and possibly unpaired examples. Aligning data distributions is an appealing strategy, since it allows to provide data spaces that are more similar to each other, regardless of the subsequent use of the transformed data. In this paper, we study a methodology that aligns data from different domains in a nonlinear way through kernelization . We introduce the Kernel Manifold Alignment (KEMA) method, which provides a flexible and discriminative projection map, exploits only a few labeled samples (or semantic ties) in each domain, and reduces to solving a generalized eigenvalue problem. We successfully test KEMA in multi-temporal and multi-source very high resolution classification tasks, as well as on the task of making a model invariant to shadowing for hyperspectral imaging. [ABSTRACT FROM AUTHOR]

Vegetation indices computed from spectral signatures are vastly used for monitoring the terrestrial biosphere. Indices are convenient proxies for canopy structure, and leaf pigment content, and consequently to estimate the photosynthetic activity of vegetation. Owing to its simplicity, the celebrated Normalized Difference Vegetation Index (NDVI) has been used as a proxy for greenness and canopy structure. Unfortunately, NDVI can only capture linear relationships of the near infrared (NIR) - red difference with the parameter of interest. To account for higher-order relations between the spectral channels, kernel NDVI (kNDVI) was proposed in (Camps-Valls et al., 2021). In this work, we give useful prescriptions for its proper use and show its good performance in a wider set of applications. We discuss the good characteristics of the index like boundedness, low error propagation. Furthermore, we give empirical evidence of performance in estimating in-situ vegetation parameters (leaf area index (LAI), gross primary productivity (GPP), leaf, and canopy chlorophyll content, green and total LAI and fraction of absorbed photosynthetically active radiation (fAPAR)) as well as the estimation of latent heat at flux tower level. We confirm the generally good performance of the index (correlation coefficient of kNDVI and canopy chlorophyll content is 0.919 and 0.933 for maize over two sites, as well as the correlation coefficient between kNDVI and carotenoid, is 0.816, 0.520 and 0.579 for three forest sites) and highlight its convenience in monitoring terrestrial ecosystems. To foster wider adoption of the new family of the index, we provide source code in 6 programming languages as well as efficient implementations in the Google Earth Engine (GEE) platform at https://github.com/IPL-UV/kNDVI. • The nonlinear kNDVI generalizes standard vegetation indices. • kNDVI propagates lower uncertainty and addresses saturation issues. • Prescriptions for the correct use of the index are given. • kNDVI yields improved performance to estimate in-situ vegetation parameters. • kNDVI improves estimation of latent heat at flux tower level. [ABSTRACT FROM AUTHOR]

• A general framework of empirical risk minimization with fairness regularizers and an analysis of its risk and fairness statistical consistency results are presented. • A Gaussian Process (GP) formulation of the fairness regularization framework is derived, which allows uncertainty quantification and principled hyperparameter selection. • A normalized version of the fairness regularizer which makes it less sensitive to the choice of kernel parameters is derived. Current adoption of machine learning in industrial, societal and economical activities has raised concerns about the fairness, equity and ethics of automated decisions. Predictive models are often developed using biased datasets and thus retain or even exacerbate biases in their decisions and recommendations. Removing the sensitive covariates, such as gender or race, is insufficient to remedy this issue since the biases may be retained due to other related covariates. We present a regularization approach to this problem that trades off predictive accuracy of the learned models (with respect to biased labels) for the fairness in terms of statistical parity, i.e. independence of the decisions from the sensitive covariates. In particular, we consider a general framework of regularized empirical risk minimization over reproducing kernel Hilbert spaces and impose an additional regularizer of dependence between predictors and sensitive covariates using kernel-based measures of dependence, namely the Hilbert-Schmidt Independence Criterion (HSIC) and its normalized version. This approach leads to a closed-form solution in the case of squared loss, i.e. ridge regression. We also provide statistical consistency results for both risk and fairness bound for our approach. Moreover, we show that the dependence regularizer has an interpretation as modifying the corresponding Gaussian process (GP) prior. As a consequence, a GP model with a prior that encourages fairness to sensitive variables can be derived, allowing principled hyperparameter selection and studying of the relative relevance of covariates under fairness constraints. Experimental results in synthetic examples and in real problems of income and crime prediction illustrate the potential of the approach to improve fairness of automated decisions. [ABSTRACT FROM AUTHOR]