Search

In this work, we propose a supervised framework for spectral unmixing of binary intimate mixtures. The core idea is based on geodesic distance measurements and regression to estimate the fractional abundances. The main assumption is that spectral reflectances of binary mixtures form a curve between the two endmembers, and the mixture's relative position on this curve serves as an indicator of its fractional abundances. We propose four novel approaches to approximate this relative position. From this, the fractional abundances are obtained using Gaussian process regression. The proposed framework simultaneously copes with the spectral variability by hypersphere and high-dimensional simplex projections. The approach is extensively validated on real datasets, including binary mineral mixtures and industrial clay powder mixtures produced in a laboratory setting, comprising 60 binary mixtures derived from five types of clay powders: Kaolin, Roof clay, Red clay, mixed clay, and Calcium hydroxide, measured by a variety of hyperspectral sensors in the VNIR-SWIR and mid-and longwave infrared regions. A comparison with the linear mixing model and several nonlinear mixing models demonstrates the superiority of the proposed approach.

Since many materials behave as heterogeneous intimate mixtures with which each photon interacts differently, the relationship between spectral reflectance and material composition is very complex. Quantitative validation of spectral unmixing algorithms requires high-quality ground truth fractional abundance data, which are very difficult to obtain.In this work, we generated a comprehensive hyperspectral dataset of intimate mineral powder mixtures by homogeneously mixing five different clay powders (Kaolin, Roof clay, Red clay, mixed clay, and Calcium hydroxide). In total 325 samples were prepared. Among the 325 samples, 60 mixtures were binary, 150 were ternary, 100 were quaternary, and 15 were quinary. For each mixture (and pure clay powder), reflectance spectra are acquired by 13 different sensors, with a broad wavelength range between the visible and the long-wavelength infrared regions (i.e., between 350 nm and 15385 nm) and with a large variation in sensor types, platforms, and acquisition conditions. We will make this dataset public, to be used by the community for the validation of nonlinear unmixing methodologies (https://github.com/VisionlabUA/Multisensor_datasets)

Optical hyperspectral cameras capture the spectral reflectance of materials. Since many materials behave as heterogeneous intimate mixtures with which each photon interacts differently, the relationship between spectral reflectance and material composition is very complex. Quantitative validation of spectral unmixing algorithms requires high-quality ground truth fractional abundance data, which are very difficult to obtain. In this work, we generated a comprehensive laboratory ground truth dataset of intimately mixed mineral powders. For this, five clay powders (Kaolin, Roof clay, Red clay, mixed clay, and Calcium hydroxide) were mixed homogeneously to prepare 325 samples of 60 binary, 150 ternary, 100 quaternary, and 15 quinary mixtures. Thirteen different hyperspectral sensors have been used to acquire the reflectance spectra of these mixtures in the visible, near, short, mid, and long-wavelength infrared regions (350-15385) nm. Overlaps in wavelength regions due to the operational ranges of each sensor and variations in acquisition conditions resulted in a large amount of spectral variability. Ground truth composition is given by construction, but to verify that the generated samples are sufficiently homogeneous, XRD and XRF elemental analysis is performed. We believe these data will be beneficial for validating advanced methods for nonlinear unmixing and material composition estimation, including studying spectral variability and training supervised unmixing approaches. The datasets can be downloaded from the following link: https://github.com/VisionlabHyperspectral/Multisensor_datasets.

Hyperspectral imaging is an important technology in the field of geosciences and remote sensing. However, the high-dimensional nature of hyperspectral images (HSIs) together with the limited availability of training/labeled samples challenge an efficient processing of HSIs. To alleviate these challenges, we propose a deep multi-resolution clustering network (DMC-Net) to analyze HSIs. DMC-Net, without requiring training/labeled samples for the training process, captures the non-linear intrinsic relation within data points in an HSI and analyzes the image at various resolutions by applying atrous convolutions. Furthermore, DMC-Net preserves the spectral information by directly incorporating extracted features from the original HSI into the reconstruction phase. In terms of clustering accuracy, experimental results on two real HSIs demonstrate the superior performance of DMC-Net compared to the state-of-the-art deep learning-based clustering approaches.

This article proposes a blind nonlinear unmixing technique for intimate mixtures using the Hapke model and convolutional neural networks (HapkeCNN). We use the Hapke model and a fully convolutional encoder–decoder deep network for the nonlinear unmixing. Additionally, we propose a novel loss function that includes three terms; 1) a quadratic term based on the Hapke model, that captures the nonlinearity; 2) the reconstruction error of the reflectances, to ensure the fidelity of the reconstructed reflectance; and 3) a minimum volume total variation (TV) term that exploits the geometrical information to estimate the endmembers in the absence of pure pixels in the hyperspectral data. The proposed method is evaluated using two simulated and two real datasets. We compare the results of endmember and abundance estimation with a number of nonlinear, and projection-based linear unmixing techniques. The experimental results confirm that HapkeCNN considerably outperforms the state-of-the-art nonlinear approaches. The proposed method was implemented in Python (3.9) using PyTorch as the platform for the deep network and is available at: https://github.com/BehnoodRasti/HapkeCNN .

In this article, we introduce a deep learning-based technique for the linear hyperspectral unmixing problem. The proposed method contains two main steps. First, the endmembers are extracted using a geometric endmember extraction method, i.e., a simplex volume maximization in the subspace of the data set. Then, the abundances are estimated using a deep image prior. The main motivation of this work is to boost the abundance estimation and make the unmixing problem robust to noise. The proposed deep image prior uses a convolutional neural network to estimate the fractional abundances, relying on the extracted endmembers and the observed hyperspectral data set. The proposed method is evaluated on simulated and three real remote sensing data for a range of SNR values (i.e., from 20 to 50 dB). The results show considerable improvements compared to state-of-the-art methods. The proposed method was implemented in Python (3.8) using PyTorch as the platform for the deep network and is available online: https://github.com/BehnoodRasti/UnDIP.

Remote sensing hyperspectral cameras acquire high spectral-resolution data that reveal valuable composition information on the targets (e.g., for Earth observation and environmental applications). The intrinsic high dimensionality and the lack of sufficient numbers of labeled/training samples prevent efficient processing of hyperspectral images (HSIs). HSI clustering can alleviate these limitations. In this study, we propose a multiscale spectral–spatial association network (MS 2 A-Net) to cluster HSIs. The backbone of MS 2 A-Net is an autoencoder architecture that allows the network to capture the nonlinear relation between data points in an unsupervised manner. The network applies a multistream approach. One stream extracts spectral information by deploying a spectral association unit. The other stream derives multiscale contextual and spatial information by employing dilated (atrous) convolutional kernels. The obtained feature representation generated by MS 2 A-Net is fed into a standard k-means clustering algorithm to produce the final clustering result. Extensive experiments on four HSIs for different types of applications (i.e., geological-, rural-, and urban-mapping) demonstrate the superior performance of MS 2 A-Net over the state-of-the-art shallow/deep learning-based clustering approaches in terms of clustering accuracy.

In this work, we introduce a multi-sensor subspace-based clustering algorithm that benefits from fine spectral-resolution hyperspectral images (HSIs) and fine spatial resolution RGB images. In order to extract spatial information, a hidden Markov random field (HMRF) is employed on the fine spatial-resolution RGB image, whereas, spectral information is derived from an HSI using an advanced sparse subspace clustering algorithm. The proposed algorithm is validated on two real geological data sets. The experimental results in this study show that the proposed algorithm outperforms the state-of-the-art clustering algorithms in terms of clustering accuracy.

The ever-growing developments in technology to capture different types of image data (e.g., hyperspectral imaging and Light Detection and Ranging (LiDAR)-derived digital surface model (DSM)), along with new processing techniques, have led to a rising interest in imaging applications for Earth observation. However, analyzing such datasets in parallel remains a challenging task. In this paper, we propose a multi-sensor deep clustering (MDC) algorithm for the joint processing of multi-source imaging data. The architecture of MDC is inspired by autoencoder (AE)-based networks. The MDC paradigm includes three parallel networks, a spectral network using an autoencoder structure, a spatial network using a convolutional autoencoder structure, and lastly, a fusion network that reconstructs the concatenated image information from the concatenated latent features from the spatial and spectral network. The proposed algorithm combines the reconstruction losses obtained by the aforementioned networks to optimize the parameters (i.e., weights and bias) of all three networks simultaneously. To validate the performance of the proposed algorithm, we use two multi-sensor datasets from different applications (i.e., geological and rural sites) as benchmarks. The experimental results confirm the superiority of our proposed deep clustering algorithm compared to a number of state-of-the-art clustering algorithms. The code will be available at: https://github.com/Kasra2020/MDC.

Remote sensing data is contaminated with different types of noise that can severely affect the analysis of this data. Generally, in modern treatment chains of satellite and aerial data, denoising techniques are applied to atmospherically corrected images prior to further analysis (e.g., classification). However, since the noise contaminates the measured radiance at the sensor, it can influence the atmospheric correction in itself and consequently the remaining of the processing chain. In this paper, we compare the performance of a denoising technique, when applied before or after atmospheric correction. Our observations challenge the current de facto paradigm of denoising in a processing chain of spaceborne and airborne remotely sensed images.

Hyperspectral linear unmixing and denoising are highly related hyperspectral image (HSI) analysis tasks. In particular, with the assumption of Gaussian noise, the linear model assumed for the HSI in the case of low-rank denoising is often the same as the one used in HSI unmixing. However, the optimization criterion and the assumptions on the constraints are different. Additionally, noise reduction as a preprocessing step in hyperspectral data analysis is often ignored. The main goal of this paper is to study experimentally the influence of noise on the process of hyperspectral unmixing by: (1) investigating the effect of noise reduction as a preprocessing step on the performance of hyperspectral unmixing; (2) studying the relation between noise and different endmember selection strategies; (3) investigating the performance of HSI unmixing as an HSI denoiser; (4) comparing the denoising performance of spectral unmixing, state-of-the-art HSI denoising techniques, and the combination of both. All experiments are performed on simulated and real datasets.

The increasing amount of information acquired by imaging sensors in Earth Sciences results in the availability of a multitude of complementary data (e.g., spectral, spatial, elevation) for monitoring of the Earth’s surface. Many studies were devoted to investigating the usage of multi-sensor data sets in the performance of supervised learning-based approaches at various tasks (i.e., classification and regression) while unsupervised learning-based approaches have received less attention. In this paper, we propose a new approach to fuse multiple data sets from imaging sensors using a multi-sensor sparse-based clustering algorithm (Multi-SSC). A technique for the extraction of spatial features (i.e., morphological profiles (MPs) and invariant attribute profiles (IAPs)) is applied to high spatial-resolution data to derive the spatial and contextual information. This information is then fused with spectrally rich data such as multi- or hyperspectral data. In order to fuse multi-sensor data sets a hierarchical sparse subspace clustering approach is employed. More specifically, a lasso-based binary algorithm is used to fuse the spectral and spatial information prior to automatic clustering. The proposed framework ensures that the generated clustering map is smooth and preserves the spatial structures of the scene. In order to evaluate the generalization capability of the proposed approach, we investigate its performance not only on diverse scenes but also on different sensors and data types. The first two data sets are geological data sets, which consist of hyperspectral and RGB data. The third data set is the well-known benchmark Trento data set, including hyperspectral and LiDAR data. Experimental results indicate that this novel multi-sensor clustering algorithm can provide an accurate clustering map compared to the state-of-the-art sparse subspace-based clustering algorithms.

The mechanical and chemical properties of a compound material are determined by the fractional abundances of its components. In this work, we present a spectral unmixing technique to estimate the fractional abundances of the components of mixed and compound materials from hyperspectral images. The estimation of fractional abundances in mixed materials faces the main challenge of intimate mixing. In compound materials, the mixing with water causes changes in chemical properties resulting in spectral variability and non-linearity. To address these challenges, a supervised method is proposed that learns a mapping from the hyperspectral data to spectra that follow the linear mixing model. Then, a linear unmixing technique is applied on the mapped spectra to estimate the fractional abundances. To demonstrate the potential of the proposed method, experiments are conducted on hyperspectral images from mixtures of red and yellow clay powders and hardened mortar samples with varying water to cement ratios.

Due to the complex interaction of light with the Earth’s surface, reflectance spectra can be described as highly nonlinear mixtures of the reflectances of the material constituents occurring in a given resolution cell of hyperspectral data. Our aim is to estimate the fractional abundance maps of the materials from the nonlinear hyperspectral data. The main disadvantage of using nonlinear mixing models is that the model parameters are not properly interpretable in terms of fractional abundances. Moreover, not all spectra of a hyperspectral dataset necessarily follow the same particular mixing model. In this work, we present a supervised method for nonlinear spectral unmixing. The method learns a mapping from a true hyperspectral dataset to corresponding linear spectra, composed of the same fractional abundances. A simple linear unmixing then reveals the fractional abundances. To learn this mapping, ground truth information is required, in the form of actual spectra and corresponding fractional abundances, along with spectra of the pure materials, obtained from a spectral library or available in the dataset. Three methods are presented for learning nonlinear mapping, based on Gaussian processes, kernel ridge regression, and feedforward neural networks. Experimental results conducted on an artificial dataset, a data set obtained by ray tracing, and a drill core hyperspectral dataset shows that this novel methodology is very promising.