Search

In hyperspectral unmixing (HU), spectral variability in hyperspectral images (HSIs) is a major challenge which has received a lot of attention over the last few years. Here, we propose a method utilizing a generative adversarial network (GAN) for creating synthetic HSIs having a controllable degree of realistic spectral variability from existing HSIs with established ground truth abundance maps. Such synthetic images can be a valuable tool when developing HU methods that can deal with spectral variability. We use a variational autoencoder (VAE) to investigate how the variability in the synthesized images differs from the original images and perform blind unmixing experiments on the generated images to illustrate the effect of increasing the variability. [ABSTRACT FROM AUTHOR]

Multispectral remote-sensing images often have band-dependent image resolution due to cost and technical limitations. To address this, we developed a method that sharpens low-resolution (LR) images using high-resolution (HR) images. In this letter, we propose a novel unsupervised deep-learning (DL) approach that involves unrolling an iterative algorithm into a deep neural network and training it using a loss function based on Stein’s risk unbiased estimate (SURE) to sharpen the LR bands (20 and 60 m) of Sentinel-2 (S2) to their highest resolution (10 m). This approach views traditional optimization model-based methods through a DL framework, improving interpretability and clarifying connections between the two approaches. Results from both simulated and real S2 datasets demonstrate that the proposed method outperforms competitive methods and produces high-quality sharpened images for the 20- and 60-m bands. The codes are available at https://github.com/hvn2/S2-Unrolling. [ABSTRACT FROM AUTHOR]

The Copernicus Sentinel-2 (S2) constellation comprises of two satellites in a sun-synchronous orbit. The S2 sensors have three spatial resolutions: 10, 20, and 60 m. The Landsat 8 (L8) satellite has sensors that provide seasonal coverage at spatial resolutions of 15, 30, and 60 m. Many remote sensing applications require the spatial resolutions of all data to be at the highest resolution possible, i.e., 10 m for S2. To address this demand, researchers have proposed various methods that exploit the spectral and spatial correlations within multispectral data to sharpen the S2 bands to 10 m. In this study, we combined S2 and L8 data. An S2 sharpening method called Sentinel-2 Sharpening (S2Sharp) was modified to include the 30 m and 15 m spectral bands from L8 and to sharpen all bands (S2 and L8) to the highest resolution of the data, which was 10 m. The method was evaluated using both real and simulated data. [ABSTRACT FROM AUTHOR]

This article proposes a denoising method based on sparse spectral–spatial and low-rank representations (SSSLRR) using the 3-D orthogonal transform (3-DOT). SSSLRR can be effectively used to remove the Gaussian and mixed noise. SSSLRR uses 3-DOT to decompose noisy HSI to sparse transform coefficients. The 3-D discrete orthogonal wavelet transform (3-D DWT) is a representative 3-DOT suitable for denoising since it concentrates on the signal in few transform coefficients, and the 3-D discrete orthogonal cosine transform (3-D DCT) is another example. An SSSLRR using 3-D DWT will be called SSSLRR-DWT. SSSLRR-DWT is an iterative algorithm based on the alternating direction method of multipliers (ADMM) that uses sparse and nuclear norm penalties. We use an ablation study to show the effectiveness of the penalties we employ in the method. Both simulated and real hyperspectral datasets demonstrate that SSSLRR outperforms other comparative methods in quantitative and visual assessments to remove the Gaussian and mixed noise. [ABSTRACT FROM AUTHOR]

Hyperspectral images (HSIs) are useful for many remote sensing applications. However, they are usually affected by noise that degrades the HSIs quality. Therefore, HSI denoising is important to improve the performance of subsequent HSI processing and analysis. In this article, we propose an HSI denoising method called Stein’s unbiased risk estimate-convolutional neural network (SURE-CNN). The method is based on an unsupervised CNN and SURE. The main difference of SURE-CNN from existing supervised learning methods is that the SURE-based loss function can be computed only from noisy data. Since SURE is an unbiased estimate of the mean squared error (MSE) of an estimator, training a CNN using the SURE loss can yield similar results as using the MSE with ground truth in supervised learning. Also, a subspace version of SURE-CNN is proposed to reduce the running time. Extensive experimental results with both simulated and real data sets show that the SURE-CNN method outperforms the competitive methods in both objective and subjective assessments. [ABSTRACT FROM AUTHOR]

Blind hyperspectral unmixing is the process of expressing the measured spectrum of a pixel as a combination of a set of spectral signatures called endmembers and simultaneously determining their fractional abundances in the pixel. Most unmixing methods are strictly spectral and do not exploit the spatial structure of hyperspectral images (HSIs). In this article, we present a new spectral–spatial linear mixture model and an associated estimation method based on a convolutional neural network autoencoder unmixing (CNNAEU). The CNNAEU technique exploits the spatial and the spectral structure of HSIs both for endmember and abundance map estimation. As it works directly with patches of HSIs and does not use any pooling or upsampling layers, the spatial structure is preserved throughout and abundance maps are obtained as feature maps of a hidden convolutional layer. We compared the CNNAEU method to four conventional and three deep learning state-of-the-art unmixing methods using four real HSIs. Experimental results show that the proposed CNNAEU technique performs particularly well and consistently when it comes to endmembers’ extraction and outperforms all the comparison methods. [ABSTRACT FROM AUTHOR]

Observation of the Earth using satellites mounted with optical sensors is an important application of remote sensing. Owing to physical constraints, multispectral (MS) sensors acquire images of lower spatial resolution than a single-band panchromatic (PAN) sensor that acquires images of the same scene. Pansharpening fuses the MS and PAN images to obtain an MS image with the same spatial resolution as the PAN image. In this letter, we propose to expand a method, initially developed for Sentinel-2 single-sensor sharpening, for pansharpening. The expanded method is based on solving a non-convex MS acquisition model using optimization methods based on cyclic decent and manifold optimization. The tuning parameters of the method are chosen using Bayesian optimization with reduced-scale evaluation. The proposed method is compared with a number of established pansharpening methods and is validated using both synthetic and real data sets. [ABSTRACT FROM AUTHOR]

Recently, the Sentinel-2 (S2) satellite constellation was deployed for mapping and monitoring the Earth environment. Images acquired by the sensors mounted on the S2 platforms have three levels of spatial resolution: 10, 20, and 60 m. In many remote sensing applications, the availability of images at the highest spatial resolution (i.e., 10 m for S2) is often desirable. This can be achieved by generating a synthetic high-resolution image through data fusion. To this end, researchers have proposed techniques exploiting the spectral/spatial correlation inherent in multispectral data to sharpen the lower resolution S2 bands to 10 m. In this paper, we propose a novel method that formulates the sharpening process as a solution to an inverse problem. We develop a cyclic descent algorithm called S2Sharp and an associated tuning parameter selection algorithm based on generalized cross validation and Bayesian optimization. The tuning parameter selection method is evaluated on a simulated data set. The effectiveness of S2Sharp is assessed experimentally by comparisons to state-of-the-art methods using both simulated and real data sets. [ABSTRACT FROM AUTHOR]

Blind hyperspectral unmixing jointly estimates spectral signatures and abundances in hyperspectral ima-ges (HSIs). Hyperspectral unmixing is a powerful tool for analyzing hyperspectral data. However, the usual huge size of HSIs may raise difficulties for classical unmixing algorithms, namely, due to limitations of the hardware used. Therefore, some researchers have considered distributed algorithms. In this paper, we develop a distributed hyperspectral unmixing algorithm that uses the alternating direction method of multipliers and \ell 1 sparse regularization. The hyperspectral unmixing problem is split into a number of smaller subproblems that are individually solved, and then the solutions are combined. A key feature of the proposed algorithm is that each subproblem does not need to have access to the whole HSI. The algorithm may also be applied to multitemporal HSIs with due adaptations accounting for variability that often appears in multitemporal images. The effectiveness of the proposed algorithm is evaluated using both simulated data and real HSIs. [ABSTRACT FROM PUBLISHER]

In this letter, we propose a method using a 3-D convolutional neural network to fuse together multispectral and hyperspectral (HS) images to obtain a high resolution HS image. Dimensionality reduction of the HS image is performed prior to fusion in order to significantly reduce the computational time and make the method more robust to noise. Experiments are performed on a data set simulated using a real HS image. The results obtained show that the proposed approach is very promising when compared with conventional methods. This is especially true when the HS image is corrupted by additive noise. [ABSTRACT FROM PUBLISHER]

In this paper, a novel feature extraction method, called orthogonal total variation component analysis (OTVCA), is proposed for remotely sensed hyperspectral data. The features are extracted by minimizing a total variation (TV) penalized optimization problem. The TV penalty promotes piecewise smoothness of the extracted features which is useful for classification. A cyclic descent algorithm called OTVCA-CD is proposed for solving the minimization problem. In the experiments, OTVCA is applied on a rural hyperspectral image having low spatial resolution and an urban hyperspectral image having high spatial resolution. The features extracted by OTVCA show considerable improvements in terms of classification accuracy compared with features extracted by other state-of-the-art methods. [ABSTRACT FROM PUBLISHER]

Blind hyperspectral unmixing involves jointly estimating endmembers and fractional abundances in hyperspectral images. An endmember is the spectral signature of a specific material in an image, while an abundance map specifies the amount of a material seen in each pixel in an image. In this paper, a new cyclic descent algorithm for blind hyperspectral unmixing using total variation (TV) and \boldsymbol\ell_q sparse regularization is proposed. Abundance maps are both spatially smooth and sparse. Their sparsity derives from the fact that each material in the image is not represented in all pixels. The abundance maps are assumed to be piecewise smooth since adjacent pixels in natural images tend to be composed of similar material. The TV regularizer is used to encourage piecewise smooth images, and the \boldsymbol\ell_q regularizer promotes sparsity. The dyadic expansion decouples the problem, making a cyclic descent procedure possible, where one abundance map is estimated, followed by the estimation of one endmember. A novel debiasing technique is also employed to reduce the bias of the algorithm. The algorithm is evaluated using both simulated and real hyperspectral images. [ABSTRACT FROM PUBLISHER]

Big data applications, such as medical imaging and genetics, typically generate datasets that consist of few observations \boldsymbol n on many more variables \boldsymbol p, a scenario that we denote as \boldsymbol p\gg\boldsymbol n. Traditional data processing methods are often insufficient for extracting information out of big data. This calls for the development of new algorithms that can deal with the size, complexity, and the special structure of such datasets. In this paper, we consider the problem of classifying \boldsymbol p\gg\boldsymbol n data and propose a classification method based on linear discriminant analysis (LDA). Traditional LDA depends on the covariance estimate of the data, but when \boldsymbol p\gg\boldsymbol n, the sample covariance estimate is singular. The proposed method estimates the covariance by using a sparse version of noisy principal component analysis (nPCA). The use of sparsity in this setting aims at automatically selecting variables that are relevant for classification. In experiments, the new method is compared to state-of-the art methods for big data problems using both simulated datasets and imaging genetics datasets. [ABSTRACT FROM PUBLISHER]

Pansharpening is the fusion of low-resolution multispectral (MS) images and high-resolution panchromatic (PAN) images to yield a high-resolution MS image. The component substitution (CS) and multiresolution analysis (MRA) methods are usually computationally efficient, making them able to handle large datasets. However, these methods often produce images that suffer from spectral and spatial distortions. The CS and MRA methods can be described using general injection schemes where details extracted from the PAN image, modulated by a band-dependent gain constant, are added to the MS image, which has been interpolated to the size of the PAN image. In this paper, we propose a simple modification of these schemes where the interpolated MS image is replaced by its deblurred version, where the deblurring kernel is matched to the modulation transfer function (MTF) of the MS sensor. This can significantly enhance the quality of the fused image. Using two real datasets and one simulated dataset, our experimental results show that using the proposed preprocessing method can significantly increase both the spectral and spatial quality of the fused image according to quantitative quality metrics. [ABSTRACT FROM PUBLISHER]

Pansharpening is the process of fusing a high-resolution panchromatic image and a low-spatial-resolution multispectral image to yield a high-spatial-resolution multispectral image. This is a typical ill-posed inverse problem, and in the past two decades, many methods have been proposed to solve it. Still, there is no general consensus on the best way to quantitatively evaluate the spectral and spatial quality of the fused image. In this paper, we compare the two most widely used and accepted methods for quality evaluation. The first method is the verification of the synthesis property which states that the fused image should be as identical as possible to the multispectral image that the sensor would observe at a higher resolution. This is impossible to verify unless the observed images are spatially degraded so that the original observed multispectral image can be used as reference. The second method is to use metrics that do not use a reference, such as the quality no reference (QNR) metrics. However, there is another property, i.e., the consistency property, which states that the fused image reduced to the resolution of the original multispectral image should be as identical to the original image as possible. This has generally been considered a necessary condition that does not have to imply correct fusion. Using real WorldView-2 and QuickBird data and a total of 18 component substitution and multiresolution analysis methods, we demonstrate that the consistency property can indeed be used to give reliable assessment of the relative performance of pansharpening methods and is superior to using the QNR metrics. [ABSTRACT FROM PUBLISHER]

The identification of the signal subspace is a very important first step for most hyperspectral algorithms. In this letter, we investigate the important problem of identifying the hyperspectral signal subspace by minimizing the mean squared error (MSE) between the true signal and an estimate of the signal. Since it is dependent on the true signal, the MSE is uncomputable in practice, and so we propose a method based on Stein's unbiased risk estimator that provides an unbiased estimate of the MSE. The resulting method is simple and fully automatic, and we evaluate it using both simulated and real hyperspectral data sets. Experimental results show that our proposed method compares well to recent state-of-the-art subspace identification methods. [ABSTRACT FROM PUBLISHER]

In remote sensing, due to cost and complexity issues, multispectral (MS) and hyperspectral (HS) sensors have significantly lower spatial resolution than panchromatic (PAN) images. Recently, the problem of fusing coregistered MS and HS images has gained some attention. In this paper, we propose a novel method for fusion of MS/HS and PAN images and of MS and HS images. MS and, more so, HS images contain spectral redundancy, whichmakes the dimensionality reduction of the data via principal component (PC) analysis very effective. The fusion is performed in the lower dimensional PC subspace; thus, we only need to estimate the first few PCs, instead of every spectral reflectance band, and without compromising the spectral and spatial quality. The benefits of the approach are substantially lower computational requirements and very high tolerance to noise in the observed data. Examples are presented using WorldView 2 data and a simulated data set based on a real HS image, with and without added noise. [ABSTRACT FROM AUTHOR]

In this paper, a new denoising method for hyperspectral images is proposed using First Order Roughness Penalty (FORP). FORP is applied in the wavelet domain to exploit the Multi-Resolution Analysis (MRA) property of wavelets. Stein's Unbiased Risk Estimator (SURE) is used to choose the tuning parameters automatically. The simulation results show that the penalized least squares using FORP can improve the Signal to Noise Ratio (SNR) compared to other denoising methods. The proposed method is also applied to a corrupted hyperspectral data set and it is shown that certain classification indices improve significantly. [ABSTRACT FROM PUBLISHER]

Hyperspectral unmixing is an important technique for analyzing remote sensing images. In this paper, we consider and examine the ℓq, 0 ≤ q ≤ 1 penalty on the abundances for promoting sparse unmixing of hyperspectral data. We also apply a first-order roughness penalty to promote piecewise smooth end-members. A novel iterative algorithm for simultaneously estimating the end-members and the abundances is developed and tested both on simulated and two real hyperspectral data sets. We present an extensive simulation study where we vary both the SNR and the sparsity of the simulated data and identify the model parameters that minimize the reconstruction errors and the spectral angle distance. We show that choosing 0 ≤ q <; 1 can outperform the ℓ1 penalty when the SNR is low or the sparsity of the underlying model is high. We also examine the effects of the imposing the abundance sum constraint using a real hyperspectral data set. [ABSTRACT FROM PUBLISHER]

In this paper, a method called wavelet-based sparse reduced-rank regression (WSRRR) is proposed for hyperspectral image restoration. The method is based on minimizing a sparse regularization problem subject to an orthogonality constraint. A cyclic descent-type algorithm is derived for solving the minimization problem. For selecting the tuning parameters, we propose a method based on Stein's unbiased risk estimation. It is shown that the hyperspectral image can be restored using a few sparse components. The method is evaluated using signal-to-noise ratio and spectral angle distance for a simulated noisy data set and by classification accuracies for a real data set. Two different classifiers, namely, support vector machines and random forest, are used in this paper. The method is compared to other restoration methods, and it is shown that WSRRR outperforms them for the simulated noisy data set. It is also shown in the experiments on a real data set that WSRRR not only effectively removes noise but also maintains more fine features compared to other methods used. WSRRR also gives higher classification accuracies. [ABSTRACT FROM PUBLISHER]

A robust framework for the classification of hyperspectral images which takes into account both spectral and spatial information is proposed. The extended multivariate attribute profile (EMAP) is used for extracting spatial information. Moreover, for solving the so-called curse of dimensionality, supervised feature extraction is carried out on both the original hyperspectral data and the output of the EMAP. After performing the dimensionality reduction, two output vectors of the original data and attributes are concatenated into one stacked vector. The final classification map is achieved by using a random-forest classifier. The main difficulties of using an EMAP is to initialize the attribute parameters. Therefore, a fully automatic scheme of the proposed method is introduced to overcome the shortcomings of using EMAP. The proposed method is tested on two widely known data sets. Experimental results confirm that the proposed method provides an accurate classification map in an acceptable CPU processing time. [ABSTRACT FROM PUBLISHER]

In this letter, we present a new method for the pansharpening of multispectral satellite imagery. Pansharpening is the process of synthesizing a high spatial resolution multispectral image from a low spatial resolution multispectral image and a high-resolution panchromatic (PAN) image. The method uses total variation to regularize an ill-posed problem dictated by a widely used explicit image formation model. This model is based on the assumptions that a linear combination of the bands of the pansharpened image gives the PAN image and that a decimation of the pansharpened image gives the original multispectral image. Experimental results are based on two real datasets and the quantitative quality of the pansharpened images is evaluated using a number of spatial and spectral metrics, some of which have been recently proposed and do not need a reference image. The proposed method compares favorably to other well-known methods for pansharpening and produces images of excellent spatial and spectral quality. [ABSTRACT FROM PUBLISHER]

The classification of high resolution urban remote sensing imagery is addressed with the focus on classification of imagery that has been pansharpened by a number of different pansharpening methods. The pansharpening process introduces some spectral and spatial distortions in the resulting fused multispectral image, the amount of which highly varies depending on which pansharpening technique is used. In the majority of the pansharpening techniques that have been proposed, there is a compromise between the spatial enhancement and the spectral consistency. Here we study the effects of the spectral and spatial distortions on the accuracy in classification of pansharpened imagery. We also study the performance in terms of accuracy of the various pansharpening techniques during classification with spatial information, obtained using mathematical morphology (MM). MM is used to derive local spatial information from the panchromatic data. Random Forests (RF) and Support Vector Machines (SVM) will be used as classifiers. Experiments are done for three different datasets that have been obtained by two different imaging sensors, IKONOS and QuickBird. These sensors deliver multispectral images that have four bands, R, G, B and near infrared (NIR). To further study the contribution of the NIR band, experiments are done using both the RGB bands and all four bands, respectively. [ABSTRACT FROM PUBLISHER]

Classification of hyperspectral data using a classifier ensemble that is based on support vector machines (SVMs) are addressed. First, the hyperspectral data set is decomposed into a few data sources according to the similarity of the spectral bands. Then, each source is processed separately by performing classification based on SVM. Finally, all outputs are used as input for final decision fusion performed by an additional SVM classifier. Results of the experiments underline how the proposed SVM fusion ensemble outperforms a standard SVM classifier in terms of overall and class accuracies, the improvement being irrespective of the size of the training sample set. The definition of the data sources resulting from the original data set is also studied. [ABSTRACT FROM AUTHOR]