Search

Multispectral remote-sensing images often have band-dependent image resolution due to cost and technical limitations. To address this, we developed a method that sharpens low-resolution (LR) images using high-resolution (HR) images. In this letter, we propose a novel unsupervised deep-learning (DL) approach that involves unrolling an iterative algorithm into a deep neural network and training it using a loss function based on Stein’s risk unbiased estimate (SURE) to sharpen the LR bands (20 and 60 m) of Sentinel-2 (S2) to their highest resolution (10 m). This approach views traditional optimization model-based methods through a DL framework, improving interpretability and clarifying connections between the two approaches. Results from both simulated and real S2 datasets demonstrate that the proposed method outperforms competitive methods and produces high-quality sharpened images for the 20- and 60-m bands. The codes are available at https://github.com/hvn2/S2-Unrolling. [ABSTRACT FROM AUTHOR]

Target detection based on hyperspectral image (HSI) representations has drawn wide attention given its wide variety of features. The matrix-based approach inevitably loses spatial information and fails to explore the intrinsic multimodal structure of an HSI cube. In this letter, we propose a regularized tensor-based model without altering the data structure. We assume that an observed third-order HSI tensor is decomposed into the sum of a total variation (TV)-regularized low-rank background tensor and a sparse (TVLrS) target tensor. The two tensors are represented as the mode-3 product of a third-order tensor, called the tensor representation coefficient (TRC), and a spectra dictionary matrix. Then, the model is coined as TVLrS-TRC. The background TRC has a low-rank property, contributing to the low-rankness characterization in our model. Moreover, as the size of the background TRC term is smaller than the background tensor, characterizing its local smoothness via TV regularization reduces the computational cost compared with that of the background tensor. Extensive experiments on two real hyperspectral datasets demonstrate the advantage of the proposed method compared with the state of the art. [ABSTRACT FROM AUTHOR]

The detection of deformation in Global Navigation Satellite System (GNSS) time series associated with (a)seismic events down to a low magnitude is still a challenging issue. The presence of a considerable amount of noise in the data makes it difficult to reveal patterns of small ground deformation. Traditional analyses and methodologies are able to effectively retrieve the deformation associated with medium to large magnitude events. However, the automatic detection and characterization of such events is still a complex task, because traditionally employed methods often separate the time series analysis from the source characterization. Here we propose a first end‐to‐end framework to characterize seismic sources using geodetic data by means of deep learning, which can be an efficient alternative to the traditional workflow, possibly overcoming its performance. We exploit three different geodetic data representations in order to leverage the intrinsic spatio‐temporal structure of the GNSS noise and the target signal associated with (slow) earthquake deformation. We employ time series, images, and image time series to account for the temporal, spatial, and spatio‐temporal domain, respectively. Thereafter, we design and develop a specific deep learning model for each dataset. We analyze the performance of the tested models both on synthetic and real data from North Japan, showing that image time series of geodetic deformation can be an effective data representation to embed the spatio‐temporal evolution, with the associated deep learning method outperforming the other two. Therefore, jointly accounting for the spatial and temporal evolution may be the key to effectively detect and characterize fast or slow earthquakes. Plain Language Summary: The continuous monitoring of ground displacement with Global Navigation Satellite System allowed, at the beginning of the 2000s, the discovery of slow earthquakes—a transient slow slippage of tectonic faults that releases stress without generating seismic waves. Nevertheless, the detection of small events is still a challenge, because they are hidden in the noise. Most of the methods which are traditionally employed are able to extract the deformation down to a certain signal‐to‐noise level. However, one can ask if deep learning can be a more efficient and powerful alternative. To this end, we address the problem by using deep learning, as it stands as a powerful way to automatize and possibly overcome traditional methods. We use and compare three data representations, that is time series, images, and image time series of deformation, which account for the temporal, spatial, and spatio‐temporal variability, respectively. We train our methods on synthetic data, since real datasets are still not enough to be effectively employed with deep learning, and we test on synthetic and real data as well, claiming that image time series and its associated deep learning model may be more effective toward the study of the slow deformation. Key Points: We develop deep learning approaches on synthetics mimicking the spatio‐temporal structure of static deformation and realistic Global Navigation Satellite System (GNSS) noiseWe design three deep learning models and we train and test them against three GNSS data representationsTransformers and image time series of deformation can effectively characterize small deformation patterns associated with the seismic source [ABSTRACT FROM AUTHOR]

Light detection and ranging (LiDAR) provides high-resolution geometric information for monitoring forests at individual tree crown (ITC) level. An important task for ITC delineation is segmentation, and previous studies showed that the adaptive 3-D mean shift (AMS3D) algorithm provides effective results. AMS3D for ITC segmentation has three components for the kernel profile: shape, weight, and size. In this letter, we present an AMS3D approach based on the adaptation of the kernel profile size through an ellipsoid crown shape model. The algorithm parameters are estimated based on allometry equations derived from 22 forest plots in two study sites. After computing the mean shift (MS) vector, we initialize the parameters of the ellipsoid crown shape model to derive the kernel profile size, and further tested two crown shape models for adapting the size of the superellipsoid (SE) kernel profile. These schemes are compared with two other MS algorithms with and without kernel profile size adaptation. We select the best algorithm output per plot based on the maximum F1-score. The ellipsoid crown shape model with a SE kernel profile of $n = 1.5$ presents the highest recall and the best Jaccard index, especially for conifers. [ABSTRACT FROM AUTHOR]

Although multimodal remote sensing data analysis can strongly improve the characterization of physical phenomena on Earth’s surface, nonidealities and estimation imperfections between records and investigation models can limit its actual information extraction ability. In this article, we aim at predicting the maximum information extraction that can be reached when analyzing a given data set. By means of an asymptotic information theory-based approach, we investigate the reliability and accuracy that can be achieved under optimal conditions for multimodal analysis as a function of data statistics and parameters that characterize the multimodal scenario to be addressed. Our approach leads to the definition of two indices that can be easily computed before the actual processing takes place. Moreover, we report in this article how they can be used for operational use in terms of image selection in order to maximize the robustness of the multimodal analysis, as well as to properly design data collection campaigns for understanding and quantifying physical phenomena. Experimental results show the consistency of our approach. [ABSTRACT FROM AUTHOR]

A method based on spectral unmixing (SU) for snow cover estimation from a time series of optical images is proposed in this letter. Specifically, we have developed an endmember estimation procedure that exploits the temporal continuity of a scene. Consecutive dates are jointly processed for a more precise description of background materials improving the estimation of a fractional snow cover map. The proposed workflow relies on up to three consecutive acquisitions over the same area to extract an appropriate set of background spectra to be considered as endmembers. In greater details, multiple sets of endmembers are extracted from different images in the time series by a geometrical automated endmember extraction algorithm and the most relevant one is selected in terms of reconstruction error. Snow cover maps are then estimated by SU considering as endmembers the snow spectra coming from a spectral library and those associated with the background materials as estimated by the proposed procedure. The proposed technique is quantitatively validated considering Moderate-Resolution Imaging Spectroradiometer Terra data over the French Alps and Moroccan High Atlas image time series and comparing the estimated snow cover maps with high-resolution reference data. The experiment clearly demonstrates the effectiveness of the generated set of endmembers using three different approaches to abundance estimation. [ABSTRACT FROM AUTHOR]

Hyperspectral video sequences (HVSs) are well suited for gas plume detection (GPD). The high spectral resolution allows the detection of chemical clouds even when they are optically thin. Processing this new type of video sequences is challenging and requires advanced image and video analysis algorithms. In this paper, we propose a novel method for GPD recorded in HVSs. Based on the assumption that the background is stationary and the gas plume is moving, the proposed method separates the background from the gas plume via a low-rank and sparse decomposition. Furthermore, taking into consideration that the gas plume is continuous in both spatial and temporal dimensions, we include total variation regularization in the constrained minimization problem, which we solve using the augmented Lagrangian multiplier method. After applying the above process to each extracted feature, a novel fusion strategy is proposed to combine the information into a final detection result. Experimental results using real data sets indicate that the proposed method achieves very promising GPD performance. [ABSTRACT FROM AUTHOR]

Multiple subpixel shifted images (MSIs) from the same area can be incorporated to improve the accuracy of soft-then-hard subpixel mapping (STHSPM). In this paper, a novel method that derives higher resolution MSIs with more spatial–spectral information (MSI-SS) is proposed. First, coarse MSIs produce two high-resolution MSIs for each class respectively by two paths at the same time. The spatial path produces the high-resolution MSIs by soft classification followed by interpolation. And, the other high-resolution MSIs are derived from the spectral path by interpolation followed by soft classification. Then the higher resolution MSIs with more spatial–spectral information for each class are derived by integrating the aforementioned two kinds of high-resolution MSIs by the appropriate weight. Finally, the integrated higher resolution MSIs for each class are used to allocate hard class labels to subpixels. The proposed method is fast and takes more spatial–spectral information of the original MSIs into account. Experiments on three real hyperspectral remote sensing images show that the proposed method produce higher SPM accuracy result. [ABSTRACT FROM PUBLISHER]

Remotely sensed images with very high spatial resolution provide a detailed representation of the surveyed scene with a geometrical resolution that, at the present, can be up to 30 cm (WorldView-3). A set of powerful image processing operators have been defined in the mathematical morphology framework. Among those, connected operators [e.g., attribute filters (AFs)] have proven their effectiveness in processing very high resolution images. AFs are based on attributes which can be efficiently implemented on tree-based image representations. In this paper, we considered the definition of \min, \max, \textdirect, and \textsubtractive filter rules for the computation of AFs over the tree-of-shapes representation. We study their performance on the classification of remotely sensed images. We compare the classification results over the tree of shapes with the results obtained when the same rules are applied on the component trees. The random forest is used as a baseline classifier, and the experiments are conducted using multispectral data sets acquired by QuickBird and IKONOS sensors over urban areas. [ABSTRACT FROM AUTHOR]

Polarimetric incoherent target decomposition aims at accessing physical parameters of illuminated scatters through the analysis of the target coherence or covariance matrix. In this framework, independent component analysis (ICA) was recently proposed as an alternative method to eigenvector decomposition to better interpret non-Gaussian heterogeneous clutter (inherent to high-resolution synthetic aperture radar systems). Until now, the two main drawbacks reported of the aforementioned method are the greater number of samples required for an unbiased estimation, when compared to the classical eigenvector decomposition, and the inability to be employed in scenarios under the Gaussian clutter assumption. In this paper, both drawbacks are analyzed. First, a Monte Carlo approach is performed in order to investigate the bias in estimating Touzi's target-scattering-vector-model parameters when ICA is employed. Simulated data and a RAMSES X-band image acquired over Brétigny, France, are taken into consideration to investigate the bias estimation under different scenarios. Finally, the performance of the algorithm is also evaluated under the Gaussian clutter assumption and when spatial correlation is introduced in the model. [ABSTRACT FROM AUTHOR]

The multiplicative model, expressed as a product between the square root of a scalar positive quantity (texture) and the description of an equivalent homogeneous surface (speckle), is one of the most appropriate and disseminated models used to describe high-resolution polarimetric synthetic aperture radar (PolSAR) clutter. Generally, the texture is assumed polarization independent, which causes PolSAR data to present a spherical symmetry property, allowing for the usage of most of the algorithms present in the literature. Nevertheless, the existence of polarization-dependent clutter has also been reported, for which specific algorithms need to be derived. Therefore, it becomes clear that the first step in SAR data analysis should be the validation of the model employed. Within this context, this paper presents a new methodological framework to assess the conformity of multivariate high-resolution SAR data with respect to the product model in terms of asymptotic statistics. More precisely, spherical symmetry is investigated by applying statistical hypothesis testing on the structure of the quadricovariance matrix. Simulated data, data from the P-band airborne data set acquired by the Office National d'Études et de Recherches Aérospatiales (ONERA) over the French Guiana in 2009 in the frame of the European Space Agency campaign TropiSAR and a RAMSES X-band image acquired over Brétigny, France, are taken into consideration to investigate the performance of the derived test. The detection results are qualitatively and quantitatively analyzed, and some important conclusions are drawn regarding the methodology employed in analyzing SAR data. [ABSTRACT FROM AUTHOR]

The intrinsic dimensionality (ID) of multivariate data is a very important concept in spectral unmixing of hyperspectral images. A good estimation of the ID is crucial for a correct retrieval of the number of endmembers (the spectral signatures of macroscopic materials) in the image, for dimensionality reduction or for subspace learning, among others. Recently, some approaches to perform spectral unmixing and superresolution locally have been proposed, which require a local estimation of the number of endmembers to use. However, the role of ID in local regions of hyperspectral images has not been properly addressed. Some important issues when dealing with small regions of hyperspectral data can seriously affect the performance of conventional hyperspectral ID estimators. We show that three factors mainly affect local ID estimation: the number of pixels in the local regions, which has to be high enough for the estimations to be relevant, the number of hyperspectral bands which complicates the estimations if the ambient space has a high dimensionality, and the noise, which can be misinterpreted as a signal when its power is important. Here, we review the hyperspectral ID estimators on the literature for local ID estimation, we show how they behave in a local setting on synthetic and real data sets, and we provide some guidelines to make proper use of these estimators in local approaches. [ABSTRACT FROM AUTHOR]

Just over a decade has passed since the concept of morphological profile was defined for the analysis of remote sensing images. Since then, the morphological profile has largely proved to be a powerful tool able to model spatial information (e.g., contextual relations) of the image. However, due to the shortcomings of using the morphological profiles, many variants, extensions, and refinements of its definition have appeared stating that the morphological profile is still under continuous development. In this case, recently introduced theoretically sound attribute profiles (APs) can be considered as a generalization of the morphological profile, which is a powerful tool to model spatial information existing in the scene. Although the concept of the AP has been introduced in remote sensing only recently, an extensive literature on its use in different applications and on different types of data has appeared. To that end, the great amount of contributions in the literature that address the application of the AP to many tasks (e.g., classification, object detection, segmentation, change detection, etc.) and to different types of images (e.g., panchromatic, multispectral, and hyperspectral) proves how the AP is an effective and modern tool. The main objective of this survey paper is to recall the concept of the APs along with all its modifications and generalizations with special emphasis on remote sensing image classification and summarize the important aspects of its efficient utilization while also listing potential future works. [ABSTRACT FROM AUTHOR]

Pansharpening aims at fusing a multispectral and a panchromatic image, featuring the result of the processing with the spectral resolution of the former and the spatial resolution of the latter. In the last decades, many algorithms addressing this task have been presented in the literature. However, the lack of universally recognized evaluation criteria, available image data sets for benchmarking, and standardized implementations of the algorithms makes a thorough evaluation and comparison of the different pansharpening techniques difficult to achieve. In this paper, the authors attempt to fill this gap by providing a critical description and extensive comparisons of some of the main state-of-the-art pansharpening methods. In greater details, several pansharpening algorithms belonging to the component substitution or multiresolution analysis families are considered. Such techniques are evaluated through the two main protocols for the assessment of pansharpening results, i.e., based on the full- and reduced-resolution validations. Five data sets acquired by different satellites allow for a detailed comparison of the algorithms, characterization of their performances with respect to the different instruments, and consistency of the two validation procedures. In addition, the implementation of all the pansharpening techniques considered in this paper and the framework used for running the simulations, comprising the two validation procedures and the main assessment indexes, are collected in a MATLAB toolbox that is made available to the community. [ABSTRACT FROM AUTHOR]

Over the last few years, several new strategies have been proposed for spectral–spatial classification of remotely sensed image data, for cases when high spatial and spectral resolutions are available. In this letter, we focus on the possibility of performing advanced spectral–spatial classification of remote sensing images with limited spectral resolution (often called multispectral). A new strategy is proposed, where the spectral dimensionality of the multispectral data is first expanded by using nonlinear feature extraction with kernel methods such as kernel principal component analysis. Then, extended multiattribute profiles (EMAPs), built on the expanded set of spectral features, are used to include spatial information. This strategy allows us to first decompose different spectral clusters into different spectral features and further improve the spatial discrimination. The resulting EMAPs are used for classification using advanced classifiers such as support vector machines and random forests. We test our proposed methodology with different multispectral data sets obtaining state-of-the-art classification results. [ABSTRACT FROM PUBLISHER]

Morphological and attribute profiles have been proven to be effective tools to fuse spectral and spatial information for classification of remote sensing data. A wide range of filters (i.e., number of levels in the profiles) is usually necessary in order to properly model the spatial information in a remote sensing scene. A dense sampling of the values of the parameters of the filters generates profiles that have both a very large dimensionality (leading to the Hughes phenomenon in classification) and a high redundancy. In this paper, a novel iterative technique based on genetic algorithms (GAs) is proposed to automatically optimize the selection of the optimal features from the profiles. The selection of the filtered images that compose the profile is performed by dividing them into three classes corresponding to high, medium, and low importance. We propose to measure the importance (modeled in terms of discriminative power in the classification task) using a random forest classifier, which provides a rank for each feature with its model. Only the set of images associated with the highest importance is selected, i.e., preserved for classification. The proposed technique is applied to the features labeled with medium importance, whereas the images with the lowest importance are removed from the profile. This method is employed to classify three hyperspectral data sets achieving significantly high classification accuracy values. A parallel computing implementation has been developed in order to significantly reduce the time required for the run of the GAs. [ABSTRACT FROM AUTHOR]

The characterization of snow extent is critical for a wide range of applications. Since 1966, snow maps at different spatial resolutions have been produced using various satellite sensor images. Nowadays, the most widely used products are likely those derived from Moderate-Resolution Imaging Spectroradiometer (MODIS) data, which cover the whole Earth at a near-daily frequency. There are a variety of snow mapping methods for MODIS data, based on different methodologies and applied at different spatial resolutions. Up to now, all these products have been tested and evaluated separately. This study aims to compare the methods currently available for retrieving snow from MODIS data. The focus is on fractional snow cover, which represents the snow cover area at the subpixel level. We examine the two main approaches available for generating such products from MODIS data; namely, linear regression of the Normalized Difference Snow Index (NDSI) and spectral unmixing (SU). These two approaches have resulted in several methods, such as MOD10A1 (the NSIDC MODIS snow product) for NDSI regression, and MODImLAB for SU. The assessment of these approaches was carried out using higher resolution binary snow maps (i.e., showing the presence or absence of snow) at spatial resolutions of 10, 20, and 30 m, produced by SPOT 4, SPOT 5, and LANDSAT-8, respectively. Three areas were selected in order to provide landscape diversity: the French Alps (117 dates), the Pyrenees (30 dates), and the Moroccan Atlas (24 dates). This study investigates the impact of reference maps on accuracy assessments, and it is suggested that NDSI-based high spatial resolution reference maps advantage NDSI medium-resolution snow maps. For MODIS snow maps, the results show that applying an NDSI approach to accurate surface reflectance corrected for topographic and atmospheric effects generally outperforms other methods for the global retrieval of snow cover area. The improvements to the newer version of MOD10A1 (Collection 6) compared to the older version (Collection 5) are significant. Products based on SU provide a good alternative and more accurate retrieval of the snow fraction where wider ranges of land covers are concerned. The fusion process and its resulting 250 m spatial resolution product improve snow line retrieval. False detection in mixed pixels, probably due to the spectral variability associated with the various materials in the spectral mixture, has been identified as an area that will require improvement. [ABSTRACT FROM AUTHOR]

Segmentation and classification are prolific research topics in the image processing community. These topics have been increasingly used in the context of analysis of cementitious materials on images acquired with a scanning electron microscope. Indeed, there is a need to be able to detect and to quantify the materials present in a cement paste in order to follow the chemical reactions occurring in the material even days after the solidification. We propose a new approach for segmentation and classification of cementitious materials based on the denoising of the data with a block-matching three-dimensional (3-D) algorithm, binary partition tree (BPT) segmentation, support vector machines (SVM) classification, and interactivity with the user. The BPT provides a hierarchical representation of the spatial regions of the data, allowing a segmentation to be selected among the admissible partitions of the image. SVMs are used to obtain a classification map of the image. This approach combines state-of-the-art image processing tools with user interactivity to allow a better segmentation to be performed, or to help the classifier discriminate the classes better. We show that the proposed approach outperforms a previous method when applied to synthetic data and several real datasets coming from cement samples, both qualitatively with visual examination and quantitatively with the comparison of experimental results with theoretical ones. [ABSTRACT FROM AUTHOR]

Multispectral image (MS) and panchromatic image (PAN) fusion, which is also named as multispectral pansharpening, aims to obtain MS with high spatial resolution and high spectral resolution. However, due to the usual neglect of noise and blur generated in the imaging and transmission phases of data during training, many deep learning (DL) pansharpening methods fail to perform on the dataset containing noise and blur. To tackle this problem, a variational optimization-guided two-stage network (VOGTNet) for multispectral pansharpening is proposed in this work, and the performance of variational optimization (VO)-based pansharpening methods relies on prior information and estimates of spatial-spectral degradation from the target image to other two original images. Concretely, we propose a dual-branch fusion network (DBFN) based on supervised learning and train it by using the datasets containing noise and blur to generate the prior fusion result as the prior information that can remove noise and blur in the initial stage. Subsequently, we exploit the estimated spectral response function (SRF) and point spread function (PSF) to simulate the process of spatial-spectral degradation, respectively, thereby making the prior fusion result and the adaptive recovery model (ARM) jointly perform unsupervised learning on the original dataset to restore more image details and results in the generation of the high-resolution MSs in the second stage. Experimental results indicate that the proposed VOGTNet improves pansharpening performance and shows strong robustness against noise and blur. Furthermore, the proposed VOGTNet can be extended to be a general pansharpening framework, which can improve the ability to resist noise and blur of other supervised learning-based pansharpening methods. The source code is available at https://github.com/HZC-1998/VOGTNet.