Search

In the context of global change, up-to-date land use land cover (LULC) maps is a major challenge to assess pressures on natural areas. These maps also allow us to assess the evolution of land cover and to quantify changes over time (such as urban sprawl), which is essential for having a precise understanding of a given territory. Few studies have combined information from Sentinel-1 and Sentinel-2 imagery, but merging radar and optical imagery has been shown to have several benefits for a range of study cases, such as semantic segmentation or classification. For this study, we used a newly produced dataset, MultiSenGE, which provides a set of multitemporal and multimodal patches over the Grand-Est region in France. To merge these data, we propose a CNN approach based on spatio-temporal and spatio-spectral feature fusion, ConvLSTM+Inception-S1S2. We used a U-Net base model and ConvLSTM extractor for spatio-temporal features and an inception module for the spatio-spectral features extractor. The results show that describing an overrepresented class is preferable to map urban fabrics (UF). Furthermore, the addition of an Inception module on a date allowing the extraction of spatio-spectral features improves the classification results. Spatio-spectro-temporal method (ConvLSTM+Inception-S1S2) achieves higher global weighted F 1 S c o r e than all other methods tested. [ABSTRACT FROM AUTHOR]

Image time series (ITS) represent complex 3D (2D+t in practice) data that are now daily produced in various domains, from medical imaging to remote sensing. They contain rich spatio-temporal information allowing the observation of the evolution of a sensed scene over time. In this work, we focus on the classification task of ITS, as often available in remote sensing tasks. An underlying problem here is to consider jointly the spatial and the temporal dimensions of the data. We present Deep-STaR, a method to learn such features from ITS data to proceed to their classification. Instead of reasoning in the original 2D+t space, we investigate novel 2D planar data representations, containing both temporal and spatial information. Such representations are a novel way to structure the ITS, compatible with deep learning architectures. They are used to feed a convolutional neural network to learn spatio-temporal features with 2D convolutions, leading ultimately to classification decision. To enhance the explainability of the results, we also propose a post-hoc attention mechanism, enabled by this new approach, providing a semantic map giving some insights for the taken decision. Deep-STaR is evaluated on a remote sensing application, for the classification of agricultural crops from satellite ITS. The results highlight the benefice of this method, compared to the literature, and its interest to make easier the interpretation of ITS to understand spatio-temporal phenomena. [Display omitted] • Classification of image time series. • Spatio-temporal planar representation of 2 D + t image data. • Learning spatio-temporal features with 2 D convolutional neural networks. • Visual post-hoc attention for spatio-temporal result interpretation. [ABSTRACT FROM AUTHOR]

The major part of the population lives in urban areas, and this is expected to increase in the future. The main challenges faced by cities currently and towards the future are the rapid urbanization, the increase in urban temperature and the urban heat island. Mapping and monitoring urban fabric (UF) to analyze the environmental impact of these phenomena is more necessary than ever. This coupled with the increased availability of Earth observation data and their growing temporal capabilities leads us to consider using temporal features for improving land use classification, especially in urban environments where the spectral overlap between classes makes it challenging. Urban land use classification thus remains a central question in remote sensing. Although some research studies have successfully used multi-temporal images such as Landsat-8 or Sentinel-2 to improve land cover classification, urban land use mapping is rarely carried using the temporal dimension. This paper explores the use of Sentinel-2 data in a deep learning framework, by firstly assessing the temporal robustness of four popular fully convolutional neural networks (FCNs) trained over single-date images for the classification of the urban footprint, and secondly, by proposing a multi-temporal FCN. A performance comparison between the proposed framework and a regular FCN is also conducted. In this study, we consider four UF classes typical of many European Western cities. Results show that training the proposed multi-date model on Sentinel 2 multi-temporal data achieved the best results with a Kappa coefficient increase of 2.72% and 6.40%, respectively for continuous UF and industrial facilities. Although a more definitive conclusion requires further testing, first results are promising because they confirm that integrating the temporal dimension with a high spatial resolution into urban land use classification may be a valuable strategy to discriminate among several urban categories. [ABSTRACT FROM AUTHOR]