Search

Abstract A major uncertainty in predicting the behaviour of marine-terminating glaciers is ice dynamics driven by non-linear calving front retreat, which is poorly understood and modelled. Using 124919 calving front positions for 149 marine-terminating glaciers in Svalbard from 1985 to 2023, generated with deep learning, we identify pervasive calving front retreats for non-surging glaciers over the past 38 years. We observe widespread seasonal cycles in calving front position for over half of the glaciers. At the seasonal timescale, peak retreat rates exhibit a several-month phase lag, with changes on the west coast occurring before those on the east coast, coincident with regional ocean warming. This spatial variability in seasonal patterns is linked to different timings of warm ocean water inflow from the West Spitsbergen Current, demonstrating the dominant role of ice-ocean interaction in seasonal front changes. The interannual variability of calving front retreat shows a strong sensitivity to both atmospheric and oceanic warming, with immediate responses to large air and ocean temperature anomalies in 2016 and 2019, likely driven by atmospheric blocking that can influence extreme temperature variability. With more frequent blocking occurring and continued regional warming, future calving front retreats will likely intensify, leading to more significant glacier mass loss.

Self-supervised learning guided by masked image modeling, such as masked autoencoder (MAE), has attracted wide attention for pretraining vision transformers in remote sensing. However, MAE tends to excessively focus on pixel details, limiting the model's capacity for semantic understanding, particularly for noisy synthetic aperture radar (SAR) images. In this article, we explore spectral and spatial remote sensing image features as improved MAE-reconstruction targets. We first conduct a study on reconstructing various image features, all performing comparably well or better than raw pixels. Based on such observations, we propose feature guided MAE (FG-MAE): reconstructing a combination of histograms of oriented gradients (HOG) and normalized difference indices (NDI) for multispectral images, and reconstructing HOG for SAR images. Experimental results on three downstream tasks illustrate the effectiveness of FG-MAE with a particular boost for SAR imagery (e.g., up to 5% better than MAE on EuroSAT-SAR). Furthermore, we demonstrate the well-inherited scalability of FG-MAE and release a first series of pretrained vision transformers for medium-resolution SAR and multispectral images.

Abstract The impact of climate change and urbanization has increased the risk of flooding. During the UN Climate Change Conference 28 (COP 28), an agreement was reached to establish “The Loss and Damage Fund” to assist low-income countries impacted by climate change. However, allocating the resources required for post-flood reconstruction and reimbursement is challenging due to the limited availability of data and the absence of a comprehensive tool. Here, we propose a novel resource allocation framework based on remote sensing and geospatial data near the flood peak, such as buildings and population. The quantification of resource distribution utilizes an exposure index for each municipality, which interacts with various drivers, including flood hazard drivers, buildings exposure, and population exposure. The proposed framework asses the flood extension using pre- and post-flood Sentinel-1 Synthetic Aperture Radar (SAR) data. To demonstrate the effectiveness of this framework, an analysis was conducted on the flood that occurred in the Thessaly region of Greece in September 2023. The study revealed that the municipality of Palamas has the highest need for resource allocation, with an exposure index rating of 5/8. Any government can use this framework for rapid decision-making and to expedite post-flood recovery.

In the intricate landscape of mapping urban slum dynamics, the significance of robust and efficient techniques is often underestimated and remains absent in many studies. This not only hampers the comprehensiveness of research but also undermines potential solutions that could be pivotal for addressing the complex challenges faced by these settlements. With this ethos in mind, we prioritize efficient methods to detect the complex urban morphologies of slum settlements. Leveraging transfer learning with minimal samples and estimating the probability of predictions for slum settlements, we uncover previously obscured patterns in urban structures. By using Monte Carlo dropout, we not only enhance classification performance in noisy datasets and ambiguous feature spaces but also gauge the uncertainty of our predictions. This offers deeper insights into the model's confidence in distinguishing slums, especially in scenarios where slums share characteristics with formal areas. Despite the inherent complexities, our custom CNN STnet stands out, delivering performance on par with renowned models like ResNet50 and Xception but with notably superior efficiency—faster training and inference, particularly with limited training samples. Combining Monte Carlo dropout, class-weighted loss function, and class-balanced transfer learning, we offer an efficient method to tackle the challenging task of classifying intricate urban patterns amidst noisy datasets. Our approach not only enhances artificial intelligence model training in noisy datasets but also advances our comprehension of slum dynamics, especially as these uncertainties shed light on the intricate intraurban variabilities of slum settlements.

In this study, we examine the potential of leveraging self-supervised learning (SSL) and transfer learning methodologies for forest disturbance mapping using Earth Observation (EO) data. Our focus is on natural disturbances caused by windthrow and snowload damages. Particularly, we investigate the potential of knowledge-distillation-based contrastive learning approaches to obtain comprehensive representations of forest structure changes using Copernicus Sentinel-1 and Sentinel-2 satellite imagery. Leveraging pretrained backbone models from knowledge distillation, we employ transfer learning based on deep change vector analysis to delineate forest changes. We demonstrate developed approaches on two use cases, namely, mapping windthown forest using bitemporal Sentinel-1 and Sentinel-2 images, and mapping forest areas damaged by excessive snowload using bitemporal Sentinel-1 images. Developed self-supervised models were compared in a benchmarking exercise. The best results were provided by pixel-level contrastive learning for Sentinel-1-based snowload damage mapping with an overall accuracy of 84% and an $F_{1}$ score of 0.567, and for Sentinel-2-based forest windthrow mapping with an overall accuracy of 76.5% and an $F_{1}$ score of 0.692. We expect that developed methodologies can be useful for mapping also other types of forest disturbances using Copernicus Sentinel images or similar EO data. Our findings underscore the potential of SSL and transfer learning for enhancing forest disturbance detection using EO.

In recent years, deep learning has emerged as the dominant approach for hyperspectral image (HSI) classification. However, deep neural networks require large annotated datasets to generalize well. This limits the applicability of deep learning for real-world HSI classification problems, as manual labeling of thousands of pixels per scene is costly and time consuming. In this article, we tackle the problem of few-shot HSI classification by leveraging state-of-the-art self-supervised contrastive learning with an improved view-generation approach. Traditionally, contrastive learning algorithms heavily rely on hand-crafted data augmentations tailored for natural imagery to generate positive pairs. However, these augmentations are not directly applicable to HSIs, limiting the potential of self-supervised learning in the hyperspectral domain. To overcome this limitation, we introduce two positive pair-mining strategies for contrastive learning on HSIs. The proposed strategies mitigate the need for high-quality data augmentations, providing an effective solution for few-shot HSI classification. Through extensive experiments, we show that the proposed approach improves accuracy and label efficiency on four popular HSI classification benchmarks. Furthermore, we conduct a thorough analysis of the impact of data augmentation in contrastive learning, highlighting the advantage of our positive pair-mining approach.

Exploiting machine learning techniques to automatically classify multispectral remote sensing imagery plays a significant role in deriving changes on the Earth’s surface. However, the computation power required to manage large Earth observation data and apply sophisticated machine learning models for this analysis purpose has become an intractable bottleneck. Leveraging quantum computing provides a possibility to tackle this challenge in the future. This article focuses on land cover classification by analyzing Sentinel-2 images with quantum computing. Two hybrid quantum-classical deep learning frameworks are proposed. Both models exploit quantum computing to extract features efficiently from multispectral images and classical computing for final classification. As proof of concept, numerical simulation results on the LCZ42 dataset through the TensorFlow Quantum platform verify our models' validity. The experiments indicate that our models can extract features more effectively compared with their classical counterparts, specifically, the convolutional neural network (CNN) model. Our models demonstrated improvements, with an average test accuracy increase of 4.5% and 3.3%, respectively, in comparison to the CNN model. In addition, our proposed models exhibit better transferability and robustness than CNN models.

Accurate and up-to-date mapping of the human population is fundamental for a wide range of disciplines, from effective governance and establishing policies to disaster management and crisis dilution. The traditional method of gathering population data through census is costly and time-consuming. Recently, with the availability of large amounts of Earth observation data sets, deep learning methods have been explored for population estimation; however, they are either limited by census data availability, inter-regional evaluations, or transparency. In this paper, we present an end-to-end interpretable deep learning framework for large-scale population estimation at a resolution of 1km that uses only the publicly available data sets and does not rely on census data for inference. The architecture is based on a modification of the common ResNet-50 architecture tailored to analyze both image-like and vector-like data. Our best model outperforms the baseline random forest model by improving the RMSE by around 9% and also surpasses the community standard product, GHS-POP, thus yielding promising results. Furthermore, we improve the transparency of the proposed model by employing an explainable AI technique that identified land use information to be the most relevant feature for population estimation. We expect the improved interpretation of the model outcome will inspire both academic and non-academic end users, particularly those investigating urbanization or sub-urbanization trends, to have confidence in the deep learning methods for population estimation.

Abstract The fusion of synthetic aperture radar (SAR) and optical satellite data is widely used for deep learning based scene classification. Counter-intuitively such neural networks are still sensitive to changes in single data sources, which can lead to unexpected behavior and a significant drop in performance when individual sensors fail or when clouds obscure the optical image. In this paper we incorporate source-wise out-of-distribution (OOD) detection into the fusion process at test time in order to not consider unuseful or even harmful information for the prediction. As a result, we propose a modified training procedure together with an adaptive fusion approach that weights the extracted information based on the source-wise in-distribution probabilities. We evaluate the proposed approach on the BigEarthNet multilabel scene classification data set and several additional OOD test cases as missing or damaged data, clouds, unknown classes, and coverage by snow and ice. The results show a significant improvement in robustness to different types of OOD data affecting only individual data sources. At the same time the approach maintains the classification performance of the baseline approaches compared. The code for the experiments of this paper is available on GitHub: https://github.com/JakobCode/OOD_DataFusion

The aim of change detection in remote sensing usually is not to find all differences between the observations, but rather only specific types of change, such as urban development, deforestation, or even more specialized categories like roadwork. However, often there are no large public datasets available for very fine-grained tasks, and to collect the amount of training data needed for most supervised learning methods is very costly and often prohibitive. For this reason, we formulate the problem of few-shot filtering, where we are provided with a relatively large change detection dataset and, at test time, a few instances of one particular change type that we try to “filter out” of the learned changes. For example, we might train on data of general urban change, and, given some samples of building construction, aim to only predict instances of these on the test set, all without any explicit labels for buildings in the training data. We further investigate a fine-tuning approach to this problem and assess its performance on a public dataset that we adapt to be used in this novel setting.

Change detection (CD) is an important yet challenging task in remote sensing. In this article, we underline that the combination of unsupervised and supervised methods in a semisupervised framework improves CD performance. We rely on half-sibling regression for optical change detection (SiROC) as an unsupervised teacher model to generate pseudolabels (PLs) and select only the most confident PLs for pretraining different student models. Our results are robust to three different competitive student models, two semisupervised PL baselines, two benchmark datasets, and a variety of loss functions. While the performance gains are highest with a limited number of labels, a notable effect of PL pretraining persists when more labeled data are used. Further, we outline that the confidence selection of SiROC is indeed effective and that the performance gains generalize to scenes that were not used for PL training. Through the PL pretraining, SemiSiROC allows student models to learn more refined shapes of changes and makes them less sensitive to differences in acquisition conditions.

We propose a tree-level biomass estimation model approximating allometric equations by LiDAR data. Since tree crown diameter estimation is challenging from spaceborne LiDAR measurements, we develop a model to correlate tree height with biomass on the individual-tree levels employing a Gaussian process regressor. In order to validate the proposed model, a set of 8342 samples on tree height, trunk diameter, and biomass has been assembled. It covers seven biomes globally present. We reference our model to four other models based on both, the Jucker data and our own dataset. Although our approach deviates from standard biomass–height–diameter models, we demonstrate the Gaussian process regression model as a viable alternative. In addition, we decompose the uncertainty of tree biomass estimates into the model- and fitting-based contributions. We verify the Gaussian process regressor has the capacity to reduce the fitting uncertainty down to below 5%. Exploiting airborne LiDAR measurements and a field inventory survey on the ground, a stand-level (or plot-level) study confirms a low relative error of below 1% for our model. The data used in this study are available at https://github.com/zhu-xlab/BiomassUQ.

We introduce an unsupervised learning method that aims to identify building anomalies using Interferometric Synthetic Aperture Radar (InSAR) time-series data. Specifically, we leverage data obtained from the European Ground Motion Service to develop our proposed approach, which employs a long short-term memory autoencoder model and a reconstruction loss function based on a soft variant of the dynamic time warping, namely “soft-DTW”. We deliberately utilize this loss function for its ability to compare time-series that are not aligned in time, unlike the other conventional reconstruction losses that do not account for time shifts. Moreover, we enhance the performance of anomaly detection by smoothing inputs with a Hann window and defining the learning objective to reconstruct the time order of randomly permuted input series. Our experimental findings, based on persistent scatterer data from Rome, Italy, reveal that our method outperforms several unsupervised machine learning and deep learning methods in detecting various types of building displacement, such as trend, noise, and step anomalies. Additionally, quantitative and qualitative evaluations validate the efficacy of our approach in identifying potentially anomalous buildings. Thus, our method offers a promising solution for detecting anomalies in PS-InSAR time-series, which could have substantial implications in the fields of urban monitoring and infrastructure management.

Measurement(s) human population distribution Technology Type(s) remote sensing Factor Type(s) remote sensing data Sample Characteristic - Location Europe

The function of plastics is an important issue, especially since it determines whether or not they can be recycled. This study presents a two-stage workflow to identify the functions of plastic materials on land surfaces using a deep learning model trained with Sentinel-2 satellite images. First, a classification map identifying 10 distinct plastic types was obtained by evaluating spaceborne hyperspectral PRISMA data. Then, different deep learning algorithms were used to assign functions to the initially classified plastic targets based on the RGB information extracted from Sentinel-2 satellite images. A total of 1,645 plastic polygons were manually labeled on RGB images of Sentinel-2, and the following five main function types were identified: plastic cover sheeting for construction areas, greenhouse structures, photovoltaic panels (PVs), roof materials, and sport field floorings. By comparing three state-of-the-art deep learning models, including GoogLeNet, VGGNet, and ResNet, an overall accuracy of 78% was achieved on the test dataset using the VGG-13 network. The model performed well in identifying PVs, greenhouses, and construction sites, with F1 scores of 0.85, 0.77, and 0.71 respectively. The performance of the model in identifying roofs and sport field floorings was lower, with respective F1 scores of 0.57 and 0.59. Overall, the results show that the proposed workflow using deep learning algorithms trained on Sentinel-2 images has a great potential to identify the function of plastic materials on land surfaces.

This paper describes a deep transfer model which consists of multiple sub-networks which are independently optimized by a supervised task-oriented loss and an unsupervised consistency loss. The former loss function utilizes annotations to accomplish the designed goal. The latter loss prompts the network to learn the target domain data distribution and encourages multiple sub-networks to share learned knowledge. We utilize the proposed model to work with a global local climate zone classification. For the dataset, the source domain includes 352,366 training samples from 42 cities, and the target domain has 48,307 samples from 10 other cities. According to our experiments, the proposed model improves 3.46% overall accuracy and 2.97% average accuracy when compared with other state-of-the-art domain adaptation methodologies. Besides, the classification maps also visualize the outstanding performance of the proposed deep transfer network.

Change detection from traditional 2D optical images has limited capability to model the changes in the height or shape of objects. Change detection using 3D point cloud from photogrammetry or LiDAR surveying can fill this gap by providing critical depth information. While most existing machine learning based 3D point cloud change detection methods are supervised, they severely depend on the availability of annotated training data, which is in practice a critical point. To circumnavigate this dependence, we propose an unsupervised 3D point cloud change detection method mainly based on self-supervised learning using deep clustering and contrastive learning. The proposed method also relies on an adaptation of deep change vector analysis to 3D point cloud via nearest point comparison. Experiments conducted on an aerial LiDAR survey dataset show that the proposed method obtains higher performance in comparison to the traditional unsupervised methods, with a gain of about 9% in mean accuracy (to reach more than 85%). Thus, it appears to be a relevant choice in scenario where prior knowledge (labels) is not ensured. The code will be made available at https://github.com/IdeGelis/torch-points3d-SSL-DCVA.

Sentinel-2 imagery has garnered significant attention in many earth system studies due to free access and high revisit frequency. Since its spatial resolution is insufficient for many applications, e.g., fine-grained land cover mapping, some studies employ fusion technique that combines high-resolution RGB images with Sentinel-2 multispectral images to improve the resolution of the latter. However, there are two issues in the existing image fusion methods. First, these methods usually assume that the time intervals between images are short (within several days), which is a strong assumption for large-scale high-resolution images and many real-world applications. Second, the spectral discrepancy between multispectral and RGB images could induce spectral aberrations in Sentinel-2 imagery upon fusion. To alleviate these issues, we propose an adaptive image fusion approach named S2IFNet, adaptively fusing images with long-time intervals (from months to years) and spectral inconsistency, thereby increasing the multispectral band resolution of Sentinel-2 imagery. Building on top of the feature extraction and fusion modules, we propose a spectral feature compensation module and a change-aware feature reconstruction module. The former alleviates the possible degradation of spectral attributes in Sentinel-2 imagery resulting from feature fusion. The latter integrates semantic and texture information to avoid adding fake textures caused by land cover changes over time. The experiments demonstrate that S2IFNet surpasses existing image fusion and reference-based super-resolution methods on synthetic and real datasets, yielding fusion results that are clearer and more reliable.

Three-dimensional (3D) building structures are vital to understanding urban dynamics. Monocular remote sensing imagery is a cost-effective data source for large-scale building height retrieval when compared to LiDAR data and multi-view imagery. Existing methods learn building footprints and height maps per pixel via either a multi-task network or two separate networks, however, failing to consider the information of neighboring pixels that belong to the identical building. Therefore, we propose learning a novel representation for 3D buildings, namely 3D centripetal shifts, a unified representation of individual building instances. Our method is termed as 3DCentripetalNet and learns the 3D centripetal shift representation that incorporates planar and vertical structures of buildings. Afterward, a decoupling module is devised to learn building corner points. Finally, a 3D modeling module is designed to retrieve building height from the learned 3D centripetal shift map and corner points. We investigate the proposed 3DCentripetalNet on two datasets with different spatial resolutions, i.e., the ISPRS Vaihingen dataset (9 cm/pixel) and the Urban 3D dataset (50 cm/pixel). Experimental results suggest that 3DCentripetalNet is able to preserve sharp building boundaries, largely alleviate false detections, and significantly outperform other competitors. Thus, we believe that 3DCentripetalNet is a robust solution for the task of building height retrieval from monocular imagery.

Estimating solar rooftop potential at a national level is a fundamental building block for every country to utilize solar power efficiently. Solar rooftop potential assessment relies on several features such as building geometry, location, and surrounding facilities. Hence, national-level approximations that do not take these factors into deep consideration are often inaccurate. This article introduces Lebanon’s first comprehensive footprint and solar rooftop potential maps using deep learning-based instance segmentation to extract buildings’ footprints from satellite images. A photovoltaic panels placement algorithm that considers the morphology of each roof is proposed. We show that the average rooftop’s solar potential can fulfill the yearly electric needs of a single-family residence while using only 5% of the roof surface. The usage of 50% of a residential apartment rooftop area would achieve energy security for up to 8 households. We also compute the average and total solar rooftop potential per district to localize regions corresponding to the highest and lowest solar rooftop potential yield. Factors such as size, ground coverage ratio and $PV_{out}$ are carefully investigated for each district. Baalbeck district yielded the highest total solar rooftop potential despite its low built-up area. While Beirut capital city has the highest average solar rooftop potential due to its extremely populated urban nature. Reported results and analysis reveal solar rooftop potential urban patterns and provides policymakers and key stakeholders with tangible insights. Lebanon’s total solar rooftop potential is about 28.1 $TWh/year$, two times larger than the national energy consumption in 2019.

The scientific outcomes of the 2022 Landslide4Sense (L4S) competition organized by the Institute of Advanced Research in Artificial Intelligence are presented here. The objective of the competition is to automatically detect landslides based on large-scale multiple sources of satellite imagery collected globally. The 2022 L4S aims to foster interdisciplinary research on recent developments in deep learning (DL) models for the semantic segmentation task using satellite imagery. Over the past few years, DL-based models have achieved performance that meets expectations on image interpretation due to the development of convolutional neural networks. The main objective of this article is to present the details and the best-performing algorithms featured in this competition. The winning solutions are elaborated with state-of-the-art models, such as the Swin Transformer, SegFormer, and U-Net. Advanced machine learning techniques and strategies, such as hard example mining, self-training, and mix-up data augmentation, are also considered. Moreover, we describe the L4S benchmark dataset in order to facilitate further comparisons and report the results of the accuracy assessment online. The data are accessible on Future Development Leaderboard for future evaluation at https://www.iarai.ac.at/landslide4sense/challenge/, and researchers are invited to submit more prediction results, evaluate the accuracy of their methods, compare them with those of other users, and, ideally, improve the landslide detection results reported in this article.

Change detection using satellite/aerial images is used to quantify the impacts of many natural and man-made disasters. At the occurrence of such events, both prechange optical and synthetic aperture radar (SAR) images can be obtained by going back in time. However, the availability of the postchange optical image is often hindered by the presence of artifacts like clouds. To circumnavigate this, we propose a novel change detection data setting that uses both optical and SAR images prechange, yet only SAR imagery postchange. For this challenging scenario, we propose a Siamese network that processes the prechange and postchange SAR inputs using a shared set of weights, while the prechange optical input is processed using a network that do not share the weights with the SAR inputs. The encoded weights from the three networks are fused and finally decoded using a common decoder to obtain the change map. Our model effectively fuses multisensor information and can obtain satisfactory result despite the absence of the postchange optical image. Experimental results on a multisensor urban dataset demonstrate the effectiveness of the proposed approach.

Most of Earth is covered by haze or clouds, impeding the constant monitoring of our planet. Preceding works have documented the detrimental effects of cloud coverage on remote sensing applications and proposed ways to approach this issue. However, up to now, little effort has been spent on understanding how exactly atmospheric disturbances impede the application of modern machine learning methods to Earth observation data. Specifically, we consider the effects of haze and cloud coverage on a scene classification task. We provide a thorough investigation of how classifiers trained on cloud-free data fail once they encounter noisy imagery—a common scenario encountered when deploying pretrained models for remote sensing to real use cases. We show how and why remote sensing scene classification suffers from cloud coverage. Based on a multistage analysis, including explainability approaches applied to the predictions, we work out four different types of effects that clouds have on scene prediction. The contribution of our work is to deepen the understanding of the effects of clouds on common remote sensing applications and consequently guide the development of more robust methods.

Solar power is a clean and renewable energy source. Promoting solar technology can not only offer all people affordable, reliable, and modern energy, but also mitigate energy-related emissions and pollutants. This significantly contributes to sustainable development goals. Aerial imagery can provide a cost-effective way for large-scale rooftop solar potential analysis when compared to other data sources. Existing studies mainly utilize aerial imagery and convolutional neural networks to learn the roof segmentation mask or the rooftop geometry map, which are the preliminary input for rooftop solar potential estimation. However, these methods fail to achieve precise solar potential analysis results. To address this issue, we propose a framework, which is termed as SolarNet for rooftop solar potential estimation. A novel multi-task learning network is devised in SolarNet to learn our proposed novel representation for rooftop geometry that incorporates 6 roof segments and orientations. Specifically, this network first learns a roof segmentation map, and then together with the extracted multiscale and contextual features to learn a roof geometry map. Finally, the solar potential can be estimated from the learned roof geometry map. The effectiveness of SolarNet is validated on two datasets: DeepRoof and RID datasets. Experimental results demonstrate that SolarNet can improve not only rooftop geometry prediction accuracy but also solar potential estimation precision, which significantly outperforms other competitors.

Many deep learning approaches make extensive use of backbone networks pretrained on large datasets like ImageNet, which are then fine-tuned. In remote sensing, the lack of comparable large annotated datasets and the diversity of sensing platforms impedes similar developments. In order to contribute towards the availability of pretrained backbone networks in remote sensing, we devise a self-supervised approach for pretraining deep neural networks. By exploiting the correspondence between co-located imagery and audio recordings, this is done completely label-free, without the need for manual annotation. For this purpose, we introduce the SoundingEarth dataset, which consists of co-located aerial imagery and crowd-sourced audio samples all around the world. Using this dataset, we then pretrain ResNet models to map samples from both modalities into a common embedding space, encouraging the models to understand key properties of a scene that influence both visual and auditory appearance. To validate the usefulness of the proposed approach, we evaluate the transfer learning performance of pretrained weights obtained against weights obtained through other means. By fine-tuning the models on a number of commonly used remote sensing datasets, we show that our approach outperforms existing pretraining strategies for remote sensing imagery. The dataset, code and pretrained model weights are available at https://github.com/khdlr/SoundingEarth.

Although deep learning techniques have brought unprecedented accuracy to automatic building extraction, several main issues still constitute an obstacle to effective and practical applications. The industry is eager for higher accuracy and more flexible data usage. In this paper, we present a co-learning framework applicable to building extraction from optical images and photogrammetric point clouds, which can take the advantage of 2D/3D multimodality data. Instead of direct information fusion, our co-learning framework adaptively exploits knowledge from another modality during the training phase with a soft connection, via a predefined loss function. Compared to conventional data fusion, this method is more flexible, as it is not mandatory to provide multimodality data in the test phase. We propose two types of co-learning: a standard version and an enhanced version, depending on whether unlabeled training data are employed. Experimental results from two data sets show that the methods we present can enhance the performance of both image and point cloud networks in few-shot tasks, as well as image networks when applying fully labeled training data sets.

Buildings are the predominant objects that characterize the urban structure. For many cities, local governments establish building databases for administration as well as urban planning and monitoring. However, newly constructed buildings are often only included with a considerable time delay in the official digital cadastral maps due to processes in the acquisition of data, so-called undocumented buildings. In this regard, detecting undocumented buildings using remote sensing techniques would support the construction of update-to-date building databases with complementary information. In-depth studies on undocumented buildings and their number and location, however, are scarce. Therefore, we exploit a deep learning-based framework to detect undocumented buildings in remote sensing data and propose to derive 2D and 3D morphological parameters as well as landscape metrics., which are capable of depicting the physical forms and spatial structures of undocumented buildings. Furthermore, we exemplify the variabilities of undocumented buildings across space by the differences in morphology and landscape metrics between high and low building density regions. Upon analysis of undocumented buildings in 15 cities in the state of Bavaria, Germany, both state- and city-scale results reveal that most undocumented buildings are located in lower dense regions. This reveals that fragmentation of the landscape by building structures in the state of Bavaria is probably greater than official geospatial data currently documented.

Deep Learning usually requires large amounts of labeled training data. In remote sensing, deep learning is often applied for land cover and land use classification as well as street network and building segmentation. In case of the latter, a common way of obtaining training labels is to leverage crowdsourced datasets which can provide numerous types of spatial information on a global scale. However, labels from crowdsourced datasets are often limited in the sense that they potentially contain high levels of noise. Understanding how such noisy labels impede the predictive performance of Deep Neural Networks (DNNs) is crucial for evaluating if crowdsourced data can be an answer to the need for large training sets by DNNs. One way towards this understanding is to identify the factors which affect the relationship between label noise and predictive performance of a model. The size of the training set could be one of these factors since it is well known for being able to greatly influence a model’s predictive performance. In this work we pick the size of the training set and study its influence on the robustness of a model against a common type of label noise known as omission noise. To this end, we utilize a dataset of aerial images for building segmentation and create several versions of the training labels by introducing different amounts of omission noise. We then train a state-of-the-art model on subsets of varying size of those versions. Our results show that the training set size does play a role in affecting the robustness of our model against label noise: A large training set improves the robustness of our model against omission noise.

Deep transfer-learning-based change detection methods are dependent on the availability of sensor-specific pretrained feature extractors. Such feature extractors are not always available due to lack of training data, especially for hyperspectral sensors and other hyperdimensional images. Moreover models trained on easily available multispectral (RGB/RGB-NIR) images cannot be reused on such hyperdimensional images due to their irregular number of bands. While hyperdimensional images show large number of spectral bands, they generally show much less spatial complexity, thus reducing the requirement of large receptive fields of convolution filters. Recent works in the computer vision have shown that even untrained deep models can yield remarkable result in some tasks like super-resolution and surface reconstruction. This motivates us to make a bold proposition that untrained lightweight deep model, initialized with some weight initialization strategy, can be used to extract useful semantic features from bi-temporal hyperdimensional images. Based on this proposition, we design a novel change detection framework for hyperdimensional images by extracting bitemporal features using an untrained model and further comparing the extracted features using deep change vector analysis to distinguish changed pixels from the unchanged ones. We further use the deep change hypervectors to cluster the changed pixels into different semantic groups. We conduct experiments on four change detection datasets: three hyperspectral datasets and a hyperdimensional polarimetric synthetic aperture radar dataset. The results clearly demonstrate that the proposed method is suitable for change detection in hyperdimensional remote sensing data.

The past years have witnessed great progress on remote sensing (RS) image interpretation and its wide applications. With RS images becoming more accessible than ever before, there is an increasing demand for the automatic interpretation of these images. In this context, the benchmark datasets serve as an essential prerequisites for developing and testing intelligent interpretation algorithms. After reviewing existing benchmark datasets in the research community of RS image interpretation, this article discusses the problem of how to efficiently prepare a suitable benchmark dataset for RS image interpretation. Specifically, we first analyze the current challenges of developing intelligent algorithms for RS image interpretation with bibliometric investigations. We then present the general guidances on creating benchmark datasets in efficient manners. Following the presented guidances, we also provide an example on building RS image dataset, i.e., Million Aerial Image Dataset (Online. Available: https://captain-whu.github.io/DiRS/), a new large-scale benchmark dataset containing a million instances for RS image scene classification. Several challenges and perspectives in RS image annotation are finally discussed to facilitate the research in benchmark dataset construction. We do hope this article will provide the RS community an overall perspective on constructing large-scale and practical image datasets for further research, especially data-driven ones.

As a unique classification scheme for urban forms and functions, the local climate zone (LCZ) system provides essential general information for any studies related to urban environments, especially on a large scale. Remote sensing data-based classification approaches are the key to large-scale mapping and monitoring of LCZs. The potential of deep learning-based approaches is not yet fully explored, even though advanced convolutional neural networks (CNNs) continue to push the frontiers for various computer vision tasks. One reason is that published studies are based on different datasets, usually at a regional scale, which makes it impossible to fairly and consistently compare the potential of different CNNs for real-world scenarios. This article is based on the big So2Sat LCZ42 benchmark dataset dedicated to LCZ classification. Using this dataset, we studied a range of CNNs of varying sizes. In addition, we proposed a CNN to classify LCZs from Sentinel-2 images, Sen2LCZ-Net. Using this base network, we propose fusing multilevel features using the extended Sen2LCZ-Net-MF. With this proposed simple network architecture, and the highly competitive benchmark dataset, we obtain results that are better than those obtained by the state-of-the-art CNNs, while requiring less computation with fewer layers and parameters. Large-scale LCZ classification examples of completely unseen areas are presented, demonstrating the potential of our proposed Sen2LCZ-Net-MF as well as the So2Sat LCZ42 dataset. We also intensively investigated the influence of network depth and width, and the effectiveness of the design choices made for Sen2LCZ-Net-MF. This article will provide important baselines for future CNN-based algorithm developments for both LCZ classification and other urban land cover land use classification. Code and pretrained models are available at https://github.com/ChunpingQiu/benchmark-on-So2SatLCZ42-dataset-a-simple-tour.

In the course of global urbanization, poverty in cities has been observed to increase, especially in the Global South. Poverty is one of the major challenges for our society in the upcoming decades, making it one of the most important issues in the Sustainable Development Goals defined by the United Nations. Satellite-based mapping can provide valuable information about slums where insights about the location and size are still missing. Large-scale slum mapping remains a challenge, fuzzy feature spaces between formal and informal settlements, significant imbalance of slum occurrences opposed to formal settlements, and various categories of multiple morphological slum features. We propose a transfer learned fully convolutional Xception network (XFCN), which is able to differentiate between formal built-up structures and the various categories of slums in high-resolution satellite data. The XFCN is trained on a large sample of globally distributed slums, located in cities of Cape Town, Caracas, Delhi, Lagos, Medellin, Mumbai, Nairobi, Rio de Janeiro, São Paulo, and Shenzhen. Slums in these cities are greatly heterogeneous inits morphological feature space and differ to a varying degree to formal settlements. Transfer learning can help to improve segmentation results when learning on a variety of slum morphologies, with high F1 scores of up to 89%.

By the year 2050, it is expected that about 68% of global population will live in cities. To understand the emerging changes in urban structures, new data sources like social media must be taken into account. In this work, we conduct a feature space analysis of geo-tagged Twitter text messages from the Los Angeles area and a geo-spatial text mining approach to classify buildings types into commercial and residential. To create the feature space, broadly accepted word embedding models like word2vec, fastText and GloVe as well as more traditional models based on TF-IDF have been considered. A visual analysis of the word embeddings shows that the two examined classes yield several word clusters. However, the classification results produced by Naïve Bayes support vector machines, and a convolutional neural network indicates that building classification from pure social media text is quite challenging. Furthermore, this work illustrates a base toward fusing text features and remote sensing images to classify urban building types.

Accurate and reliable building footprint maps are of great interest in many applications, e.g., urban monitoring, 3D building modeling, and geographical database updating. When compared to traditional methods, the deep-learning-based semantic segmentation networks have largely boosted the performance of building footprint generation. However, they still are not capable of delineating structured building footprints. Most existing studies dealing with this issue are based on two steps, which regularize building boundaries after the semantic segmentation networks are implemented, making the whole pipeline inefficient. To address this, we propose an end-to-end network for the building footprint generation with boundary regularization, which is termed RegGAN. Our method is based on a generative adversarial network (GAN). Specifically, a multiscale discriminator is proposed to distinguish the input between false and true, and a generator is utilized to learn from the discriminator’s response to generate more realistic building footprints. We propose to incorporate regularized loss in the objective function of RegGAN, in order to further enhance sharp building boundaries. The proposed method is evaluated on two datasets with varying spatial resolutions: the INRIA dataset (30 cm/pixel) and the ISPRS dataset (5 cm/pixel). Experimental results show that RegGAN is able to well preserve regular shapes and sharp building boundaries, which outperforms other competitors.

With the objective of exploiting hardware capabilities and preparing the ground for the next-generation X-band synthetic aperture radar (SAR) missions, TerraSAR-X and TanDEM-X are now able to operate in staring spotlight mode, which is characterized by an increased azimuth resolution of approximately 0.24 m compared with 1.1 m of the conventional sliding spotlight mode. In this paper, we demonstrate for the first time its potential for SAR tomography (TomoSAR). To this end, we tailored our interferometric and tomographic processors for the distinctive features of the staring spotlight mode, which will be analyzed accordingly. By means of its higher spatial resolution, the staring spotlight mode will not only lead to a denser point cloud but also to more accurate height estimates due to the higher signal-to-clutter ratio. As a result of a first comparison between sliding and staring spotlight TomoSAR, first, the density of the staring spotlight point cloud is approximately 5.1-5.5 times as high; and, second, the relative height accuracy of the staring spotlight point cloud is approximately 1.7 times as high.

Local manifold learning has been successfully applied to hyperspectral dimensionality reduction in order to embed nonlinear and nonconvex manifolds in the data. Local manifold learning is mainly characterized by affinity matrix construction, which is composed of two steps: neighbor selection and computation of affinity weights. There is a challenge in each step: First, the neighbor selection is sensitive to complex spectral variability due to nonuniform data distribution, illumination variations, and sensor noise; second, the computation of affinity weights is challenging due to highly correlated spectral signatures in the neighborhood. To address the two issues, in this paper, a novel manifold learning methodology based on locally linear embedding is proposed through learning a robust local manifold representation. More specifically, a hierarchical neighbor selection is designed to progressively eliminate the effects of complex spectral variability using joint normalization and to robustly compute affinity (or reconstruction) weights reducing multicollinearity via the refined neighbor selection. Additionally, an idea that combines spatial-spectral information is introduced into the proposed manifold learning methodology to further improve the robustness of affinity calculations. Classification is explored as a potential application for validating the proposed algorithm. The classification accuracy in the use of different dimensionality reduction methods is evaluated and compared, while two kinds of strategies are applied in selecting the training and test samples: random sampling and region-based sampling. Experimental results show the classification accuracy obtained by the proposed method is superior to those state-of-the-art dimensionality reduction methods.

Interferometric synthetic aperture radar (InSAR) techniques, such as persistent scatterer interferometry (PSI) or SAR tomography (TomoSAR), deliver three-dimensional (3-D) point clouds of the scatterers' positions together with their motion information relative to a reference point. Due to the SAR side-looking geometry, minimum of two point clouds from cross-heading orbits, i.e., ascending and descending, are required to achieve a complete monitoring over an urban area. However, these two point clouds are usually not coregistered due to their different reference points with unknown 3-D positions. In general, no exact identical points from the same physical object can be found in such two point clouds. This article describes a robust algorithm for fusing such two point clouds of urban areas. The contribution of this paper is finding the theoretically exact point correspondence, which is the end positions of façades, where the two point clouds close. We explicitly define this algorithm as “L-shape detection and matching,” in this paper, because the façades commonly appear as L-shapes in InSAR point cloud. This algorithm introduces a few important features for a reliable result, including point density estimation using adaptive directional window for better façade points detection and L-shape extraction using weighed Hough transform. The algorithm is fully automatic. Its accuracy is evaluated using simulated data. Furthermore, the proposed method is applied on two TomoSAR point clouds over Berlin with ascending and descending geometry. The result is compared with the first PSI point cloud fusion method (S. Gernhardt and R. Bamler, “Deformation monitoring of single buildings using meter-resolution SAR data in PSI,” ISPRS J. Photogramm. Remote Sens., vol. 73, pp. 68-79, 2012.) for urban area. Submeter consistency is achieved.

The remote sensing based mapping of land cover at extensive scales, e.g., of whole continents, is still a challenging task because of the need for sophisticated pipelines that combine every step from data acquisition to land cover classification. Utilizing the Google Earth Engine (GEE), which provides a catalog of multi-source data and a cloud-based environment, this research generates a land cover map of the whole African continent at 10 m resolution. This land cover map could provide a large-scale base layer for a more detailed local climate zone mapping of urban areas, which lie in the focus of interest of many studies. In this regard, we provide a free download link for our land cover maps of African cities at the end of this paper. It is shown that our product has achieved an overall accuracy of 81% for five classes, which is superior to the existing 10 m land cover product FROM-GLC10 in detecting urban class in city areas and identifying the boundaries between trees and low plants in rural areas. The best data input configurations are carefully selected based on a comparison of results from different input sources, which include Sentinel-2, Landsat-8, Global Human Settlement Layer (GHSL), Night Time Light (NTL) Data, Shuttle Radar Topography Mission (SRTM), and MODIS Land Surface Temperature (LST). We provide a further investigation of the importance of individual features derived from a Random Forest (RF) classifier. In order to study the influence of sampling strategies on the land cover mapping performance, we have designed a transferability analysis experiment, which has not been adequately addressed in the current literature. In this experiment, we test whether trained models from several cities contain valuable information to classify a different city. It was found that samples of the urban class have better reusability than those of other natural land cover classes, i.e., trees, low plants, bare soil or sand, and water. After experimental evaluation of different land cover classes across different cities, we conclude that continental land cover mapping results can be considerably improved when training samples of natural land cover classes are collected and combined from areas covering each Köppen climate zone.

This article addresses the question of mapping building functions jointly using both aerial and street view images via deep learning techniques. One of the central challenges here is determining a data fusion strategy that can cope with heterogeneous image modalities. We demonstrate that geometric combinations of the features of such two types of images, especially in an early stage of the convolutional layers, often lead to a destructive effect due to the spatial misalignment of the features. Therefore, we address this problem through a decision-level fusion of a diverse ensemble of models trained from each image type independently. In this way, the significant differences in appearance of aerial and street view images are taken into account. Compared to the common multi-stream end-to-end fusion approaches proposed in the literature, we are able to increase the precision scores from 68% to 76%. Another challenge is that sophisticated classification schemes needed for real applications are highly overlapping and not very well defined without sharp boundaries. As a consequence, classification using machine learning becomes significantly harder. In this work, we choose a highly compact classification scheme with four classes, commercial, residential, public, and industrial because such a classification has a very high value to urban geography being correlated with socio-demographic parameters such as population density and income.

In remote sensing, hyperspectral and polarimetric synthetic aperture radar (PolSAR) images are the two most versatile data sources for a wide range of applications such as land use land cover classification. However, the fusion of these two data sources receive less attention than many other, because of their scarce data availability, and relatively challenging fusion task caused by their distinct imaging geometries. Among the existing fusion methods, including manifold learning-based, kernel-based, ensemble-based, and matrix factorization, manifold learning is one of most celebrated techniques for the fusion of heterogeneous data. Therefore, this paper aims to promote the research in hyperspectral and PolSAR data fusion, by providing a comprehensive comparison between existing manifold learning-based fusion algorithms. We conducted experiments on 16 state-of-the-art manifold learning algorithms that embrace two important research questions in manifold learning-based fusion of hyperspectral and PolSAR data: (1) in which domain should the data be aligned—the data domain or the manifold domain; and (2) how to make use of existing labeled data when formulating a graph to represent a manifold—supervised, semi-supervised, or unsupervised. The performance of the algorithms were evaluated via multiple accuracy metrics of land use land cover classification over two data sets. Results show that the algorithms based on manifold alignment generally outperform those based on data alignment (data concatenation). Semi-supervised manifold alignment fusion algorithms performs the best among all. Experiments using multiple classifiers show that they outperform the benchmark data alignment-based algorithms by ca. 3% in terms of the overall classification accuracy.

Global Local Climate Zone (LCZ) maps, indicating urban structures and land use, are crucial for Urban Heat Island (UHI) studies and also as starting points to better understand the spatio-temporal dynamics of cities worldwide. However, reliable LCZ maps are not available on a global scale, hindering scientific progress across a range of disciplines that study the functionality of sustainable cities. As a first step towards large-scale LCZ mapping, this paper tries to provide guidance about data/feature choice. To this end, we evaluate the spectral reflectance and spectral indices of the globally available Sentinel-2 and Landsat-8 imagery, as well as the Global Urban Footprint (GUF) dataset, the OpenStreetMap layers buildings and land use and the Visible Infrared Imager Radiometer Suite (VIIRS)-based Nighttime Light (NTL) data, regarding their relevance for discriminating different Local Climate Zones (LCZs). Using a Residual convolutional neural Network (ResNet), a systematic analysis of feature importance is performed with a manually-labeled dataset containing nine cities located in Europe. Based on the investigation of the data and feature choice, we propose a framework to fully exploit the available datasets. The results show that GUF, OSM and NTL can contribute to the classification accuracy of some LCZs with relatively few samples, and it is suggested that Landsat-8 and Sentinel-2 spectral reflectances should be jointly used, for example in a majority voting manner, as proven by the improvement from the proposed framework, for large-scale LCZ mapping.

Recently, convolutional neural networks (CNN) have been intensively investigated for the classification of remote sensing data by extracting invariant and abstract features suitable for classification. In this paper, a novel framework is proposed for the fusion of hyperspectral images and LiDAR-derived elevation data based on CNN and composite kernels. First, extinction profiles are applied to both data sources in order to extract spatial and elevation features from hyperspectral and LiDAR-derived data, respectively. Second, a three-stream CNN is designed to extract informative spectral, spatial, and elevation features individually from both available sources. The combination of extinction profiles and CNN features enables us to jointly benefit from low-level and high-level features to improve classification performance. To fuse the heterogeneous spectral, spatial, and elevation features extracted by CNN, instead of a simple stacking strategy, a multi-sensor composite kernels (MCK) scheme is designed. This scheme helps us to achieve higher spectral, spatial, and elevation separability of the extracted features and effectively perform multi-sensor data fusion in kernel space. In this context, a support vector machine and extreme learning machine with their composite kernels version are employed to produce the final classification result. The proposed framework is carried out on two widely used data sets with different characteristics: an urban data set captured over Houston, USA, and a rural data set captured over Trento, Italy. The proposed framework yields the highest OA of 92 . 57 % and 97 . 91 % for Houston and Trento data sets. Experimental results confirm that the proposed fusion framework can produce competitive results in both urban and rural areas in terms of classification accuracy, and significantly mitigate the salt and pepper noise in classification maps.

Persistent Scatterer Interferometry (PSI) is an advanced multitemporal InSAR technique that is capable of retrieving the 3D coordinates and the underlying deformation of time-coherent scatterers. Various factors degrade the localization accuracy of PSI point clouds in the geocoding process, which causes problems for interpretation of deformation results and also making it difficult for the point clouds to be compared with or integrated into data from other sensors. In this study, we employ the SAR imaging geodesy method to perform geodetic corrections on SAR timing observations and thus improve the positioning accuracy in the horizontal components. We further utilize geodetic stereo SAR to extract large number of highly precise ground control points (GCP) from SAR images, in order to compensate for the unknown height offset of the PSI point cloud. We demonstrate the applicability of the approach using TerraSAR-X high resolution spotlight images over the city of Berlin, Germany. The corrected results are compared with a reference LiDAR point cloud of Berlin, which confirms the improvement in the geocoding accuracy.

The concept of the local climate zone (LCZ) has been recently proposed as a generic land-cover/land-use classification scheme. It divides urban regions into 17 categories based on compositions of man-made structures and natural landscapes. Although it was originally designed for temperature study, the morphological structure concealed in LCZs also reflects economic status and population distribution. To this end, global LCZ classification is of great value for worldwide studies on economy and population. Conventional classification approaches are usually successful for an individual city using optical remote sensing data. This paper, however, attempts for the first time to produce global LCZ classification maps using polarimetric synthetic aperture radar (PolSAR) data. Specifically, we first produce polarimetric features, local statistical features, texture features, and morphological features and compare them, with respect to their classification performance. Here, an ensemble classifier is investigated, which is trained and tested on already separated transcontinental cities. Considering the challenging global scope this work handles, we conclude the classification accuracy is not yet satisfactory. However, Sentinel-1 dual-Pol SAR data could contribute the classification for several LCZ classes. According to our feature studies, the combination of local statistical features and morphological features yields the best classification results with 61.8% overall accuracy (OA), which is 3% higher than the OA produced by the second best features combination. The 3% is considerably large for a global scale. Based on our feature importance analysis, features related to VH polarized data contributed the most to the eventual classification result.

In this paper, we propose a generative framework to produce similar yet novel samples for a specified image. We then propose the use of these images as hard-negatives samples, within the framework of hard-negative mining, in order to improve the performance of classification networks in applications which suffer from sparse labelled training data. Our approach makes use of a variational autoencoder (VAE) which is trained in an adversarial manner in order to learn a latent distribution of the training data, as well as to be able to generate realistic, high quality image patches. We evaluate our proposed generative approach to hard-negative mining on a synthetic aperture radar (SAR) and optical image matching task. Using an existing SAR-optical matching network as the basis for our investigation, we compare the performance of the matching network trained using our approach to the baseline method, as well as to two other hard-negative mining methods. Our proposed generative architecture is able to generate realistic, very high resolution (VHR) SAR image patches which are almost indistinguishable from real imagery. Furthermore, using the patches as hard-negative samples, we are able to improve the overall accuracy, and significantly decrease the false positive rate of the SAR-optical matching task—thus validating our generative hard-negative mining approaches’ applicability to improve training in data sparse applications.

Since its launch in 2007, TerraSAR-X has continuously provided spaceborne synthetic aperture radar (SAR) images of our planet with unprecedented spatial resolution, geodetic, and geometric accuracy. This has brought life to the once inscrutable SAR images, which deterred many researchers. Thanks to merits like higher spatial resolution and more precise orbit control, we are now able to indicate individual buildings, even individual floors, to pinpoint targets within centimeter accuracy. As a result, multi-baseline SAR interferometric (InSAR) techniques are flourishing, from point target-based algorithms, to coherent stacking techniques, to absolute positioning of the former techniques. This article reviews the recent advances of multi-baseline InSAR techniques using TerraSAR-X images. Particular focus was put on our own development of persistent scatterer interferometry, SAR tomography, robust estimation in distributed scatterer interferometry and absolute positioning using geodetic InSAR. Furthermore, by introducing the applications associated with these techniques, such as 3D reconstruction and deformation monitoring, this article is also intended to give guidance to wider audiences who would like to resort to SAR data and related techniques for their applications.

Every city is—quoting Plato—divided into two, one city of the poor, the other of the rich. In this study we test whether the economic urban divide is reflected in the digital sphere of cities. Because, especially in dynamically growing cities, ready-to-use comprehensive data sets on the urban poor, as well as on the digital divide, are not existent, we use proxies: we spatially delimit the urban poor using settlement characteristics derived from remote sensing data. The digital divide is targeted by geolocated Twitter data. Based on a sample of eight cities across the globe, we spatially test whether areas of the urban poor are more likely to be digital cold spots. Over the course of time, we analyze whether temporal signatures in poor urban areas differ from formal environments. We find that the economic divide influences digital participation in public life. Less residents of morphological slums are found to be digitally oriented (“are digitally left behind”) as compared to residents of formal settlements. However, among the few twitter users in morphological slums, we find their temporal behavior similar to the twitter users in formal settlements. In general, we conclude this discussion, this study exemplifies that the combination of both heterogeneous data sets allows for extending the capabilities of individual disciplines for research towards urban poverty.

Urbanization is a substantial contributor to anthropogenic environmental change, and often occurs at a rapid pace that demands frequent and accurate monitoring. Time series of satellite imagery collected at fine spatial resolution using stable spectral bands over decades are most desirable for this purpose. In practice, however, temporal spectral variance arising from variations in atmospheric conditions, sensor calibration, cloud cover, and other factors complicates extraction of consistent information on changes in urban land cover. Moreover, the construction and application of effective training samples is time-consuming, especially at continental and global scales. Here, we propose a new framework for satellite-based mapping of urban areas based on transfer learning and deep learning techniques. We apply this method to Landsat observations collected during 1984–2016 and extract annual records of urban areas in four cities in the temperate zone (Beijing, New York, Melbourne, and Munich). The method is trained using observations of Beijing collected in 1999, and then used to map urban areas in all target cities for the entire 1984–2016 period. The method addresses two central challenges in long term detection of urban change: temporal spectral variance and a scarcity of training samples. First, we use a recurrent neural network to minimize seasonal urban spectral variance. Second, we introduce an automated transfer strategy to maximize information gain from limited training samples when applied to new target cities in similar climate zones. Compared with other state-of-the-art methods, our method achieved comparable or even better accuracy: the average change detection accuracy during 1984–2016 is 89% for Beijing, 94% for New York, 93% for Melbourne, and 89% for Munich, and the overall accuracy of single-year urban maps is approximately 96 ± 3% among the four target cities. The results demonstrate the practical potential and suitability of the proposed framework. The method is a promising tool for detecting urban change in massive remote sensing data sets with limited training data.