Search

Conditional (cGAN)-based methods proposed so far for synthetic aperture radar (SAR)-to-optical image synthesis tend to produce noisy and unsharp optical outcomes. In this work, we propose the atrous-cGAN, a novel cGAN architecture that improves the SAR-to-optical image translation. The proposed generator and discriminator networks rely on atrous convolutions and incorporate an atrous spatial pyramid pooling (ASPP) module to enhance fine details in the generated optical image by exploiting spatial context at multiple scales. This letter reports experiments carried out to assess the performance of atrous-cGAN for the synthesis of Landsat-8 images from Sentinel-1A data based on three public data sets. The experimental analysis indicated that the atrous-cGAN consistently outperformed the classical pix2pix counterpart in terms of visual quality, similar to the true optical image, and as a feature learning tool for semantic segmentation.

Changes in environmental conditions, geographical variability and different sensor properties typically make it almost impossible to employ previously trained classifiers for new data without a significant drop in classification accuracy. Domain adaptation (DA) techniques been proven useful to alleviate that problem. In particular, appearance adaptation techniques may be used to adapt images from a specific dataset in such a way that the generated images have a style that is similar to the images from another dataset. Such techniques are, however, prone to creating artifacts that hinder proper classification of the adapted images. In this work we propose an unsupervised DA approach for change detection tasks, which is based on a particular appearance adaptation method: the Cycle-Consistent Generative Adversarial Network (CycleGAN). Specifically, we extend that method by introducing additional constraints in the training phase of the model components, which make it preserve the semantic structure and class transitions in the adapted images. We evaluate the proposed approach on a deforestation detection application, considering different sites in the Amazon rain-forest and in the Brazilian Cerrado (savanna) using Landsat-8 images. In the experiments, each site corresponds to a domain, and the accuracy of a classifier trained with images and references from one (source) domain is measured in the classification of another (target) domain. The results show that the proposed approach is successful in producing artifact-free adapted images, which can be satisfactory classified by the pre-trained source classifiers. On average, the accuracies achieved in the classification of the adapted images outperformed the baselines (when no adaptation was made) by 7.1% in terms of mean average precision, and 9.1% in terms of F1-Score. To the best of our knowledge, the proposed method is the first unsupervised domain adaptation approach devised for change detection.

The increasing food demand is regarded as the main threat to nature today. In this scenario, Remote Rensing is an essential technology to assess and monitor the extent and productivity of cultivate...

Fully convolutional neural networks (FCN) are successfully used for pixel-wise land cover classification - the task of identifying the physical material of the Earth’s surface for every pixel in an image. The acquisition of large training datasets is challenging, especially in remote sensing, but necessary for a FCN to perform well. One way to circumvent manual labelling is the usage of existing databases, which usually contain a certain amount of label noise when combined with another data source. As a first part of this work, we investigate the impact of training data on a FCN. We experiment with different amounts of training data, varying w.r.t. the covered area, the available acquisition dates and the amount of label noise. We conclude that the more data is used for training, the better is the generalization performance of the model, and the FCN is able to mitigate the effect of label noise to a high degree. Another challenge is the imbalanced class distribution in most real-world datasets, which can cause the classifier to focus on the majority classes, leading to poor classification performance for minority classes. To tackle this problem, in this paper, we use the cosine similarity loss to force feature vectors of the same class to be close to each other in feature space. Our experiments show that the cosine loss helps to obtain more similar feature vectors, but the similarity of the cluster centers also increases.

This paper reports the results of a study that aims to develop semi-automatic methods for assessing the degree of corrosion in industrial plant. We evaluated two fully convolutional networks (U-Net and DeepLab v3 +) to segment corroded areas in panoramic images of offshore platforms. The experimental analysis was based on two datasets built for this study. The datasets comprise 9,112 2D images and 3,732 panoramic images. Both FCNs trained on 2D images were tested on 2D images and cubic projections of panoramic images. In addition to pointing out encouraging results, the experiments indicated that most prediction errors concentrated in corrosion defects with a small pixel area.

Deforestation in the Amazon rainforest is an alarming problem of global interest. Environmental impacts of this process are countless, but probably the most significant concerns regard the increase in CO2 emissions and global temperature rise. Currently, the assessment of deforested areas in the Amazon region is a manual task, where people analyse multiple satellite images to quantify the deforestation. We propose a method for automatic deforestation detection based on Deep Learning Neural Networks with dual-attention mechanisms. We employed a siamese architecture to detect deforestation changes between optical images in 2018 and 2019. Experiments were performed to evaluate the relevance and sensitivity of hyperparameter tuning of the loss function and the effects of dual-attention mechanisms (spatial and channel) in predicting deforestation. Experimental results suggest that a proper tuning of the loss function might bring benefits in terms of generalisation. We also show that the spatial attention mechanism is more relevant for deforestation detection than the channel attention mechanism. When both mechanisms are combined, the greatest improvements are found, and we reported an increase of 1.06% in the mean average precision over a baseline.

Crop recognition in tropical regions is a challenging task because of the highly complex crop dynamics, with multiple crops per year. Nevertheless, most automatic methods proposed thus far are devoted to temperate areas where normally a single crop is cultivated along the crop year. This paper introduces convolutional recurrent networks for crop recognition in areas characterized by complex spatiotemporal dynamics typical of tropical agriculture, where a per date classification is required. The proposed networks consist of two sequential steps. First, a deep network simultaneously models spatial and temporal contexts. Second, a post-processing algorithm enforces prior knowledge about the crop dynamics in the target area based on the posterior probabilities computed in the first step. The paper proposes deep network architectures that join a fully convolutional network (FCN) for modeling spatial context at multiple levels and a bidirectional recurrent neural network to explore the temporal context. The recurrent network is configured as N-to-N, where N is the sequence length. This allows it to produce classification outcomes for the entire sequence of multi-temporal images using a single network. Different network designs are proposed based on three FCN architectures: U-Net, dense network, and Atrous Spatial Pyramid Pooling. A convolutional Long-Short-Term-Memory (ConvLSTM) accounts for sequence modeling, whereas the Most Likely Class Sequence (MLCS) algorithm is adopted for enforcing prior knowledge. The paper finally reports experiments conducted on Sentinel-1 data of two publicly available datasets from different tropical regions. The experimental results indicated that the proposed architectures outperformed state-of-the-art methods based on recurrent networks in terms of Overall Accuracy and per-class F1 score.

The classification of tree species can significantly benefit from high spatial and spectral information acquired by unmanned aerial vehicles (UAVs) associated with advanced feature extraction and classification methods. Different from the traditional feature extraction methods, that highly depend on user’s knowledge, the convolutional neural network (CNN)-based method can automatically learn and extract the spatial-related features layer by layer. However, in order to capture significant features of the data, the CNN classifier requires a large number of training samples, which are hardly available when dealing with tree species in tropical forests. This study investigated the following topics concerning the classification of 14 tree species in a subtropical forest area of Southern Brazil: i) the performance of the CNN method associated with a previous step to increase and balance the sample set (data augmentation) for tree species classification as compared to the conventional machine learning methods support vector machine (SVM) and random forest (RF) using the original training data; ii) the performance of the SVM and RF classifiers when associated with a data augmentation step and spatial features extracted from a CNN. Results showed that the CNN classifier outperformed the conventional SVM and RF classifiers, reaching an overall accuracy (OA) of 84.37% and Kappa of 0.82. The SVM and RF had a poor accuracy with the original spectral bands (OA 62.67% and 59.24%) but presented an increase between 14% and 21% in OA when associated with a data augmentation and spatial features extracted from a CNN.

Made available in DSpace on 2020-12-12T01:54:59Z (GMT). No. of bitstreams: 0 Previous issue date: 2020-04-02 Fundação de Amparo à Pesquisa e Inovação do Estado de Santa Catarina Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq) Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES) The classification of tree species can significantly benefit from high spatial and spectral information acquired by unmanned aerial vehicles (UAVs) associated with advanced classification methods. This study investigated the following topics concerning the classification of 16 tree species in two subtropical forest fragments of Southern Brazil: i) the potential integration of UAV-borne hyperspectral images with 3D information derived from their photogrammetric point cloud (PPC); ii) the performance of two machine learning methods (support vector machine–SVM and random forest–RF) when employing different datasets at a pixel and individual tree crown (ITC) levels; iii) the potential of two methods for dealing with the imbalanced sample set problem: a new weighted SVM (wSVM) approach, which attributes different weights to each sample and class, and a deep learning classifier (convolutional neural network–CNN), associated with a previous step to balance the sample set; and finally, iv) the potential of this last classifier for tree species classification as compared to the above mentioned machine learning methods. Results showed that the inclusion of the PPC features to the hyperspectral data provided a great accuracy increase in tree species classification results when conventional machine learning methods were applied, between 13 and 17% depending on the classifier and the study area characteristics. When using the PPC features and the canopy height model (CHM), associated with the majority vote (MV) rule, the SVM, wSVM and RF classifiers reached accuracies similar to the CNN, which outperformed these classifiers for both areas when considering the pixel-based classifications (overall accuracy of 84.4% in Area 1, and 74.95% in Area 2). The CNN was between 22% and 26% more accurate than the SVM and RF when only the hyperspectral bands were employed. The wSVM provided a slight increase in accuracy not only for some lesser represented classes, but also some major classes in Area 2. While conventional machine learning methods are faster, they demonstrated to be less stable to changes in datasets, depending on prior segmentation and hand-engineered features to reach similar accuracies to those attained by the CNN. To date, CNNs have been barely explored for the classification of tree species, and CNN-based classifications in the literature have not dealt with hyperspectral data specifically focusing on tropical environments. This paper thus presents innovative strategies for classifying tree species in subtropical forest areas at a refined legend level, integrating UAV-borne 2D hyperspectral and 3D photogrammetric data and relying on both deep and conventional machine learning approaches. Division of Remote Sensing National Institute for Space Research (INPE) Department of Forest Engineering Santa Catarina State University (UDESC) Department of Electrical Engineering Pontifical Catholic University of Rio de Janeiro (PUC) Department of Sustainable Agro-Ecosystems and Bioresources Research and Innovation Centre Department of Geography Santa Catarina State University (UDESC) Department of Cartography São Paulo State University (UNESP) Department of Cartography São Paulo State University (UNESP) Fundação de Amparo à Pesquisa e Inovação do Estado de Santa Catarina: 2017TR1762 CNPq: 313887/2018-7 CAPES: 88882.330700/2018-01

This work explores the integration of airborne Light Detection and Ranging (LiDAR) data and WorldView-2 (WV2) images to classify the land cover of a subtropical forest area in Southern Brazil. Diff...

Recently, recurrent neural networks have been proposed for crop mapping from multitemporal remote sensing data. Most of these proposals have been designed and tested in temperate regions, where a single harvest per season is the rule. In tropical regions, the favorable climate and local agricultural practices, such as crop rotation, result in more complex spatio-temporal dynamics, where the single harvest per season assumption does not hold. In this context, a demand arises for methods capable of recognizing agricultural crops at multiple dates along the multitemporal sequence. In the present work, we propose to adapt two recurrent neural networks, originally conceived for single harvest per season, for multidate crop recognition. In addition, we propose a novel multidate approach based on bidirectional fully convolutional recurrent neural networks. These three architectures were evaluated on public Sentinel-1 data sets from two tropical regions in Brazil. In our experiments, all methods achieved state-of-the-art accuracies with a clear superiority of the proposed architecture. It outperformed its counterparts in up to 3.8% and 7.4%, in terms of per-month overall accuracy, and it was the best performing method in terms of F1-score for most crops and dates on both regions.

This work follows a line of research dedicated to the parallelization of image segmentation algorithms on distributed computing environments, which is motivated by the increasing resolutions and availability of Remote Sensing (RS) images. Here we focus on region-growing segmentation, which is regarded as a time consuming and demanding approach in terms of computational resources. Its parallelization is a complex problem since it usually affects the final outcome in comparison to what would be delivered by a sequential solution. This is due to the fact that subdividing an image to perform segmentation of its tiles concurrently usually introduces undesirable artifacts near to the borders of the image tiles. Additional processing steps are then required to properly stitch together the segments alongside tiles borders in order to eliminate such artifacts. In this work we evaluated alternative implementations of a previously proposed region-growing distributed segmentation approach, which was originally built on top of the Hadoop distributed computing framework. We developed a new implementation of the approach, which was built with the Spark framework, and compared its performance with that of the original implementation. In this investigation RS images of various sizes were processed using different configurations of a physical computer cluster. We evaluated computational performances and accessed the differences among the segmentation outcomes generated by the alternative implementations. We also assessed the stability of the implementations by comparing the segmentations produced with different cluster configurations. Although the approach is, in principle, suitable to any region growing algorithm, the experiments were performed with a particular segmentation method, and the results showed that the Spark implementation consistently outperformed the Hadoop counterpart, bringing in most cases a significant improvement in terms of processing time. The experiment results also attested the stability of the distributed segmentation approach, as very similar results were produced with the alternative implementations, running on different cluster configurations.

This work aims at investigating unsupervised and semi-supervised representation learning methods based on generative adversarial networks for remote sensing scene classification. The work introduces a novel approach, which consists in a semi-supervised extension of a prior unsupervised method, known as MARTA-GAN. The proposed approach was compared experimentally with two baselines upon two public datasets, UC-MERCED and NWPU-RESISC45. The experiments assessed the performance of each approach under different amounts of labeled data. The impact of fine-tuning was also investigated. The proposed method delivered in our analysis the best overall accuracy under scarce labeled samples, both in terms of absolute value and in terms of variability across multiple runs.

Remote sensing imagery from different sensors enables accurate crop monitoring and mapping, supporting efficient and sustainable agricultural practices. This paper proposes a flexible multi-modal Encoder-Decoder architecture to implement and investigate different fusion strategies for crop segmentation using synthetic-aperture radar (SAR) and optical image sequences. The trained model handles each modal individually or both, allowing us to investigate scenarios where one modal is not available. Our architecture consists of two modality-specific encoders, a shared decoder, a fusion module, and three classifiers, one for each modal and one for the fusion output. Also, we propose using a partial loss function that allows training the network with scarce ground truths. The proposed approach was evaluated in a public dataset comprising seven multitemporal SAR images and four multitemporal optical images from a tropical agricultural region in Brazil. We report results for feature and decision fusion strategies and discuss the benefits of using each of them for multi-modal crop segmentation.

Reliable crop mapping is an essential tool for agricultural monitoring and food security. In the tropics, where cloud cover massively affects optical imagery, synthetic-aperture radar (SAR) imagery emerged as a cost-effective alternative for discriminating crops in large scale agricultural regions. Recently, unsupervised deep clustering approaches have emerged as a competitive alternative in several different applications with labeling restrictions. This paper explores this literature and evaluates the feasibility of such methods applied to crop classification in a tropical region from multi-temporal SAR image sequences. We focus on the k-Means-related deep clustering methods, specifically, on Deep Embedding Clustering and Deep K-Means. We report experiments conducted on a public dataset from a tropical region with highly complex crop dynamics. In our experiments the tested unsupervised approaches managed to deliver nearly 78% of supervised counterparts for this task11The source codes are available at https:/github.com/DLoboT/Project_DL_2020.

Early detection of deforestation processes is vital to maintain and regulate tropical rainforests, such as in the Amazon region. Most of them rely on optical imagery. Approaches based on Synthetic Aperture Radar (SAR) data are comparatively unexplored, in particular for deforestation detection in tropical rainforests. This work addresses this gap and evaluates Fully Convolutional Networks based on the U-Net, Res-Unet and Siamese Network, for deforestation detection using images from three different sensors, Landsat-8, Sentinel-2, and Sentinel-1. Experiments conducted on a dataset of the Amazon rainforest indicated that Fully Convolutional Networks working on Sentinel-1 data can achieve sufficient accuracy for detecting deforestation in tropical rainforests when clouds prevent the use of optical data11The source code is available in https://github.zcom/MabelOrtega/Comparison-of-Optical-and-SAR-data-for-deforestation-mapping-in-the-Amazon-Forest-with-FCN.

This paper introduces a novel hybrid N-to-N bidirectional ConvLSTM for multi-temporal crop recognition in areas characterized by highly complex crop dynamics as in tropical regions. The proposed method seeks to explore both spatial and temporal context of the data by adding a bidirectional ConvLSTM to every skip connection in the U-Net and applying the fully connected conditional random fields (CRF) methodology as a post-processing of the predictions. We evaluated our method on a publicly available tropical region dataset, from Sentinel-1 data, obtaining an average improvement of almost 4% in the Average F1 Score and 1.7% in Overall Accuracy, when comparing with a state-of-the art method.

Recent works have studied crop recognition in regions with highly complex spatio-temporal dynamics typical of a tropical climate. However, most proposals have only been evaluated in a single agricultural year, and their capabilities to generalize to dates outside the temporal sequence have not been properly addressed thus far. This work assesses the generalization capabilities of a recent convolutional recurrent architecture, testing it in a temporal sequence two years ahead of the sequence with which it was trained. Furthermore, a N-to-1 variant of such network is proposed, which is able to produce classification outcomes for every month in the agricultural year, and it is compared with two baselines designed in a more traditional approach, in which a separate specific network is trained for each month of the year. The approaches are evaluated on two public datasets from a tropical region. The first dataset comprehends the period from June 2017 to May 2018, while the second goes from October 2019 to September 2020. Results show a decrease of up to 24.6% in per-date average F1 score when training the network with data of an agricultural year different from the one it is tested on, which indicates a domain shift that demands further research. Additionally, the proposed approach presented only a slight decrease in performance compared to its baseline when trained on the same dataset, with a 2.7% drop in average F1 score. This performance drop is a small cost in exchange for its operational advantages, such as reduced training time and a more straightforward pipeline.

Although very efficient in a number of application fields, deep learning based models are known to demand large amounts of labeled data for training. Particularly for remote sensing applications, responding to that demand is generally expensive and time consuming. Moreover, supervised training methods tend to perform poorly when they are tested with a set of samples that does not match the general characteristics of the training set. Domain adaptation methods can be used to mitigate those problems, especially in applications where labeled data is only available for a particular region or epoch, i.e., for a source domain, but not for a target domain on which the model should be tested. In this work we introduce a domain adaptation approach based on representation matching for the deforestation detection task. The approach follows the Adversarial Discriminative Domain Adaptation (ADDA) framework, and we introduce a margin-based regularization constraint in the learning process that promotes a better convergence of the model parameters during training. The approach is evaluated using three different domains, which represent sites in different forest biomes. The experimental results show that the approach is successful in the adaptation of most of the domain combination scenarios, usually with considerable gains in relation to the baselines.

This work proposes a multi-task fully convolutional architecture for tree species mapping in dense forests from sparse and scarce polygon-level annotations using hyperspectral UAV-borne data. Our model implements a partial loss function that enables dense tree semantic labeling outcomes from non-dense training samples, and a distance regression complementary task that enforces tree crown boundary constraints and substantially improves the model performance. Our multi-task architecture uses a shared backbone network that learns common representations for both tasks and two task-specific decoders, one for the semantic segmentation output and one for the distance map regression. We report that introducing the complementary task boosts the semantic segmentation performance compared to the single-task counterpart in up to 11% reaching an average user's accuracy of 88.63% and an average producer's accuracy of 88.59%, achieving state-of-art performance for tree species classification in tropical forests., Full version of preprint accepted at ISPRS Journal of Photogrammetry and Remote Sensing

Visual object tracking aims to deliver accurate estimates about the state of the target in a sequence of images or video frames. Nevertheless, tracking algorithms are sensitive to different kinds of image perturbations that frequently cause tracking failures. Indeed, tracking failures result from the insertion of imprecise target-related data into the trackers’ appearance models, which leads the trackers to lose the target or drift away from it. Here, we propose a tracking fusion approach, which incorporates feedback and re-initialization mechanisms to improve overall tracking performance. Our fusion technique, called SymTE-TR, enhances trackers’ overall performance by updating their appearances models with reliable information of the target’s states, while resets the imprecise trackers. We evaluated our approach on a facial video dataset, which characterizes a particular challenging tracking application under different imaging conditions. The experimental results indicate that our approach contributes to enhancing individual tracker performances by providing stable results across the video sequences and, consequently, contributes to stable overall tracking fusion performances.

Knowing the spatial distribution of endangered tree species in a forest ecosystem or forest remnants is a valuable information to support environmental conservation practices. The use of Unmanned Aerial Vehicles (UAVs) offers a suitable alternative for this task, providing very high-resolution images at low costs. In parallel, recent advances in the computer vision field have led to the development of effective deep learning techniques for end-to-end semantic image segmentation. In this scenario, the DeepLabv3+ is well established as the state-of-the-art deep learning method for semantic segmentation tasks. The present paper proposes and assesses the use of DeepLabv3+ for mapping the threatened Dipteryx alata Vogel tree, popularly also known as cumbaru. We also compare two backbone networks for feature extraction in the DeepLabv3+ architecture: the Xception and MobileNetv2. Experiments carried out on a dataset consisting of 225 UAV/RGB images of an urban area in Midwest Brazil demonstrated that DeepLabv3+ was able to achieve in mean overall accuracy and Fl-score above 90%, and IoU above 80%. The experimental analysis also pointed out that the MobileNetv2 backbone overcame its counterpart by a wide margin due to its comparatively simpler architecture in view of the available training data.