Search

Climate change and the subsequent biodiversity loss are two of the greatest threats we face in our time. To tackle these problems, we require timely and localised monitoring of specific metrics such as (1) agricultural expansion, its relation to forest loss and degradation; and (2) the characterisation of species with decreased populations, at the brink of extinction, or extinct; among many others. Earth observation and terrestrial images are crucial for large-scale and fine-grained monitoring of such metrics. In the increased availability of such data, machine learning models enable the understanding of the massive amount of information that is continuously available. Most successful applications rely on a fully labelled and well-balanced training set with a low noise level. Such datasets in practice are not always possible to obtain, as several factors might arise. In this thesis, we have proposed methods to deal with scenarios commonly found in practice and to efficiently leverage the available resources. We focused specifically on two important types of datasets: First, with Earth observation datasets, we used medium resolution satellite images to perform tree density estimation, this allows us to make large-scale estimates of certain tree crops despite not having universal access to high-resolution images. Furthermore, we proposed an active learning approach that is well suited for country-scale applications and saves resources in terms of computation, time, and labelling effort. With this method, we obtained country-scale density maps of oil palms and coconuts in Southeast Asia. Furthermore, after a Typhoon passed over large plantations of coconuts in the Philippines, we relied on our method and a robustness analysis to understand the changes observed before and after the Typhoon in a relatively short time window. Second, using terrestrial images from animal species, we explored how to use side-information to deal with unbalanced and biased datasets. We explored two types of side-information: (1) Keypoint annotations of training samples to guide the model towards discriminative aspects of the image for fine-grained classification. This proved to be helpful in cases of scarce samples, long-tailed class distributions or biased backgrounds. And (2) side-information at a class level in the form of (bird) field-guides. We used them in a Zero-shot learning setting to classify samples from unseen classes by leveraging the information from the field-guides and their similarities to the seen ones. Our work shows that it is possible to carry out efficient labelling campaigns with large-scale areas of interest and to deal with biased datasets and unbalanced class distributions. Future lines of research include sensor fusion, meta-learning and the role of datasets overall to better assess generalisation beyond a single dataset. This work shows some of the potential that efficient machine learning models have for ecosystem monitoring under common and realistic scenarios., ISBN:978-3-03837-014-7

Mapping vegetation properties globally is crucial to understand the role of terrestrial ecosystems in the global carbon cycle. Spatially explicit, high-resolution data are needed to manage terrestrial ecosystems so that climate change can be mitigated and biodiversity loss prevented. Since no current single data source can provide such data with global coverage and high spatial resolution, new solutions must be found. This thesis aims to develop novel data-driven tools based on state-of-the-art deep learning to advance the mapping of vegetation properties, in particular canopy height, at global scale. Two ongoing space missions, namely the Copernicus Sentinel-2 mission and NASA’s GEDI LIDAR mission, deliver publicly available data that form the basis of the methods presented in this thesis. While GEDI is a key climate mission that provides sparse vegetation structure measurements at global scale (between 51.6° N & S), Sentinel-2 delivers dense optical images with global coverage, but cannot directly measure vertical vegetation structure. The presented work is a holistic approach based on gradually extended methods towards the large-scale fusion of Sentinel-2 and GEDI for the global mapping of canopy top height with high spatial resolution. Furthermore, since transparency of the modelling limitations is critical to build trust and to inform downstream applications about the reliability of the estimates, probabilistic deep learning techniques are integrated to quantify the predictive uncertainty. In a first step, a novel approach based on deep convolutional neural networks (CNNs) was developed to estimate dense canopy height maps from Sentinel-2 optical images by training with local dense reference data from airborne measurement campaigns (LIDAR and photogrammetry) in Gabon and Switzerland. By exploiting textural image features, the model achieved low error, even for canopies up to 50 m height. However, its applicability is limited to regions represented by the available training data. The launch of the spaceborne GEDI full waveform LIDAR in December 2018 promised to provide sparse reference data of vegetation structure measurements at global scale. Since interpreting on-orbit GEDI LIDAR waveforms proved to be a difficult task due to unknown noise in the data, a novel probabilistic deep learning approach was developed to retrieve canopy top height globally and quantify the predictive uncertainty from GEDI. Given these footprint-level estimates, the Sentinel-2 based canopy height mapping approach could be extended to be trained with sparse supervision. After demonstrating that this approach allows to estimate canopy top height suitable to map indicative high carbon stocks in tropical Southeast Asia, a global probabilistic model was developed to retrieve canopy top height anywhere on Earth. Ultimately, the first global, wall-to-wall canopy top height map at 10 m ground sampling distance was computed for the year 2020., ISBN:978-3-03837-013-0

Dissertation, RWTH Aachen University, 2021; Aachen : RWTH Aachen University 1 Online-Ressource : Illustrationen (2022). = Dissertation, RWTH Aachen University, 2021, Computer vision has made tremendous leaps during the past decade. One of the key factors behind this growth is the vast amount of data that we can generate today: millions of pictures are shared online daily and new specialized sensors allow to easily capture 3D data. Along with the recent advances in deep learning and increased availability of computational power, it is now possible to take advantage of these large amounts of high-quality data. As a result, computer vision achieved impressive performance gains across numerous fields and applications. However, the increased amount of available data also introduces new challenges. To exploit the large body of available data, we either need efficient unsupervised algorithms to learn patterns from unlabeled data, or we require efficient labeling tools to allow the creation of large-scale labeled datasets. These are essential for the success of most deep learning models. In this thesis, we deal with issues arising from these different aspects of computer vision: unsupervised algorithms for landmark recognition, fully-supervised methods for semantic segmentation on large-scale 3D point clouds and interactive object segmentation for out-of-domain dataset labeling. More specifically, the main contributions of this thesis are organized into three parts, each one covering an individual computer vision topic: In the first part, we address the problem of object discovery in time - varying, large - scale image collections. We propose a novel tree structure that closely approximates the Minimum Spanning Tree and present an efficient construction approach to incrementally update the tree structure when new data is added to the image database. This happens either in online-streaming or batch form. Our proposed tree structure is created in a local neighborhood of the matching graph during image retrieval and can be efficiently updated whenever the image database is extended. We show how our tree structure can be incorporated in existing clustering approaches such as Single-Link and Iconoid Shift for efficient large-scale object discovery in image collections. In the second part of the thesis, we focus on defining novel 3D convolutional and recurrent operators over unstructured 3D point clouds. The goal is to learn point representations for the task of 3D semantic segmentation. The recurrent consolidation unit layer operates on multi-scale and grid neighborhoods along and allows our model to learn long-range dependencies. Additionally, we introduce two types of local neighborhoods for each 3D point that encode local geometry to facilitate the definition and use of convolutions on 3D point clouds. Finally, in the third part, we address the task interactive object segmentation. Aided by an algorithm, a user segments an object mask in a given image by clicking inside or outside the object. We present a method that significantly reduces the number of required user clicks compared to previous work. In particular, we look at out-of-domain settings where the test datasets are significantly different from the datasets used to train our deep model. We propose to treat user corrections as sparse supervision to adapt our model parameters on-the-fly. Our adaptive method can significantly reduce the number of required clicks to segment an object and handle distribution shifts from small to large, specialize to a new class of objects introduced during test time, and can even handle large domain changes from commercial images to medical and aerial data., Published by RWTH Aachen University, Aachen

High-resolution 3D velocimetry estimation of fluids is a key problem in experimental fluid mechanics. It offers important applications for aero- and hydrodynamic measurements in academia and industry and facilitates fundamental research in turbulent flows. By injecting tracer particles into a fluid and observing their displacements over time from multiple view-points, a dense velocity field can be obtained. However, with increased particle seeding densities, ambiguities in the 3D particle reconstruction arise, which, in turn, affect the reconstruction accuracy of the underlying flow field. Existing approaches are limited to low seeding densities, which limit the spatial resolution of the flow field, or require long time sequences to heuristically resolve ambiguities of large, self-occluding particle sets. This thesis focuses on novel, physically-motivated approaches for high accuracy 3D flow estimation from few time steps. Multiple contributions are presented to tackle high seeding densities and, thus, facilitate high-resolution velocity field estimation. First, a variational formulation of the 3D flow estimation problem is introduced. The coarse- to-fine optimization scheme allows incorporation of physical priors, such as the incompressible stationary Stokes equations. To counteract the high memory requirement of voxel-based representations, a sparse particle reconstruction approach is subsequently proposed, combined with a sparse descriptor for 3D correspondence matching. Finally, a joint energy formulation is presented that optimizes both the sparse 3D particle locations and the dense motion field. Taking into account all available input views and both time steps jointly results in a better disambiguation of particle reconstructions and a more detailed flow field estimation. This favorable formulation is further extended to multiple time steps, allowing to further improve the particle reconstruction. Hence, even higher particle seeding densities can be supported. The proposed approaches were quantitatively validated on synthetic fluid simulations and delivered compelling results on experiments in water and air., ISBN:978-3-03837-010-9

We introduce a novel generative autoencoder network model that learns to encode and reconstruct images with high quality and resolution, and supports smooth random sampling from the latent space of the encoder. Generative adversarial networks (GANs) are known for their ability to simulate random high-quality images, but they cannot reconstruct existing images. Previous works have attempted to extend GANs to support such inference but, so far, have not delivered satisfactory high-quality results. Instead, we propose the Progressively Growing Generative Autoencoder (Pioneer) network which achieves high-quality reconstruction with images without requiring a GAN discriminator. We merge recent techniques for progressively building up the parts of the network with the recently introduced adversarial encoder–generator network. The ability to reconstruct input images is crucial in many real-world applications, and allows for precise intelligent manipulation of existing images. We show promising results in image synthesis and inference, with state-of-the-art results in CelebA inference tasks.

IGP Mitteilungen, 119, ISBN:978-3-03837-007-9

Sampling-based algorithms in the mould of RANSAC have emerged as one of the most successful methods for the fully automated registration of point clouds acquired by terrestrial laser scanning (TLS). Sampling methods in conjunction with 3D keypoint extraction, have shown promising results, e.g. the recent K-4PCS (Theiler et al., 2013). However, they still exhibit certain improbable failures, and are computationally expensive and slow if the overlap between scans is low. Here, we examine several variations of the basic K-4PCS framework that have the potential to improve its runtime and robustness. Since the method is inherently parallelizable, straight-forward multi-threading already brings down runtimes to a practically acceptable level (seconds to minutes). At a conceptual level, replacing the RANSAC error function with the more principled MSAC function (Torr and Zisserman, 2000) and introducing a minimum-distance prior to counter the near-field bias reduce failure rates by a factor of up to 4. On the other hand, replacing the repeated evaluation of the RANSAC error function with a voting scheme over the transformation parameters proved not to be generally applicable for the scan registration problem. All these possible extensions are tested experimentally on multiple challenging outdoor and indoor scenarios., ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, II-3, ISSN:2194-9042, ISSN:2194-9050

Traffic monitoring plays an important role in transportation management. In addition, airborne acquisition enables a flexible and realtime mapping for special traffic situations e.g. mass events and disasters. Also the automatic extraction of vehicles from aerial imagery is a common application. However, many approaches focus on the target object only. As an extension to previously developed car detection techniques, a validation scheme is presented. The focus is on exploiting the background of the vehicle candidates as well as their color properties in the HSV color space. Therefore, texture of the vehicle background is described by color co-occurrence histograms. From all resulting histograms a likelihood function is calculated giving a quantity value to indicate whether the vehicle candidate is correctly classified. Only a few robust parameters have to be determined. Finally, the strategy is tested with a dataset of dense urban areas from the inner city of Munich, Germany. First results show that certain regions which are often responsible for false positive detections, such as vegetation or road markings, can be excluded successfully.

This paper investigates the performance of SIFT-based image matching regarding large differences in image scaling and rotation, as this is usually the case when trying to match images captured from UAVs and airplanes. This task represents an essential step for image registration and 3d-reconstruction applications. Various real world examples presented in this paper show that SIFT, as well as A-SIFT perform poorly or even fail in this matching scenario. Even if the scale difference in the images is known and eliminated beforehand, the matching performance suffers from too few feature point detections, ambiguous feature point orientations and rejection of many correct matches when applying the ratio-test afterwards. Therefore, a new feature matching method is provided that overcomes these problems and offers thousands of matches by a novel feature point detection strategy, applying a one-to-many matching scheme and substitute the ratio-test by adding geometric constraints to achieve geometric correct matches at repetitive image regions. This method is designed for matching almost nadir-directed images with low scene depth, as this is typical in UAV and aerial image matching scenarios. We tested the proposed method on different real world image pairs. While standard SIFT failed for most of the datasets, plenty of geometrical correct matches could be found using our approach. Comparing the estimated fundamental matrices and homographies with ground-truth solutions, mean errors of few pixels can be achieved.