Search

Novel view synthesis (NVS) of urban scenes enables the exploration of cities virtually and interactively, which can further be used for urban planning, navigation, digital tourism, etc. However, many current NVS methods require a large amount of images from known views as input and are sensitive to intrinsic and extrinsic camera parameters. In this paper, we propose a new unified framework for NVS of urban scenes with fewer required views via the integration of scene priors and the joint optimization of camera parameters under an geometric constraint along with NeRF weights. The integration of scene priors makes full use of the priors from the neighbor reference views to reduce the number of required known views. The joint optimization can correct the errors in camera parameters, which are usually derived from algorithms like Structure-from-Motion (SfM), and then further improves the quality of the generated novel views. Experiments show that our method achieves about 25. 375 dB and 25. 512 dB in average in terms of peak signal-to-noise (PSNR) on synthetic and real data, respectively. It outperforms popular state-of-the-art methods (i.e., BungeeNeRF and MegaNeRF) by about 2– 4 dB in PSNR. Notably, our method achieves better or competitive results than the baseline method with only one third of the known view images required for the baseline. The code and dataset are available at https://github.com/Dongber/PriNeRF. [ABSTRACT FROM AUTHOR]

Multimodal image matching is a core basis for information fusion, change detection, and image-based navigation. However, multimodal images may simultaneously suffer from severe nonlinear radiation distortion (NRD) and complex geometric differences, which pose great challenges to existing methods. Although deep learning-based methods had shown potential in image matching, they mainly focus on same-source images or single types of multimodal images such as optical-synthetic aperture radar (SAR). One of the main obstacles is the lack of public data for different types of multimodal images. In this paper, we make two major contributions to the community of multimodal image matching: First, we collect six typical types of images, including optical-optical, optical-infrared, optical-SAR, optical-depth, optical-map, and nighttime, to construct a multimodal image dataset with a total of 1200 pairs. This dataset has good diversity in image categories, feature classes, resolutions, geometric variations, etc. Second, we propose a scale and rotation invariant feature transform (SRIF) method, which achieves good matching performance without relying on data characteristics. This is one of the advantages of our SRIF over deep learning methods. SRIF obtains the scales of FAST keypoints by projecting them into a simple pyramid scale space, which is based on the study that methods with/without scale space have similar performance under small scale change factors. This strategy largely reduces the complexity compared to traditional Gaussian scale space. SRIF also proposes a local intensity binary transform (LIBT) for SIFT-like feature description, which can largely enhance the structure information inside multimodal images. Extensive experiments on these 1200 image pairs show that our SRIF outperforms current state-of-the-arts by a large margin, including RIFT, CoFSM, LNIFT, and MS-HLMO. Both the created dataset and the code of SRIF will be publicly available in https://github.com/LJY-RS/SRIF. [ABSTRACT FROM AUTHOR]

[Display omitted] • A feature extraction module is proposed to extract the geometric and semantic features of each point efficiently. Then, the attention mechanism is deployed to aggregate the learned features. • The proposed ResDLPS-Net is optimized by joint training of residual connections and dense convolutional connections. • Experiments demonstrate that ResDLPS-Net outperforms the state-of-the-art deep learning networks on the indoor dataset S3DIS and the outdoor large-scale dataset Toronto-3D. Significantly, the Mean Intersection over Union (mIoU) of ResDLPS-Net on the Toronto-3D dataset is 80.27%. Semantic segmentation methods based on three-dimensional (3D) point clouds are mostly limited to input point clouds that have been divided into blocks for training. This is mainly attributed to the requirement of constant trade-offs between computational resources and accuracy for directly processing large-scale point clouds. Specifically, the block dividing strategy will add the data preprocessing time to some extent and may disturb the complete geometry of the object. Therefore, this paper proposes a large-scale point cloud semantic segmentation network without block dividing operation, referred to as ResDLPS-Net. This network can take the complete point cloud of the whole large scene as input and process up to nearly a million points on one single GPU. In particular, a novel feature extraction module is designed to efficiently extract neighbor, geometric, and semantic features. The learned features are then aggregated through the attention mechanism to form local feature descriptors. In addition, the proposed ResDLPS-Net is jointly trained by residual connections and dense convolutional connections to optimize the feature aggregation operation. As a result, the ResDLPS-Net performs brilliantly on multiple objects, such as windows, road markings, fences, etc. For example, the Mean Intersection over Union (mIoU) of road markings on the Toronto-3D dataset is 37.76% higher than the state-of-the-art algorithm. Moreover, this paper outperforms most deep learning methods on three well-known benchmark datasets, including the indoor dataset S3DIS and the outdoor large-scale scene datasets Semantic3D and Toronto-3D. The proposed ResDLPS-Net achieves the best performance on the S3DIS dataset. The average accuracy (mA) and overall accuracy (OA) are 82.3% and 88.1%, respectively. Notably, the proposed ResDLPS-Net attains a mIoU of 80.27% on the Toronto-3D dataset, which is 6.00% higher than the best results published currently. [ABSTRACT FROM AUTHOR]

Point cloud registration is a fundamental problem in 3D computer vision. This paper addresses scan matching for reliable Mobile LiDAR Systems (MLSs) that can traverse different environments and be robust against outliers and noise. High reliability in multiple scenarios is vital to many applications, such as autonomous driving, but poor feature representations often compromise it. This paper introduces an expressive feature called the rectangle-flattening representation to enhance reliability. First, we propose a clustering method based on density, direction and flattening that allows regions to grow in a "planes first, lines second, less flattened structures last" manner. This method can extract rectangles from environments where planes are scarce. Second, we develop a squared point-to-rectangle distance function that is piecewise yet continuously differentiable to leverage the rectangle-flattening representation for scan matching. Unlike the traditional point-to-plane or plane-to-plane residual functions that rely on planar surfaces in other directions to provide translational information, our point-to-rectangle distance function is intrinsically translation-aware. Extensive experiments are conducted on three aspects: scan matching accuracy, robustness, and odometry and mapping on MLSs. We compare our algorithm to several state-of-the-art methods using KITTI and Ford datasets in scan matching accuracy test with environments covering residential areas, highways, rural areas, downtown areas and campuses. Rigorous experiments show that among all of the methods compared, only RectMatch has an overall scan matching success rate surpassing 90% and even 95% across the two datasets. The robustness tests demonstrate that RectMatch can better deal with random outliers and Gaussian noise. For a comprehensive evaluation of RectMatch for MLSs, the third test incorporates five publicly available datasets using different laser scanners on multiple platforms traversing different environments. The results show high algorithm reliability and accuracy. [ABSTRACT FROM AUTHOR]

A depth map fusion algorithm fuses depth maps from different perspectives into a unified coordinate framework and performs surface calculations to generate dense point clouds of the entire scene. The existing algorithms ensure the quality of these dense point clouds by eliminating inconsistencies between depth maps, but the problem of many redundant calculations often arises. In this paper, a depth map fusion algorithm based on pixel region prediction is proposed. First, the image combination is calculated to select a set of candidate neighbor images for each reference image in the scene. Second, voxels and measure estimates are constructed on a coarse scale, and an inference strategy and a corrector are proposed to merge pixel regions at a fine scale guided by the coarse scale. Finally, the deduced pixel regions at the fine scale are used as the image-space constraints for depth fusion. Public and actual oblique images datasets are used for experimental verification. Compared with the famous COLMAP, OPENMVS, Gipuma and ACMP methods, the number of redundant calculations is significantly reduced; according to Data1 ∼ Data9 in the experiment, as the number of images increases, the fusion efficiency is increased by 47.5% to 156.6%; at the same time, the point cloud accuracy is comparable to other methods. [ABSTRACT FROM AUTHOR]

How can we perform an efficient 3D reconstruction with high accuracy and completeness, in the presence of non-Lambertian surface and low textured regions? This paper aims at fast quality 3D reconstruction, best near real time. While deep learning approaches perform very well in multi-view stereo (MVS), the high complexity of models makes them inapplicable in practical applications. Few works were explored to accelerate deep learning-based 3D reconstruction approaches. In this paper, we take an unprecedented attempt to compress and accelerate these models via pruning their redundant parameters. We introduce an efficient channel pruning method for 2D convolutional neural networks (CNNs) based on a mixed back propagation process , where a soft mask is learned to prune the channels using a fast iterative shrinkage-thresholding algorithm. While in 3D CNNs, we train a large multi-scale CNNs architecture and observe that only utilizing one module enough for the 3D reconstruction, which can still maintain the performance of the full-precision model. We achieve an efficient MVS reconstruction system up to 2 times faster , in contrast to the state-of-the-arts, while maintaining comparable model accuracy and even better completeness. [ABSTRACT FROM AUTHOR]

This paper presents a novel hierarchical approach to solving structure-from-motion (SFM) problems. The algorithm begins with small local reconstructions based on nonlinear bundle adjustment (BA). These are then joined in a hierarchical manner using a strategy that requires solving a linear least squares optimization problem followed by a nonlinear transform. The algorithm can handle ordered monocular and stereo image sequences. Two stereo images or three monocular images are adequate for building each initial reconstruction. The bulk of the computation involves solving a linear least squares problem and, therefore, the proposed algorithm avoids three major issues associated with most of the nonlinear optimization algorithms currently used for SFM: the need for a reasonably accurate initial estimate, the need for iterations, and the possibility of being trapped in a local minimum. Also, by summarizing all the original observations into the small local reconstructions with associated information matrices, the proposed Linear SFM manages to preserve all the information contained in the observations. The paper also demonstrates that the proposed problem formulation results in a sparse structure that leads to an efficient numerical implementation. The experimental results using publicly available datasets show that the proposed algorithm yields solutions that are very close to those obtained using a global BA starting with an accurate initial estimate. The C/C++ source code of the proposed algorithm is publicly available at https://github.com/LiangZhaoPKUImperial/LinearSFM . [ABSTRACT FROM AUTHOR]

Multi-view depth map reconstruction is a popular approach to generate 3D information with good flexibility and scalability. However, texture-weak regions and repeated textures challenge these methods in urban scenes. To address this need, this paper proposes an image-triangle based multi-view 3D reconstruction (IMGTR) method. Starting from constructing a density-adaptive image triangulation for each image, the main procedure is to determine the corresponding object plane for each image triangle under an objective function consisting of the image similarity measure, the smoothness constraint and the continuity constraint on the edges between adjacent triangles. Qualitative and quantitative experiments show that the proposed method can reconstruct complex urban structures more accurate and achieve higher fidelity in planar urban structures than some recently released multi-view stereo algorithms. Specifically, for the Vaihingen dataset IMGTR achieves an average improvement of 4.27% compared to PMVD, 50.94% to SURE and 54.76% to COLMAP in position accuracy. [ABSTRACT FROM AUTHOR]

Infrared dim target detection under complex ocean environment plays a key role in military and civilian fields. Many state-of-the-art methods have disadvantages such as low generalization ability, poor robustness to noise and stubborn edges, high time complexity, and the existence of background residuals or target defects in detection results. To further solve these shortcomings, based on the infrared patch tensor (IPT) model, a robust infrared dim target detection algorithm is proposed in this paper, which converts the target detection task into a convex optimization problem. Aiming at the current situation that the tensor rank approximation in the IPT model has not been well resolved, a new vector form of tensor rank named mode- k 1 k 2 extension tensor tubal rank (METTR) is defined, the elements of which include the tubal ranks of all tensors expanded via mode- k 1 k 2 extension. Through the mode- k 1 k 2 extension of the tensor, the hidden information among the different modes of the tensor is better mined. To minimize the METTR efficiently, we propose its convex approximation norm METTR, and establish a tensor robust principal component analysis (TRPCA) model joint l 1 norm. Then we use the alternating direction multiplier method (ADMM) and set the optimal parameters to solve the proposed model. A series of experimental results show that the proposed algorithm outperforms the baselines in terms of background suppression and target detection. [ABSTRACT FROM AUTHOR]

[Display omitted] • We implement more accurate optical flow tracking by the assistance of IMU. We use the rotated tracked points obtained by an IMU to initialize the estimation of the optical flow, and this can improve the reliability of the initial feature points. To obtain accurate feature matching, a stereo baseline constraint and ring matching are used to remove outlier points. The experiments show that the information fused with a stereo camera and an IMU achieves real-time optical flow tracking and more accurate feature matching than the LK optical flow method. Compared with the feature matching method in ORB-SLAM3, the proposed method is more lightweight and faster. • We develop a high-precision SLAM framework that integrates stereo vision and an IMU to tackle the problem of pose inaccuracy that results from the fast movement of a stereo camera and insufficient view overlap between frames. We build a new objective function based on VINS to reduce the complexity of the computation. We adopt a sliding window to ensure the real time performance of the system, and the marginalized information is added as a prior in the objective function. The experimental results demonstrate that the localization accuracy of the proposed SLAM framework is better than that of OKVIS, VINS-Mono and VINS-Fusion. • We propose a fast method for constructing dense point maps to estimate depth values based on stereo vision and an IMU. The depth value computed by SGBM (semi-global block matching) is regarded as the initial value to update the depth of the deep filter, which can improve the convergence rate. We further adopt the TSDF (truncated signed distance function) to fuse the depth images obtained by stereo matching and build a dense map. The experiments show that the proposed stereo dense reconstruction method can obtain a deeper image, less convergence time for the estimated picture and fewer updating frames than the REMODE (probabilistic, monocular dense reconstruction) method. Mobile vehicles require accurate localization and dense mapping for motion planning. In this paper, we propose a dense map construction algorithm based on a light-and-fast stereo visual-inertial navigation system (VINS). A tightly coupled nonlinear optimization method is used to calculate the position of adjacent keyframes. An optical flow tracking method fused with IMU information and ring matching constraints is used to improve the matching accuracy and speed of the feature points. In addition, we obtain the pose and depth values using the semi-global block matching (SGBM) method, which are used as the initial values of the depth filter to update the depth image and improve the convergence speed. Then, we further use the Truncated Signed Distance Function (TSDF) method to construct the dense map. We compare our algorithm with state-of-the-art algorithms on the EuRoc dataset and then compare the estimated depth image using the proposed algorithm and the point cloud construction with the probabilistic monocular dense reconstruction (REMODE). The experiments show that the proposed algorithm can obtain more accurate localization than VINS and OKVIS, as well as a faster tracking speed, a better depth map, a lower convergence time for the estimated image and a lower number of updated frames than REMODE. [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel approach that reconstructs the indoor scene in a structure-aware manner and produces two meshes with different levels of abstraction. To be precise, we start from the raw triangular mesh of indoor scene and decompose it into two parts: structure and non-structure objects. On the one hand, structure objects are defined as significant permanent parts in the indoor environment such as floors, ceilings and walls. In the proposed algorithm, structure objects are abstracted by planar primitives and assembled into a polygonal structure mesh. This step produces a compact structure-aware watertight model that decreases the complexity of original mesh by three orders of magnitude. On the other hand, non-structure objects are movable objects in the indoor environment such as furniture and interior decoration. Meshes of these objects are repaired and simplified according to their relationship with respect to structure primitives. Finally, the union of all the non-structure meshes and structure mesh comprises the scene mesh. Note that structure mesh and scene mesh preserve various levels of abstraction and can be used for different applications according to user preference. Our experiments on both LIDAR and RGBD data scanned from simple to large scale indoor scenes indicate that the proposed framework generates structure-aware results while being robust and scalable. It is also compared qualitatively and quantitatively against popular mesh approximation, floorplan generation and piecewise-planar surface reconstruction methods to demonstrate its performance. [ABSTRACT FROM AUTHOR]

A compensation of atmospheric effects is essential for mm-sensitivity in differential interferometric synthetic aperture radar ( DInSAR ) techniques. Numerical weather predictions are used to compensate these disturbances allowing a reduction in the number of required radar scenes. Practically, predictions are solutions of partial differential equations which never can be precise due to model or initialisation uncertainties. In order to deal with the chaotic nature of the solutions, ensembles of predictions are computed. From a stochastic point of view, the ensemble mean is the expected prediction, if all ensemble members are equally likely. This corresponds to the typical assumption that all ensemble members are physically correct solutions of the set of partial differential equations. DInSAR allows adding to this knowledge. Observations of refractivity can now be utilised to check the likelihood of a solution and to weight the respective ensemble member to estimate a better expected prediction. The objective of the paper is to show the synergy between ensemble weather predictions and differential interferometric atmospheric correction. We demonstrate a new method first to compensate better for the atmospheric effect in DInSAR and second to estimate an improved numerical weather prediction ( NWP ) ensemble mean. Practically, a least squares fit of predicted atmospheric effects with respect to a differential interferogram is computed. The coefficients of this fit are interpreted as likelihoods and used as weights for the weighted ensemble mean. Finally, the derived weighted prediction has minimal expected quadratic errors which is a better solution compared to the straightforward best-fitting ensemble member. Furthermore, we propose an extension of the algorithm which avoids the systematic bias caused by deformations. It makes this technique suitable for time series analysis, e.g. persistent scatterer interferometry ( PSI ). We validate the algorithm using the well known Netherlands-DInSAR test case and first show that the atmospheric compensation improves by nearly 40% compared to the straightforward technique. Second, we compare our results with independent sea level pressure data. In our test case, the mean squared error is reduced by 29% compared to the averaged ensemble members with equal weights. An application demonstration using actual Sentinel-1 data and a typical test site with significant subsidence (Mexico City) completes the paper. [ABSTRACT FROM AUTHOR]

Forthcoming superspectral satellite missions dedicated to land monitoring, as well as planned imaging spectrometers, will unleash an unprecedented data stream. The processing requirements for such large data streams involve processing techniques enabling the spatio-temporally explicit quantification of vegetation properties. Typically retrieval must be accurate, robust and fast. Hence, there is a strict requirement to identify next-generation bio-geophysical variable retrieval algorithms which can be molded into an operational processing chain. This paper offers a review of state-of-the-art retrieval methods for quantitative terrestrial bio-geophysical variable extraction using optical remote sensing imagery. We can categorize these methods into (1) parametric regression , (2) non-parametric regression , (3) physically-based and (4) hybrid methods . Hybrid methods combine generic capabilities of physically-based methods with flexible and computationally efficient methods, typically non-parametric regression methods. A review of the theoretical basis of all these methods is given first and followed by published applications. This paper focusses on: (1) retrievability of bio-geophysical variables, (2) ability to generate multiple outputs, (3) possibilities for model transparency description, (4) mapping speed, and (5) possibilities for uncertainty retrieval. Finally, the prospects of implementing these methods into future processing chains for operational retrieval of vegetation properties are presented and discussed. [ABSTRACT FROM AUTHOR]

Oriented object detection in remote sensing images is an active yet challenging task. State-of-the-art detectors adopt oriented bounding box as a basic representation of an object. The design of them always multiplies the number of anchor boxes with arbitrary angles. Coupled with the rotated Non-Maximum-Suppression (NMS) algorithm, their computational complexities and design difficulties are dramatically increased. In this paper, we propose a novel representation form of oriented objects as pairs of middle lines. Along with the new representation, Oriented Objects Detection Network (O 2 -DNet) is proposed to encode and detect these paired middle lines. By eliminating the pre-designed set of anchor boxes and complicated rotated-NMS post-processing, our model runs in an anchor-free and NMS-free manner. Thereby, compared with state-of-the-art bounding-box and anchor-NMS based detectors, O 2 -DNet is simpler and more efficient in the oriented object detection task. Experiments show that O 2 -DNet achieves excellent performances on the DOTA dataset. In fact, the horizontal objects can also be regarded as a unique form of oriented ones with an angle of 90 degrees. O 2 -DNet can also achieve competitive results on the horizontal box dataset, such as DIOR. [ABSTRACT FROM AUTHOR]

Hyperspectral anomaly detection methods based on representation model have attracted much attention in recent years. In the method, a background dictionary is used to represent each pixel linearly, and the residual is taken as the abnormal level of the pixel. Therefore, building a functional model with strong representation ability and a matching background dictionary is an important factor for success of the method. In the existing methods, the lack of feature utilization, the contamination of background dictionary and the dependence on prior knowledge lead to the instability of anomaly detection results, which are difficult to be applied in practice. To address the issues, this paper proposed a novel hyperspectral anomaly detection method which consists of two interconnected components: a new anomaly detection function model through the combination of LRR and CR, and the supporting background dictionary which does not need prior knowledge. The new anomaly detection model decomposed a two-dimensional normalized hyperspectral image into background and anomaly components, and reconstructed the background through a background dictionary and the corresponding coefficient matrix. The representation coefficient matrix was constrained by global low-rank and local collaborative attributes which helps background modeling. A distance weight matrix was also included to enhance the dense representation of background dictionary atoms which are similar to the testing pixel. The anomaly part was constrained by column sparsity simultaneously due to the global sparsity of anomalous targets in hyperspectral images. To further clarify the physical meaning of the model, another version that imposes non-negative and sum-to-one constraints on coefficient matrix was also proposed. The supporting background dictionary was designed for practical and reliable purposes and was implemented by dual mean shift clustering which automatically estimates parameter without prior-knowledge. The experimental results show that the proposed method can effectively improve the result of anomaly detection, and has the best ROC, AUC, and separation degree between background and anomalies for four real datasets. The average AUC of four datasets of the proposed algorithm is 2.6% higher than the method based on CR and 3.2% higher than the method based on LRR. Moreover, the proposed algorithm has stability and reliability under complex background, as well as the application on a larger hyperspectral scene shows that it offers great potential in practical use. [ABSTRACT FROM AUTHOR]

This paper proposes a novel domain adaptation algorithm to handle the challenges posed by the satellite and aerial imagery, and demonstrates its effectiveness on the built-up region segmentation problem. Built-up area estimation is an important component in understanding the human impact on the environment, effect of public policy and in general urban population analysis. The diverse nature of aerial and satellite imagery (capturing different geographical locations, terrains and weather conditions) and lack of labeled data covering this diversity makes machine learning algorithms difficult to generalize for such tasks, especially across multiple domains. Re-training for new domain is both computationally and labor expansive mainly due to the cost of collecting pixel level labels required for the segmentation task. Domain adaptation algorithms have been proposed to enable algorithms trained on images of one domain (source) to work on images from other dataset (target). Unsupervised domain adaptation is a popular choice since it allows the trained model to adapt without requiring any ground-truth information of the target domain. On the other hand, due to the lack of strong spatial context and structure, in comparison to the ground imagery, application of existing unsupervised domain adaptation methods results in the sub-optimal adaptation. We thoroughly study limitations of existing domain adaptation methods and propose a weakly-supervised adaptation strategy where we assume image level labels are available for the target domain. More specifically, we design a built-up area segmentation network (as encoder-decoder), with image classification head added to guide the adaptation. The devised system is able to address the problem of visual differences in multiple satellite and aerial imagery datasets, ranging from high resolution (HR) to very high resolution (VHR), by investigating the latent space as well as the structured output space. A realistic and challenging HR dataset is created by hand-tagging the 73.4 sq-km of Rwanda, capturing a variety of build-up structures over different terrain. The developed dataset is spatially rich compared to existing datasets and covers diverse built-up scenarios including built-up areas in forests and deserts, mud houses, tin and colored rooftops. Extensive experiments are performed by adapting from the single-source domain datasets, such as Massachusetts Buildings Dataset, to segment out the target domain. We achieve high gains ranging 11.6–52% in IoU over the existing state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Accurate and detailed geographical information digitizing human activity patterns plays an essential role in response to natural disasters. Volunteered geographical information, in particular OpenStreetMap (OSM), shows great potential in providing the knowledge of human settlements to support humanitarian aid, while the availability and quality of OSM remains a major concern. The majority of existing works in assessing OSM data quality focus on either extrinsic or intrinsic analysis, which is insufficient to fulfill the humanitarian mapping scenario to a certain degree. This paper aims to explore OSM missing built-up areas from an integrative perspective of social sensing and remote sensing. First, applying hierarchical DBSCAN clustering algorithm, the clusters of geo-tagged tweets are generated as proxies of human active regions. Then a deep learning based model fine-tuned on existing OSM data is proposed to further map the missing built-up areas. Hit by Cyclone Idai and Kenneth in 2019, the Republic of Mozambique is selected as the study area to evaluate the proposed method at a national scale. As a result, 13 OSM missing built-up areas are identified and mapped with an over 90% overall accuracy, being competitive compared to state-of-the-art products, which confirms the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Abstract: The primary contribution of this paper is an efficient feature extraction and matching implementation for large images in large-scale aerial photogrammetry experiments. First, a Block-SIFT method is designed to overcome the memory limitation of SIFT for extracting and matching features from large photogrammetric images. For each pair of images, the original large image is split into blocks and the possible corresponding blocks in the other image are determined by pre-estimating the relative transformation between the two images. Because of the reduced memory requirement, features can be extracted and matched from the original images without down-sampling. Next, a red-black tree data structure is applied to create a feature relationship to reduce the search complexity when matching tie points. Meanwhile, tree key exchange and segment matching methods are proposed to match the tie points along-track and across-track. Finally, to evaluate the accuracy of the features extracted and matched from the proposed -SIFT algorithm, a bundle adjustment with parallax angle feature parametrization (ParallaxBA [1] The ParallaxBA source code is available open-source at OpenSLAM http://openslam.org/ParallaxBA.html. The features of the test datasets in this paper as the input to ParallaxBA are also available at OpenSLAM. 1 ) is applied to obtain the Mean Square Error (MSE) of the feature reprojections, where the feature extraction and matching result is the only information used in the nonlinear optimisation system. Seven different experimental aerial photogrammetric datasets are used to demonstrate the efficiency and validity of the proposed algorithm. It is demonstrated that more than 33 million features can be extracted and matched from the Taian dataset with 737 images within 21h using the -SIFT algorithm. In addition, the ParallaxBA involving more than 2.7 million features and 6 million image points can easily converge to an MSE of 0.03874. The C/C++ source code for the proposed algorithm is available at http://services.eng.uts.edu.au/sdhuang/research.htm. [Copyright &y& Elsevier]

Abstract: Scene comparison and matching is a fundamental operation in geoinformatics. However, existing solutions are rather inadequate to support scene similarity assessment when comparing datasets collected from diverse sources especially ones that are available in diverse modalities (e.g. comparing image to vector datasets), or represent different time instances and thus differ partially in their content. In this paper we introduce a two-stage scene similarity assessment and matching framework that makes use of spatial scene content to compare and match two scenes as they may be captured in two different datasets (e.g. an aerial image and a map). At first stage our approach makes use of a matching algorithm based on the comparison of attributed graphs, where linear feature networks (e.g. road networks) are transformed into graphs and network properties are expressed through graph-embedded invariant attributes. By matching these graphs we can assess the similarity between two scenes. At the second stage, we proceed with an invariant scene comparison metric that incorporates additional scene content in the form of object configurations present within individual road network loops (e.g. building arrangements within city squares). By combining diverse but co-located pieces of information (e.g. roads and buildings) in an integrated process, our algorithm supports scene comparison and matching even when comparing heterogeneous datasets. In this paper we present key theoretical concepts and provide experimental results to demonstrate the performance of the proposed approach. [Copyright &y& Elsevier]

Abstract: That the multi-source remote sensing information integrates knowledge-based geospatial constraints to develop efficient and practical Land cover classification algorithm has become one of the most important developing directions in the field of remote sensing ground object classification. Remote sensing classification is a strictly incompatible problem, but the spectra distribution of remote sensing data has compatible attributes especially in mountainous regions, and such contradiction is one of the main reasons leading to uncertainties in remote sensing classification. In this paper, the remote sensing spectra feature compatible information is transformed into the probability of the association degree firstly, and then the matter-element theory is introduced to establish models to achieve the integrated classification of multi-source data to fuse knowledge-based geographical constrained condition probability. Taking the grassland–wetland fragile ecosystem in Ruoergai plateau of China as a case study, this paper selected the multi-source data including images of Landsat TM and CBERS, ASTER-GDEM and MODIS-NDVI to construct a comprehensive classifier, in which the relationship between topography and land cover, and the prior knowledge on vegetation growth difference were taken as constraints to support the decision-making. The classification accuracy was evaluated by a field investigation and existing land cover map. The test result shows that, the overall accuracy (89.89%) and Kappa coefficient (0.8870) are better than those derived by the Maximum Likelihood method. It indicates that the proposed classification method is not subject to the dimensionality and form of data sources, and it can integrate the data source information to improve the classification accuracy, so that it is very useful to apply multi-source data and prior knowledge to land cover classification in mountainous regions. [Copyright &y& Elsevier]

Abstract Filtering of low vegetation with height less than approximately 1.5 m is a challenging problem, especially in mountainous areas covered by heavy low foliage, bushes and sub-shrubberies, etc. The paper proposes an approach for obtaining a more accurate Digital Elevation Model (DEM) by removing low vegetation from point cloud. The approach combines point cloud with full waveform data, and begins by filtering point cloud by way of progressive TIN densification (PTD) method. Ground points are thus extracted, but mixed with false ground points, which are mainly from low vegetation and other manmade low objects. Gaussian decomposition by grouping Levenberg–Marquardt (LM) algorithm with F test is performed for the full waveforms corresponding to the extracted ground points. Echo widths and backscattering coefficients are calculated based on the parameters extracted from the decomposition, and used to discriminate points of low vegetation from points of other low objects, allowing the false ground points reflected from low vegetation to be labeled. New elevation values are calculated from the last echoes of the waveforms from low vegetation, and the DEM is updated by replacing the original elevations with the calculated ones. The resultants are assessed both quantitatively by check points and qualitatively by rendered DEM and contour lines generated from it. The accuracy of the refined DEM with low vegetation removal increases by 31% compared with the original DEM in the experiment, showing the effectiveness of the proposed approach. [ABSTRACT FROM AUTHOR]

Abstract Cloud and cloud shadow detection is a necessary preprocessing step for optical remote sensing applications because of the huge negative influence cloud and cloud shadow can have on data analysis. However, most of the existing cloud/shadow detection methods are designed based on specific band configurations of certain sensors, and their working mechanisms are relatively complex and computationally complicated, which limits their application. In view of this, in this paper, a unified cloud/shadow detection algorithm based on spectral indices (CSD-SI) is proposed for most of the widely used multi/hyperspectral optical remote sensing sensors with both visible and infrared spectral channels. On the one hand, the cloud index (CI) and cloud shadow index (CSI) are proposed to indicate the potential clouds and cloud shadows based on their physical reflective characteristics. In addition, considering the spatial coexistence of cloud and cloud shadow, a spatial matching strategy is utilized to remove the pseudo cloud shadows. The effectiveness of the proposed CSD-SI algorithm is demonstrated on eight different types of widely used multi/hyperspectral optical sensors, with different spectral and spatial resolution levels. Overall, in the experiments undertaken in this study, CSD-SI achieved a mean overall accuracy of 98.52% for cloud, with a mean producer’s accuracy of 93.13% and a mean user’s accuracy of 98.13%. For cloud shadow, CSD-SI achieved a means producer’s accuracy of 84.33% and a mean user’s accuracy of 89.12%. The experimental results show that the proposed CSD-SI method based on spectral indices can obtain a comparable cloud/shadow detection performance to that of the other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Abstract The last two decades have witnessed increasing awareness of the potential of terrestrial laser scanning (TLS) in forest applications in both public and commercial sectors, along with tremendous research efforts and progress. It is time to inspect the achievements of and the remaining barriers to TLS-based forest investigations, so further research and application are clearly orientated in operational uses of TLS. In such context, the international TLS benchmarking project was launched in 2014 by the European Spatial Data Research Organization and coordinated by the Finnish Geospatial Research Institute. The main objectives of this benchmarking study are to evaluate the potential of applying TLS in characterizing forests, to clarify the strengths and the weaknesses of TLS as a measure of forest digitization, and to reveal the capability of recent algorithms for tree-attribute extraction. The project is designed to benchmark the TLS algorithms by processing identical TLS datasets for a standardized set of forest attribute criteria and by evaluating the results through a common procedure respecting reliable references. Benchmarking results reflect large variances in estimating accuracies, which were unveiled through the 18 compared algorithms and through the evaluation framework, i.e., forest complexity categories, TLS data acquisition approaches, tree attributes and evaluation procedures. The evaluation framework includes three new criteria proposed in this benchmarking and the algorithm performances are investigated through combining two or more criteria (e.g., the accuracy of the individual tree attributes are inspected in conjunction with plot-level completeness) in order to reveal algorithms’ overall performance. The results also reveal some best available forest attribute estimates at this time, which clarify the status quo of TLS-based forest investigations. Some results are well expected, while some are new, e.g., the variances of estimating accuracies between single-/multi-scan, the principle of the algorithm designs and the possibility of a computer outperforming human operation. With single-scan data, i.e., one hemispherical scan per plot, most of the recent algorithms are capable of achieving stem detection with approximately 75% completeness and 90% correctness in the easy forest stands (easy plots: 600 stems/ha, 20 cm mean DBH). The detection rate decreases when the stem density increases and the average DBH decreases, i.e., 60% completeness with 90% correctness (medium plots: 1000 stem/ha, 15 cm mean DBH) and 30% completeness with 90% correctness (difficult plots: 2000 stems/ha, 10 cm mean DBH). The application of the multi-scan approach, i.e., five scans per plot at the center and four quadrant angles, is more effective in complex stands, increasing the completeness to approximately 90% for medium plots and to approximately 70% for difficult plots, with almost 100% correctness. The results of this benchmarking also show that the TLS-based approaches can provide the estimates of the DBH and the stem curve at a 1–2 cm accuracy that are close to what is required in practical applications, e.g., national forest inventories (NFIs). In terms of algorithm development, a high level of automation is a commonly shared standard, but a bottleneck occurs at stem detection and tree height estimation, especially in multilayer and dense forest stands. The greatest challenge is that even with the multi-scan approach, it is still hard to completely and accurately record stems of all trees in a plot due to the occlusion effects of the trees and bushes in forests. Future development must address the redundant yet incomplete point clouds of forest sample plots and recognize trees more accurately and efficiently. It is worth noting that TLS currently provides the best quality terrestrial point clouds in comparison with all other technologies, meaning that all the benchmarks labeled in this paper can also serve as a reference for other terrestrial point clouds sources. [ABSTRACT FROM AUTHOR]

Abstract: Road detection has been a topic of great interest in the photogrammetric and remote sensing communities since the end of the 70s. Many approaches dealing with various sensor resolutions, the nature of the scene or the wished accuracy of the extracted objects have been presented. This topic remains challenging today as the need for accurate and up-to-date data is becoming more and more important. Based on this context, we will study in this paper the road network from a particular point of view, focusing on road marks, and in particular dashed lines. Indeed, they are very useful clues, for evidence of a road, but also for tasks of a higher level. For instance, they can be used to enhance quality and to improve road databases. It is also possible to delineate the different circulation lanes, their width and functionality (speed limit, special lanes for buses or bicycles...). In this paper, we propose a new robust and accurate top–down approach for dashed line detection based on stochastic geometry. Our approach is automatic in the sense that no intervention from a human operator is necessary to initialise the algorithm or to track errors during the process. The core of our approach relies on defining geometric, radiometric and relational models for dashed lines objects. The model also has to deal with the interactions between the different objects making up a line, meaning that it introduces external knowledge taken from specifications. Our strategy is based on a stochastic method, and in particular marked point processes. Our goal is to find the objects configuration minimising an energy function made-up of a data attachment term measuring the consistency of the image with respect to the objects and a regularising term managing the relationship between neighbouring objects. To sample the energy function, we use Green algorithm’s; coupled with a simulated annealing to find its minimum. Results from aerial images at various resolutions are presented showing that our approach is relevant and accurate as it can handle the most frequent layouts of dashed lines. Some issues, for instance, such as the relative weighting of both terms of the energy are also discussed in the conclusion. [Copyright &y& Elsevier]

Abstract: The continued advancements in satellite sensor technologies have increased the number of objects that can be discriminated within satellite imagery. Effective segmentation of high resolution satellite imagery is currently a hot topic of research. Existing segmentation algorithms and applications contain many parameters and options which require the operator to select a proper set of parameters for a given data set. The setting of these parameters can be quite tedious and the same set of parameters may or may not work from one high resolution satellite image scene to the next. This paper presents a modification of a region based approach for unsupervised segmentation of high resolution satellite imagery as a solution to segmentation of land use coverage in QuickBird multispectral 2.44 m imagery. This type of segmentation is important to a variety of applications such as land use classification and urban planning. All region based segmentation approaches require a method for representing image regions/segments and judging the similarity between two given image regions/segments. In the proposed modification of this paper, region description is provided through the integration of band ratios. Region similarity measures are performed using Fuzzy Logic. The Hierarchical Split Merge Refinement (HSMR) algorithmic framework for unsupervised image segmentation is the foundation for this modification. In addition, this paper improves upon the merging and refinement processes of the HSMR algorithm. Test results demonstrate stable segmentation of land use areas across a variety of high resolution satellite images. [Copyright &y& Elsevier]

Abstract: Coastal change is a major issue in many regions of the world, and is often driven by geohazard processes such as landslides and rockfalls. Effective assessment of such phenomena is essential for successful management of coastal ecosystems, and is often reliant on GIS-based analysis. However, while it is crucial that multi-temporal datasets can be accurately registered to a common reference system, traditionally, the dynamic nature of the coastal environment has hampered this process. This paper presents a robust surface matching technique which overcomes the requirement for physical control points, and instead derives control directly from the DEM surfaces. Although surface matching procedures are well established, performance can be sub-optimal where the surfaces contain regions of difference, such as those associated with geohazard activity or vegetation effects. The crucial aspect of the least squares matching approach developed here, is the incorporation of a robust estimation function which allows the effects of surface discrepancies to be mitigated through outlier handling. Aerial photogrammetry is an established technique for coastal monitoring, and extensive archival collections exist. However, archival datasets are particularly affected by the difficulties associated with acquisition of ground control. Conversely, the maturing technique of airborne laser scanning is less influenced by such problems, and instead is capable of producing a high quality representation of coastal terrain. This paper describes the application of the robust surface matching technique to test sites located on the east coast of England. Photogrammetric DEMs are approximately oriented, before being matched to control surfaces derived from higher order datasets, including airborne laser scanning DEMs. The robust matching algorithm is shown to produce significantly improved results over ordinary surface matching. Analysis indicates the effectiveness of this technique for exploitation of archival datasets, revealing a signature of extensive geohazard activity over the twenty-five year study period. Robust matching of airborne laser scanning datasets has also enabled the quantification of short-term geohazard activity, demonstrating the flexibility of this strategy. [Copyright &y& Elsevier]

Abstract: More and more frequently, the populating of 3D MRDB’s (Multi Representations Data Base) is required to support advanced cartographic applications and advanced geospatial decision analysis. We previously proposed a manual photogrammetric process based on the Multi Representation Acquisition Pattern (MRAP) concept to tackle simultaneously Fine Level Geometries (FLG) extraction and Coarse Level Geometries (CLG) extraction. This paper presents our progress, from an automation standpoint, regarding our approach to populate MRDBs containing building geometries through a photogrammetric multiple representations acquisition process. Two new algorithms dedicated to Multi Representation (MR) acquisition are introduced and constitute two contributions. These combine passive imagery and Digital Surface Model analysis in order to address automation issues. The first algorithm allows for the automatic determination of the MRAP parameters from a single click initialization. The second algorithm aims at supporting part of the MR acquisition when MRAP are not relevant and allows the automatic building footprints extraction. This paper describes the project motivation and its actual progress. It is divided into four parts. The first part concerns the MR data acquisition specifications. A description of the manual prototype is provided and the MRAP concept is described. In the second part, the first algorithm allowing the automatic determination of the MRAP parameters is introduced. The third part is dedicated to the review of the automation performances through the study of three test sites. Finally, our second algorithm, allowing the automatic building footprints extraction is introduced with preliminary results. [Copyright &y& Elsevier]

Abstract: 3D surface matching would be an ill conditioned problem when the curvature of the object surface is either homogenous or isotropic, e.g. for plane or spherical types of objects. A reliable solution can only be achieved if supplementary information or functional constraints are introduced. In a previous paper, an algorithm for the least squares matching of overlapping 3D surfaces, which were digitized/sampled point by point using a laser scanner device, by the photogrammetric method or other techniques, was proposed [Gruen, A., and Akca, D., 2005. Least squares 3D surface and curve matching. ISPRS Journal of Photogrammetry and Remote Sensing 59 (3), 151–174.]. That method estimates the transformation parameters between two or more fully 3D surfaces, minimizing the Euclidean distances instead of z-differences between the surfaces by least squares. In this paper, an extension to the basic algorithm is given, which can simultaneously match surface geometry and its attribute information, e.g. intensity, colour, temperature, etc. under a combined estimation model. Three experimental results based on terrestrial laser scanner point clouds are presented. The experiments show that the basic algorithm can solve the surface matching problem provided that the object surface has at least the minimal information. If not, the laser scanner derived intensities are used as supplementary information to find a reliable solution. The method derives its mathematical strength from the least squares image matching concept and offers a high level of flexibility for many kinds of 3D surface correspondence problem. [Copyright &y& Elsevier]

Abstract: This paper presents a novel version of the five-point relative orientation algorithm given in Nister [Nister, D., 2004. An efficient solution to the five-point relative pose problem, IEEE Transactions on Pattern Analysis and Machine Intelligence, 26 (6), 756–770]. The name of the algorithm arises from the fact that it can operate even on the minimal five-point correspondences required for a finite number of solutions to relative orientation. For the minimal five correspondences, the algorithm returns up to 10 real solutions. The algorithm can also operate on many points. Like the previous version of the five-point algorithm, our method can operate correctly even in the face of critical surfaces, including planar and ruled quadric scenes. The paper presents comparisons with other direct methods, including the previously developed five-point method, two different six-point methods, the seven-point method, and the eight-point method. It is shown that the five-point method is superior in most cases among the direct methods. The new version of the algorithm was developed from the perspective of algebraic geometry and is presented in the context of computing a Gröbner basis. The constraints are formulated in terms of polynomial equations in the entries of the fundamental matrix. The polynomial equations generate an algebraic ideal for which a Gröbner basis is computed. The Gröbner basis is used to compute the action matrix for multiplication by a single variable monomial. The eigenvectors of the action matrix give the solutions for all the variables and thereby also relative orientation. Using a Gröbner basis makes the solution clear and easy to explain. [Copyright &y& Elsevier]

Over the past years, several filters have been developed to extract bare-Earth points from point clouds. ISPRS Working Group III/3 conducted a test to determine the performance of these filters and the influence of point density thereon, and to identify directions for future research. Twelve selected datasets have been processed by eight participants. In this paper, the test results are presented. The paper describes the characteristics of the provided datasets and the used filter approaches. The filter performance is analysed both qualitatively and quantitatively. All filters perform well in smooth rural landscapes, but all produce errors in complex urban areas and rough terrain with vegetation. In general, filters that estimate local surfaces are found to perform best. The influence of point density could not well be determined in this experiment. Future research should be directed towards the usage of additional data sources, segment-based classification, and self-diagnosis of filter algorithms. [Copyright &y& Elsevier]

High-resolution imagery is now routinely utilized to acquire geometric and attribute data for road networks. Frequently, these data must be integrated with data from other sources to meet application requirements. A critical first step in this process of data integration is to identify the correspondences between networks, so that data can be merged or transferred from one network to another. This paper introduces a semiautomated method that combines an automated algorithm and an interactive procedure to match networks that are represented differently. The automated algorithm establishes robust correspondences for nodes, edges, and segments between two networks using a cluster-based matching mechanism. The interactive procedure allows a user to visually check and correct correspondences that are mismatched by the automated algorithm. Two matching examples are presented in the paper to demonstrate the applicability of the proposed method. The results of these two matching examples illustrate that this semiautomated method is highly effective and can meet the needs of real-world applications. The automated algorithm is capable of generating reliable matching measures, resolving difficult matches, and in both cases, correctly identifying the overwhelming majority of the matching counterparts. The interactive procedure, on the other hand, is useful for checking and correcting mismatched counterparts during the early stage of the matching process, which significantly improves performance and reliability. [Copyright &y& Elsevier]

Land surface temperature (LST) is one of the most important parameters in land surface processes. Although satellite-derived LST can provide valuable information, the value is often limited by cloud contamination. In this paper, a two-step satellite-derived LST reconstruction framework is proposed. First, a multi-temporal reconstruction algorithm is introduced to recover invalid LST values using multiple LST images with reference to corresponding remotely sensed vegetation index. Then, all cloud-contaminated areas are temporally filled with hypothetical clear-sky LST values. Second, a surface energy balance equation-based procedure is used to correct for the filled values. With shortwave irradiation data, the clear-sky LST is corrected to the real LST under cloudy conditions. A series of experiments have been performed to demonstrate the effectiveness of the developed approach. Quantitative evaluation results indicate that the proposed method can recover LST in different surface types with mean average errors in 3–6 K. The experiments also indicate that the time interval between the multi-temporal LST images has a greater impact on the results than the size of the contaminated area. [ABSTRACT FROM AUTHOR]

Due to the complexity of background scenarios and the variation of target appearance, it is difficult to achieve high accuracy and fast speed for object tracking. Currently, correlation filters based trackers (CFTs) show promising performance in object tracking. The CFTs estimate the target’s position by correlation filters with different kinds of features. However, most of CFTs can hardly re-detect the target in the case of long-term tracking drifts. In this paper, a feature integration object tracker named correlation filters and online learning (CFOL) is proposed. CFOL estimates the target’s position and its corresponding correlation score using the same discriminative correlation filter with multi-features. To reduce tracking drifts, a new sampling and updating strategy for online learning is proposed. Experiments conducted on 51 image sequences demonstrate that the proposed algorithm is superior to the state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

The large number of object categories and many overlapping or closely neighboring objects in large-scale urban scenes pose great challenges in point cloud classification. In this paper, a novel framework is proposed for classification and reconstruction of airborne laser scanning point cloud data. To label point clouds, we present a rectified linear units neural network named ReLu-NN where the rectified linear units (ReLu) instead of the traditional sigmoid are taken as the activation function in order to speed up the convergence. Since the features of the point cloud are sparse, we reduce the number of neurons by the dropout to avoid over-fitting of the training process. The set of feature descriptors for each 3D point is encoded through self-taught learning, and forms a discriminative feature representation which is taken as the input of the ReLu-NN. The segmented building points are consolidated through an edge-aware point set resampling algorithm, and then they are reconstructed into 3D lightweight models using the 2.5D contouring method ( Zhou and Neumann, 2010 ). Compared with deep learning approaches, the ReLu-NN introduced can easily classify unorganized point clouds without rasterizing the data, and it does not need a large number of training samples. Most of the parameters in the network are learned, and thus the intensive parameter tuning cost is significantly reduced. Experimental results on various datasets demonstrate that the proposed framework achieves better performance than other related algorithms in terms of classification accuracy and reconstruction quality. [ABSTRACT FROM AUTHOR]

This paper studies the problem of color consistency correction for sequential images with diverse color characteristics. Existing algorithms try to adjust all images to minimize color differences among images under a unified energy framework, however, the results are prone to presenting a consistent but unnatural appearance when the color difference between images is large and diverse. In our approach, this problem is addressed effectively by providing a guided initial solution for the global consistency optimization, which avoids converging to a meaningless integrated solution. First of all, to obtain the reliable intensity correspondences in overlapping regions between image pairs, we creatively propose the histogram extreme point matching algorithm which is robust to image geometrical misalignment to some extents. In the absence of the extra reference information, the guided initial solution is learned from the major tone of the original images by searching some image subset as the reference, whose color characteristics will be transferred to the others via the paths of graph analysis. Thus, the final results via global adjustment will take on a consistent color similar to the appearance of the reference image subset. Several groups of convincing experiments on both the synthetic dataset and the challenging real ones sufficiently demonstrate that the proposed approach can achieve as good or even better results compared with the state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Local feature detectors are widely used in many photogrammetry and remote sensing applications. The quantity and distribution of the local features play a critical role in the quality of the image matching process, particularly for multi-sensor high resolution remote sensing image registration. However, conventional local feature detectors cannot extract desirable matched features either in terms of the number of correct matches or the spatial and scale distribution in multi-sensor remote sensing images. To address this problem, this paper proposes a novel method for uniform and robust local feature extraction for remote sensing images, which is based on a novel competency criterion and scale and location distribution constraints. The proposed method, called uniform competency (UC) local feature extraction, can be easily applied to any local feature detector for various kinds of applications. The proposed competency criterion is based on a weighted ranking process using three quality measures, including robustness, spatial saliency and scale parameters, which is performed in a multi-layer gridding schema. For evaluation, five state-of-the-art local feature detector approaches, namely, scale-invariant feature transform (SIFT), speeded up robust features (SURF), scale-invariant feature operator (SFOP), maximally stable extremal region (MSER) and hessian-affine, are used. The proposed UC-based feature extraction algorithms were successfully applied to match various synthetic and real satellite image pairs, and the results demonstrate its capability to increase matching performance and to improve the spatial distribution. The code to carry out the UC feature extraction is available from https://www.researchgate.net/publication/317956777_UC-Feature_Extraction . [ABSTRACT FROM AUTHOR]

The S-NPP VIIRS Land Surface Temperature (LST) product attained the stage V1 of validation maturity (provisional validated) at the end of 2014. This paper evaluates the current VIIRS V1 LST product versus concurrent ground data acquired at a rice paddy site from December 2014 to August 2016. The experimental site has three different seasonal and homogeneous land covers through the year, which makes the site interesting for validation activities. An autonomous and multiangular system was used to record continuous ground data at the site. The data acquired at zenith angles similar to the VIIRS viewing angles were used for the validation to avoid possible differences between satellite and ground views due to angular dependences of the LSTs. Concurrently to surface data, downwelling sky radiances were measured at different incidence angles by the system, which were used to improve the cloud screening of the validation dataset, since cloud leakage was identified in previous validation studies as an important issue for further improvement. The validation results show good performance for the VIIRS V1 LST product at zenith angles ≤40°, with systematic uncertainties within ±0.5 K and accuracies around 1.2 K. These values are within the threshold requirements established for the VIIRS LST product, and they are better than the validation results published previously for the beta version of the product or using VIIRS data reprocessed with the calibrated algorithm coefficients implemented from April 2014. As the VIIRS LST algorithm has regression coefficients dependent on land cover type, the impact of land cover misclassifications on VIIRS LST data accuracy was also evaluated. It was expected that changing the surface type assigned by the VIIRS product to more appropriate types at the site pixels should improve the validation results. However, the improvement was limited, likely due to the reduced range of variability of the emissivities considered for the different land cover types in the regression process of the VIIRS LST algorithm coefficients. The results reveal the difficulties and uncertainties involved in the LST retrieval when using a LST algorithm with surface type dependent coefficients. [ABSTRACT FROM AUTHOR]

Since forest planning is increasingly taking an ecological, diversity-oriented perspective into account, remote sensing technologies are becoming ever more important in assessing existing resources with reduced manual effort. While the light detection and ranging (LiDAR) technology provides a good basis for predictions of tree height and biomass, tree species identification based on this type of data is particularly challenging in structurally heterogeneous forests. In this paper, we analyse existing approaches with respect to the geometrical scale of feature extraction (whole tree, within crown partitions or within laser footprint) and conclude that currently features are always extracted separately from the different scales. Since multi-scale approaches however have proven successful in other applications, we aim to utilize the within-tree-crown distribution of within-footprint signal characteristics as additional features. To do so, a spin image algorithm, originally devised for the extraction of 3D surface features in object recognition, is adapted. This algorithm relies on spinning an image plane around a defined axis, e.g. the tree stem, collecting the number of LiDAR returns or mean values of returns attributes per pixel as respective values. Based on this representation, spin image features are extracted that comprise only those components of highest variability among a given set of library trees. The relative performance and the combined improvement of these spin image features with respect to non-spatial statistical metrics of the waveform (WF) attributes are evaluated for the tree species classification of Scots pine ( Pinus sylvestris L.), Norway spruce ( Picea abies (L.) Karst.) and Silver/Downy birch ( Betula pendula Roth/ Betula pubescens Ehrh.) in a boreal forest environment. This evaluation is performed for two WF LiDAR datasets that differ in footprint size, pulse density at ground, laser wavelength and pulse width. Furthermore, we evaluate the robustness of the proposed method with respect to internal parameters and tree size. The results reveal, that the consideration of the crown-internal distribution of within-footprint signal characteristics captured in spin image features improves the classification results in nearly all test cases. [ABSTRACT FROM AUTHOR]

The ability of remote sensing-based surface energy balance (SEB) models to track water stress in rain-fed switchgrass ( Panicum virgatum L.) has not been explored yet. In this paper, the theoretical framework of crop water stress index (CWSI; 0 = extremely wet or no water stress condition and 1 = extremely dry or no transpiration) was utilized to estimate CWSI in rain-fed switchgrass using Landsat-derived evapotranspiration (ET) from five remote sensing based single-source SEB models, namely Surface Energy Balance Algorithm for Land (SEBAL), Mapping ET with Internalized Calibration (METRIC), Surface Energy Balance System (SEBS), Simplified Surface Energy Balance Index (S-SEBI), and Operational Simplified Surface Energy Balance (SSEBop). CWSI estimates from the five SEB models and a simple regression model that used normalized difference vegetation index (NDVI), near-surface temperature difference, and measured soil moisture (SM) as covariates were compared with those derived from eddy covariance measured ET (CWSI EC ) for the 32 Landsat image acquisition dates during the 2011 (dry) and 2013 (wet) growing seasons. Results indicate that most SEB models can predict CWSI reasonably well. For example, the root mean square error (RMSE) ranged from 0.14 (SEBAL) to 0.29 (SSEBop) and the coefficient of determination ( R 2 ) ranged from 0.25 (SSEBop) to 0.72 (SEBAL), justifying the added complexity in CWSI modeling as compared to results from the simple regression model ( R 2  = 0.55, RMSE = 0.16). All SEB models underestimated CWSI in the dry year but the estimates from SEBAL and S-SEBI were within 7% of the mean CWSI EC and explained over 60% of variations in CWSI EC . In the wet year, S-SEBI mostly overestimated CWSI (around 28%), while estimates from METRIC, SEBAL, SEBS, and SSEBop were within 8% of the mean CWSI EC . Overall, SEBAL was the most robust model under all conditions followed by METRIC, whose performance was slightly worse and better than SEBAL in dry and wet years, respectively. Underestimation of CWSI under extremely dry soil conditions and the substantial role of SM in the regression model suggest that integration of SM in SEB models could improve their performances under dry conditions. These insights will provide useful guidance on the broader applicability of SEB models for mapping water stresses in switchgrass under varying geographical and meteorological conditions. [ABSTRACT FROM AUTHOR]

Robust image matching is crucial for many applications of remote sensing and photogrammetry, such as image fusion, image registration, and change detection. In this paper, we propose a robust feature matching method based on support-line voting and affine-invariant ratios. We first use popular feature matching algorithms, such as SIFT, to obtain a set of initial matches. A support-line descriptor based on multiple adaptive binning gradient histograms is subsequently applied in the support-line voting stage to filter outliers. In addition, we use affine-invariant ratios computed by a two-line structure to refine the matching results and estimate the local affine transformation. The local affine model is more robust to distortions caused by elevation differences than the global affine transformation, especially for high-resolution remote sensing images and UAV images. Thus, the proposed method is suitable for both rigid and non-rigid image matching problems. Finally, we extract as many high-precision correspondences as possible based on the local affine extension and build a grid-wise affine model for remote sensing image registration. We compare the proposed method with six state-of-the-art algorithms on several data sets and show that our method significantly outperforms the other methods. The proposed method achieves 94.46% average precision on 15 challenging remote sensing image pairs, while the second-best method, RANSAC, only achieves 70.3%. In addition, the number of detected correct matches of the proposed method is approximately four times the number of initial SIFT matches. [ABSTRACT FROM AUTHOR]

This paper presents an improved multi-scale Retinex (MSR) based enhancement for ariel images under low visibility. For traditional multi-scale Retinex, three scales are commonly employed, which limits its application scenarios. We extend our research to a general purpose enhanced method, and design an MSR with more than three scales. Based on the mathematical analysis and deductions, an explicit multi-scale representation is proposed that balances image contrast and color consistency. In addition, a histogram truncation technique is introduced as a post-processing strategy to remap the multi-scale Retinex output to the dynamic range of the display. Analysis of experimental results and comparisons with existing algorithms demonstrate the effectiveness and generality of the proposed method. Results on image quality assessment proves the accuracy of the proposed method with respect to both objective and subjective criteria. [ABSTRACT FROM AUTHOR]

Paddy rice agriculture plays an important role in various environmental issues including food security, water use, climate change, and disease transmission. However, regional and global paddy rice maps are surprisingly scarce and sporadic despite numerous efforts in paddy rice mapping algorithms and applications. With the increasing need for regional to global paddy rice maps, this paper reviewed the existing paddy rice mapping methods from the literatures ranging from the 1980s to 2015. In particular, we illustrated the evolution of these paddy rice mapping efforts, looking specifically at the future trajectory of paddy rice mapping methodologies. The biophysical features and growth phases of paddy rice were analyzed first, and feature selections for paddy rice mapping were analyzed from spectral, polarimetric, temporal, spatial, and textural aspects. We sorted out paddy rice mapping algorithms into four categories: (1) Reflectance data and image statistic-based approaches, (2) vegetation index (VI) data and enhanced image statistic-based approaches, (3) VI or RADAR backscatter-based temporal analysis approaches, and (4) phenology-based approaches through remote sensing recognition of key growth phases. The phenology-based approaches using unique features of paddy rice (e.g., transplanting) for mapping have been increasingly used in paddy rice mapping. Current applications of these phenology-based approaches generally use coarse resolution MODIS data, which involves mixed pixel issues in Asia where smallholders comprise the majority of paddy rice agriculture. The free release of Landsat archive data and the launch of Landsat 8 and Sentinel-2 are providing unprecedented opportunities to map paddy rice in fragmented landscapes with higher spatial resolution. Based on the literature review, we discussed a series of issues for large scale operational paddy rice mapping. [ABSTRACT FROM AUTHOR]

This paper proposes an accurate multi-view stereo method for image-based 3D reconstruction that features robustness in the presence of occlusions. The new method offers improvements in dealing with two fundamental image matching problems. The first concerns the selection of the support window model, while the second centers upon accurate visibility estimation for each pixel. The support window model is based on an approximate 3D support plane described by a depth and two per-pixel depth offsets. For the visibility estimation, the multi-view constraint is initially relaxed by generating separate support plane maps for each support image using a modified PatchMatch algorithm. Then the most likely visible support image, which represents the minimum visibility of each pixel, is extracted via a discrete Markov Random Field model and it is further augmented by parameter clustering. Once the visibility is estimated, multi-view optimization taking into account all redundant observations is conducted to achieve optimal accuracy in the 3D surface generation for both depth and surface normal estimates. Finally, multi-view consistency is utilized to eliminate any remaining observational outliers. The proposed method is experimentally evaluated using well-known Middlebury datasets, and results obtained demonstrate that it is amongst the most accurate of the methods thus far reported via the Middlebury MVS website. Moreover, the new method exhibits a high completeness rate. [ABSTRACT FROM AUTHOR]

This paper introduces a novel aerial building detection method based on region orientation as a new feature, which is used in various steps throughout the presented framework. As building objects are expected to be connected with each other on a regional level, exploiting the main orientation obtained from the local gradient analysis provides further information for detection purposes. The orientation information is applied for an improved edge map design, which is integrated with classical features like shadow and color. Moreover, an orthogonality check is introduced for finding building candidates, and their final shapes defined by the Chan–Vese active contour algorithm are refined based on the orientation information, resulting in smooth and accurate building outlines. The proposed framework is evaluated on multiple data sets, including aerial and high resolution optical satellite images, and compared to six state-of-the-art methods in both object and pixel level evaluation, proving the algorithm’s efficiency. [ABSTRACT FROM AUTHOR]

Target detection is one of the most important problems in hyperspectral image (HSI) processing. However, the classical algorithms depend on the specific statistical hypothesis test, and the algorithms may only perform well under certain conditions, e.g., the adaptive matched subspace detector algorithm assumes that the background covariance matrices do not include the target signatures, which seldom happens in the real world. How to develop a proper metric for measuring the separability between targets and backgrounds becomes the key in target detection. This paper proposes an efficient maximum margin metric learning (MMML) based target detection algorithm, which aims at exploring the limited samples in metric learning and transfers the metric learning problem for hyperspectral target detection into a maximum margin problem which can be optimized via a cutting plane method, and maximally separates the target samples from the background ones. The extensive experimental results with different HSIs demonstrate that the proposed method outperforms both the state-of-the-art target detection algorithms and the other classical metric learning methods. [ABSTRACT FROM AUTHOR]

Robust, well-distributed and accurate feature matching in multi-sensor remote sensing image is a difficult task duo to significant geometric and illumination differences. In this paper, a robust and effective image matching approach is presented for multi-sensor remote sensing images. The proposed approach consists of three main steps. In the first step, UR-SIFT (Uniform robust scale invariant feature transform) algorithm is applied for uniform and dense local feature extraction. In the second step, a novel descriptor namely Distinctive Order Based Self Similarity descriptor , DOBSS descriptor, is computed for each extracted feature. Finally, a cross matching process followed by a consistency check in the projective transformation model is performed for feature correspondence and mismatch elimination. The proposed method was successfully applied for matching various multi-sensor satellite images as: ETM+, SPOT 4, SPOT 5, ASTER, IRS, SPOT 6, QuickBird, GeoEye and Worldview sensors, and the results demonstrate its robustness and capability compared to common image matching techniques such as SIFT, PIIFD, GLOH, LIOP and LSS. [ABSTRACT FROM AUTHOR]

In this paper we present an approach to perform relative spectral alignment between optical cross-sensor acquisitions. The proposed method aims at projecting the images from two different and possibly disjoint input spaces into a common latent space, in which standard change detection algorithms can be applied. The system relies on the regularized kernel canonical correlation analysis transformation (kCCA), which can accommodate nonlinear dependencies between pixels by means of kernel functions. To learn the projections, the method employs a subset of samples belonging to the unchanged areas or to uninteresting radiometric differences. Since the availability of ground truth information to perform model selection is limited, we propose a completely automatic strategy to select the hyperparameters of the system as well as the dimensionality of the transformed (latent) space. The proposed scheme is fully automatic and allows the use of any change detection algorithm in the transformed latent space. A synthetic problem built from real images and a case study involving a real cross-sensor change detection problem illustrate the capabilities of the proposed method. Results show that the proposed system outperforms the linear baseline and provides accuracies close the ones obtained with a fully supervised strategy. We provide a MATLAB implementation of the proposed method as well as the real cross-sensor data we prepared and employed at https://sites.google.com/site/michelevolpiresearch/codes/cross-sensor . [ABSTRACT FROM AUTHOR]

Airborne laser scanning data perfectly represent flat or gently sloped areas; to date, however, accurate breakline detection is the main drawback of this technique. This issue becomes particularly important in the case of modeling buildings, where accuracy higher than the footprint size is often required. This article covers several issues related to full-waveform data registered on building step edges. First, the full-waveform data simulator was developed and presented in this paper. Second, this article provides a full description of the changes in echo amplitude, echo width and returned power caused by the presence of edges within the laser footprint. Additionally, two important properties of step edge echoes, peak shift and echo asymmetry, were noted and described. It was shown that these properties lead to incorrect echo positioning along the laser center line and can significantly reduce the edge points’ accuracy. For these reasons and because all points are aligned with the center of the beam, regardless of the actual target position within the beam footprint, we can state that step edge points require geometric corrections. This article presents a novel algorithm for the refinement of step edge points. The main distinguishing advantage of the developed algorithm is the fact that none of the additional data, such as emitted signal parameters, beam divergence, approximate edge geometry or scanning settings, are required. The proposed algorithm works only on georeferenced profiles of reflected laser energy. Another major advantage is the simplicity of the calculation, allowing for very efficient data processing. Additionally, the developed method of point correction allows for the accurate determination of points lying on edges and edge point densification. For this reason, fully automatic localization of building roof step edges based on LiDAR full-waveform data with higher accuracy than the size of the lidar footprint is feasible. [ABSTRACT FROM AUTHOR]

Airborne Laser Scanning (ALS) was introduced to provide rapid, high resolution scans of landforms for computational processing. More recently, ALS has been adapted for scanning urban areas. The greater complexity of urban scenes necessitates the development of novel methods to exploit urban ALS to best advantage. This paper presents occlusion images: a novel technique that exploits the geometric complexity of the urban environment to improve visualisation of small details for better feature recognition. The algorithm is based on an inversion of traditional occlusion techniques. [ABSTRACT FROM AUTHOR]

This paper introduces a novel, region-growing algorithm for the fast surface patch segmentation of three-dimensional point clouds of urban environments. The proposed algorithm is composed of two stages based on a coarse-to-fine concept. First, a region-growing step is performed on an octree-based voxelized representation of the input point cloud to extract major (coarse) segments. The output is then passed through a refinement process. As part of this, there are two competing factors related to voxel size selection. To balance the constraints, an adaptive octree is created in two stages. Empirical studies on real terrestrial and airborne laser scanning data for complex buildings and an urban setting show the proposed approach to be at least an order of magnitude faster when compared to a conventional region growing method and able to incorporate semantic-based feature criteria, while achieving precision, recall, and fitness scores of at least 75% and as much as 95%. [ABSTRACT FROM AUTHOR]