Search

Pole-like objects provide important street infra- structure for road inventory and road mapping. In this article, we proposed a novel pole-like object extraction algorithm based on plane filtering from mobile Light Detection and Ranging (LiDAR) data. The proposed approach is composed of two parts. In the first part, a novel octree-based split scheme was proposed to fit initial planes from off-ground points. The results of the plane fitting contribute to the extraction of pole-like objects. In the second part, we proposed a novel method of pole-like object extraction by plane filtering based on local geometric feature restriction and isolation detection. The proposed approach is a new solution for detecting pole-like objects from mobile LiDAR data. The innovation in this article is that we assumed that each of the pole-like objects can be represented by a plane. Thus, the essence of extracting pole-like objects will be converted to plane selecting problem. The proposed method has been tested on three data sets captured from different scenes. The average completeness, correctness, and quality of our approach can reach up to 87.66%, 88.81%, and 79.03%, which is superior to state-of-the-art approaches. The experimental results indicate that our approach can extract pole-like objects robustly and efficiently. [ABSTRACT FROM AUTHOR]

It is a classical task to automatically extract road networks from very high-resolution (VHR) images in remote sensing. This paper presents a novel method for extracting road networks from VHR remotely sensed images in complex urban scenes. Inspired by image segmentation, edge detection, and object skeleton extraction, we develop a multitask convolutional neural network (CNN), called RoadNet, to simultaneously predict road surfaces, edges, and centerlines, which is the first work in such field. The RoadNet solves seven important issues in this vision problem: 1) automatically learning multiscale and multilevel features [gained by the deeply supervised nets (DSN) providing integrated direct supervision] to cope with the roads in various scenes and scales; 2) holistically training the mentioned tasks in a cascaded end-to-end CNN model; 3) correlating the predictions of road surfaces, edges, and centerlines in a network model to improve the multitask prediction; 4) designing elaborate architecture and loss function, by which the well-trained model produces approximately single-pixel width road edges/centerlines without nonmaximum suppression postprocessing; 5) cropping and bilinear blending to deal with the large VHR images with finite-computing resources; 6) introducing rough and simple user interaction to obtain desired predictions in the challenging regions; and 7) establishing a benchmark data set which consists of a series of VHR remote sensing images with pixelwise annotation. Different from the previous works, we pay more attention to the challenging situations, in which there are lots of shadows and occlusions along the road regions. Experimental results on two benchmark data sets show the superiority of our proposed approaches. [ABSTRACT FROM AUTHOR]

This paper presents a hierarchical method for matching line segments from two images. Line segments are matched first in groups and then in individuals. While matched in groups, the line segments lying adjacently are intersected to form junctions. At the places of the junctions, the structures are constructed called Line–Junction–Line (LJL), which consists of two adjacent line segments and their intersecting junction. To reliably deal with the possible scale change between the two images to be matched, we propose to describe LJLs by a robust descriptor in the multi-scale pyramids of images constructed from two original images. By evaluating the description vector distances of LJLs from the two images, some candidate LJL matches are obtained, which are then refined and expanded by an effective match-propagation strategy. The line segments used to construct LJLs are matched when the LJLs they formed are matched. For those left unmatched line segments, we match them in individuals by utilizing the local homographies estimated from their neighboring matched LJLs. Experiments demonstrate the superiorities of the proposed method to the state-of-the-art methods for its robustness in more difficult situations, the larger amounts of correct matches, and the higher accuracy in most cases. [ABSTRACT FROM AUTHOR]

The automatic classification of 3-D point clouds is publicly known as a challenging task in a complex road environment. Specifically, each point is automatically classified into a unique category label, and then, the labels are used as clues for semantic analysis and scene recognition. Instead of heuristically extracting handcrafted features in traditional methods to classify all points, we put forward an end-to-end octree-based fully convolutional network (FCN) to classify 3-D point clouds in an urban road environment. There are four contributions in this paper. The first is that the integration and comprehensive uses of OctNet and FCN greatly decrease the computing time and memory demands compared with a dense 3-D convolutional neural network (CNN). The second is that the octree-based network is strengthened by means of modifying the cross-entropy loss function to solve the problems of an unbalanced category distribution. The third is that an Inception-ResNet block is united with our network, which enables our 3-D CNN to effectively learn how to classify scenes containing objects at multiple scales and improve classification accuracy. The last is that an open source data set (HuangshiRoad data set) with ten different classes is introduced for 3-D point cloud classification. Three representative data sets [Semantic3D, WHU_MLS (blocks I and II), and HuangshiRoad] with different covered areas and numbers of points and classes are selected to evaluate our proposed method. The experimental results show that the overall classification accuracy is appreciable, with 89.4% for Semantic3D, 82.9% for WHU_MLS block I, 91.4% for WHU_MLS block II, and 94% for HuangshiRoad. Our deep learning approach can efficiently classify 3-D dense point clouds in an urban road environment measured by a mobile laser scanning (MLS) system or static LiDAR. [ABSTRACT FROM AUTHOR]

3D object recognition is a fundamental task in 3D computer vision. View-based methods have received considerable attention due to their high efficiency and superior performance. To better capture the long-range dependencies among multi-view images, Transformer has recently been introduced into view-based 3D object recognition and achieved excellent performance. However, the information among views on multiple scales is not utilized sufficiently in the existing Transformer-based methods. To address this limitation, we proposed a 3D object recognition method named iMVS to i ntegrate M ulti- V iew information on multiple S cales. Specifically, for the single-view image/features at each scale, we adopt a hybrid feature extraction module consisting of CNN and Transformer to jointly capture local and non-local information. For the extracted multi-view image features at each scale, we develop a feature transfer module including a view Transformer block to achieve the information transfer across views. Following a sequential process of the single-view feature extraction and multi-view feature transfer on multiple scales, the multi-view information is sufficiently interacted. Subsequently, the multi-scale features with multi-view information are fed into our designed feature aggregation module to generate a category-specific descriptor, where the adopted channel Transformer block facilitates the descriptor to be more expressive. Coupling with these designs, our method can fully exploit the information embedded within multi-view images. Experimental results on ModelNet40, ModelNet10 and a real-world dataset MVP-N demonstrate the superior performance of our method. • The view-based 3D object recognition method makes full use of multi-view information • The transfer module achieves multi-view feature interaction on multiple scales • The channel Transformer-based module effectively aggregates multi-scale features • Experimental results demonstrate the effectiveness of our method iMVS [ABSTRACT FROM AUTHOR]

This paper investigates the generation of large field-of-view (FOV) and high-resolution (HR) panorama images for smartphones. Existing techniques like image stitching struggle to produce satisfactory results due to geometry misalignment and inconsistent appearance. To circumvent the inherent challenges of image stitching methods and generate high-quality panoramas, we treat the image stitching problem as a multiple-reference-based super-resolution problem. Specifically, one large-FOV low-resolution (LR) image and several overlapped small-FOV HR images are taken as inputs, where the LR image acts as a base and multi-view references provide rich HR information. Building on this foundation, a novel multi-view references image super-resolution framework (MVRefSR) is proposed. Within this framework, to address the residual geometric misalignment between the LR-Ref image pairs after coarse alignment, a flow-based RefSR network (FlowSRNet) is proposed, which super-resolves LR patches with corresponding HR references. To facilitate adaptive feature fusion and minimize distortion in structured regions, a fusion weight estimation module and a gradient branch are introduced in FlowSRNet. Finally, the large-FOV HR image is generated by combining these SR patches together. Furthermore, the lack of real-world RefSR datasets for smartphones is addressed by designing an innovative dataset construction pipeline. Extensive experiments demonstrate the superior performance of FlowSRNet and MVRefSR over the compared SR methods and image stitching software, which proves the effectiveness of generating panoramas from a new perspective. • A novel multi-view references image super-resolution framework (MVRefSR) is proposed to generate the large field-of-view and high-resolution panoramas from a new perspective. • Two modules, a fusion weight estimation module and a gradient branch, are adopted to enhance the performance of our reference-based super-resolution (RefSR) methods within our MVRefSR framework. • A new RefSR dataset generation pipeline and a real-world RefSR dataset are presented to address the lack of datasets and facilitate the study of real-world RefSR. • Experimental results demonstrate the superiority of our RefSR network and our framework MVRefSR, which proves the effectiveness of generating panoramas from a new perspective. [ABSTRACT FROM AUTHOR]

In this paper, we propose a thresholding method utilizing the mutual relation between the two used features, which should be both robust and of low correlation. The mutual relation is exploited through a histogram subtraction transformation, which tries to reduce only the background part of each of the histograms for the features so that we can easily differentiate between the background part and the target part. A speedup scheme that requires prior knowledge to exclude the possible local minimum(s) is also presented. To sufficiently validate the effectiveness of our histogram subtraction transformation, the two developed thresholding methods are applied to shadow detection and vegetation detection and compared with some state-of-the-art methods in both fields. The comparative results on multiple data sets indicate that with the thresholds automatically provided by the proposed thresholding methods, even the simple binarization methods can obtain good detection results. [ABSTRACT FROM AUTHOR]

Recent deep learning models have demonstrated strong capabilities for classifying text and non-text components in natural images. They extract a high-level feature globally computed from a whole image component (patch), where the cluttered background information may dominate true text features in the deep representation. This leads to less discriminative power and poorer robustness. In this paper, we present a new system for scene text detection by proposing a novel text-attentional convolutional neural network (Text-CNN) that particularly focuses on extracting text-related regions and features from the image components. We develop a new learning mechanism to train the Text-CNN with multi-level and rich supervised information, including text region mask, character label, and binary text/non-text information. The rich supervision information enables the Text-CNN with a strong capability for discriminating ambiguous texts, and also increases its robustness against complicated background components. The training process is formulated as a multi-task learning problem, where low-level supervised information greatly facilitates the main task of text/non-text classification. In addition, a powerful low-level detector called contrast-enhancement maximally stable extremal regions (MSERs) is developed, which extends the widely used MSERs by enhancing intensity contrast between text patterns and background. This allows it to detect highly challenging text patterns, resulting in a higher recall. Our approach achieved promising results on the ICDAR 2013 data set, with an F-measure of 0.82, substantially improving the state-of-the-art results. [ABSTRACT FROM AUTHOR]