Search

Forest digitisation is one of the next major challenges to be tackled in the forestry domain. As a consequence of tremendous advances in 3D scanning technologies, broad areas of forest can be mapped in 3D dramatically faster than 20 years ago. Consequently, capturing 3D forest point clouds with the use of 3D sensing technologies – such as lidar – is becoming predominant in the field of forestry. However, the processing of 3D point clouds to bring semantics to the 3D forestry data – e.g. by linking them with ecological values – has not seen similar advancements. Therefore, in this paper we consider a novel approach based on the use of VR (Virtual reality) as a potential solution for deriving biodiversity from 3D point clouds acquired in the field. That is, we developed a VR labelling application to visualise forest point clouds and to perform the segmentation of several biodiversity components on tree stems e.g., mosses, lichens and bark pockets. Furthermore, the VR segmented point cloud was analysed with standard accuracy and precision metrics. Namely, the proposed VR application managed to achieve an IoU (Intersection over Union) rate value of 98.74% for the segmentation of bark pockets and resp. 93.71% for the moss and lichen classes. These encouraging results reinforce the potential for the proposed VR labelling method for other purposes in the future, for example for AI (Artificial Intelligence) training dataset creation., International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLVIII-2/W1, ISSN:1682-1750, ISSN:2194-9034, ISSN:1682-1777

Learning from 3D point-cloud data has rapidly gained momentum, motivated by the success of deep learning on images and the increased availability of 3D~data. In this paper, we aim to construct anisotropic convolution layers that work directly on the surface derived from a point cloud. This is challenging because of the lack of a global coordinate system for tangential directions on surfaces. We introduce DeltaConv, a convolution layer that combines geometric operators from vector calculus to enable the construction of anisotropic filters on point clouds. Because these operators are defined on scalar- and vector-fields, we separate the network into a scalar- and a vector-stream, which are connected by the operators. The vector stream enables the network to explicitly represent, evaluate, and process directional information. Our convolutions are robust and simple to implement and match or improve on state-of-the-art approaches on several benchmarks, while also speeding up training and inference., Comment: 8 pages, 5 figures, 7 tables; ACM Transactions on Graphics 41, 4, Article 105 (SIGGRAPH 2022)

Two important features of the points in the LiDAR point clouds are the spatial and the color features. The spatial feature is mostly used in the point cloud processing field due to its 3D informative and distinctive characteristic. The local geometric difference derived from the spatial features of the points is usually benefited by graph-based point cloud segmentation methods, because the geometric features of the local point groups are highly distinctive. In this paper, we use both the geometric and color differences of the adjacent local point groups at the impact rates 0.3, 0.5, and 0.7 and cooperate the Euclidean and the vector color differences within several averaging techniques for the color difference. The difference forms have been tested within a graph-based segmentation method on four point cloud segmentation datasets, two indoor and two outdoor, using their spatial and color information. The geometric mean as an averaging techniques increases the segmentation success for the all datasets except one outdoor when the color differences are used in the segmentation at the impact rate 0.3, while the harmonic mean increases the success for the all datasets the successes except the other outdoor at the same impact rate. According to the test results, the cooperating of the Euclidean and vector angular color difference measurements can considerable increase the segmentation success on the point clouds with color information in a high quality.

As a basic and important research, point cloud segmentation plays an important role in many fields. However, the traditional point cloud segmentation algorithm still suffers from two problems. The ...

The work presents a survey campaign specifically designed to formulate an effective restoration project in a Cultural Heritage context, the Military Shrine in Cima Grappa (Italy). Several outputs have been generated by exploring the most advanced laser scanning survey technique and some specific point cloud analysis algorithms. A detailed geometrical 3D reconstruction of human-made and natural tunnels coating materials, a geo-mechanical survey of the rock mass, a map of rock collapses and cinematic analysis of instability processes.Integrating Laser Scanning technique with the Scan-line survey allowed to perform advanced analysis and rock-mass characterisation in a predominant subterranean developed area. Most of the tunnels and underground spaces displayed rock collapses and diffuse active instability processes that certainly could have drastically slowed down surveys and analysis. The adopted techniques allowed both to proceed in acquiring data end in delivering sound outputs rapidly.

Point cloud segmentation is a key problem in 3D content understanding. The existing methods based on deep neural network for point cloud segmentation mainly train the network in a supervised fashion, which heavily rely on a large amount of high-quality manual-labeled training point clouds. However, it is very tedious and time-consuming to manually assign part labels for each point in point clouds. Meanwhile, a lot of unlabeled point clouds can easily be obtained from 3D scanners, Internet or reconstruction. Therefore, we introduce self-training to utilize these unlabeled point clouds. So the proposed semi-supervised point cloud segmentation method can employ both labeled point clouds and unlabeled point clouds for training. Moreover, in order to make better use of unlabeled point clouds, the adopted adversarial architecture proposes confidence discrimination of label prediction for unlabeled point clouds. Thus, pseudo labels on unlabeled point clouds with higher reliability can be picked out to participate the network training, which further improves segmentation performance. The experiments show that the proposed method can make full use of the unlabeled point clouds in training. In addition, segmentation performance improves by self-training with label confidence prediction.

Among different 3D data representations, point cloud stands out for its efficiency and flexibility. Hence, many researchers have been involved in the point cloud analysis recently. Existing approaches for point cloud segmentation task typically suffer from two limitations: 1) They usually treat different neighbor points as equals which cannot characterize the correlation between the center point and its neighborhoods well. Moreover, different parts may have different local structures for a point cloud, but they just learn a single representation space which is not sufficient and stable. 2) They often capture hierarchical information by heuristic sampling approaches which cannot reveal the spatial relationships of points well to learn global features. To overcome these limitations, we propose a novel hierarchical attentive pooling graph network (HAPGN) which utilizes the gated graph attention network (GGAN) and hierarchical graph pooling module (HiGPool) as building blocks for point cloud segmentation. Specifically, GGAN can highlight not only the importance of different neighbor points but also the importance of different representation spaces to enhance the local feature extraction. HiGPool is a novel pooling module that can capture the spatial layouts of points to learn the hierarchical features adequately. Experimental results on the ShapeNet part dataset and S3DIS dataset show that HAPGN can achieve superior performance over the state-of-the-art segmentation approaches. Furthermore, we also combine our proposed HiGPool with some recent approaches for point cloud classification and achieve better results on the ModelNet40 dataset.

Voxel grid is widely used in point cloud segmentation due to its regularity. However, the memory consumption caused by high resolution restricts the performance of voxel grid. This paper proposes an improved voxel grid deep network (IVDN) model to represent more comprehensive point cloud features at the same resolution, thus improving the segmentation performance of point cloud. Firstly, the point cloud data are structured within a voxel bounding box to correspond with the three-dimensional(3D) convolution kernel, and a fixed number of point coordinates are selected to generate the point feature vector. Then, in order to consider the distribution characteristics, the reliability coefficient is used as an equivalent descriptor of the point cloud distribution density. Finally, a corresponding deep network is constructed to deal with the above features. Experimental results show that the proposed IVDN model can improve the mean classification accuracy and segmentation index mIoU(Mean Intersection over Union) effectively, with a 0.45% and 0.3% improvement on Shape net16 dataset respectively.

Room segmentation is a matter of ongoing interesting for indoor navigation and reconstruction in robotics and AEC. While in robotics field, the problem room segmentation has been typically addressed on 2D floorplan, interest in enrichment 3D models providing more detailed representation of indoors has been growing in the AEC. Point clouds make available more realistic and update but room segmentation from point clouds is still a challenging topic. This work presents a method to carried out point cloud segmentation into rooms based on 3D mathematical morphological operations. First, the input point cloud is voxelized and indoor empty voxels are extracted by CropHull algorithm. Then, a morphological erosion is performed on the 3D image of indoor empty voxels in order to break connectivity between voxels belonging to adjacent rooms. Remaining voxels after erosion are clustered by a 3D connected components algorithm so that each room is individualized. Room morphology is retrieved by individual 3D morphological dilation on clustered voxels. Finally, unlabelled occupied voxels are classified according proximity to labelled empty voxels after dilation operation. The method was tested in two real cases and segmentation performance was evaluated with encouraging results. Xunta de Galicia Ref. ED481B-2019-061 Xunta de Galicia Ref. ED481D 2019/020 Ministerio de Ciencia Innovación y Universidades (España) Ref. RTI2018-095893-B-C21 Ministerio de Ciencia Innovación y Universidades (España) Ref. PID2019-105221RB-C43

Point cloud segmentation is the premise and basis of many 3D perception tasks, such as intelligent driving, object detection and recognition, scene recognition and understanding. In this paper, we present an improved PointNet for 3D object part Segmentation, and named the proposed PointNet as Deep Residual Neural Network Based PointNet (DResNet-PointNet). The architecture of DResNet- PointNet was desigined based on the idea of residual networks. Residual networks can increase the depth of the DResNet-PointNet without network degradation. The depth of DResNet-PointNet is twice as deep as that of original PointNet model. Increasing the depth of DResNet-PointNet can improve its ability to express complex functions and generalization ability of complex classification problems, and achieve better approximation of complex functions, thus improving the accuracy of segmentation. The experimental results of part segmentation verify the feasibility and effectiveness of DResNet-PointNet.

The leaf phenotypic traits of plants have a significant impact on the efficiency of canopy photosynthesis. However, traditional methods such as destructive sampling will hinder the continuous monitoring of plant growth, while manual measurements in the field are both time-consuming and laborious. Nondestructive and accurate measurements of leaf phenotypic parameters can be achieved through the use of 3D canopy models and object segmentation techniques. This paper proposed an automatic branch–leaf segmentation pipeline based on lidar point cloud and conducted the automatic measurement of leaf inclination angle, length, width, and area, using pear canopy as an example. Firstly, a three-dimensional model using a lidar point cloud was established using SCENE software. Next, 305 pear tree branches were manually divided into branch points and leaf points, and 45 branch samples were selected as test data. Leaf points were further marked as 572 leaf instances on these test data. The PointNet++ model was used, with 260 point clouds as training input to carry out semantic segmentation of branches and leaves. Using the leaf point clouds in the test dataset as input, a single leaf instance was extracted by means of a mean shift clustering algorithm. Finally, based on the single leaf point cloud, the leaf inclination angle was calculated by plane fitting, while the leaf length, width, and area were calculated by midrib fitting and triangulation. The semantic segmentation model was tested on 45 branches, with a mean Precisionsem, mean Recallsem, mean F1-score, and mean Intersection over Union (IoU) of branches and leaves of 0.93, 0.94, 0.93, and 0.88, respectively. For single leaf extraction, the Precisionins, Recallins, and mean coverage (mCoV) were 0.89, 0.92, and 0.87, respectively. Using the proposed method, the estimated leaf inclination, length, width, and area of pear leaves showed a high correlation with manual measurements, with correlation coefficients of 0.94 (root mean squared error: 4.44°), 0.94 (root mean squared error: 0.43 cm), 0.91 (root mean squared error: 0.39 cm), and 0.93 (root mean squared error: 5.21 cm2), respectively. These results demonstrate that the method can automatically and accurately measure the phenotypic parameters of pear leaves. This has great significance for monitoring pear tree growth, simulating canopy photosynthesis, and optimizing orchard management.

[Abstract] The point clouds acquired with a mobile LiDAR scanner (MLS) have high density and accuracy, which allows one to identify different elements of the road in them, as can be found in many scientific references, especially in the last decade. This study presents a methodology to characterize the urban space available for walking, by segmenting point clouds from data acquired with MLS and automatically generating impedance surfaces to be used in pedestrian accessibility studies. Common problems in the automatic segmentation of the LiDAR point cloud were corrected, achieving a very accurate segmentation of the points belonging to the ground. In addition, problems caused by occlusions caused mainly by parked vehicles and that prevent the availability of LiDAR points in spaces normally intended for pedestrian circulation, such as sidewalks, were solved in the proposed methodology. The innovation of this method lies, therefore, in the high definition of the generated 3D model of the pedestrian space to model pedestrian mobility, which allowed us to apply it in the search for shorter and safer pedestrian paths between the homes and schools of students in urban areas within the Big-Geomove project. Both the developed algorithms and the LiDAR data used are freely licensed for their use in further research. This research study was funded by the Directorate-General for Traffic of Spain, grant number SPIP2017-02340

Rain spots on green plum are superficial micro-defects. Defect detection based on a two-dimensional image is easily influenced by factors such as placement position and light and is prone to misjudgment and omission, which are the main problems affecting the accuracy of defect screening of green plum. In this paper, using computer vision technology, an improved structure from motion (SFM) and patch-based multi-view stereo (PMVS) algorithm based on similar graph clustering and graph matching is proposed to perform three-dimensional sparse and dense reconstruction of green plums. The results show that, compared with the traditional algorithm, the running time of this algorithm is lower, at only 26.55 s, and the mean values of camera optical center error and pose error are 0.019 and 0.631, respectively. This method obtains a higher reconstruction accuracy to meet the subsequent plum micro-defect detection requirements. Aiming at the dense point cloud model of green plums, through point cloud preprocessing, the improved adaptive segmentation algorithm based on the Lab color space realizes the effective segmentation of the point cloud of green plum micro-defects. The experimental results show that the average running time of the improved adaptive segmentation algorithm is 2.56 s, showing a faster segmentation speed and better effect than the traditional K-means and K-means++ algorithms. After clustering the micro-defect point cloud, the micro-defect information of green plums was extracted on the basis of random sample consensus (RANSAC) plane fitting, which provides a theoretical model for further improving the accuracy of sorting the appearance quality of green plums.

Easily accessible depth sensors have enabled using point-cloud data to augment tabletop surfaces in everyday environments. However, point-cloud operations are computationally expensive and challenging to perform in real-time, particularly when targeting embedded systems without a dedicated GPU. In this paper, we propose mitigating the high computational costs by segmenting candidate interaction regions near real-time. We contribute an open-source solution for variable depth cameras using CPU-based architectures. For validation, we employ Microsoft’s Azure Kinect and report achieved performance. Our initial findings show that our approach takes under to segment candidate interaction regions on a tabletop surface and reduces the data volume by up to 70%. We conclude by contrasting the performance of our solution against a model-fitting approach implemented by the SurfaceStreams toolkit. Our approach outperforms the RANSAC-based strategy within the context of our test scenario, segmenting a tabletop’s interaction region up to 94% faster. Our results show promise for point-cloud-based approaches, even when targeting embedded solutions with limited resources.

Three-dimensional high throughput plant phenotyping techniques provide an opportunity to measure plant organ-level traits which can be highly useful to plant breeders. The number and locations of cotton bolls, which are the fruit of cotton plants and an important component of fiber yield, are arguably among the most important phenotypic traits but are complex to quantify manually. Hence, there is a need for effective and efficient cotton boll phenotyping solutions to support breeding research and monitor the crop yield leading to better production management systems. We developed a novel methodology for 3D cotton boll mapping within a plot in situ. Point clouds were reconstructed from multi-view images using the structure from motion algorithm. The method used a region-based classification algorithm that successfully accounted for noise due to sunlight. The developed density-based clustering method could estimate boll counts for this situation, in which bolls were in direct contact with other bolls. By applying the method to point clouds from 30 plots of cotton plants, boll counts, boll volume and position data were derived. The average accuracy of boll counting was up to 90% and the R2 values between fiber yield and boll number, as well as fiber yield and boll volume were 0.87 and 0.66, respectively. The 3D boll spatial distribution could also be analyzed using this method. This method, which was low-cost and provided improved site-specific data on cotton bolls, can also be applied to other plant/fruit mapping analysis after some modification.

Light detection and ranging (LIDAR) scanning is a common method of substation scene modeling that extracts point clouds of electrical equipment from the point cloud scene of a substation. The extraction effect is limited by uncertainty regarding the noise level, nonuniform point cloud density, and the computational complexity. In this paper, we propose a point cloud extraction solution for electrical equipment models. First, a statistical analysis of substation ground elevation is performed to obtain the point clouds of devices at the feature height and remove large numbers of redundant underground point clouds. Second, based on the statistically derived power equipment feature heights, the point cloud data are sliced according to the featured elevation intervals. Based on voxelization, the point cloud slices are then clustered using horizontal hierarchical clustering. The clustering results at different elevation intervals are then reclustered using vertical hierarchical clustering. Finally, we use filters combined with the DBSCAN algorithm to perform fine segmentation on the point cloud data. The results show that our slice clustering approach reduces the computational burden involved in point cloud processing, and the comprehensive clustering strategy ensures the accuracy of the clustering results.

Most of the existing simultaneous localization and mapping (SLAM) methods are based on the static environment assumption. The presence of moving objects in the scene will lead to much uncertainty in SLAM results, which also hinders the loop-closure detection (LCD). Although moving object tracking (MOT) is necessary for planning and decisions, it is often accomplished separately. To jointly solve SLAM and MOT for complex urban driving scenarios, this paper presents a high performance method named 4D-SLAM based on the fusion of LiDAR and IMU. The integration of SLAM and MOT is formulated as a joint posterior probability problem based on a dynamic Bayesian network (DBN) and is implemented with the following four sequential stages: preprocessing, moving object detection and tracking, odometry estimation and mapping. In the preprocessing stage, the motion distortion uncertainty caused by LiDAR scanning is first compensated for, and the initial LiDAR motion is estimated. In the moving object detection and tracking stage, we exploit a CNN-based segmentation network to detect the potential moving objects first, then the states of the potential moving objects are optimized by an unscented Kalman filter (UKF). In the odometry estimation stage, the distinctive planar and edge features extracted from a static background point cloud are used for the odometry estimation, and a two-step Levenberg-Marquardt optimization method is adopted to solve the 6DOF pose across consecutive scans. In the mapping stage, the mapping based on the pose estimation result and LCD are realized, and graph-based global optimization is exploited to further improve the map consistency for large scale environment. The comprehensive experiments with the open source dataset KITTI and the data collected by us show that the presented method not only outperforms the SOTA SLAM methods in terms of trajectory and mapping accuracy but can also detect and track moving objects efficiently.

Point cloud segmentation is a significant process to organise an unstructured point cloud. In this study, RGB point cloud was generated with the help of images acquired from an Unmanned Aerial Vehicle (UAV). A dense urban area was considered with varying planar features in the built-up environment along with buildings with different floors. Initially, using Cloth Simulation Filter (CSF) filter, the ground and the non-ground features in the Point Cloud Data (PCD) were segmented, with non-ground features comprising trees and buildings and ground features comprising roads, ground vegetation, and open land. Subsequently, using CANUPO classifier the trees and building points were classified. Noise filtering removed the points which have less density in clusters. Point cloud normals were generated for the building points. For segmentation building elements, normal vector components in different directions (X component, Y component and Z component) were used to segment out the facade, and the roof points of the buildings as the surface normals corresponding to the roof will have a higher contribution in the z component of the normal vector. The validation of the segmentation is done by comparing the results with manually identified roof points and façade points in the point cloud. Overall accuracies obtained for building roof and building facade segmentation are 90.86 % and 84.83 % respectively.

Point clouds acquired using Mobile Laser Scanning (MLS) are applied to extract road information such as curb stones, road markings, and road side objects. In this paper, we present a scanline-based MLS point cloud segmentation method for various road and road side objects. First, end points of the scanline, jump edge points, and corner points are extracted as feature points. The feature points are then interpolated to accurately extract irregular parts consisting of irregularly distributed points such as vegetation. Next, using a point reduction method, additional feature points on a smooth surface are extracted for segmentation at the edges of the curb cut. Finally, points between the feature points are extracted as flat segments on the scanline, and continuing feature points are extracted as irregular segments on the scanline. Furthermore, these segments on the scanline are integrated as flat or irregular regions. In the extraction of the feature points, neighboring points based on the spatial distance are used to avoid being influenced by the difference in the point density. Based on experiments, the effectiveness of the proposed method was indicated based on an application to an MLS point cloud.

Place recognition is an essential part of simultaneous localization and mapping (SLAM). LiDAR-based place recognition relies almost exclusively on geometric information. However, geometric information may become unreliable when faced with environments dominated by unstructured objects. In this paper, we explore the role of segmentation for extracting key structured information. We propose STV-SC, a novel segmentation and temporal verification enhanced place recognition method for unstructured environments. It contains a range image-based 3D point segmentation algorithm and a three-stage process to detect a loop. The three-stage method consists of a two-stage candidate loop search process and a one-stage segmentation and temporal verification (STV) process. Our STV process utilizes the time-continuous feature of SLAM to determine whether there is an occasional mismatch. We quantitatively demonstrate that the STV process can trigger false detections caused by unstructured objects and effectively extract structured objects to avoid outliers. Comparison with state-of-art algorithms on public datasets shows that STV-SC can run online and achieve improved performance in unstructured environments (Under the same precision, the recall rate is 1.4∼16% higher than Scan context). Therefore, our algorithm can effectively avoid the mismatching caused by the original algorithm in unstructured environment and improve the environmental adaptability of mobile agents.

The integrity of shoreline is disrupted by cliffs, posing obstacles to marine surveying and chart mapping, particularly in research related to the cliff section of the sea area. The present study proposes a method to solve this problem. In the proposed method, we first extract the boundary of the cliff to segment the point cloud according to a designed rule, then calculate the centroid coordinates of each point cloud block, followed by the coordinates of side points on both sides of the boundary of each block from the centroid, and finally create a side point cloud projected to the water surface corrected for elevation. The corrected point cloud is considered a point cloud dataset of the innermost shoreline. Combined with a point cloud projected from the boundary of the cliff section to the water surface, we developed a calculation method for the optimal shoreline position. Our experiment proved that the method could effectively extract the shoreline of the study area. Moreover, compared with that of the commonly used shoreline extraction method, the absolute error of the isoline tracking method was very large, up to several meters. However, the proposed method achieved smaller standard deviation and variance (0.1254 and 0.0157, respectively) than the isoline tracking method (0.9837 and 0.9677, respectively).

Existing point cloud semantic segmentation approaches do not perform well on details, especially for the boundary regions. However, supervised-learning-based methods depend on costly artificial annotations for performance improvement. In this paper, we bridge this gap by designing a self-supervised pretext task applicable to point clouds. Our main innovation lies in the mixed feature prediction strategy during the pretraining stage, which facilitates point cloud feature learning with boundary-aware foundations. Meanwhile, a dynamic feature aggregation module is proposed to regulate the range of receptive field according to the neighboring pattern of each point. In this way, more spatial details are preserved for discriminative high-level representations. Extensive experiments across several point cloud segmentation datasets verify the superiority of our proposed method, including ShapeNet-part, ScanNet v2, and S3DIS. Furthermore, transfer learning on point cloud classification and object detection tasks demonstrates the generalization ability of our method.

Building information models (BIM) in the civil industry are very popular nowadays. The basic information of these models is the 3D geometric model of a building structure. The most applied methodology to model the existing buildings is by generating 3D geometric information from point clouds provided by laser scanners. The fundamental principle of this methodology is the recognition of structures shaped in basic geometric primitives, e.g., planes, spheres, and cylinders. The basic premise of the efficiency of this methodology is the automation of detection, since manual segmentation of a point cloud can be challenging, time-consuming, and, therefore, inefficient. This paper presents a novel algorithm for the automated segmentation of geometric shapes in point clouds without needing pre-segmentation. With the designed algorithm, structures formed in three types of basic geometrical primitive can be identified and segmented: planar (e.g., walls, floors, ceilings), spherical (e.g., laser scanner reference targets), and cylindrical (e.g., columns, pillars, piping). The RANSAC paradigm partially inspires the proposed algorithm; however, various modifications must be made. The algorithm was tested on several point clouds and was compared with the standard RANSAC algorithm; this part is described in the last section of the paper. One of the tests was performed on a double cylinder-shaped test object, the parameters (radius and height) of this object were available with high accuracy (0.1 mm), and the differences between the known and estimated parameters were below 0.5 mm in each case, indicating the correctness of the proposed algorithm. Also, a comparison with the standard RANSAC algorithm was performed, where the algorithm proposed showed better results than the standard RANSAC algorithm. The segmentation quality was, on average, increased from 50% to 100%.

PointNet, which enables end-to-end learning for scattered/unordered point data, is a popular neural network architecture. However, in many applications, large amounts of complete point clouds are hardly available for non-rigid objects such as the human body. To generate the training data of PointNet, in this study, we propose to generate human body point clouds of various postures by uniformly sampling point clouds from meshes with respect to multiple human mesh model datasets. Experiments show that the model trained with the point clouds generated from mesh data is effective in the task of human body segmentation.

A reliable and robust visual perception system is often a key-enabler for autonomous robotic systems working in at least partially unknown environment. For autonomous robotic rock breaking system, boulder detection and localization system is an essential part that must be able to operate under challenging environments. The problem can be summarized as detecting unstructured objects in a structured environment. A key to robust boulder detection is an effective clustering algorithm, that can segment a point cloud, captured by e.g., a Time-of-Flight (TOF) camera, into clusters of individual boulders. This information is crucial for an autonomous robotic secondary breaking system, that uses it to break each individual boulder by a hydraulic impact hammer. This study proposes a novel algorithm for point cloud segmentation that can be used for the boulder detection application. The method is designed using features of the Point Cloud Library (PCL), and it is benchmarked against other readily available algorithms in the PCL. The results indicate robust performance with an impressive 97.4% accuracy on our dataset. acceptedVersion

Accurately detecting the tooth profile parameters of the synchronous belt is crucial for the transmission’s load distribution and service life. However, the existing detection methods have low efficiency, are greatly affected by the manual experience, and cannot realize automatic detection. A measurement method based on point cloud data is proposed to solve this issue. The surface space points of the synchronous belt are acquired by a line-structured light sensor, and the raw point clouds are preprocessed to remove outliers and reduce the number of points. Then, the point clouds are divided into plane and arc regions, and different methods are used for fitting. Finally, the parameters of each tooth are calculated. The experimental results show that the method has high measurement accuracy and reliable stability and can replace the original detection method to realize automatic detection.

Effectively integrating the local features and their spatial distribution information for more effective point cloud analysis is a subject that has been explored for a long time. Inspired by convolutional neural networks (CNNs), this paper studies the relationship between local features and their spatial characteristics and proposes a concise architecture to effectively integrate them instead of designing more sophisticated feature extraction modules. Different positions in the feature map of the 2D image correspond to different weights in the convolution kernel, making the obtained features that are sensitive to local distribution characteristics. Thus, the spatial distribution of the input features of the point cloud within the receptive field is critical for capturing abstract regional aggregated features. We design a lightweight structure to extract local features by explicitly supplementing the distribution information of the input features to obtain distinctive features for point cloud analysis. Compared with the baseline, our model shows improvements in accuracy and convergence speed, and these advantages facilitate the introduction of the snapshot ensemble. Aiming at the shortcomings of the commonly used cosine annealing learning schedule, we design a new annealing schedule that can be flexibly adjusted for the snapshot ensemble technology, which significantly improves the performance by a large margin. Extensive experiments on typical benchmarks verify that, although it adopts the basic shared multi-layer perceptrons (MLPs) as feature extractors, the proposed model with a lightweight structure achieves on-par performance with previous state-of-the-art (SOTA) methods (e.g., MoldeNet40 classification, 0.98 million parameters and 93.5% accuracy; S3DIS segmentation, 1.4 million parameters and 68.7% mIoU).

Vision-based measurement has been studied extensively in recent decades for its potential applications in robotics. However, there still remain challenges when we aim at a fast and robust detecting system even in the presence of damaged cartons. This paper implements a non-contact volume measurement system based on point cloud geometric properties of objects for logistics parcels, which recognizes and measures different types of objects or even deformed cartons in various scenarios. Our proposed method is robust to handle the different sizes of objects using the same system. Experimental verification is performed with different cartons on the conveyor belt, results indicate that our proposed system with error percentage in volume is 5.92% with an affordable 3D camera (ASUS Xtion Pro Live).

Photogrammetry is one of the widest techniques used to monitor terrain changes which occur due to natural process and geological natural risk zones. In order to carry out terrain monitoring, it is necessary to eliminate all the non-ground elements. One of the most variable elements in this monitoring is the presence of vegetation, which obscures the ground and can significantly mislead any multitemporal analysis to detect terrain changes. Therefore, the focus of this paper is about how best to filter the vegetation to attain an accurate reading of the terrain. There are several methods to filter it based on colourising an excessive greenness vegetation index or non-visible channels as the IR in the well-known index NVDI. However, achieving this kind of information is not always possible because its high cost. Instead this channel we can add new information using the HSV colour space obtained from the RGB information. In this paper, we propose a double possibility, on one hand work wit h RGB+HSV for a supervised segmentation on images. On the other, to use excessive greenness vegetation indices and RGB+HSV for the segmentation of point clouds. The results shown that the use of additional channels HSV can significantly improve the segmentation in both studies, and therefore render a much more accurate assessment of the underlying terrain

In this paper, a novel deep learning-based point cloud segmentation model, referred to as the hierarchical grow network, is proposed. This model is based on the PointNet segmentation model.Different from PointNet, the entire point cloud is divided into several clusters, which are then combined at different hierarchical levels. This way, information from different receptive fields is used to make the final segmentation. The proposed hierarchical growing approach can also be used with other 3D feature extraction networks. Experiments performed on the indoor S3DIS database show that our proposed approach provides improvement and higher accuracy compared to the PointNet model.

The US Army is paying increased attention to the development of rapid three-dimensional (3D) reconstruction using photogrammetry and unmanned aerial vehicle (UAV) technologies for creating ...

The k-nearest neighbors (KNN) algorithm has been widely adopted to construct graph convolutional networks (GCNs) for point cloud segmentation. However, the $\ell_{2}$ norm cannot discriminate the multi-dimensional structures within a point cloud. In this paper, we propose a novel structure-aware graph construction for point clouds that compensates the $\ell_{2}$ norm with per-dimension differences of the signal. The proposed method dynamically calculates the similarity ratio to determine the dimension-based proximity of the pair of points. Consequently, it improves both the spatial and spectral GCNs with the capability of aggregating information from relevant neighbors for point cloud segmentation. As a model-agnostic method, it can be seamlessly embedded into arbitrary GCN architectures during the graph construction phase. Experimental results demonstrate that the proposed method can improve classification accuracy around thejoint areas of objects.

Road potholes affect comfort, safety, traffic condition and vehicle stability. Accurately detecting these potholes is vital for assessing the degree of pavement distress and developing road maintenance plan accordingly. This paper proposes a simple and effective pothole detection method based on 3D point cloud segmentation. Using binocular stereo vision to acquire 3D point clouds, fitting the pavement plane and then eliminating it from the 3D point clouds of road scene, we could roughly extract the pothole. K-means clustering and region growing algorithms were adopted to extract the potholes precisely. The experimental results demonstrate that our proposed method has a very good segmentation effect on scenes involving plane and target object.

Point cloud analysis has received much attention recently; and segmentation is one of the most important tasks. The success of existing approaches is attributed to deep network design and large amount of labelled training data, where the latter is assumed to be always available. However, obtaining 3d point cloud segmentation labels is often very costly in practice. In this work, we propose a weakly supervised point cloud segmentation approach which requires only a tiny fraction of points to be labelled in the training stage. This is made possible by learning gradient approximation and exploitation of additional spatial and color smoothness constraints. Experiments are done on three public datasets with different degrees of weak supervision. In particular, our proposed method can produce results that are close to and sometimes even better than its fully supervised counterpart with 10X fewer labels.

Despite photogrammetry has been longly used for 3D model production, the recent development of structure from motion based reconstructions and the spread of unmanned aerial vehicles (UAV) are increasing its usage even further. Actually, the combined use of nadir and oblique UAV photogrammetry allows to cross the bridge between close range and aerial photogrammetry, enabling the quick acquisition of images for detailed 3D model production. This paper assesses the accuracy of UAV photogrammetry in the reconstruction of a cultural heritage building, namely the Girifalco fortress (Cortona, Italy), and it evaluates advantages and issues of this kind of 3D model generation. Furthermore, this work aims also at investigating the possible use of the generated model in order to extract geometric information about the building. First, elementary geometric shapes are extracted from the photogrammetric point cloud: the results of this step can be exploited for enabling the automatic generation of compact models of the building, which can be useful for instance in 3D city modeling applications. Then, a proper analysis of the geometric information included in the extracted shapes can be used in order to extract semantic information (e.g. to classify windows and doors) and to detect damaged areas on the building walls.

Photogrammetric techniques have dramatically improved over the last few years, enabling the creation of visually compelling three-dimensional (3D) meshes using unmanned aerial vehicle image...

Laser scanning generates a point cloud from which geometries can be extracted, but most methods struggle to do this automatically, especially for the entirety of an architecturally complex building (as opposed to that of a single façade). To address this issue, this paper introduces the Improved Slicing Method (ISM), an innovative and computationally-efficient method for three-dimensional building segmentation. The method is also able to detect opening boundaries even on roofs (e.g. chimneys), as well as a building’s overall outer boundaries using a local density analysis technique. The proposed procedure is validated by its application to two architecturally complex, historic brick buildings. Accuracies of at least 86% were achieved, with computational times as little as 0.53 s for detecting features from a data set of 5.0 million points. The accuracy more than rivalled the current state of the art, while being up to six times faster and with the further advantage of requiring no manual intervention or reliance on a priori information. European Research Council

In this paper, we present a novel 3D segmentation approach operating on point clouds generated from overlapping images. The aim of the proposed hybrid approach is to effectively segment co-planar objects, by leveraging the structural information originating from the 3D point cloud and the visual information from the 2D images, without resorting to learning based procedures. More specifically, the proposed hybrid approach, H-RANSAC, is an extension of the well-known RANSAC plane-fitting algorithm, incorporating an additional consistency criterion based on the results of 2D segmentation. Our expectation that the integration of 2D data into 3D segmentation will achieve more accurate results, is validated experimentally in the domain of 3D city models. Results show that HRANSAC can successfully delineate building components like main facades and windows, and provide more accurate segmentation results compared to the typical RANSAC plane-fitting algorithm.

With widely used LiDAR sensors included in consumer electronic devices, it is increasingly convenient to acquire point cloud data, but it is also difficult to segment the point cloud data obtained from these unprofessional LiDAR devices, due to their low accuracy and high noise. To address the issue, a point cloud segmentation method using the tensor feature is proposed. The normal vectors of the point cloud are computed based on initial tensor encoding, which are further encoded into the tensor of each point. Using the tensor from a nearby point, the tensor of the center point is aggregated in all dimensions from its neighborhood. Then, the tensor feature in the point is decomposed and different dimensional shape features are detected, and the point cloud dataset is segmented based on the clustering of the tensor feature. Using the point cloud dataset acquired from the iPhone-based LiDAR sensor, experiments were conducted, and results show that both normal vectors and tensors are computed, then the dataset is successfully segmented.

Point cloud segmentation is the foundation of 3D environmental perception for modern intelligent systems. To solve this problem and image segmentation, conditional random fields (CRFs) are usually formulated as discrete models in label space to encourage label consistency, which is actually a kind of postprocessing. In this paper, we reconsider the CRF in feature space for point cloud segmentation because it can capture the structure of features well to improve the representation ability of features rather than simply smoothing. Therefore, we first model the point cloud features with a continuous quadratic energy model and formulate its solution process as a message-passing graph convolution, by which it can be easily integrated into a deep network. We theoretically demonstrate that the message passing in the graph convolution is equivalent to the mean-field approximation of a continuous CRF model. Furthermore, we build an encoder-decoder network based on the proposed continuous CRF graph convolution (CRFConv), in which the CRFConv embedded in the decoding layers can restore the details of high-level features that were lost in the encoding stage to enhance the location ability of the network, thereby benefiting segmentation. Analogous to the CRFConv, we show that the classical discrete CRF can also work collaboratively with the proposed network via another graph convolution to further improve the segmentation results. Experiments on various point cloud benchmarks demonstrate the effectiveness and robustness of the proposed method. Compared with the state-of-the-art methods, the proposed method can also achieve competitive segmentation performance., 20 pages + 8 pages (supplemental material)

In order to accurately extract the effective area of microhardness indentation obtained by laser scanning confocal microscope, the indentation point cloud segmentation method is studied based on over-segmentation using voxel cloud connectivity segmentation. Further, a microhardness indentation point cloud segmentation algorithm is proposed. The proposed method can determine the belonging area of the point cloud by combining multiple feature score models, and can achieve the segmentation and extraction of ultra-microhardness indentation data where the characteristics of indentation area are not obvious, or the data point is dense. The experimental results show that the method can extract the indentation area accurately, and the repeatability and consistency of extraction is good.

Currently, most rescue robots are mainly teleoperated and integrate some level of autonomy to reduce the operator’s workload, allowing them to focus on the primary mission tasks. One of the main causes of mission failure are human errors and increasing the robot’s autonomy can increase the probability of success. For this reason, in this work, a stair detection and characterization pipeline is presented. The pipeline is tested on a differential drive robot using the ROS middleware, YOLOv4-tiny and a region growing based clustering algorithm. The pipeline’s staircase detector was implemented using the Neural Compute Engines (NCEs) of the OpenCV AI Kit with Depth (OAK-D) RGB-D camera, which allowed the implementation using the robot’s computer without a GPU and, thus, could be implemented in similar robots to increase autonomy. Furthermore, by using this pipeline we were able to implement a Fuzzy controller that allows the robot to align itself, autonomously, with the staircase. Our work can be used in different robots running the ROS middleware and can increase autonomy, allowing the operator to focus on the primary mission tasks. Furthermore, due to the design of the pipeline, it can be used with different types of RGB-D cameras, including those that generate noisy point clouds from low disparity depth images.

As vital infrastructures, high-speed railways support the development of transportation. To maintain the punctuality and safety of railway systems, researchers have employed manual and computer vision methods to monitor overhead contact systems (OCSs), but they have low efficiency. Investigators have also used light detection and ranging (LiDAR) to generate point clouds by emitting laser beams. The point cloud is segmented for automatic OCS recognition, which improves recognition efficiency. However, existing LiDAR point cloud segmentation methods have high computational/model complexity and latency. In addition, they cannot adapt to embedded devices with different architectures. To overcome these issues, this article presents a lightweight neural network EffNet consisting of three modules: ExtractA, AttenA, and AttenB. ExtractA extracts the features from the disordered and irregular point clouds of an OCS. AttenA keeps information flowing in EffNet while extracting useful features. AttenB uses channel and spatialwise statistics to enhance important features and suppress unimportant ones efficiently. To further speed up EffNet and match it with diverse architectures, we optimized it with a generation framework of tensor programs and deployed it on embedded systems with different architectures. Extensive experiments demonstrated that EffNet has at least a 0.57% higher mean accuracy, but with 25.00% and 9.30% lower computational and model complexity for OCS recognition than others, respectively. The optimized EffNet can be adapted to different architectures. Its latency decreased by 51.97%, 56.47%, 63.63%, 82.58%, 85.85%, and 91.97% on the NVIDIA Nano CPU, TX2 CPU, UP Board CPU, Nano GPU, TX2 GPU, and RTX 2,080 Ti GPU, respectively.

Point cloud segmentation is a fundamental problem. Due to the complexity of real-world scenes and the limitations of 3D scanners, interactive segmentation is currently the only way to cope with all kinds of point clouds. However, interactively segmenting complex and large-scale scenes is very time-consuming. In this paper, we present a novel interactive system for segmenting point cloud scenes. Our system automatically suggests a series of camera views, in which users can conveniently specify segmentation guidance. In this way, users may focus on specifying segmentation hints instead of manually searching for desirable views of unsegmented objects, thus significantly reducing user effort. To achieve this, we introduce a novel view preference model, which is based on a set of dedicated view attributes, with weights learned from a user study. We also introduce support relations for both graph-cut-based segmentation and finding similar objects. Our experiments show that our segmentation technique helps users quickly segment various types of scenes, outperforming alternative methods.

Point cloud processing is a struggled field because the points in the clouds are three-dimensional and irregular distributed signals. For this reason, the points in the point clouds are mostly sampled into regularly distributed voxels in the literature. Voxelization as a pretreatment significantly accelerates the process of segmenting surfaces. The geometric cues such as plane directions (normals) in the voxels are mostly used to segment the local surfaces. However, the sampling process may include a non-planar point group (patch), which is mostly on the edges and corners, in a voxel. These voxels can cause misleading the segmentation process. In this paper, we separate the non-planar patches into planar sub-patches using k-means clustering. The largest one among the planar sub-patches replaces the normal and barycenter properties of the voxel with those of itself. We have tested this process in a successful point cloud segmentation method and measure the effects of the proposed method on two point cloud segmentation datasets (Mosque and Train Station). The method increases the accuracy success of the Mosque dataset from 83.84% to 87.86% and that of the Train Station dataset from 85.36% to 87.07%.

Publishers Note: This paper, originally published on 18 November 2019, was withdrawn 26 February 2020 for ethics violations.