Search

Graph Convolutional Neural Networks (GCNs) have demonstrated significant efficiency and flexibility in processing irregular data. And since point clouds are essentially irregular points discretely distributed in space, GCNs have great potential for application in point cloud segmentation tasks. However, current GCNs struggle to learn global structure and local details efficiently. Furthermore, their excessive use of Max pooling results in the substantial loss of contextual structural information. To address the above problems, we present a novel hierarchical graph structure learning network (SGSLNet), which mainly consists of structure-aware Adaptive Graph Convolution (GAdaptive Conv) and Stratified Contextual Graph Pooling (SCGP). GAdaptive Conv is used to dynamically learn local geometric structures, while SCGP is employed to aggregate features and model global contextual structures. Our method not only learns global structure and local details concurrently but also reduces the loss of contextual structure information. We conduct extensive experiments on various datasets, including ShapeNetPart, S3DIS, and ScanNet v2. The results show that SGSLNet achieves state-of-the-art segmentation performance. [ABSTRACT FROM AUTHOR]

Identifying shapeless granular materials in complex construction scenarios is critical for achieving automation in engineering equipment such as wheel loaders. The challenges of segmenting point clouds for granular materials involve dealing with sparsity, real-time processing requirements, the lack of distinct shape representation, and the issue of different materials sharing similar shapes. This paper proposes MVSRF, a real-time multi-view based point cloud semantic segmentation method incorporating a single-frame re-segmentation component and a multi-frame semantic filter to enhance accuracy and robustness. First, the segmentation system generates a sparse pixel-depth grid map via semantic projection to encapsulate global points and their behaviors, while employing an edge detector to label boundary points around objects. Second, a zero-shot re-segmentation algorithm involving seed extension, novel one-dimensional DBSCAN, Delaunay triangulation, and semantic reassignment corrects mis-segmented points caused by mapping bias. Finally, a lightweight semantic filter is designed to suppress semantic noise during multiple observations. We have built a multi-sensor platform on a wheel loader and collected experimental data to verify the effectiveness of our method. Two optimization components illustrated exceptional performance on the annotated dataset. The MVSRF method possesses strong robustness against external calibration errors, camera pose estimation errors, and inaccurate image segmentation, providing a practical solution for real-time perception of granular materials. [ABSTRACT FROM AUTHOR]

Film cooling is an effective technology for improving the thrust-to-weight ratio of an aero-engine. The axial orientation of film cooling holes is one of the key factors determining the film cooling effect. However, the machining quality of cooling holes is still carried out by manual inspection, whose accuracy and consistency cannot be guaranteed. To address this problem, this paper proposes an axial orientation extraction algorithm based on laser point cloud processing. Firstly, the blade surface is scanned by a line laser sensor and a blade point cloud is obtained. Secondly, a single film hole can be segmented from the blade point cloud based on scanning line fitting residual. Thirdly, an improved Gaussian map is to complement the arc belt obtained by Gaussian map to obtain a complete circle along an error-sensitive direction. Finally, the axial orientation is estimated by random sample consensus with an initial value. The accuracy and consistency of the algorithm are verified by comparing the extraction result by the algorithm with those by the Geomagic software. The result shows that the axial accuracy extracted by the algorithm can reach 0.405°. [ABSTRACT FROM AUTHOR]

Weakly-supervised point cloud segmentation with extremely limited labels is highly desirable to alleviate the expensive costs of collecting densely annotated 3D points. This paper explores applying the consistency regularization that is commonly used in weakly-supervised learning, for its point cloud counterpart with multiple data-specific augmentations, which has not been well studied. We observe that the straightforward way of applying consistency constraints to weakly-supervised point cloud segmentation has two major limitations: noisy pseudo labels due to the conventional confidence-based selection and insufficient consistency constraints due to discarding unreliable pseudo labels. Therefore, we propose a novel Reliability-Adaptive Consistency Network (RAC-Net) to use both prediction confidence and model uncertainty to measure the reliability of pseudo labels and apply consistency training on all unlabeled points while with different consistency constraints for different points based on the reliability of corresponding pseudo labels. Experimental results on the S3DIS and ScanNet-v2 benchmark datasets show that our model achieves superior performance in weakly-supervised point cloud segmentation. The code will be released publicly at https://github.com/wu-zhonghua/RAC-Net. [ABSTRACT FROM AUTHOR]

Traditional manual teaching or offline programming welding modes may lead to long teaching time, low efficiency, and inability to adapt to changing welding environments when facing complex trajectory workpieces or Multi-welded workpieces. In this paper, a teaching-free welding method based on visual sensing system for robotic is proposed. Firstly, the three-dimensional information of the workpiece surface is captured by a monocular structured light camera and characterized by point cloud data. Point cloud stitching is performed on multiple local images of large-size workpieces to reconstruct the welding surface. Then, statistical filtering and deep learning methods are used to preprocess and segment the point cloud to obtain the reference points of the welding path. Finally, according to the shape characteristics of the workpiece, the auxiliary projection method is used to automatically generate the robot surfacing path. Experimental results show that under the condition of camera accuracy of ± 0.05 mm, the maximum planning path error is less than 1 mm, which meets the actual welding needs. This method is significant for achieving welding automation and improving production efficiency. [ABSTRACT FROM AUTHOR]

Point cloud classification and segmentation is a crucial yet challenging step towards 3D scene understanding. In response to the fact that 3D point clouds cannot be directly applied to traditional 2D image convolutional neural networks and sparse point cloud features are difficult to capture, an improved segmentation network Point-Attention Net is proposed. In comparison with existing techniques, the proposed method models on point cloud directly, avoiding the extra computational and memory cost of converting point cloud to other forms. It also employs a novel proportional discrete dilation model for analyzing the features of a point cloud on different scales simultaneously. Additionally, the proposed method combines the graph attention convolution and adaptive weight assignment techniques, yielding better segmentation performance on the edge point cloud and more accurate analysis of the adjacency and spatial geometric distribution. Experimental results show that our algorithm achieves 64.56 % and 74.1 % IoU on average on S3DIS and Semantic3D datasets respectively, outperforming the state of the art. [ABSTRACT FROM AUTHOR]

The problem-solving success of an Evolutionary Computing algorithm is too sensitive to the structures of mutation and crossover operators it uses. The mutation operator generates the trial vectors necessary for the relevant Evolutionary Computing method to perform efficient global and local search in the search space. Partial elitist mutation operators can produce more efficient trial vectors than isotropic mutation strategies. Another numerical genetic operator that affects the process of producing efficient trial vectors is crossover. Due to the dominant effect of the Evolutionary Computing algorithms of mutation and crossover operators on problem solving capacity, new numerical-genetic operators are constantly being developed. When solving a problem with the Differential Evolution Algorithm determining the ideal mutation operator and setting the initial values of the internal parameters of the crossover operator is quite time-consuming and difficult. In this paper, the Multi-population Based Differential Evolution Algorithm (MDE) has been proposed to solve real-valued numerical optimization problems with its convergence proof. The mutation operator of MDE is partial—elitist and its crossover operator is parameter-free, in practice. In this paper, 28 benchmark problems of CEC2013 with Dim = 20 and one real-world geometric optimization problem have been used in the experiments performed to examine the numerical problem-solving success of MDE. MDE's success in solving related benchmark problems has been statistically compared with ABC, CK, SOS and GWO. Statistical analysis of the results obtained from the experiments exposed that MDE is statistically more successful than comparison methods in solving numerical optimization problems used. [ABSTRACT FROM AUTHOR]

The data-hungry nature of deep learning and the high cost of annotating point-level labels make it difficult to apply semantic segmentation methods to indoor point cloud scenes. Therefore, exploring how to make point cloud segmentation methods less rely on point-level labels is a promising research topic. In this paper, we introduce a weakly supervised framework for semantic segmentation on indoor point clouds. To reduce the labor cost in data annotation, we use image-level weak labels that only indicate the classes that appeared in the rendered images of point clouds. The experiments validate the effectiveness and scalability of our framework. Our segmentation results on both ScanNet and S3DIS datasets outperform the state-of-the-art method using a similar level of weak supervision. [ABSTRACT FROM AUTHOR]

Point cloud segmentation is the premise and basis of many 3D perception tasks, such as intelligent driving, object detection and recognition, scene recognition and understanding. In this paper, we present an improved PointNet for 3D object part Segmentation, and named the proposed PointNet as Deep Residual Neural Network Based PointNet (DResNet-PointNet). The architecture of DResNet- PointNet was desigined based on the idea of residual networks. Residual networks can increase the depth of the DResNet-PointNet without network degradation. The depth of DResNet-PointNet is twice as deep as that of original PointNet model. Increasing the depth of DResNet-PointNet can improve its ability to express complex functions and generalization ability of complex classification problems, and achieve better approximation of complex functions, thus improving the accuracy of segmentation. The experimental results of part segmentation verify the feasibility and effectiveness of DResNet-PointNet. [ABSTRACT FROM AUTHOR]

Tree species segmentation is an essential condition for research forestry and has a large impact on forest resource monitoring, sustainable forest management, and biodiversity research. Recently, the development of hardware and software has been rapidly increasing. Regarding hardware, the active remote sensing system LiDAR can be used to obtain many point clouds and can significantly improve the tree segmentation accuracy compared with traditional optical remote sensing hardware. With respect to software, deep learning theory is effectively utilized to process 3D point clouds, such as extracting the features of data. However, deep learning-based methods are underutilized in tree species point cloud segmentation. Therefore, it is extremely important to combine current technological advantages for this application. In this article, we construct a point cloud processing dataset that comprises substantial tree information and 5 tree species, including willow, fir, bamboo, palm, and rubber. The novel representation of point clouds via a superpoint graph is utilized to pre-process the point clouds in a large outdoor scene. We propose to apply state-of-the-art deep learning frameworks, including PointNet network and graph convolution networks, to process tree species point clouds in complex forest scenes. We also discuss the effectiveness of the method and the situations influenced by different parameters. The experimental results finally verify the effectiveness of the framework in tree species segmentation. [ABSTRACT FROM AUTHOR]

In this paper, we propose a method for generating a hierarchical, volumetric topological map from 3D point clouds. There are three basic hierarchical levels in our map: s t o r e y - r e g i o n - v o l u m e . The advantages of our method are reflected in both input and output. In terms of input, we accept multi-storey point clouds and building structures with sloping roofs or ceilings. In terms of output, we can generate results with metric information of different dimensionality, that are suitable for different robotics applications. The algorithm generates the volumetric representation by generating volumes from a 3D voxel occupancy map. We then add passages (connections between volumes), combine small volumes into a big region and use a 2D segmentation method for better topological representation. We evaluate our method on several freely available datasets. The experiments highlight the advantages of our approach. [ABSTRACT FROM AUTHOR]

Segmentation of 3D point clouds is still an open issue in the case of unbalanced and in-homogeneous data-sets. In the application context of the modeling of botanical trees, a fundamental challenge consists in separating the leaves from the wood. Based on deep learning and a class decision process, we propose an innovative method designed to separate leaf points from wood points in terrestrial LiDAR point clouds of trees. Although simple, our approach learns trees characteristic point patterns efficiently and robustly. To train our 3D deep learning model, we constructed a 3D labeled point cloud data-set of different tree species. Experiments show that our 3D deep representation together with our geometric approach leads to significant improvement over the state-of-the-art methods in segmentation task. [ABSTRACT FROM AUTHOR]

The vertical structure of forests affects energy transfer and material exchange within forest ecosystems and is of great significance to scientific forestry and ecology research. In this paper, the Shangri-La forest plot in northwestern Yunnan Province of China was the study area. The forest sample plot point cloud data were obtained by terrestrial laser scanning technology. A new method for classifying the vertical structure of forest sample plots based on point cloud data is proposed. The method comprehensively utilizes morphological filtering and comparative shortest-path (CSP) algorithm point cloud segmentation technology. Additionally, the method proposes the concept of secondary CSP segmentation that precisely classifies three types of vertical features in forest point cloud data: trees, shrubs and the ground. Finally, an accuracy analysis showed that the error rate of the tree results was 1.87%, and the error rate of the shrub results was 16.23%. [ABSTRACT FROM AUTHOR]

In order to realize real-time 3D environment perception of UAVs and autopilot in low-altitude complex road scenes, a neural network model RobNet based on SqueezeNet and cyclic CRF for real-time 3D point cloud segmentation is proposed to segment the road objects in real time. Firstly, the unordered, scattered 3D point cloud data are preprocessed into a standard data format similar to an image by a spherical mapping method. Then, at the macro-level of the model design, the SqueezeNet network with the residual connection optimization is selected as the basic network of the model, and then, the conditional random field (CRF) algorithm which is processed into the cyclic network structure is used to refine the segmentation result. Finally, the construction of the basic network, the cyclic network and the network parameter settings in the model is elaborated at the micro-level. The experimental results show that the RobNet model proposed in this paper can segment the target in the road scene better. The segmentation callback rate of the three types of vehicles, pedestrians and cyclists is increased by 28, 2 and 17%, respectively, compared with the VoxelNet network. The higher callback rate is in line with the safe movement specifications for drones and autonomous driving. At the same time, the proposed model parameters are small, 98.5% smaller than the classic network AlexNet, and are easy to deploy on a platform with limited computing resources. The RobNet model in the Robot Operating System (ROS) framework engineering deployment and implementation experimental data shows that the model meets the real-time and stability requirements of the drone and automatic driving application, engineering code can run in real time at 12 Hz, the standard deviation of each frame's running time is around 4.5 ms. [ABSTRACT FROM AUTHOR]

This paper presents a system for online learning of human classifiers by mobile service robots using 3D LiDAR sensors, and its experimental evaluation in a large indoor public space. The learning framework requires a minimal set of labelled samples (e.g. one or several samples) to initialise a classifier. The classifier is then retrained iteratively during operation of the robot. New training samples are generated automatically using multi-target tracking and a pair of "experts" to estimate false negatives and false positives. Both classification and tracking utilise an efficient real-time clustering algorithm for segmentation of 3D point cloud data. We also introduce a new feature to improve human classification in sparse, long-range point clouds. We provide an extensive evaluation of our the framework using a 3D LiDAR dataset of people moving in a large indoor public space, which is made available to the research community. The experiments demonstrate the influence of the system components and improved classification of humans compared to the state-of-the-art. [ABSTRACT FROM AUTHOR]

In this paper, a transmission method of the 3D point cloud data of the object upper body is proposed. The key idea of the method is to reduce the amount of transmission data in the condition of retaining the necessary information. In our system, the unnecessary information is removed by using filters and the segmentation algorithm. Three P frames (Predicted frame) are inserted between two I frames (Intra frame) to further improve the transmission rate. The I frame of large size includes all the vision information that is related to the object upper body, while the P frame includes only the information of the small change in the current frame, whose size is smaller than the former. In order to acquire the information of the P frame, first, a human face detection algorithm based on the machine learning technology is implemented. Then, a coarse-to-fine approach is used to detect the nose tip which helps to improve the precision of acquiring the human faces information. The experiment result demonstrates that the proposed method is able to reduce the time delay of processing and transmission and to lower the compression ratio with an allowable distortion rate. [ABSTRACT FROM AUTHOR]

A novel strategy is presented to determine the next-best view for a robot arm, equipped with a depth camera in eye-in-hand configuration, which is oriented to autonomous exploration of unknown objects. Instead of maximizing the total size of the expected unknown volume that becomes visible, the next-best view is chosen to observe the border of incomplete objects. Salient regions of space that belong to the objects are detected, without any prior knowledge, by applying a point cloud segmentation algorithm. The system uses a Kinect V2 sensor, which has not been considered in previous works on next-best view planning, and it exploits KinectFusion to maintain a volumetric representation of the environment. A low-level procedure to reduce Kinect V2 invalid points is also presented. The viability of the approach has been demonstrated in a real setup where the robot is fully autonomous. Experiments indicate that the proposed method enables the robot to actively explore the objects faster than a standard next-best view algorithm. [ABSTRACT FROM AUTHOR]

Graph-based variational methods have recently shown to be highly competitive for various classification problems of high-dimensional data, but are inherently difficult to handle from an optimization perspective. This paper proposes a convex relaxation for a certain set of graph-based multiclass data segmentation models involving a graph total variation term, region homogeneity terms, supervised information and certain constraints or penalty terms acting on the class sizes. Particular applications include semi-supervised classification of high-dimensional data and unsupervised segmentation of unstructured 3D point clouds. Theoretical analysis shows that the convex relaxation closely approximates the original NP-hard problems, and these observations are also confirmed experimentally. An efficient duality-based algorithm is developed that handles all constraints on the labeling function implicitly. Experiments on semi-supervised classification indicate consistently higher accuracies than related non-convex approaches and considerably so when the training data are not uniformly distributed among the data set. The accuracies are also highly competitive against a wide range of other established methods on three benchmark data sets. Experiments on 3D point clouds acquire by a LaDAR in outdoor scenes and demonstrate that the scenes can accurately be segmented into object classes such as vegetation, the ground plane and human-made structures. [ABSTRACT FROM AUTHOR]

Point cloud segmentation is a fundamental problem. Due to the complexity of real-world scenes and the limitations of 3D scanners, interactive segmentation is currently the only way to cope with all kinds of point clouds. However, interactively segmenting complex and large-scale scenes is very time-consuming. In this paper, we present a novel interactive system for segmenting point cloud scenes. Our system automatically suggests a series of camera views, in which users can conveniently specify segmentation guidance. In this way, users may focus on specifying segmentation hints instead of manually searching for desirable views of unsegmented objects, thus significantly reducing user effort. To achieve this, we introduce a novel view preference model, which is based on a set of dedicated view attributes, with weights learned from a user study. We also introduce support relations for both graph-cut-based segmentation and finding similar objects. Our experiments show that our segmentation technique helps users quickly segment various types of scenes, outperforming alternative methods. [ABSTRACT FROM AUTHOR]