Search

Three-dimensional reconstruction from point clouds is an important research topic in computer vision and computer graphics. However, the discrete nature, sparsity, and noise of the original point cloud contribute to the results of 3D surface generation based on global features often appearing jagged and lacking details, making it difficult to describe shape details accurately. We address the challenge of generating smooth and detailed 3D surfaces from point clouds. We propose an adaptive octree partitioning method to divide the global shape into local regions of different scales. An iterative loop method based on GRU is then used to extract features from local voxels and learn local smoothness and global shape priors. Finally, a moving least-squares approach is employed to generate the 3D surface. Experiments demonstrate that our method outperforms existing methods on benchmark datasets (ShapeNet dataset, ABC dataset, and Famous dataset). Ablation studies confirm the effectiveness of the adaptive octree partitioning and GRU modules. [ABSTRACT FROM AUTHOR]

An essential task for 3D visual world understanding is 3D object detection in lidar point clouds. To predict directly bounding box parameters from point clouds, existing voting-based methods use Hough voting to obtain the centroid of each object. However, it may be difficult for the inaccurately voted centers to regress boxes accurately, leading to the generation of redundant bounding boxes. For objects in indoor scenes, there are several co-occurrence patterns for objects in indoor scenes. Concurrently, semantic relations between object layouts and scenes can be used as prior context to guide object detection. We propose a simple, yet effective network, RSFF-Net, which adds refined voting and scene feature fusion for indoor 3D object detection. The RSFF-Net consists of three modules: geometric function, refined voting, and scene constraint. First, a geometric function module is used to capture the geometric features of the nearest object of the voted points. Then, the coarse votes are revoted by a refined voting module, which is based on the fused feature between the coarse votes and geometric features. Finally, a scene constraint module is used to add the association information between candidate objects and scenes. RSFF-Net achieves competitive results on indoor 3D object detection benchmarks: ScanNet V2 and SUN RGB-D. [ABSTRACT FROM AUTHOR]

Landsat-like moderate resolution remote sensing images are widely used in land use and land cover (LULC) classification. Limited by coarser resolutions, most of the traditional LULC classifications that are based on moderate resolution remote sensing images focus on the spectral features of a single pixel. Inspired by the spatial evaluation methods in landscape ecology, this study proposed a new method to extract neighborhood characteristics around a pixel for moderate resolution images. 3 landscape-metric-like indexes, i.e., mean index, standard deviation index, and distance weighted value index, were defined as adjacent region features to include the surrounding environmental characteristics. The effects of the adjacent region features and the different feature set configurations on improving the LULC classification were evaluated by a series of well-controlled LULC classification experiments using K nearest neighbor (KNN) and support vector machine (SVM) classifiers on a Landsat 8 Operational Land Imager (OLI) image. When the adjacent region features were added, the overall accuracies of both the classifiers were higher than when only spectral features were used. For the KNN and SVM classifiers that used only spectral features, the overall accuracies of the LULC classification were 85.45% and 88.87%, respectively, and the accuracies were improved to 94.52% and 96.97%. The classification accuracies of all the LULC types improved. Highly heterogeneous LULC types that are easily misclassified achieved greater improvements. As comparisons, the grey-level co-occurrence matrix (GLCM) and convolutional neural network (CNN) approaches were also implemented on the same dataset. The results revealed that the new method outperformed GLCM and CNN approaches and can significantly improve the classification performance that is based on moderate resolution data. [ABSTRACT FROM AUTHOR]

Breast tumor segmentation of ultrasound images provides valuable tumor information for early detection and diagnosis. However, speckle noise and blurred boundaries in breast ultrasound images present challenges for tumor segmentation, especially for malignant tumors with irregular shapes. Recent vision transformers have shown promising performance in handling the variation through global context modeling. Nevertheless, they are often dominated by features of large patterns and lack the ability to recognize negative information in ultrasound images, which leads to the loss of breast tumor details (e.g., boundaries and small objects). In this paper, we propose a novel noise suppression and boundary refinement network, NSBR-Net, to simultaneously alleviate speckle noise interference and blurred boundary problems of breast tumor segmentation. Specifically, we propose two innovative designs, namely, the Noise Suppression Module (NSM) and the Boundary Refinement Module (BRM). The NSM filters noise information from the coarse-grained feature maps, while the BRM progressively refines the boundaries of significant lesion objects. Our method demonstrates superior accuracy over state-of-the-art deep learning models, achieving significant improvements of 3.67% on Dataset B and 2.30% on the BUSI dataset in mDice for testing malignant tumors. [ABSTRACT FROM AUTHOR]

Multi-view Stereo (MVS) meshes suffer from occlusions or missing data, making most surface reconstruction methods invalid. Due to data imperfections, existing methods, just like PolyFit, have made a trade-off between reconstruction accuracy and time consumption. We propose a novel approach that automatically reconstructs a building surface model from raw triangular mesh with data incompleteness. Unlike existing methods that extract high-quality primitives, our method focuses on assemblies of primitives to control the level of geometric detail in the reconstructed models. We design a topological relationship of the primitives to form a plane connection graph. To be precise, we first combine the scalability of shapes with plane primitives. Then strong and soft connections between primitives are constructed by calculating the confidence of the plane intersection in space. Furthermore, the topological relations of all primitives are encoded into an undirected graph. Finally, a watertight and manifold model is extracted from the faces of a candidate set by energy minimization. Experiments on the Helsinki 3D dataset demonstrate the superiority of our method in time consumption and reconstruction error, as measured by Hausdorff distance. Our method outperforms other primitive-based algorithms in handling non-planar structures. Even when dealing with imperfect data, a watertight model is still obtained. [Display omitted] • Due to the occlusion caused by various reasons such as trees, we designed a lightweight data structure and general strategy to rectify the building data defects. A confidence strategy combining graph structure is proposed, which reduces the time consumption compared with the state-of-art methods and captures more accurate geometric details within the reconstruction pipeline to generate building models with higher levels of detail. [ABSTRACT FROM AUTHOR]

Although learning-based stereo matching methods have achieved remarkable performance, accurately recovering disparity maps for boundary areas (e.g., thin structures) remains an intractable issue. Existing stereo pipelines usually employ the standard L1 loss function, averaging pixel-wise losses equally despite variable difficulty. This mechanism inevitably overwhelms thin structures due to their limited proportion. In this paper, we propose reweighting the L1 loss to focus on pixels of varying hardness. First, we explicitly model the uncertainty of each pixel to gauge the confidence in its prediction. By aggregating the volume, uncertainty is obtained effortlessly. Second, uncertainty is mapped to weights, which reweight the L1 loss accordingly. The core of our approach lies in leveraging uncertainty to personalize the loss map adjustment, progressively optimizing challenging regions during training. Notably, our method requires no extra parameters or inference computations. Finally, we introduce the Boundary Pixel Error (BPE), a novel metric targeting boundary quality. Extensive experiments with the SceneFlow, KITTI 2012, and KITTI 2015 datasets demonstrate the effectiveness and universality of our elegant framework, seamlessly integrating it into existing models as a plug-and-play component, leading to substantial performance improvements. [Display omitted] [ABSTRACT FROM AUTHOR]

[Display omitted] • A feature extraction module is proposed to extract the geometric and semantic features of each point efficiently. Then, the attention mechanism is deployed to aggregate the learned features. • The proposed ResDLPS-Net is optimized by joint training of residual connections and dense convolutional connections. • Experiments demonstrate that ResDLPS-Net outperforms the state-of-the-art deep learning networks on the indoor dataset S3DIS and the outdoor large-scale dataset Toronto-3D. Significantly, the Mean Intersection over Union (mIoU) of ResDLPS-Net on the Toronto-3D dataset is 80.27%. Semantic segmentation methods based on three-dimensional (3D) point clouds are mostly limited to input point clouds that have been divided into blocks for training. This is mainly attributed to the requirement of constant trade-offs between computational resources and accuracy for directly processing large-scale point clouds. Specifically, the block dividing strategy will add the data preprocessing time to some extent and may disturb the complete geometry of the object. Therefore, this paper proposes a large-scale point cloud semantic segmentation network without block dividing operation, referred to as ResDLPS-Net. This network can take the complete point cloud of the whole large scene as input and process up to nearly a million points on one single GPU. In particular, a novel feature extraction module is designed to efficiently extract neighbor, geometric, and semantic features. The learned features are then aggregated through the attention mechanism to form local feature descriptors. In addition, the proposed ResDLPS-Net is jointly trained by residual connections and dense convolutional connections to optimize the feature aggregation operation. As a result, the ResDLPS-Net performs brilliantly on multiple objects, such as windows, road markings, fences, etc. For example, the Mean Intersection over Union (mIoU) of road markings on the Toronto-3D dataset is 37.76% higher than the state-of-the-art algorithm. Moreover, this paper outperforms most deep learning methods on three well-known benchmark datasets, including the indoor dataset S3DIS and the outdoor large-scale scene datasets Semantic3D and Toronto-3D. The proposed ResDLPS-Net achieves the best performance on the S3DIS dataset. The average accuracy (mA) and overall accuracy (OA) are 82.3% and 88.1%, respectively. Notably, the proposed ResDLPS-Net attains a mIoU of 80.27% on the Toronto-3D dataset, which is 6.00% higher than the best results published currently. [ABSTRACT FROM AUTHOR]

Object detection is a challenging computer vision task with numerous real-world applications. In recent years, the concept of the object relationship model has become helpful for object detection and has been verified and realized in deep learning. Nonetheless, most approaches to modeling object relations are limited to using the anchor-based algorithms; they cannot be directly migrated to the anchor-free frameworks. The reason is that the anchor-free algorithms are used to eliminate the complex design of anchors and predict heatmaps to represent the locations of keypoints of different object categories, without considering the relationship between keypoints. Therefore, to better fuse the information between the heatmap channels, it is important to model the visual relationship between keypoints. In this paper, we present a knowledge-driven network (KDNet)—a new architecture that can aggregate and model keypoint relations to augment object features for detection. Specifically, it processes a set of keypoints simultaneously through interactions between their local and geometric features, thereby allowing the modeling of their relationship. Finally, the updated heatmaps were used to obtain the corners of the objects and determine their positions. The experimental results conducted on the RIDER dataset confirm the effectiveness of the proposed KDNet, which significantly outperformed other state-of-the-art object detection methods. [ABSTRACT FROM AUTHOR]

The semantic segmentation of small objects in point clouds is currently one of the most demanding tasks in photogrammetry and remote sensing applications. Multi-resolution feature extraction and fusion can significantly enhance the ability of object classification and segmentation, so it is widely used in the image field. For this motivation, we propose a point cloud semantic segmentation network based on multi-scale feature fusion (MSSCN) to aggregate the feature of a point cloud with different densities and improve the performance of semantic segmentation. In our method, random downsampling is first applied to obtain point clouds of different densities. A Spatial Aggregation Net (SAN) is then employed as the backbone network to extract local features from these point clouds, followed by concatenation of the extracted feature descriptors at different scales. Finally, a loss function is used to combine the different semantic information from point clouds of different densities for network optimization. Experiments were conducted on the S3DIS and ScanNet datasets, and our MSSCN achieved accuracies of 89.80% and 86.3%, respectively, on these datasets. Our method showed better performance than the recent methods PointNet, PointNet++, PointCNN, PointSIFT, and SAN. [ABSTRACT FROM AUTHOR]