Search

The three-dimensional (3D) modeling and recognition of 3D objects have been traditionally performed using local features to represent the underlying 3D surface. Extraction of features requires cropping of several local surface patches around detected keypoints. Although an important step, the extraction and representation of such local patches adds to the computational complexity of the algorithms. This paper proposes a novel Keypoints-based Surface Representation (KSR) technique. The proposed technique has the following two characteristics: ( 1 ) It does not rely on the computation of features on a small surface patch cropped around a detected keypoint. Rather, it exploits the geometrical relationship between the detected 3D keypoints for local surface representation. ( 2 ) KSR is computationally efficient, requiring only seconds to process 3D models with over 50,000 points with a MATLAB implementation. Experimental results on the UWA and Stanford 3D models dataset suggest that it can accurately perform pairwise and multiview range image registration (3D modeling). KSR was also tested for 3D object recognition with occluded scenes. Recognition results on the UWA dataset show that the proposed technique outperforms existing methods including 3D-Tensor, VD-LSD, keypoint-depth based feature, spherical harmonics and spin image with a recognition rate of 95.9%. The proposed approach also achieves a recognition rate of 93.5% on the challenging Ca'Fascori dataset compared to 92.5% achieved by game-theoretic. The proposed method is computationally efficient compared to state-of-the-art local feature methods. [ABSTRACT FROM AUTHOR]

• A novel concept and framework is introduced to directly match 2D and 3D features. • The framework is used for 6-DOF pose estimation, and camera and visual localization. • Current techniques usually rely on Structure from Motion models for localization. • Proposed framework can be used for any type of 3D models and point clouds. • The method outperforms state-of-the-art techniques on benchmark datasets. This paper presents a novel approach for cross-domain descriptor matching between 2D and 3D modalities. The 2D-3D matching is applied to localize 2D images in 3D point clouds. Direct cross-domain matching allows our technique to localize images in any type of 3D point cloud without any constraints on the nature or mechanism by which it is obtained. We propose a learning based framework, called Desc-Matcher, to directly match features between the two modalities. A dataset of 2D and 3D features with corresponding locations in images and point clouds is generated to train the Desc-Matcher. To estimate the pose of an image in any 3D cloud, keypoints and feature descriptors are extracted from the query image and the point cloud. The trained Desc-Matcher is then used to match the features from the image and the point cloud. A robust pose estimator is used to predict the location and orientation of the query image from the corresponding positions of the matched 2D and 3D features. We carried out an extensive evaluation of the proposed method for indoor and outdoor scenarios and with different types of point clouds to verify the feasibility of our approach. Experimental results show that the proposed approach can reliably estimate the 6-DOF poses of query cameras in any type of 3D point cloud with high precision. We achieved average median errors of 1.09 c m / 0. 27 ∘ and 19 c m / 0. 39 ∘ on the Stanford and Cambridge datasets, respectively. [ABSTRACT FROM AUTHOR]