Search

The three-dimensional (3D) modeling and recognition of 3D objects have been traditionally performed using local features to represent the underlying 3D surface. Extraction of features requires cropping of several local surface patches around detected keypoints. Although an important step, the extraction and representation of such local patches adds to the computational complexity of the algorithms. This paper proposes a novel Keypoints-based Surface Representation (KSR) technique. The proposed technique has the following two characteristics: ( 1 ) It does not rely on the computation of features on a small surface patch cropped around a detected keypoint. Rather, it exploits the geometrical relationship between the detected 3D keypoints for local surface representation. ( 2 ) KSR is computationally efficient, requiring only seconds to process 3D models with over 50,000 points with a MATLAB implementation. Experimental results on the UWA and Stanford 3D models dataset suggest that it can accurately perform pairwise and multiview range image registration (3D modeling). KSR was also tested for 3D object recognition with occluded scenes. Recognition results on the UWA dataset show that the proposed technique outperforms existing methods including 3D-Tensor, VD-LSD, keypoint-depth based feature, spherical harmonics and spin image with a recognition rate of 95.9%. The proposed approach also achieves a recognition rate of 93.5% on the challenging Ca'Fascori dataset compared to 92.5% achieved by game-theoretic. The proposed method is computationally efficient compared to state-of-the-art local feature methods. [ABSTRACT FROM AUTHOR]

This paper tackles the problem of feature matching and range image registration. Our approach is based on a novel set of discriminating three-dimensional (3D) local features, named 3D-Vor ( Vor ticity). In contrast to conventional local feature representation techniques, which use the vector field (i.e. surface normals) to just construct their local reference frames, the proposed feature representation exploits the vorticity of the vector field computed at each point of the local surface to capture the distinctive characteristics at each point of the underlying 3D surface. The 3D-Vor descriptors of two range images are then matched using a fully automatic feature matching algorithm which identifies correspondences between the two range images. Correspondences are verified in a local validation step of the proposed algorithm and used for the pairwise registration of the range images. Quantitative results on low resolution Kinect 3D data (Washington RGB-D dataset) show that our proposed automatic registration algorithm is accurate and computationally efficient. The performance evaluation of the proposed descriptor was also carried out on the challenging low resolution Washington RGB-D (Kinect) object dataset, for the tasks of automatic range image registration. Reported experimental results show that the proposed local surface descriptor is robust to resolution, noise and more accurate than state-of-the-art techniques. It achieves 90% registration accuracy compared to 50%, 69.2% and 52% for spin image, 3D SURF and SISI/LD-SIFT descriptors, respectively. [ABSTRACT FROM AUTHOR]

Deep neural networks (DNNs) can be easily deceived by imperceptible alterations known as adversarial examples. These examples can lead to misclassification, posing a significant threat to the reliability of deep learning systems in real-world applications. Adversarial training (AT) is a popular technique used to enhance robustness by training models on a combination of corrupted and clean data. However, existing AT-based methods often struggle to handle transferred adversarial examples that can fool multiple defense models, thereby falling short of meeting the generalization requirements for real-world scenarios. Furthermore, AT typically fails to provide interpretable predictions, which are crucial for domain experts seeking to understand the behavior of DNNs. To overcome these challenges, we present a novel approach called Jacobian norm and Selective Input Gradient Regularization (J-SIGR). Our method leverages Jacobian normalization to improve robustness and introduces regularization of perturbation-based saliency maps, enabling interpretable predictions. By adopting J-SIGR, we achieve enhanced defense capabilities and promote high interpretability of DNNs. We evaluate the effectiveness of J-SIGR across various architectures by subjecting it to powerful adversarial attacks. Our experimental evaluations provide compelling evidence of the efficacy of J-SIGR against transferred adversarial attacks, while preserving interpretability. The project code can be found at https://github.com/Lywu-github/jJ-SIGR.git. • An approach based on Jacobian norm and selective input gradient regularization. • The proposed approach can improve robustness and interpretability of DNNs. • Insights to reveal the relationship between Jacobian norm and linear robustness. • Experiments are conducted on a variety of attacks to prove the effectiveness. [ABSTRACT FROM AUTHOR]

• A novel concept and framework is introduced to directly match 2D and 3D features. • The framework is used for 6-DOF pose estimation, and camera and visual localization. • Current techniques usually rely on Structure from Motion models for localization. • Proposed framework can be used for any type of 3D models and point clouds. • The method outperforms state-of-the-art techniques on benchmark datasets. This paper presents a novel approach for cross-domain descriptor matching between 2D and 3D modalities. The 2D-3D matching is applied to localize 2D images in 3D point clouds. Direct cross-domain matching allows our technique to localize images in any type of 3D point cloud without any constraints on the nature or mechanism by which it is obtained. We propose a learning based framework, called Desc-Matcher, to directly match features between the two modalities. A dataset of 2D and 3D features with corresponding locations in images and point clouds is generated to train the Desc-Matcher. To estimate the pose of an image in any 3D cloud, keypoints and feature descriptors are extracted from the query image and the point cloud. The trained Desc-Matcher is then used to match the features from the image and the point cloud. A robust pose estimator is used to predict the location and orientation of the query image from the corresponding positions of the matched 2D and 3D features. We carried out an extensive evaluation of the proposed method for indoor and outdoor scenarios and with different types of point clouds to verify the feasibility of our approach. Experimental results show that the proposed approach can reliably estimate the 6-DOF poses of query cameras in any type of 3D point cloud with high precision. We achieved average median errors of 1.09 c m / 0. 27 ∘ and 19 c m / 0. 39 ∘ on the Stanford and Cambridge datasets, respectively. [ABSTRACT FROM AUTHOR]

• We introduce a multi-stage information diffusion strategy to achieve interactions for joint-task learning through the proposed end-to-end deep neural network (MIDNet). • We design a Cross-Correlation Attention Module (CCAM) to effectively capture interactive information at different levels between depth and surface normal estimations. • Our MIDNet boasts state-of-the-art performance for joint monocular depth estimation and surface normal estimation on the NYU Depth V2 and Stanford 2D-3D-Semantic datasets, and achieves similar performance to prior art on the KITTI dataset. Depth and surface normal estimations are important for 3D geometric perception, which has numerous applications including autonomous vehicles and robots. In this paper, we propose a lightweight Multi-stage Information Diffusion Network (MIDNet) for the simultaneous prediction of depth and surface normals from a single RGB image. To obtain semantic and detail-preserving features, we adopt a high-resolution network as our backbone to learn multi-scale features, which are then fused into shared features for the two tasks. To mutually boost each task, a Cross-Correlation Attention Module (CCAM) is proposed to adaptively integrate information for the prediction of the two tasks in multiple stages, including feature-level information interaction and task-level information interaction. Ablation studies show that the proposed multi-stage information diffusion strategy can boost the performance gain for the two tasks at different levels. Compared to current state-of-the-art methods on the NYU Depth V2, Stanford 2D-3D-Semantic and KITTI datasets, our method achieves superior performance for both monocular depth and surface normal estimation. [ABSTRACT FROM AUTHOR]