Search

3D object detection is a crucial and complex undertaking in the realm of 3D scene comprehension. Monocular-based 3D detectors, in comparison to LiDAR 3D detectors that utilize point clouds as input, often exhibit a significant performance gap. Incorporating guidance from LiDAR-based detectors has led to notable advancements in monocular 3D detection. Nevertheless, some current approaches focus solely on transferring feature or response knowledge from LiDAR data, neglecting the valuable geometric structural information present in the LiDAR data. As a result, there is untapped potential to fully exploit the geometric relationships within the LiDAR data to further improve monocular 3D detection performance. In this paper, we propose a High-order Structural Relation Distillation Networks, which establishes the geometric structure information in the BEV feature level and distills the relation from LiDAR Detectors to Monocular Image Detectors. Specifically, to better align the structural relation with the LiDAR BEV features, we propose the Image BEV Feature Enhancing Module, which can capture essential spatial depth context via explicitly modeling the interdependence between the depth channels of BEV features to improve the representations. In addition, instead of only distillation at feature level, our distillation model contains point, line, and surface relation distillation, which captures high-order structural relations from LiDAR BEV features and enhances the detection of the image-based student model, particularly for non-rigid object detection. Our method achieves state-of-the-art performance, especially in the case of non-rigid Cyclist class.

Deep learning techniques have largely solved the problem of rail surface defect detection (SDD), however, two aspects have yet to be addressed. In most existing approaches, two red–green–blue and depth (RGB-D) streams are indiscriminately fused across modalities, ignoring the fact that RGB and depth images produce different feature qualities in different scenes. Additionally, in their focus on performance, previous studies have overlooked the fact that models produce several parameters, resulting in unrealistic practical applications. To address these challenges, we designed a modal evaluation network (MENet) via knowledge distillation (KD) (MENet-S*) for a no-service rail SDD to adaptively manage information in each scenario and achieve model compression. First, to dynamically adjust the feature distribution and quality, dynamic and static feature coding ideas are introduced. Second, modal evaluation distillation is introduced, which allows a compact model (MENet-S) to learn the feature evaluation process of a complex model (MENet-T). Third, to enable MENet-S to learn the dynamic encoding process of MENet-T and to improve the feature representation of MENet-S, we propose accessible knowledge distillation. Furthermore, multitiered KD is introduced to facilitate the learning of MENet-S. Based on extensive experiments using the industrial RGB-D dataset NEU RSDDS-AUG, we observed that MENet-S* (MENet-S with KD) outperformed 16 state-of-the-art methods. In addition, to demonstrate the generalization capability of MENet-S*, we evaluated the proposed network on three additional public datasets, and MENet-S* achieved competitive results. The source codes and results are available at https://github.com/hjklearn/MENet-KD.

Owing to the expansion in processing of industrial information through advances in machine learning, the demand for accurate crowd counting in various applications is increasing. We propose a multimodality cross-guided compensation coordination network (MC3Net) for accurate red–green–blue and thermal (RGB-T) crowd counting. The network includes modules of intricate interactive fusion, feature difference compensation, and complementary attention enhancement. We use ConvNext as the backbone and process the three streams from RGB, thermal, and spliced RGB-T inputs. The multimodality data are sequentially guided and fused hierarchically, fully combining features extracted from the RGB and thermal images. Thereafter, difference compensation is applied to compress fusion and splicing features. Redundant information is removed. Then, feature mismatch is mitigated to enhance complementary information, reduce the loss of details, and finally obtain crowd statistics. Results from extensive experiments on the RGBT-CC dataset indicate the robustness and effectiveness of MC3Net, which also achieves high performance on the DroneRGBT dataset and ShanghaiTechRGBD dataset, outperforming existing crowd counting methods. The code and models are available at: https://github.com/WBangG/MC3Net.

Detecting 3D pedestrian from point cloud data in real-time while accounting for scale is crucial in various robotic and autonomous driving applications. Currently, the most successful methods for 3D object detection rely on voxel-based techniques, but these tend to be computationally inefficient for deployment in aerial scenarios. Conversely, the pillar-based approach exclusively employs 2D convolution, requiring fewer computational resources, albeit potentially sacrificing detection accuracy compared to voxel-based methods. Previous pillar-based approaches suffered from inadequate pillar feature encoding. In this letter, we introduce a real-time and scale-aware 3D Pedestrian Detection, which incorporates a robust encoder network designed for effective pillar feature extraction. The Proposed TriFocus Attention module (TriFA), which integrates external attention and similar attention strategies based on Squeeze and Exception. By comprehensively supervising the point-wise, channel-wise, and pillar-wise of pillar features, it enhances the encoding ability of pillars, suppresses noise in pillar features, and enhances the expression ability of pillar features. The proposed Bidirectional Scale-Aware Feature Pyramid module (BiSAFP) integrates a scale-aware module into the multi-scale pyramid structure. This addition enhances its ability to perceive pedestrian within low-level features. Moreover, it ensures that the significance of feature maps across various feature levels is fully taken into account. BiSAFP represents a lightweight multi-scale pyramid network that minimally impacts inference time while substantially boosting network performance. Our approach achieves real-time detection, processing up to 30 frames per second (FPS).