Research on the Monocular 3D Perception System for Autonomous-rail Rapid Tram

The 3D perception system functions as one of the core components for the safe operation of autonomous-rail rapid tram. Addressing the limitations of LiDAR systems used for autonomous-rail rapid tram in complex urban environments, such as challenges in perceiving distant objects, insensitivity to color, and potential failure, this paper proposes a pure-vision monocular 3D perception system for autonomous-rail rapid tram. Comprising data preprocessing, model training and model deployment, this vision-based scheme with a full coverage from data collection to on-board deployment enables the 3D perception of obstacles around autonomous-rail rapid tram, thereby improving the operational reliability in complex urban environments. The test results based on the autonomous-rail rapid transit dataset and Waymo open dataset show that the system can effectively perceive the complex road scenes in autonomous-rail rapid transit, achieving a final 3D average precision (AP) of 0.53, reasoning a single-frame of image about 56 ms, and meeting the real-time requirements of autonomous-rail rapid tram for the obstacle perception algorithm.