Search

Stones are one of the primary objects that impede the normal activity of underground pipelines. As human intervention is difficult inside a narrow underground pipe, a robot with a machine vision system is required. In order to remove the stones during regular robotic inspections, precise stone detection, segmentation, and measurement of their distance from the robot are needed. We applied Mask R-CNN to perform an instant segmentation of stones. The distance between the robot and the segmented stones was calculated using spatial information obtained from a lidar camera. Artificial light was used for both image acquisition and testing, as natural light is not available inside the underground pipe. ResNet101 was chosen as the foundation of the Mask R-CNN, and transfer learning was utilized to shorten the training time. The experimental results of our model showed that the average detection precision rate reached 92.0; the recall rate was 90.0%; and the F1 score rate reached 91.0%. The distance values were calculated efficiently with an error margin of 11.36 mm. Moreover, the Mask R-CNN-based stone detection model can detect asymmetrically shaped stones in complex background and lighting conditions. [ABSTRACT FROM AUTHOR]

Stones are one of the primary objects that impede the normal activity of underground pipelines. As human intervention is difficult inside a narrow underground pipe, a robot with a machine vision system is required. In order to remove the stones during regular robotic inspections, precise stone detection, segmentation, and measurement of their distance from the robot are needed. We applied Mask R-CNN to perform an instant segmentation of stones. The distance between the robot and the segmented stones was calculated using spatial information obtained from a lidar camera. Artificial light was used for both image acquisition and testing, as natural light is not available inside the underground pipe. ResNet101 was chosen as the foundation of the Mask R-CNN, and transfer learning was utilized to shorten the training time. The experimental results of our model showed that the average detection precision rate reached 92.0; the recall rate was 90.0%; and the F1 score rate reached 91.0%. The distance values were calculated efficiently with an error margin of 11.36 mm. Moreover, the Mask R-CNN-based stone detection model can detect asymmetrically shaped stones in complex background and lighting conditions.

• Computer vision-based displacement measurement system is prone to errors induced by the camera ego-motion. • This study proposes a novel method for measuring structural displacement in full-scale civil structures, allowing for the repositioning of the camera using RGB and LiDAR cameras. • Displacement is measured with an RGB camera and compensated for errors induced by camera motion using a LiDAR camera. • The proposed method is validated in semi-outdoor tests by comparing it with the conventional computer vision-based method. Displacement measurements based on computer vision techniques have shown significant potential in structural health monitoring. However, as the inevitable camera ego-motion results in a drift in the measured displacement, compensation for such errors must be performed carefully. Although an additional RGB camera is considered in conventional methods for error compensation, the need for the sub-target limits its application. This study presents a dual-camera system, consisting of RGB and light detection and ranging (LiDAR) cameras, that enables accurate camera ego-motion compensation for long-term structural displacement measurements. In particular, the LiDAR camera of the proposed system can measure the six-degrees-of-freedom (6-DOF) ego-motion of the camera body regardless of the sub-target and its elaborate design. The accuracy of the proposed method is evaluated by comparing it with that of a conventional RGB camera. The resulting displacement can be employed to monitor the long-term behavior of full-scale bridges. [ABSTRACT FROM AUTHOR]