Search

Construction sites are highly hazardous due to the dynamic interaction between workers and moving equipment, with high fatality rates caused by collision and falling from height, etc. Hence, identifying unsafe behaviors among workers is crucial for enhancing site safety, such as tracking their on-site movement and personal protective equipment (PPE). Vision-based video processing has been actively used to automatically recognize workers and their behaviors on construction sites. However, existing studies mainly monitor workers within a single camera capturing only a small sub-region. As workers typically move around fairly large sites, continuously tracking their movement across multiple cameras would enable more comprehensive behavioral analyses. Hence, this paper proposes a framework for monitoring safety compliance among workers, by combining worker re-identification (ReID) and PPE classification. Deep learning-based approaches are developed to address the challenges for these two tasks respectively. For ReID, a new loss function named similarity loss is designed to encourage deep learning models to learn more discriminative human features, realizing a more robust tracking of individual workers. For classifying PPE statuses, a weighted-class strategy is proposed to mitigate model bias when given imbalanced samples among classes, for improved performance despite limited training samples. By combining the ReID and PPE classification results, a workflow is developed to log any incident of workers not wearing the necessary PPEs. With an actual construction site dataset, the proposed methods improve worker ReID and PPE classification by 4% and 13% accuracies respectively, which will facilitate site video analytics and inspection of site safety compliance among workers. • Designed a similarity loss to learn discriminative human features for worker ReID • Designed a weighted-class strategy for PPE classification with imbalanced samples • Monitored worker PPE statuses with consistent identities across multiple cameras • Developed an incident logging workflow to monitor worker PPE compliance on site • Improved ReID and PPE classification by 4% and 13% accuracies over existing CNNs [ABSTRACT FROM AUTHOR]

Construction sites are among the most hazardous places with various safety issues. The high rate of hazards on construction sites can be attributed to the dynamic and complex characteristics of construction-related entities, such as the movement of construction equipment and workers as well as the interactions among them. Tracking construction equipment and workers can help avoid potential collisions and other accidents to achieve safer on-site conditions. As construction equipment (e.g. excavators, trucks, cranes, and bulldozers) plays a significant role in construction projects, it is important to track the location, pose and movement of construction equipment. Currently, with the wide installation of surveillance cameras on construction sites, computer vision techniques are explored to process the captured surveillance videos and images, such as to monitor the site conditions and prevent potential hazards. Previous studies have attempted to identify and locate different types of construction equipment on construction sites based on surveillance videos using computer vision techniques. However, there are limited studies that automatically estimate the full body pose and movement of on-site construction equipment, which can greatly influence the safety condition of construction sites and the utilization of the equipment itself. In this study, a methodology framework is developed for automatically estimating the poses of different construction equipment in videos captured on construction sites using computer vision and deep learning techniques. Firstly, keypoints of equipment are defined, based on which the images collected from the surveillance cameras are annotated to generate the ground truth labels. 70%, 10%, and 20% of the annotated image dataset are used for training, validation and testing, respectively. Then, the architectures of three types of deep learning networks i.e. Stacked Hourglass Network (HG), Cascaded Pyramid Network (CPN), and an ensemble model (HG-CPN) integrating Stacked Hourglass and Cascaded Pyramid Network are constructed and trained in the same training environment. After training, the three models are evaluated on the testing dataset in terms of normalized errors (NE), percentage of correct keypoints (PCK), area under the curve (AUC), detection speed, and training time. The experiment results demonstrate the promising performance of our proposed methodology framework for automatically estimating different full body poses of construction equipment with high accuracy and fast speed. It is indicated by experiments that both HG and CPN can achieve relative high accuracy, with a PCK value of 91.19% and 91.78% respectively for estimating the equipment full body poses. In addition, the ensemble model with online data augmentation can further improve the accuracy, achieving a NE of 14.57 × 10−3, a PCK of 93.43%, and an AUC of 39.72 × 10−3 at the detection speed of 125 millisecond (ms) per image. This study lays the foundation for applying computer vision and deep learning techniques in the full body pose estimation of construction equipment, which can contribute to the real-time safety monitoring on construction sites. • A methodology framework is proposed to estimate on-site equipment full body poses. • Equipment keypoints are defined and an equipment pose dataset is built and labeled. • Two models (HG and CPN) achieved accuracies over 90% for estimating equipment full body pose. • The ensemble model (HG-CPN) further improved the full body pose estimation accuracy. • HG, CPN, and HG-CPN are compared, and insights are provided for potential improvement. [ABSTRACT FROM AUTHOR]

To reduce machine-related accidents on sites, automatically monitoring the full-body poses of operating heavy machines is crucial. Conventional pose estimation systems relying on homogeneous sensors are vulnerable to negative environmental impacts, leading to inaccurate and unstable estimation of machine states. Hence, a full-body pose estimation framework is proposed for excavators, with a data fusion strategy to utilize different types of onboard sensors for enhanced accuracy and robustness. Specifically, a non-invasive onboard visual-inertial sensor system is designed for data fusion. Then, through competitive and complementary data fusion, the keypoints describing the full-body poses of the excavator are tracked in 3D space. Especially, an EKF-based localization algorithm is developed for optimized multi-keypoint tracking, which is verified to improve the accuracy and robustness of pose estimation by a real-world excavator case study. The proposed sensor-fusion method can effectively improve operational safety, by accurately monitoring the motion of heavy machines operating on construction sites. • A pose estimation framework for excavators proposed using visual-inertial sensor fusion method. • A non-invasive onboard multi-sensor system designed for data fusion to track motions. • An EKF-based algorithm developed to track multi-keypoint of excavators competitively. • Improved accuracy and robustness in keypoint tracking, confirmed by real-life excavator case study. [ABSTRACT FROM AUTHOR]

Monitoring unsafe operations of construction machines is important for reducing accidents on sites. To automate safety monitoring, this paper studies pose estimation of construction machines, which describes their motions in 3D space. Inertial Measurement Unit (IMU) is considered accurate, cost-saving and user-friendly for estimating the pose of machines. Nevertheless, the complexity and instability of construction sites hinder the application of IMU to construction machines. Hence, this study first defines the information requirements of IMU-based full-body pose estimation for construction machines. Then, an efficient method is developed to generate the trajectories of a machine based on kinematics. Moreover, an optimal installation scheme for IMU sensors is investigated systematically to optimize the location and number of IMUs to install. Overall, this study contributes an IMU-based method to provide full-body poses of machines without environmental constraints on real construction sites, and a theoretical basis of optimal IMU installation strategy for reliable motion-related analyses. • The information requirements are defined on IMU-based analysis of construction machine poses. • A trajectory generation method is proposed to describe the change of joint angles. • The practicability and performance of the proposed approach are fully verified in real scenarios. • The average RMSE between the estimated pose and ground truth is 1.71 degrees. • An optimal IMU installation strategy is proposed including location and number. [ABSTRACT FROM AUTHOR]