Search

The ever increasing volume of visual data due to recent advances in smart devices and camera-equipped platforms provides an unprecedented opportunity to visually capture actual status of construction sites at a fraction of cost compared to other alternatives methods. Most efforts on documenting as-built status, however, stay at collecting visual data and updating BIM. Hundreds of images and videos are captured but most of them soon become useless without properly being localized with plan document and time. To take full advantage of visual data for construction performance analytics, three aspects (reliability, relevance, and speed) of capturing, analyzing, and reporting visual data are critical. This paper 1) investigates current strategies for leveraging emerging big visual data and BIM in construction performance monitoring from these three aspects, 2) characterizes gaps in knowledge via case studies and structures a road map for research in visual sensing and analytics. [ABSTRACT FROM AUTHOR]

This paper presents a new appearance-based material classification method for monitoring construction progress deviations at the operational-level. The method leverages 4D Building Information Models (BIM) and 3D point cloud models generated from site photologs using Structure-from-Motion techniques. To initialize, a user manually assigns correspondences between the point cloud model and BIM, which automatically brings in the photos and the 4D BIM into alignment from all camera viewpoints. Through reasoning about occlusion, each BIM element is back-projected on all images that see that element. From these back-projections, several 2D patches are sampled per element and are classified into different material types. To perform material classification, the expected material type information is derived from BIM. Then the image patches are convolved with texture and color filters and their concatenated vector-quantized responses are compared with multiple discriminative material classification models that are relevant to the expected progress of that element. For each element, a quantized histogram of the observed material types is formed and the material type with the highest appearance frequency infers the appearance and thus the state of progress. To validate, four new datasets of incomplete and noisy point cloud models are introduced which are assembled from real-world construction site images and BIMs. An extended version of the Construction Material Library (CML) is also introduced for training/testing the material classifiers. The material classification shows an average accuracy of 92.4% for CML image patches of 100 × 100 pixels. The experiments on those four datasets show an accuracy of 95.9%, demonstrating the potential of appearance-based recognition method for inferring the actual state of construction progress for BIM elements. [ABSTRACT FROM AUTHOR]

Accurate and rapidly produced 3D models of the as-built environment can be significant assets for a variety of Engineering scenarios. Starting with a point cloud of a scene—generated using laser scanners or image-based reconstruction methods—the user must first identify collections of points that belong to individual surfaces, and then, fit surfaces and solid geometry objects appropriate for the analysis. When performed manually, this task is often prohibitively time consuming and, in response, several research groups have recently focused on developing methods for automating the modeling process. Due to the limitations of the data collection processes as well as the complexity of as-built scenes, automated 3D modeling still presents many challenges. To overcome existing limitations, in this paper, we propose a new region growing method for robust context-free segmentation of unordered point clouds based on geometrical continuities. In our method, the user sets a single parameter which accounts for the desired level of abstraction. We treat this parameter as a locally adaptive threshold to account for local context. Our method of segmentation starts with a multi-scale feature detection, describing surface roughness and curvature around each 3D point, and is followed by seed finding and region growing steps. Experimental results from seven challenging point clouds of the built environment demonstrate that our method can account for variability in point cloud density, surface roughness, curvature, and clutter within a single scene. [ABSTRACT FROM AUTHOR]

The ability to import building geometric and construction thermal data from building information models (BIM) has significant potential to reduce the time and uncertainty in building energy modeling process. In today's BIM-based energy modeling practice, thermal properties are mainly derived from generic building construction types in BIM. However, for energy modeling of existing buildings, such assumptions are often inaccurate as they do not account for diminishing thermal resistances of building materials instigated by their deteriorations. To improve the reliability of BIM-based energy modeling, we present a system, together with new methods for automated association and updating of actual thermal property measurements with BIM elements in gbXML schema. By leveraging collections of digital and thermal images and based on environmental measurements, our system first produces a 3D thermal model for the building under inspection and then derives the actual thermal resistances of the building assemblies at the level of 3D vertexes. By associating these measurements with their corresponding elements in gbXML, thermal properties of the BIM elements are automatically updated. Our experiments in real-world residential and instructional buildings show how actual thermal properties can be automatically associated with BIM elements and updated in gbXML. The proposed method shortens the gap between architectural information in BIM and the actual data needed for energy performance simulation, and enables reliable BIM-based energy analysis for retro-commissioning, continuous commissioning, and retrofit. [ABSTRACT FROM AUTHOR]

This paper presents a non-parametric image parsing method for segmentation and recognition of roadway assets such as traffic signs, traffic lights, pavement markings, and guardrails from 2D car-mounted video streams. The method can be easily scaled to thousands of video frames captured during data collection and does not need training. Instead, it retrieves a set of most relevant video frames (e.g. highway vs. secondary road) which serve as candidates for superpixel-level annotation. It then obtains superpixels from the video frames and using the retrieval set encodes their visual characteristics using a histogram of different shape, appearance, and color descriptors. Neighborhood contexts are incorporated by using Markov Random Field (MRF) optimization and two types of semantic (e.g. guardrail) and geometric (e.g. horizontal) labels are simultaneously assigned to the superpixels. We introduce a new dataset from I-57 together with its ground truth and present experimental results on both I-57 and SmartRoad datasets. Experimental results with an average accuracy of 88.24% for recognition and 82.02% for segmentation show that our local visual features provide acceptable performance, while the method overall does not require any significant supervised training. This scalable method has potential to reduce the time and effort required for developing road inventories, especially for those such as guardrails and traffic lights that are not typically considered in 2D asset recognition methods. [ABSTRACT FROM AUTHOR]

Highlights: [•] We propose a robust vision-based method for material detection from single images. [•] A new Construction Materials Library containing 20 typical construction materials. [•] We achieve accuracies of 97.1% for high quality 200 by 200 pixel color images. [•] We maintain accuracies above 90% for images as small as 30 by 30 pixels. [•] We maintain accuracies above 92% for highly compressed, low quality images. [ABSTRACT FROM AUTHOR]

Highlights: [•] We present a computer vision based method for equipment action recognition. [•] Our vision-based method is based on a multiple binary SVM classifier and spatio-temporal features. [•] A comprehensive real-world video dataset of excavator and truck actions is presented. [•] We achieve accuracies of 86.33% and 98.33% for excavator and truck action classes. [•] The presented method can be used for construction activity analysis using long sequences of videos. [Copyright &y& Elsevier]

Highlights: [•] Image-based 3D modeling method of actual building energy performance is presented. [•] Building environments and their energy performances are jointly visualized in 3D. [•] Automatically 3D-registered thermal images assist with localizing energy problems. [•] 3D spatio-thermal models can facilitate energy building diagnostics and retrofit analysis. [Copyright &y& Elsevier]

Abstract: Building energy performance simulation tools such as EnergyPlus, Ecotect, and eQuest are widely used to model energy performance of existing buildings and assess retrofit alternatives. Nevertheless, predictions from simulations typically deviate from actual measurements. Monitoring actual performance and measuring deviations from simulated data in 3D can help improve simulation accuracy through model calibrations, and in turn facilitate identification of energy performance problem. To do that, this paper presents Energy Performance Augmented Reality (EPAR) modeling that leverages collections of unordered digital and thermal imagery, in addition to computational fluid dynamics (CFD) models. First, users collect large numbers of digital and thermal imagery from the building under inspection using a single thermal camera. Through an image-based reconstruction pipeline, actual 3D spatio-thermal models are automatically generated and are superimposed with expected building energy performance models generated using CFD analysis through a user-driven process. The outcomes are EPAR models which visualize actual and expected models in a common 3D environment. Within the EPAR models, actual measurements and simulated results can be systematically compared and analyzed. The method is validated on typical residential and instructional buildings. The results demonstrate that EPAR models facilitate calibration of building energy performance models and support detection and analysis of building performance deviations. [Copyright &y& Elsevier]

Abstract: This paper presents a computer vision based algorithm for automated 2D detection of construction workers and equipment from site video streams. The state-of-the-art research proposes semi-automated detection methods for tracking of construction workers and equipment. Considering the number of active equipment and workers on jobsites and their frequency of appearance in a camera''s field of view, application of semi-automated techniques can be time-consuming. To address this limitation, a new algorithm based on Histograms of Oriented Gradients and Colors (HOG+C) is proposed. Our proposed detector uses a single sliding window at multiple scales to identify the potential candidates for the location of equipment and workers in 2D. Each detection window is first divided into small spatial regions and then the gradient orientations and hue–saturation colors are locally histogrammed and concatenated to form the HOG+C descriptors. Tiling the sliding detection window with a dense and overlapping grid of formed descriptors and using a binary Support Vector Machine (SVM) classifier for each resource enables automated 2D detection of workers and equipment. A new comprehensive benchmark dataset containing over 8000 annotated video frames including equipment and workers from different construction projects is introduced. This dataset contains a large range of pose, scale, background, illumination, and occlusion variation. Our preliminary results on detection of standing workers, excavators and dump trucks with an average accuracy of 98.83%, 82.10%, and 84.88% respectively indicate the applicability of the proposed method for automated activity analysis of workers and equipment from single video cameras. Unlike other state-of-the-art algorithms in automated resource tracking, this method particularly detects idle resources and does not need manual or semi-automated initialization of the resource locations in 2D video frames. The experimental results and the perceived benefits of the proposed method are discussed in detail. [Copyright &y& Elsevier]

Abstract: Accurate and rapid assessment of the as-built status on any construction site provides the opportunity to understand the current performance of a project easily and quickly. Rapid project assessment further identifies discrepancies between the as-built and as-planned progress, and facilitates decision making on the necessary remedial actions. Currently, manual visual observations and surveying are the most dominant data capturing techniques but they are time-consuming, error-prone, and infrequent, making quick and reliable decision-making difficult. Therefore, research on new approaches that allow automatic recognition of as-built performance and visualization of construction progress is essential. This paper presents and compares two methods for obtaining point cloud models for detection and visualization of as-built status for construction projects: (1) A new method of automated image-based reconstruction and modeling of the as-built project status using unordered daily construction photo collections through analysis of Structure from Motion (SfM); (2) 3D laser scanning and analysis of the as-built dense point cloud models. These approaches provide robust means for recognition of progress, productivity, and quality on a construction site. In this paper, an overview of the newly developed automated image-based reconstruction approach and exclusive features which distinct it from other image-based or conventional photogrammetric techniques is presented. Subsequently the terrestrial laser scanning approach carried out for reconstruction and comparison of as-built scenes is presented. Finally the accuracy and usability of both of these techniques for metric reconstruction, automated production of point cloud models, 3D CAD shape modeling, and as-built visualizations is evaluated and compared on eight different case studies. It is shown that for precise defect detection or alignment tasks, image-based point cloud models may not be as accurate and dense as laser scanners'' point cloud models. Nonetheless image-based point cloud models provide an opportunity to extract as-built semantic information (i.e., progress, productivity, quality and safety) through the content of the images, are easy to use, and do not need add burden on the project management teams by requiring expertise for data collection or analysis. Finally image-based reconstruction automatically provides photo alignment with point cloud models and enables image-based renderings which can remarkably impact automated performance monitoring and as-built visualizations. [Copyright &y& Elsevier]

Generating 3D models from point cloud data is a common Virtual Design and Construction (VDC) service. Research has focused on automating several key steps, including segmenting point clouds based on appearance and geometric attributes. These methods dominantly use Manhattan-World assumptions in their approach. Hence, in cases where building systems are close to architectural/structural elements, these methods result in over-segmentation and require significant fine-tuning by the users. To overcome these limitations, this paper presents a learning method based on Markov Random Field (MRF) that assigns semantic labels to point cloud segment. The MRF enforces coherence between the semantic (e.g., beam, column, wall, ceiling, floor, pipe) and geometric labels (e.g., horizontal, vertical, cylindrical), and it uses the neighborhood context to enhance the semantic labeling accuracy. Experimental results show an average accuracy of 90% on semantic labeling, achieving state-of-the-art performance on labeling beam, ceiling, column, floor, pipe, and wall elements. [Display omitted] • MRF-based method for semantic segmentation of indoor point cloud scenes. • Inference via neighborhood context improves semantic segmentation of point clouds. • Coherence between semantic and geometric information enhances labeling accuracy. • The pipeline, CRF and MRF, reduces over-segmentation in region-growing methods. [ABSTRACT FROM AUTHOR]

• A method to learn planning & scheduling knowledge as Dynamic Process Templates (DPT). • A sequential machine learning learns planning and scheduling knowledge for the DPTs. • For an input sequence of activities, DPTs generate predecessor & successor activities. • The DPTs predict previously seen & unseen activity dependencies at 98% & 72% accuracy. Today, construction planning and scheduling is almost always performed manually, by experienced practitioners. The knowledge of those individuals is materialized, maintained, and propagated through master schedules and look-ahead plans. While historical project schedules are available, manually mining their embedded knowledge to create generic work templates for future projects or revising look-ahead schedules is very difficult, time-consuming and error-prone. The rigid work templates from prior research are also not scalable to cover the inter and intra-class variability in historical schedule activities. This paper aims at fulfilling these needs via a new method to automatically learn construction knowledge from historical project planning and scheduling records and digitize such knowledge in a flexible and generalizable data schema. Specifically, we present Dynamic Process Templates (DPTs) based on a novel vector representation for construction activities where the sequencing knowledge is modeled with generative Long Short-Term Memory Recurrent Neural Networks (LSTM-RNNs). Our machine learning models are exhaustively tested and validated on a diverse dataset of 32 schedules obtained from real-world projects. The experimental results show our method is capable of learning planning and sequencing knowledge at high accuracy across different projects. The benefits for automated project planning and scheduling, schedule quality control, and automated generation of project look-aheads are discussed in detail. [ABSTRACT FROM AUTHOR]

This paper presents a new benchmark dataset for validating vision-based methods that automatically identifies visually distinctive working activities of excavators and dump trucks from individual frames of a video sequence. Our dataset consists of 10 videos of interacting pairs of construction equipment filmed at ground level with accompanying ground truth annotations. These annotations consist of per-equipment and per-frame equipment bounding boxes that also have associated identities and activity labels. Our videos depict an excavator interacting with 1 or more dump trucks. We also propose a deep learning-based method for detecting and tracking objects based on Convolutional Neural Networks (CNNs). The tracking trajectories are fed into a Hidden Markov Model (HMM) that automatically discovers and assigns activity labels for any observed object. Our HMM method leverages trajectories to train a Gaussian Mixture Model (GMM) with which we estimate the probability density function of each activity using Support Vector Machine (SVM) classifiers. The proposed HMM also models activity duration and the transition between activities. We show that our method can accurately distinguish between individual equipment working activities. Results show 97.43% detection Average Precision (AP) for excavators and 75.29% AP for dump trucks, as well as cross-category tracking accuracy of 81.94% and tracking precision of 87.45%. Separate experiment results show activity analysis results of 86.8% accuracy for excavators and 88.5% for dump trucks. Our results show that our method can accurately conduct activity analysis and can be fused with methods that detect motion trajectories to scale to the needs of practical applications. Unlabelled Image • We introduce a benchmark dataset for detection, tracking and frame-level action recognition of earthmoving operations. • We also develop a method for detecting, tracking and categorizing actions of excavators and dump trucks from videos. • Our method is potent for all three subtasks and requires neither part annotations nor manual object track initialization. [ABSTRACT FROM AUTHOR]

Creating consistency among project schedule data, BIM, and payment applications requires activities in a construction schedule to be mapped with the most relevant ASTM Uniformat classifications. To do so, we introduce UniformatBridge , a new transformer-based natural language processing model, that automatically labels activities in a project schedule with Uniformat classification. Our model introduces construction sequencing tokens that capture logistically-constrained predecessor and successor activities into BERT architecture. We also introduce a dataset of real-world construction project schedules with their ground-truth Uniformat classifications for validation. Experimental results using this dataset achieve F1-scores of 0.93 and 0.87 when matching unstructured schedule data to Uniformat Level 2 and 3 classifications, respectively. We share how our method unlocks development of new techniques to (1) automatically create 4D BIM, and (2) computer-vision progress monitoring to tie semantic segmentation of reality capture data based on Uniformat classes against schedule or payment application data structures, with/without BIM. • A new NLP transformer model to automatically map schedule activities to ASTM UniFormat classes. • Sequencing knowledge is embedded in the model via concatenating [pred] and [succ] tokens. • Method achieves 93% and 87% F1-Scores in mapping UniFormat II Level 2 & 3 classes with schedule activities. • Our method serves as a universal identifier enabling automated creation of 4D BIMs. • Our method streamlines mapping between schedule, cost and payment application data. [ABSTRACT FROM AUTHOR]

Recent advances in reality capture technology focused on automating reality capture and devising robust computational models to convert the collected data into usable formats. However, these modern approaches are still challenged by the insufficient capacity of the resulting information to relay accurate and complete representations of the construction state due to the data's poor visual quality. In addition, the complexity of robotic path planning –especially for indoor construction– hinders automatic visual data collection due to cluttered, narrow, and dynamic construction spaces. This work targets both challenges by presenting a reinforcement learning model that optimizes indoor data collection policy for acquiring high-quality visual data using camera-equipped unmanned ground rovers. Results from three learned navigation policies show the capability of the method to provide high visual quality for the collected data. The learned policies reduced the data collection duration by 38.23% on average compared to the currently used automatic data collection strategies. The policies also provided a 31.04% average reduction in data collection distance compared to lawn-mower patterns. [Display omitted] • Reinforcement learning is used to learn a navigation policy maximizing data's visual quality and minimizing capture duration. • The optimized navigation policies are evaluated for data's visual quality and safe data collection navigation. • Divide-and-conquer algorithm reduces the optimization's computational complexity providing solutions in a timely manner. • Learnt capture plans achieve maximum visual quality in less duration than traditional collection plans. • Reinforcement learning provides ∼200% improvement in data collection duration compared to state-of-the-art NPV methods. [ABSTRACT FROM AUTHOR]

[Display omitted] • A NLP transformer method for predicting implicit construction dependencies with 91% F1-Score. • A process for augmenting schedules with the role and flexibility of activity relationships. • A method for revising lookahead plans based on the flexibility of activity dependencies. • A Constrained Condition model for creating optimal lookahead plan revisions. Revising lookahead plans is practiced in all construction projects to meet contractual deadlines, mitigate activity delays, and disentangle process bottlenecks. We present a new machine learning-based method –which we call Implicit Logic Checker (ILC)– to learn the implicit dependency constraints and the flexibility of construction schedule relationships using a Transformer language model architecture. We then leverage relationships' flexibilities to provide a basis for creating optimal lookahead plan revisions to help mitigate the effects of unavoidable activity delays. We then deploy a Constrained Conditional (CC) model that applies learned construction planning knowledge to build revised lookahead plans and optimizes them for consistency and duration. Our ILC model is trained and tested on 35,332 manually labeled predecessor-successor relationships from eight real construction projects achieving an F1-Score of 91%. Similarly, the revised lookahead plans generated by the CC model on two case studies show reduced overall plan durations. [ABSTRACT FROM AUTHOR]

Over the last few years, new methods that detect construction progress deviations by comparing laser scanning or image-based point clouds with 4D BIM are developed. To create complete as-built models, these methods require the visual sensors to have proper line-of-sight and field-of-view to building elements. For reporting progress deviations, they also require Building Information Modeling (BIM) and schedule Work-Breakdown-Structure (WBS) with high Level of Development (LoD). While certain logics behind sequences of construction activities can augment 4D BIM with lower LoDs to support making inferences about states of progress under limited visibility, their application in visual monitoring systems has not been explored. To address these limitations, this paper formalizes an ontology that models construction sequencing rationale such as physical relationships among components. It also presents a classification mechanism that integrates this ontology with BIM to infer states of progress for partially and fully occluded components. The ontology and classification mechanism are validated using a Charrette test and by presenting their application together with BIM and as-built data on real-world projects. The results demonstrate the effectiveness and generality of the proposed ontology. It also illustrates how the classification mechanism augments 4D BIM at lower LoDs and WBS to enable visual progress assessment for partially and fully occluded BIM elements and provide detailed operational-level progress information. [ABSTRACT FROM AUTHOR]

The visibility of a traffic sign at night depends on its retro-reflectivity, a property that needs to be frequently monitored to ensure transportation safety. In the U.S., Federal Highway Administration (FHWA) maintains regulations to ensure minimum retro-reflectivity levels. Current measurement techniques either (a) use vehicle mounted device during the night, or (b) use manual handheld devices during the day. The former is expensive due to nighttime labor cost. The latter is time-consuming and unsafe. To address these limitations, this paper presents a computer vision-based technique to measure retro-reflectivity during daytime using a vehicle mounted device. The presented algorithms simulate nighttime visibility of traffic signs from images taken during daytime and measure their retro-reflectivity. The technique is faster, cheaper, and safer as it neither requires nighttime operation nor requires manual sign inspection. It also satisfies FHWA measurement guidelines both in terms of granularity and accuracy. The performance of the presented technique is evaluated under various testing conditions. The results are promising and demonstrate a strong potential in lowering inspection cost and improving safety in practical applications on retro-reflectivity measurement. [ABSTRACT FROM AUTHOR]

Workface assessment – the process of determining the overall activity rates of onsite construction workers throughout a day – typically involves manual visual observations which are time-consuming and labor-intensive. To minimize subjectivity and the time required for conducting detailed assessments, and allowing managers to spend their time on the more important task of assessing and implementing improvements, we propose a new inexpensive vision-based method using RGB-D sensors that is applicable to interior construction operations. This is a particularly challenging task as construction activities have a large range of intra-class variability including varying sequences of body posture and time-spent on each individual activity. The skeleton extraction algorithms from RGB-D sequences produce noisy outputs when workers interact with tools or when there is a significant body occlusion within the camera's field-of-view. Existing vision-based methods are also limited as they can primarily classify “atomic” activities from RGB-D sequences involving one worker conducting a single activity. To address these limitations, our method includes three components: 1) an algorithm for detecting, tracking, and extracting body skeleton features from depth images; 2) a discriminative bag-of-poses activity classifier for classifying single visual activities from a given body skeleton sequence; and 3) a Hidden Markov Model to represent emission probabilities in the form of a statistical distribution of single activity classifiers. For training and testing purposes, we introduce a new dataset of eleven RGB-D sequences for interior drywall construction operations involving three actual construction workers conducting eight different activities in various interior locations. Our results with an average accuracy of 76% on the testing dataset show the promise of vision-based methods using RGB-D sequences for facilitating the activity analysis workface assessment. [ABSTRACT FROM AUTHOR]

[Display omitted] • A method to predict logical relationships between any pair of schedule activities with F1-Score of 88.3%. • A method to schedule a short list of input activities based on the learned logic. • Formalization of indirect logic dependencies between schedule activities. • A method for mining indirect dependencies from an input schedule. In construction, schedule mistakes causing delays beyond substantial completion dates cost contractors expensive liquidated damages. Hence, several industry guidelines, such as the DCMA's 14 point assessment, define schedule quality and offer systematic methods for ensuring it. These guidelines list "logic" as an essential control metric, and they require planners to ensure their schedules are free of missing or wrong logical dependencies. Checking the logic requires extensive construction domain knowledge, and planners perform it entirely manually as there are no available software solutions that support it. This paper offers a novel machine learning-based solution that learns construction scheduling domain knowledge from existing records completely automatically and applies it to validate the logic in input schedules achieving an F1 score of 88.3%. Furthermore, we tailor our method to use the learned knowledge to schedule a list of unordered activities. The details of the method, experimental results, benefits, and limitations are discussed. [ABSTRACT FROM AUTHOR]

[Display omitted] • An NLP transformer for relationship prediction between look-ahead tasks and master schedule activities with 82.1% Precision. • A generative model for creating look-ahead planning task descriptions based on a master schedule activity description. • A semi-automated human-in-the-loop method for linking look-ahead tasks to master schedule activities with 76.5% Precision. • A formalized representation for matching look-ahead planning tasks to master schedule activities. In construction, master schedules and look-ahead plans are created at different times (monthly vs. weekly), by different personas (planner vs. superintendent), with different software (scheduling solution vs. spreadsheet), and at different levels of granularity (milestones vs. production details). Their full-alignment is essential for project coordination, progress updating, and payment application reviews, and its absence may lead to costly litigation. This paper presents the first attempt to automate linking look-ahead planning tasks to master-schedule activities following an NLP-based multi-stage ranking formulation. Our model employs distance-based matching for candidate generation and a Transformer architecture for final matching. 1 1 Access to the code and a sample of the dataset can be granted upon direct request from the authors. Validation results from real-world projects demonstrate that the method helps planners match look-ahead planning tasks to master schedule activities by presenting a list of top-five matches with a precision of 76.5%. We also show that the method helps superintendents create look-ahead plans from a master schedule by generating lists of tasks based on activity descriptions. [ABSTRACT FROM AUTHOR]

Effective progress monitoring is crucial for successful construction project delivery. Visual data, comprising images and videos of construction operations, has emerged as a valuable source for gaining comprehensive insights into project status. While various vision-based methods have been developed for automated progress monitoring at the element level (e.g., columns, beams, floors, walls), challenges persist in achieving accurate schedule activity-level (e.g., formwork, reinforcement, concrete placement) progress tracking. Existing methods often struggle to report progress status beyond binary form (built/not built). To address these limitations, this research proposes a novel framework for precisely measuring the percent completion of activities associated with under-construction building elements. The framework is implemented through an Activity-level progress monitoring system (ALPMS). ALPMS takes the input of construction site images and a four-dimensional building information model (BIM) and outputs the activity-wise completion percentage required for updating project schedules. It creates as-built point clouds from images, compares them with the as-planned BIM, generates orthographic views of under-construction elements through projective transformation or neural radiance fields (NeRF), applies deep learning-based semantic segmentation for progress reasoning, and estimates completion percentages. Finally, the activity-level rich semantic information is transferred to the as-built point cloud and BIM for three-dimensional visualization of the progress status. When the framework was applied to two building construction projects, on average, the ALPMS could report the activity-wise completion percentages with <6% mean absolute error. Future research shall focus on automating the mapping of project schedule activities and activity-level progress details for timely schedule updates and prediction of project completion dates. • Activity percent complete measurement framework for outdoor and indoor construction monitoring. • Percentage completion estimation of schedule activities through semantic segmentation of orthographic view images. • Methodology for orthographic view synthesis through projective transformation and neural radiance field (NeRF). • 3D progress visualization through semantic segmentation of the as-built point cloud and color-coded BIM. • Semantic image segmentation dataset for activity level progress monitoring. [ABSTRACT FROM AUTHOR]

Today, computer vision object detection methods are used for safety inspections from site videos and images. These methods detect bounding boxes and use hand-made rules to enable personal protective equipment compliance checks. This paper presents a new method to improve the breadth and depth of vision-based safety compliance checking by explicitly classifying worker-tool interactions. A detection model is trained on a newly constructed image dataset for construction sites, achieving 52.9% average mean precision for 10 object categories and 89.4% average precision for detecting workers. Using this detector and new dataset, the proposed human-object interaction recognition model achieved 79.78% precision and 77.64% recall for hard hat checking; 79.11% precision and 75.29% recall for safety coloring checking. The new model also verifies hand protection for workers when tools are being used with 66.2% precision and 64.86% recall. The proposed model is superior in these checking tasks when compared with post-processing detected objects with hand-made rules, or applying detected objects only. • A learned Human-Object Interaction (HOI) recognition model is proposed for personal protective equipment compliance checking. • Final HOI recognition model is validated by hard hat, safety coloring, and hand protection checking. • Final HOI recognition model achieves higher precision and recall than using hand-made rules or detected objects only. [ABSTRACT FROM AUTHOR]

Timely and accurate monitoring of onsite construction operations can bring an immediate awareness on project specific issues. It provides practitioners with the information they need to easily and quickly make project control decisions. Despite their importance, the current practices are still time-consuming, costly, and prone to errors. To facilitate the process of collecting and analyzing performance data, researchers have focused on devising methods that can semi-automatically or automatically assess ongoing operations both at project level and operation level. A major line of work has particularly focused on developing computer vision techniques that can leverage still images, time-lapse photos and video streams for documenting the work in progress. To this end, this paper extensively reviews these state-of-the-art vision-based construction performance monitoring methods. Based on the level of information perceived and the types of output, these methods are mainly divided into two categories (namely project level: visual monitoring of civil infrastructure or building elements vs. operation level: visual monitoring of construction equipment and workers). The underlying formulations and assumptions used in these methods are discussed in detail. Finally the gaps in knowledge that need to be addressed in future research are identified. [ABSTRACT FROM AUTHOR]