Search

The exponential growth in reality capture and Building Information Modeling (BIM)-enabled construction workflows has created a surge in computer vision–driven solutions that automatically model and compare as-built conditions against BIM, offering project teams actionable insight into construction progress and quality. Despite their significant impact, the performance of these methods heavily relies on the accuracy and completeness of the reality capture. In addition, and especially in the case of reality captures conducted with camera-equipped unmanned aerial vehicles (UAVs), operational requirements—including battery capacity and operator's line of sight (LOS)—should be carefully considered for safe flight execution. Accounting for these technical and operational requirements during reality capture planning requires expertise. In addition, it involves a significant amount of manual tweaking that does not scale well to ongoing changes due to progress on construction projects. To address these limitations, this paper presents a novel multiobjective optimization method to improve reality capture plans aiming to maximize (1) visual coverage of the monitored structure, (2) redundant observation of the structure's components in the collected frames, (3) resolution of the structure's elements in the captured data, (4) canonical camera viewpoints to the structure's topology, and (5) stability of three-dimensional (3D) reconstruction algorithms used to process the data altogether, while (6) reducing the data collection duration. The objectives are also set to meet other technical and operational requirements, particularly for camera-equipped UAVs. Furthermore, a client-server system architecture is presented to visualize, simulate, and optimize reality capture missions in a web-based 3D environment using four-dimensional (4D) BIM to indicate the as-planned expected changes. Five conducted experiments using real-world data demonstrated the method's capability to enhance the quality of user-created reality capture plans. The optimization process resulted in a 7.65% improvement in visual coverage, 30.89% enhancement in the structure's resolution, and 8.95% more stable 3D reconstruction while ensuring the flight paths meet operational requirements. [ABSTRACT FROM AUTHOR]

This paper presents new metrics and methods for evaluating the quality of reality capture plans—commonly used to operate camera‐mounted unmanned aerial vehicles (UAVs) or ground rovers—for construction progress monitoring and inspection of as‐is conditions. Using 4D building information model (BIM) or 3D reality model as a priori, these metrics provide feedback on the quality of a plan (within a few minutes), accounting for resolution, visibility, accuracy, completeness of the capture, and satisfying battery capacity and line‐of‐sight requirements. A cloud‐based system is introduced to create and optimize UAV/rover missions in the context of prior model. Results from real‐world construction data sets demonstrate that the proposed metrics offer actionable insights into the accuracy and completeness of reality capture plans. Additionally, a capture plan—with a combination of canonical and noncanonical camera views—that satisfies the introduced metrics is statistically correlated with the quality of reconstructed reality. These metrics can improve computer‐vision progress monitoring and inspection methods that rely on the construction site's appearance and geometry. [ABSTRACT FROM AUTHOR]

This paper proposes a new method for single-worker severity level prediction from already collected site images and video clips. Onsite safety observers often assess workers' severity levels during construction activities. While risk analysis is key to improving long–term construction site safety, omnipresent monitoring is still time-consuming and costly to implement. The recent growth of visual data captured actively on construction sites has opened a new opportunity to increase the frequency of worker safety monitoring. This paper shows that a comprehensive vision-based assessment is the most informative to automatically infer worker severity level from images. Efficient computer vision models are presented to conduct this risk analysis. The method is validated on a challenging image dataset first of its kind. Specifically, the proposed method detects and evaluates the worker state from visual data, defined by (1) worker body posture, (2) the usage of personal protective equipment, (3) worker interactions with tools and materials, (4) the construction activity being performed, and (5) the presence of surrounding workplace hazards. To estimate the worker state, a multitasked recognition model is introduced that recognizes objects, activity, and keypoints from visual data simultaneously, taking 36.6% less time and 40.1% less memory while keeping comparably performances compared to a system running individual models for each subtask. Worker activity recognition is further improved with a spatio-temporal graph neural network model using recognized per-frame worker activity, detected bounding boxes of tools and materials, and estimated worker poses. Finally, severity levels are predicted by a trained classifier on a dataset of images of construction workers accompanied with ground truth severity level annotations. In the test dataset assembled from real-world projects, the severity level prediction model achieves 85.7% cross-validation accuracy in a bricklaying task and 86.6% cross-validation accuracy for a plastering task, demonstrating the potential for near real-time worker safety detection and severity assessment. [ABSTRACT FROM AUTHOR]

The architecture, engineering, and construction (AEC) industry perform thousands of scans each year. The majority of these point clouds are used for generating three-dimensional (3D) models—a process formally known as scan to building information modeling (Scan-to-BIM)—that represent the current conditions of a construction scene. Although point cloud data provide the scene's geometric information, its use presents several challenges that make the process of generating a 3D model from point cloud data time-consuming, labor-intensive, and error-prone. In order to address the mentioned challenges, this paper presents a new end-to-end deep learning method, named Scan2BIM-NET, for semantically segmenting the structural, architectural, and mechanical components present in point cloud data. It classifies beam, ceiling, column, floor, pipe, and wall elements using three main networks: two convolutional neural network (CNN) and one recurrent neural network (RNN). The method was trained and tested using 83 rooms from point cloud data representing real-world industrial and commercial buildings. The process returned an average accuracy of 86.13%, and the beam, ceiling, column, floor, pipe, and wall categories obtained an accuracy of 82.47%, 92.60%, 59.31%, 98.71%, 82.79%, and 84.46%, respectively. The experimental results showed that deep learning improves the accuracy of semantic segmentation of architectural, structural, and mechanical components. This new method has the potential of being a tool during the Scan-to-BIM process, especially for semantically segmenting underceiling areas where mechanical components are close to structural elements. [ABSTRACT FROM AUTHOR]

Current construction still suffers from significant schedule delays and budget overruns due to inefficient and retroactive project controls programs. This paper developed a visual and virtual project control system that proactively promotes team-based planning and real-time communication of progress and schedule risk. The system benefits from the growth of visual data on construction sites and the increasing maturity level of building information modeling (BIM), to map the current state of production in 3D and expose waste. By modeling production trends, reliability in the future state of production is forecasted to highlight potential delay issues in a location-driven scheme. A visual 3D interface supports collaborative decision-making to eliminate root causes of waste, enables pull flows between people and information, decentralizes work tracking, and facilitates in-process quality control and hand-overs among contractors. This system was implemented on two projects to validate the underlying algorithms. Results show that the system's functional aspects improve transparency, accountability, and traceability in project execution, and streamline communications between the field and the office. Specifically, 30% PPC increase and four weeks project completion date advance have been observed. Limitations such as resistance to workflow changes could be further improved in the future through early system introduction. [ABSTRACT FROM AUTHOR]

A key underlying challenge in today's model-driven approach to engineering is how progress and level of effort are being measured and reported. Recent definitions such as level of development enable project teams to track completeness of information in building information modeling (BIM)-driven processes; yet, there are no established processes or metrics to reliably measure progress or track productivity of a model-driven engineering process as a function of the maturity of the model and/or the data that supports the engineering process. To address this gap in knowledge, this paper presents a discrete set of model maturity index (MMI) definitions to measure progress in engineering work, per modeling discipline. Using these MMI definitions, a model maturity risk index (Model MRI) toolkit was created to track maturity of an engineering process at the granularity of work breakdown structure locations. The MMI toolkit also reports risk in achieving certain MMI levels per modeling discipline. To benchmark maturity of the modeling work per project milestone or model review session and to establish workflows for tracking progress, an addendum to the modeling execution plan (ModelXP) is presented. New productivity and project controls metrics for tracking modeling work are also introduced. The developed definitions, toolkit, and addendum to ModelXP were validated through exhaustive surveys and charrette tests with project stakeholders involved in capital projects. The role of the MMI definitions and Model MRI toolkit in providing project controls insight and proactive management opportunities in real-world projects is discussed in detail. [ABSTRACT FROM AUTHOR]

Falls, struck-bys, and caught-in/betweens are among the most common types of fatal accidents on construction sites. Despite their significance, the majority of today's accident prevention programs react passively to situations in which workers or equipment enter predefined unsafe zones. To support systems that proactively prevent these accidents, this paper presents a path prediction model for workers and equipment. The model leverages the extracted video frames to predict upcoming worker and equipment motion trajectories on construction sites. Specifically, the model takes two-dimensional (2D) tracks of workers and equipment from visual data—based on computer vision methods for detection and tracking—and uses a long short-term memory (LSTM) encoder-decoder followed by a mixture density network (MDN) to predict their locations. A multihead prediction module is introduced to predict locations at different future times. The method is validated on an existing dataset, TrajNet, and a new dataset of 105 high-definition videos recorded over 30 days from a real-world construction site. On the TrajNet dataset, the proposed model significantly outperforms Social LSTM. On the new dataset, the presented model outperforms conventional time-series models and achieves average localization errors of 7.30, 12.71, and 24.22 pixels for 10, 20, and 40 future steps, respectively. The benefits and limitations of the method to worker and equipment path prediction are discussed. [ABSTRACT FROM AUTHOR]

The ever increasing volume of visual data due to recent advances in smart devices and camera-equipped platforms provides an unprecedented opportunity to visually capture actual status of construction sites at a fraction of cost compared to other alternatives methods. Most efforts on documenting as-built status, however, stay at collecting visual data and updating BIM. Hundreds of images and videos are captured but most of them soon become useless without properly being localized with plan document and time. To take full advantage of visual data for construction performance analytics, three aspects (reliability, relevance, and speed) of capturing, analyzing, and reporting visual data are critical. This paper 1) investigates current strategies for leveraging emerging big visual data and BIM in construction performance monitoring from these three aspects, 2) characterizes gaps in knowledge via case studies and structures a road map for research in visual sensing and analytics. [ABSTRACT FROM AUTHOR]

Frequent analysis and updating the condition of traffic signs and mile markers are among the most important aspects of a highway asset management system. Today's practices mainly involve manual data collection and analysis, which have to be done for millions of miles of roads and the practice needs to be repeated regularly. While significant progress has been made on improving the data collection practice by leveraging video streams collected from car-mounted cameras, the analysis has primarily remained a manual and labor-intensive process. Automating the analysis from the collected videos is also challenging due to the interclass variability of traffic signs, expected changes in illumination, occlusion, sign position, and orientation. To address these challenges, this paper presents and evaluates the performance of three computer vision algorithms for detection and classification of traffic signs in presence of cluttered backgrounds and static and dynamic occlusions. The task particularly focuses on (1) extracting two-dimensional (2D) candidate windows from already collected video streams that potentially contain traffic signs-without making any prior assumption about their locations; (2) detecting the presence of signs in these 2D candidate windows; and (3) classifying them into warning, regulatory, stop, and yield sign categories based on their shape and color. For validation, a new comprehensive benchmark data set of over 11,000 annotated U.S. traffic sign images with a large range of pose, scale, background, illumination, and occlusion variation is introduced. Experimental results show an average accuracy of 76.20%, 89.31%, and 94.83% for the methods of (1) Haar-like features with Cascade classifiers, (2) histograms of oriented gradients (HOG) with multiple one-versus-all support vector machine (SVM) classifiers, and (3) HOG+C with the SVM classifiers, a variant of the second method with histograms of colors concatenated to HOG. The experimental results demonstrate the potential of leveraging joint representation of texture and color in HOG+C together with SVM discriminative classifiers as a viable solution for creating up-to-date and complete inventories of traffic signs for U.S. roadways. [ABSTRACT FROM AUTHOR]

The advent of affordable jobsite cameras is reshaping the way on-site construction activities are monitored. To facilitate the analysis of large collections of videos, research has focused on addressing the problem of manual workface assessment by recognizing worker and equipment activities using computer-vision algorithms. Despite the explosion of these methods, the ability to automatically recognize and understand worker and equipment activities from videos is still rather limited. The current algorithms require large-scale annotated workface assessment video data to learn models that can deal with the high degree of intraclass variability among activity categories. To address current limitations, this study proposes crowdsourcing the task of workface assessment from jobsite video streams. By introducing an intuitive web-based platform for massive marketplaces such as Amazon Mechanical Turk (AMT) and several automated methods, the intelligence of the crowd is engaged for interpreting jobsite videos. The goal is to overcome the limitations of the current practices of workface assessment and also provide significantly large empirical data sets together with their ground truth that can serve as the basis for developing video-based activity recognition methods. Six extensive experiments have shown that engaging nonexperts on AMT to annotate construction activities in jobsite videos can provide complete and detailed workface assessment results with 85% accuracy. It has been demonstrated that crowdsourcing has the potential to minimize time needed for workface assessment, provides ground truth for algorithmic developments, and most importantly allows on-site professionals to focus their time on the more important task of root-cause analysis and performance improvements. [ABSTRACT FROM AUTHOR]

Recent research efforts to improve energy modeling and diagnostics for existing buildings have focused on devising methods based on digital photogrammetry or three-dimensional (3D) laser scanning and thermal imagery. Prior research has shown that fusing actual and expected 3D spatiothermal models provide valuable information for analyzing performance gaps. Until now, these methods have primarily focused on detecting and localizing potential performance problems without analyzing their associated cost. A reliable assessment of the cost provides an opportunity to conduct cost-benefit analysis for various retrofit alternatives. It also improves current thermographic inspection which relies on visual detection and qualitative interpretation of thermal irregularities. To this end, this paper presents a new 3D thermography-based method for cost-benefit analysis of energy efficiency building envelope retrofits. The method builds on energy performance augmented reality (EPAR) modeling which generates and fuses actual and expected 3D spatiothermal models using collections of thermal and digital images. In the resulting EPAR models, first detect and calculate the building areas associated with potential thermal degradations, and then explore the as-is thermal conditions of the associated building assemblies at 3D vertex-level. Next, using the historical climatic data and the recommended R-values, the amount of the unnecessary heat transfer through the detected areas and the monetary benefits from their retrofit are estimated. Finally, the expected energy cost savings were balanced against the cost of implementing insulation retrofit alternatives. Experimental results on two real-world buildings and four hypothetical cases with different geographical conditions show potential of the proposed method in reliably estimating the return on investment from retrofitting thermal performance problems. [ABSTRACT FROM AUTHOR]

Recent research efforts to improve information management on construction sites have focused on devising mobile augmented reality methods to provide on-demand access to project information and facilitate field reporting. To track position and orientation of the users, prior research focused on radio frequency based location tracking methods, fiduciary markers, or parallel visual tracking and mapping techniques. These methods have potential in providing fast information access to field personnel, yet their application heavily depends on preinstalled infrastructure, may suffer from accumulated drift errors, and in the case of visual methods do not typically scale well. Furthermore, these methods do not provide content authoring capability in three dimensions, which is necessary for reciprocal application of augmented reality for field reporting and asynchronous personnel-to-personnel communications. To overcome these limitations, this paper presents a scalable markerless and infrastructure-independent mobile augmented reality system for commodity smartphones and tablets, with client-server architecture and cloud computing basis, to facilitate both accessing to project information and conducting field reporting in three dimensions. The method relies on structure-from-motion point clouds generated from site images and/or videos to enable image-based localization and single-image based content authoring in three dimensions. By using mobile client devices with the proposed method, onsite personnel can associate field reports with two-dimensional (2D) patches from mobile images and store that information directly in the underlying three-dimensional (3D) point cloud model. In addition, the client-server architecture of the proposed system allows an onsite access to most updated project information and field reports as they are being documented on the server. The presented architecture also handles multiple onsite information retrieval and field reporting requests simultaneously. The overall system is validated on several indoor/outdoor work areas of three real-world construction projects with varying degrees of occlusions and lighting conditions. The experimental results show the potential of the proposed method in prompt field reporting in three dimensions, minimizing the time field personnel spent on accessing project information, and communicating with others to resolve an issue/conflict on the jobsite within 1-3 min. [ABSTRACT FROM AUTHOR]

This paper presents a new appearance-based material classification method for monitoring construction progress deviations at the operational-level. The method leverages 4D Building Information Models (BIM) and 3D point cloud models generated from site photologs using Structure-from-Motion techniques. To initialize, a user manually assigns correspondences between the point cloud model and BIM, which automatically brings in the photos and the 4D BIM into alignment from all camera viewpoints. Through reasoning about occlusion, each BIM element is back-projected on all images that see that element. From these back-projections, several 2D patches are sampled per element and are classified into different material types. To perform material classification, the expected material type information is derived from BIM. Then the image patches are convolved with texture and color filters and their concatenated vector-quantized responses are compared with multiple discriminative material classification models that are relevant to the expected progress of that element. For each element, a quantized histogram of the observed material types is formed and the material type with the highest appearance frequency infers the appearance and thus the state of progress. To validate, four new datasets of incomplete and noisy point cloud models are introduced which are assembled from real-world construction site images and BIMs. An extended version of the Construction Material Library (CML) is also introduced for training/testing the material classifiers. The material classification shows an average accuracy of 92.4% for CML image patches of 100 × 100 pixels. The experiments on those four datasets show an accuracy of 95.9%, demonstrating the potential of appearance-based recognition method for inferring the actual state of construction progress for BIM elements. [ABSTRACT FROM AUTHOR]

Accurate and rapidly produced 3D models of the as-built environment can be significant assets for a variety of Engineering scenarios. Starting with a point cloud of a scene—generated using laser scanners or image-based reconstruction methods—the user must first identify collections of points that belong to individual surfaces, and then, fit surfaces and solid geometry objects appropriate for the analysis. When performed manually, this task is often prohibitively time consuming and, in response, several research groups have recently focused on developing methods for automating the modeling process. Due to the limitations of the data collection processes as well as the complexity of as-built scenes, automated 3D modeling still presents many challenges. To overcome existing limitations, in this paper, we propose a new region growing method for robust context-free segmentation of unordered point clouds based on geometrical continuities. In our method, the user sets a single parameter which accounts for the desired level of abstraction. We treat this parameter as a locally adaptive threshold to account for local context. Our method of segmentation starts with a multi-scale feature detection, describing surface roughness and curvature around each 3D point, and is followed by seed finding and region growing steps. Experimental results from seven challenging point clouds of the built environment demonstrate that our method can account for variability in point cloud density, surface roughness, curvature, and clutter within a single scene. [ABSTRACT FROM AUTHOR]

The ability to import building geometric and construction thermal data from building information models (BIM) has significant potential to reduce the time and uncertainty in building energy modeling process. In today's BIM-based energy modeling practice, thermal properties are mainly derived from generic building construction types in BIM. However, for energy modeling of existing buildings, such assumptions are often inaccurate as they do not account for diminishing thermal resistances of building materials instigated by their deteriorations. To improve the reliability of BIM-based energy modeling, we present a system, together with new methods for automated association and updating of actual thermal property measurements with BIM elements in gbXML schema. By leveraging collections of digital and thermal images and based on environmental measurements, our system first produces a 3D thermal model for the building under inspection and then derives the actual thermal resistances of the building assemblies at the level of 3D vertexes. By associating these measurements with their corresponding elements in gbXML, thermal properties of the BIM elements are automatically updated. Our experiments in real-world residential and instructional buildings show how actual thermal properties can be automatically associated with BIM elements and updated in gbXML. The proposed method shortens the gap between architectural information in BIM and the actual data needed for energy performance simulation, and enables reliable BIM-based energy analysis for retro-commissioning, continuous commissioning, and retrofit. [ABSTRACT FROM AUTHOR]

This paper presents a non-parametric image parsing method for segmentation and recognition of roadway assets such as traffic signs, traffic lights, pavement markings, and guardrails from 2D car-mounted video streams. The method can be easily scaled to thousands of video frames captured during data collection and does not need training. Instead, it retrieves a set of most relevant video frames (e.g. highway vs. secondary road) which serve as candidates for superpixel-level annotation. It then obtains superpixels from the video frames and using the retrieval set encodes their visual characteristics using a histogram of different shape, appearance, and color descriptors. Neighborhood contexts are incorporated by using Markov Random Field (MRF) optimization and two types of semantic (e.g. guardrail) and geometric (e.g. horizontal) labels are simultaneously assigned to the superpixels. We introduce a new dataset from I-57 together with its ground truth and present experimental results on both I-57 and SmartRoad datasets. Experimental results with an average accuracy of 88.24% for recognition and 82.02% for segmentation show that our local visual features provide acceptable performance, while the method overall does not require any significant supervised training. This scalable method has potential to reduce the time and effort required for developing road inventories, especially for those such as guardrails and traffic lights that are not typically considered in 2D asset recognition methods. [ABSTRACT FROM AUTHOR]

Natural disasters all too often place human lives and property at risk. Recovery efforts following a disaster can be slow and painstaking work, and potentially put responders in harm's way. A system which helps identify defects in critical building elements (e.g., concrete columns) before responders must enter a structure could save lives. In this paper we propose such a system, centered around an image-based three-dimensional (3D) reconstruction method and a new 3D crack detection algorithm. The image-based method is capable of detecting and analyzing surface damages in 3D. We also demonstrate how a robotic platform could be used to gather the set of images from which the reconstruction is created, further reducing the risk to responders. In this regard, image-based 3D reconstructions represent a convenient method of creating 3D models because most robotic platforms can carry a lightweight camera payload. Additionally, the proposed 3D crack detection algorithm also provides the advantage of being able to operate on 3D mesh models regardless of their data collection source. Our experimental results showed that the 3D crack detection algorithm performed well on several sample building elements, successfully identifying cracks, reconstructing 3D profiles, and measuring geometrical characteristics on damaged elements and not finding any cracks on intact ones. The operation and perceived benefits of the proposed method in a post-disaster situation are also discussed in detail. [ABSTRACT FROM AUTHOR]

This work provides an analysis of the applicability of two image-based three-dimensional (3D) reconstruction techniques-(1) four-dimensional augmented reality () 3D reconstruction software and (2) phototourism applications-for remote site construction project planning. Remote sites present uncertainty to construction project planning through access, material and resource availability, potential lack of technical expertise on site, and environmental considerations. Both and phototourism have the potential to minimize remote site uncertainty by using available camera technology to digitally capture existing site conditions for the development of accurate 3D modeling for construction project planning. For this study, the researchers (1) analyzed the applicability of 3D reconstruction and phototourism software applications to remote construction planning; (2) determined the appropriateness and robustness of various levels of camera technology for capturing existing conditions on remote sites; and (3) evaluated the impact of image resolution and the method of capture on 3D image-based software-all key facets to usability and access. The researchers used three cameras of various performance capabilities, i.e., a 35-mm disposable box camera, a 5.7-17.1-mm (35 mm equivalent: 37-111 mm) low-level digital camera, and a high-level digital single-lens reflex camera. The following paper outlines a methodology for testing these camera technologies and discusses their appropriateness for reducing barriers to the application of image-based reconstruction techniques for remote site construction planning. This work offers the conclusion that ordinary camera technology is an appropriate option for capturing existing conditions in remote sites (usability), while more advanced camera technology offers better total system benefits (access) of the image-based 3D reconstruction platform. offers a more robust system for modeling remote sites; however, both and phototourism have the potential to generate useful computer models using catalogs of digital images to enable better planning and execution of construction projects in remote areas. [ABSTRACT FROM AUTHOR]

Highlights: [•] We propose a robust vision-based method for material detection from single images. [•] A new Construction Materials Library containing 20 typical construction materials. [•] We achieve accuracies of 97.1% for high quality 200 by 200 pixel color images. [•] We maintain accuracies above 90% for images as small as 30 by 30 pixels. [•] We maintain accuracies above 92% for highly compressed, low quality images. [ABSTRACT FROM AUTHOR]

Quick and reliable identification of energy performance problems in buildings is a critical step in improving their efficiency. The current practice of building diagnostics typically involves nonintrusive data collection using thermal cameras. This requires large amounts of unordered and nongeo-tagged two-dimensional (2D) imagery to be manually analyzed at a later stage, which makes the analysis time-consuming and labor-intensive. Because of the absence of a benchmark for energy performance, identification of performance problems also heavily relies on the auditor's knowledge, and consequently may lead to subjective and error-prone inspections. As a step towards rapid and objective identification of performance problems, this paper presents a new method for automated analysis and visualization of deviations between buildings' actual and simulated energy performances. The proposed method is based on the recently developed energy performance augmented reality (EPAR) environments. In the EPAR modeling method, actual and expected three-dimensional (3D) spatio-thermal models are generated and superimposed in a common 3D virtual environment. The method leverages unordered collections of thermal and digital imagery for actual energy performance modeling, in addition to computational fluid dynamics (CFD) analysis for expected energy performance simulation. Based on the EPAR models which store actual and simulated thermal values at the level of 3D points, two new algorithms are developed to facilitate identification of potential performance problems: (1) 3D thermal mesh modeling using trees and nearest-neighbor searching to automate calculation of temperature deviations, and (2) automated visualization of performance deviations using a metaphor based on traffic light colors. The proposed modeling method is validated on several interior locations of instructional and residential buildings. Empirical observations show that automated analysis using EPAR models enables performance deviations to be rapidly and accurately measured. The visualization of deviations in three dimensions enables auditors to easily identify potential performance problems and, in turn, enables auditors to focus more on other important tasks of analyzing retrofit alternatives. [ABSTRACT FROM AUTHOR]

Highlights: [•] We present a computer vision based method for equipment action recognition. [•] Our vision-based method is based on a multiple binary SVM classifier and spatio-temporal features. [•] A comprehensive real-world video dataset of excavator and truck actions is presented. [•] We achieve accuracies of 86.33% and 98.33% for excavator and truck action classes. [•] The presented method can be used for construction activity analysis using long sequences of videos. [Copyright &y& Elsevier]

Highlights: [•] Image-based 3D modeling method of actual building energy performance is presented. [•] Building environments and their energy performances are jointly visualized in 3D. [•] Automatically 3D-registered thermal images assist with localizing energy problems. [•] 3D spatio-thermal models can facilitate energy building diagnostics and retrofit analysis. [Copyright &y& Elsevier]

Abstract: Building energy performance simulation tools such as EnergyPlus, Ecotect, and eQuest are widely used to model energy performance of existing buildings and assess retrofit alternatives. Nevertheless, predictions from simulations typically deviate from actual measurements. Monitoring actual performance and measuring deviations from simulated data in 3D can help improve simulation accuracy through model calibrations, and in turn facilitate identification of energy performance problem. To do that, this paper presents Energy Performance Augmented Reality (EPAR) modeling that leverages collections of unordered digital and thermal imagery, in addition to computational fluid dynamics (CFD) models. First, users collect large numbers of digital and thermal imagery from the building under inspection using a single thermal camera. Through an image-based reconstruction pipeline, actual 3D spatio-thermal models are automatically generated and are superimposed with expected building energy performance models generated using CFD analysis through a user-driven process. The outcomes are EPAR models which visualize actual and expected models in a common 3D environment. Within the EPAR models, actual measurements and simulated results can be systematically compared and analyzed. The method is validated on typical residential and instructional buildings. The results demonstrate that EPAR models facilitate calibration of building energy performance models and support detection and analysis of building performance deviations. [Copyright &y& Elsevier]

Abstract: This paper presents a computer vision based algorithm for automated 2D detection of construction workers and equipment from site video streams. The state-of-the-art research proposes semi-automated detection methods for tracking of construction workers and equipment. Considering the number of active equipment and workers on jobsites and their frequency of appearance in a camera''s field of view, application of semi-automated techniques can be time-consuming. To address this limitation, a new algorithm based on Histograms of Oriented Gradients and Colors (HOG+C) is proposed. Our proposed detector uses a single sliding window at multiple scales to identify the potential candidates for the location of equipment and workers in 2D. Each detection window is first divided into small spatial regions and then the gradient orientations and hue–saturation colors are locally histogrammed and concatenated to form the HOG+C descriptors. Tiling the sliding detection window with a dense and overlapping grid of formed descriptors and using a binary Support Vector Machine (SVM) classifier for each resource enables automated 2D detection of workers and equipment. A new comprehensive benchmark dataset containing over 8000 annotated video frames including equipment and workers from different construction projects is introduced. This dataset contains a large range of pose, scale, background, illumination, and occlusion variation. Our preliminary results on detection of standing workers, excavators and dump trucks with an average accuracy of 98.83%, 82.10%, and 84.88% respectively indicate the applicability of the proposed method for automated activity analysis of workers and equipment from single video cameras. Unlike other state-of-the-art algorithms in automated resource tracking, this method particularly detects idle resources and does not need manual or semi-automated initialization of the resource locations in 2D video frames. The experimental results and the perceived benefits of the proposed method are discussed in detail. [Copyright &y& Elsevier]

The significant advancement in digital imaging and widespread popularity of digital cameras for capturing a comprehensive visual record of construction performance in the architecture, engineering, construction, and facility management (AEC/FM) industries have triggered an extensive growth in the rate of site photography, allowing hundreds of images to be stored for a project on a daily basis. Meanwhile, collaborative AEC technologies centering around building information models (BIMs) are becoming widely applied to support various architectural, structural, and preconstruction decision-making tasks. These models, if integrated with the as-built perspective of a construction, have great potential to extensively add value during the construction phase of a project. This paper reports recent developments from research efforts in (1) automated acquisition of as-built point-cloud models from unordered daily site photo collections and geo-registration of site images, (2) automated generation of four-dimensional (4D) as-built point-cloud models, and (3) semiautomated superimposition of the integrated as-built model over 4D (three dimensions plus time) BIMs to generate integrated 4D as-built and as-planned visualizations. The limitations and benefits of each modeling approach, the motivations for development of 4D augmented-reality (D4AR) environments for integrated visualization of as-built and as-planned models, the perceived and observed applications, and the benefits in seven case studies are discussed. Not only does a D4AR model visualize construction processes and performance deviations, it can also be used as a tool for automated and remote monitoring of progress and safety, quality control, and site layout management, and enables enhanced coordination and communication. [ABSTRACT FROM AUTHOR]

Abstract: Accurate and rapid assessment of the as-built status on any construction site provides the opportunity to understand the current performance of a project easily and quickly. Rapid project assessment further identifies discrepancies between the as-built and as-planned progress, and facilitates decision making on the necessary remedial actions. Currently, manual visual observations and surveying are the most dominant data capturing techniques but they are time-consuming, error-prone, and infrequent, making quick and reliable decision-making difficult. Therefore, research on new approaches that allow automatic recognition of as-built performance and visualization of construction progress is essential. This paper presents and compares two methods for obtaining point cloud models for detection and visualization of as-built status for construction projects: (1) A new method of automated image-based reconstruction and modeling of the as-built project status using unordered daily construction photo collections through analysis of Structure from Motion (SfM); (2) 3D laser scanning and analysis of the as-built dense point cloud models. These approaches provide robust means for recognition of progress, productivity, and quality on a construction site. In this paper, an overview of the newly developed automated image-based reconstruction approach and exclusive features which distinct it from other image-based or conventional photogrammetric techniques is presented. Subsequently the terrestrial laser scanning approach carried out for reconstruction and comparison of as-built scenes is presented. Finally the accuracy and usability of both of these techniques for metric reconstruction, automated production of point cloud models, 3D CAD shape modeling, and as-built visualizations is evaluated and compared on eight different case studies. It is shown that for precise defect detection or alignment tasks, image-based point cloud models may not be as accurate and dense as laser scanners'' point cloud models. Nonetheless image-based point cloud models provide an opportunity to extract as-built semantic information (i.e., progress, productivity, quality and safety) through the content of the images, are easy to use, and do not need add burden on the project management teams by requiring expertise for data collection or analysis. Finally image-based reconstruction automatically provides photo alignment with point cloud models and enables image-based renderings which can remarkably impact automated performance monitoring and as-built visualizations. [Copyright &y& Elsevier]

The ability to effectively communicate progress information and represent as-built and as-planned progress discrepancies are identified as key components for successful project management that allow corrective decisions to be made in a timely manner. However, current formats of reporting (e.g., textual progress reports, progress curves, and photographs) may not properly and quickly communicate project progress. Current monitoring methods also require manual data collection and extensive data extraction from different construction documents, which distract managers from the important task of decision making. Therefore, to facilitate progress monitoring, this paper proposes visualization of performance metrics that aims to represent progress deviations through superimposition of four-dimensional (4D) as-planned model over time-lapsed photographs in single and comprehensive visual imagery. As a part of the developed system, registration of the 4D model with photographs, augmenting photographs, and occlusion removal for progress images are presented. While contextual information is preserved, the as-built photographs are enhanced and augmented with 4D as-planned model in which the performance metrics are visualized. The augmented photographs provide a consistent platform for representing as-planned, as-built, and progress discrepancies information and facilitate communication and reporting processes. [ABSTRACT FROM AUTHOR]

Generating 3D models from point cloud data is a common Virtual Design and Construction (VDC) service. Research has focused on automating several key steps, including segmenting point clouds based on appearance and geometric attributes. These methods dominantly use Manhattan-World assumptions in their approach. Hence, in cases where building systems are close to architectural/structural elements, these methods result in over-segmentation and require significant fine-tuning by the users. To overcome these limitations, this paper presents a learning method based on Markov Random Field (MRF) that assigns semantic labels to point cloud segment. The MRF enforces coherence between the semantic (e.g., beam, column, wall, ceiling, floor, pipe) and geometric labels (e.g., horizontal, vertical, cylindrical), and it uses the neighborhood context to enhance the semantic labeling accuracy. Experimental results show an average accuracy of 90% on semantic labeling, achieving state-of-the-art performance on labeling beam, ceiling, column, floor, pipe, and wall elements. [Display omitted] • MRF-based method for semantic segmentation of indoor point cloud scenes. • Inference via neighborhood context improves semantic segmentation of point clouds. • Coherence between semantic and geometric information enhances labeling accuracy. • The pipeline, CRF and MRF, reduces over-segmentation in region-growing methods. [ABSTRACT FROM AUTHOR]

• A method to learn planning & scheduling knowledge as Dynamic Process Templates (DPT). • A sequential machine learning learns planning and scheduling knowledge for the DPTs. • For an input sequence of activities, DPTs generate predecessor & successor activities. • The DPTs predict previously seen & unseen activity dependencies at 98% & 72% accuracy. Today, construction planning and scheduling is almost always performed manually, by experienced practitioners. The knowledge of those individuals is materialized, maintained, and propagated through master schedules and look-ahead plans. While historical project schedules are available, manually mining their embedded knowledge to create generic work templates for future projects or revising look-ahead schedules is very difficult, time-consuming and error-prone. The rigid work templates from prior research are also not scalable to cover the inter and intra-class variability in historical schedule activities. This paper aims at fulfilling these needs via a new method to automatically learn construction knowledge from historical project planning and scheduling records and digitize such knowledge in a flexible and generalizable data schema. Specifically, we present Dynamic Process Templates (DPTs) based on a novel vector representation for construction activities where the sequencing knowledge is modeled with generative Long Short-Term Memory Recurrent Neural Networks (LSTM-RNNs). Our machine learning models are exhaustively tested and validated on a diverse dataset of 32 schedules obtained from real-world projects. The experimental results show our method is capable of learning planning and sequencing knowledge at high accuracy across different projects. The benefits for automated project planning and scheduling, schedule quality control, and automated generation of project look-aheads are discussed in detail. [ABSTRACT FROM AUTHOR]

This paper presents a new benchmark dataset for validating vision-based methods that automatically identifies visually distinctive working activities of excavators and dump trucks from individual frames of a video sequence. Our dataset consists of 10 videos of interacting pairs of construction equipment filmed at ground level with accompanying ground truth annotations. These annotations consist of per-equipment and per-frame equipment bounding boxes that also have associated identities and activity labels. Our videos depict an excavator interacting with 1 or more dump trucks. We also propose a deep learning-based method for detecting and tracking objects based on Convolutional Neural Networks (CNNs). The tracking trajectories are fed into a Hidden Markov Model (HMM) that automatically discovers and assigns activity labels for any observed object. Our HMM method leverages trajectories to train a Gaussian Mixture Model (GMM) with which we estimate the probability density function of each activity using Support Vector Machine (SVM) classifiers. The proposed HMM also models activity duration and the transition between activities. We show that our method can accurately distinguish between individual equipment working activities. Results show 97.43% detection Average Precision (AP) for excavators and 75.29% AP for dump trucks, as well as cross-category tracking accuracy of 81.94% and tracking precision of 87.45%. Separate experiment results show activity analysis results of 86.8% accuracy for excavators and 88.5% for dump trucks. Our results show that our method can accurately conduct activity analysis and can be fused with methods that detect motion trajectories to scale to the needs of practical applications. Unlabelled Image • We introduce a benchmark dataset for detection, tracking and frame-level action recognition of earthmoving operations. • We also develop a method for detecting, tracking and categorizing actions of excavators and dump trucks from videos. • Our method is potent for all three subtasks and requires neither part annotations nor manual object track initialization. [ABSTRACT FROM AUTHOR]

Creating consistency among project schedule data, BIM, and payment applications requires activities in a construction schedule to be mapped with the most relevant ASTM Uniformat classifications. To do so, we introduce UniformatBridge , a new transformer-based natural language processing model, that automatically labels activities in a project schedule with Uniformat classification. Our model introduces construction sequencing tokens that capture logistically-constrained predecessor and successor activities into BERT architecture. We also introduce a dataset of real-world construction project schedules with their ground-truth Uniformat classifications for validation. Experimental results using this dataset achieve F1-scores of 0.93 and 0.87 when matching unstructured schedule data to Uniformat Level 2 and 3 classifications, respectively. We share how our method unlocks development of new techniques to (1) automatically create 4D BIM, and (2) computer-vision progress monitoring to tie semantic segmentation of reality capture data based on Uniformat classes against schedule or payment application data structures, with/without BIM. • A new NLP transformer model to automatically map schedule activities to ASTM UniFormat classes. • Sequencing knowledge is embedded in the model via concatenating [pred] and [succ] tokens. • Method achieves 93% and 87% F1-Scores in mapping UniFormat II Level 2 & 3 classes with schedule activities. • Our method serves as a universal identifier enabling automated creation of 4D BIMs. • Our method streamlines mapping between schedule, cost and payment application data. [ABSTRACT FROM AUTHOR]

Recent advances in reality capture technology focused on automating reality capture and devising robust computational models to convert the collected data into usable formats. However, these modern approaches are still challenged by the insufficient capacity of the resulting information to relay accurate and complete representations of the construction state due to the data's poor visual quality. In addition, the complexity of robotic path planning –especially for indoor construction– hinders automatic visual data collection due to cluttered, narrow, and dynamic construction spaces. This work targets both challenges by presenting a reinforcement learning model that optimizes indoor data collection policy for acquiring high-quality visual data using camera-equipped unmanned ground rovers. Results from three learned navigation policies show the capability of the method to provide high visual quality for the collected data. The learned policies reduced the data collection duration by 38.23% on average compared to the currently used automatic data collection strategies. The policies also provided a 31.04% average reduction in data collection distance compared to lawn-mower patterns. [Display omitted] • Reinforcement learning is used to learn a navigation policy maximizing data's visual quality and minimizing capture duration. • The optimized navigation policies are evaluated for data's visual quality and safe data collection navigation. • Divide-and-conquer algorithm reduces the optimization's computational complexity providing solutions in a timely manner. • Learnt capture plans achieve maximum visual quality in less duration than traditional collection plans. • Reinforcement learning provides ∼200% improvement in data collection duration compared to state-of-the-art NPV methods. [ABSTRACT FROM AUTHOR]

In recent years, computer vision algorithms have shown to effectively leverage visual data from jobsites for video-based activity analysis of construction equipment. However, earthmoving operations are restricted to site work and surrounding terrain, and the presence of other structures, particularly in urban areas, limits the number of viewpoints from which operations can be recorded. These considerations lower the degree of intra-activity and interactivity category variability to which said algorithms are exposed, hindering their potential for generalizing effectively to new jobsites. Secondly, training computer vision algorithms is also typically reliant on large quantities of hand-annotated ground truth. These annotations are burdensome to obtain and can offset the cost-effectiveness incurred from automating activity analysis. The main contribution of this paper is a means of inexpensively generating synthetic data to improve the capabilities of vision-based activity analysis methods based on virtual, kinematically articulated three-dimensional (3D) models of construction equipment. The authors introduce an automated synthetic data generation method that outputs a two-dimensional (2D) pose corresponding to simulated excavator operations that vary according to camera position with respect to the excavator and activity length and behavior. The presented method is validated by training a deep learning–based method on the synthesized 2D pose sequences and testing on pose sequences corresponding to real-world excavator operations, achieving 75% precision and 71% recall. This exceeds the 66% precision and 65% recall obtained when training and testing the deep learning–based method on the real-world data via cross-validation. Limited access to reliable amounts of real-world data incentivizes using synthetically generated data for training vision-based activity analysis algorithms. [ABSTRACT FROM AUTHOR]

[Display omitted] • A NLP transformer method for predicting implicit construction dependencies with 91% F1-Score. • A process for augmenting schedules with the role and flexibility of activity relationships. • A method for revising lookahead plans based on the flexibility of activity dependencies. • A Constrained Condition model for creating optimal lookahead plan revisions. Revising lookahead plans is practiced in all construction projects to meet contractual deadlines, mitigate activity delays, and disentangle process bottlenecks. We present a new machine learning-based method –which we call Implicit Logic Checker (ILC)– to learn the implicit dependency constraints and the flexibility of construction schedule relationships using a Transformer language model architecture. We then leverage relationships' flexibilities to provide a basis for creating optimal lookahead plan revisions to help mitigate the effects of unavoidable activity delays. We then deploy a Constrained Conditional (CC) model that applies learned construction planning knowledge to build revised lookahead plans and optimizes them for consistency and duration. Our ILC model is trained and tested on 35,332 manually labeled predecessor-successor relationships from eight real construction projects achieving an F1-Score of 91%. Similarly, the revised lookahead plans generated by the CC model on two case studies show reduced overall plan durations. [ABSTRACT FROM AUTHOR]

Accurate and efficient tracking, analysis and visualization of as-built (actual) status of buildings under construction are critical components of a successful project monitoring. Such information directly supports control decision-making and if automated, can significantly impact management of a project. This paper presents a new automated approach for recognition of physical progress based on two emerging sources of information: (1) unordered daily construction photo collections, which are currently collected at almost no cost on all construction sites; and (2) building information models (BIMs), which are increasingly turning into binding components of architecture/engineering/construction contracts. First, given a set of unordered and uncalibrated site photographs, an approach based on structure-from-motion, multiview stereo, and voxel coloring and labeling algorithms is presented that calibrates cameras, photorealistically reconstructs a dense as-built point cloud model in four dimensions (three dimensions + time), and traverses and labels the scene for occupancy. This strategy explicitly accounts for occlusions and allows input images to be taken far apart and widely distributed around the environment. An Industry Foundation Class-based (IFC-based) BIM is subsequently fused into the as-built scene by a robust registration step and is traversed and labeled for expected progress visibility. Next, a machine-learning scheme built upon a Bayesian probabilistic model is proposed that automatically detects physical progress in the presence of occlusions and demonstrates that physical progress monitoring at schedule activity level could be fully automated. Finally, the system enables the expected and reconstructed elements to be explored with an interactive, image-based, three-dimensional (3D) viewer where deviations are automatically color-coded over the IFC-based BIM. To that extent, the underlying hypotheses and algorithms for generating integrated four-dimensional (4D) as-built and as-planned models plus automated progress monitoring are presented. Experimental results are reported for challenging image data sets collected under different lighting conditions and severe occlusions from two ongoing building construction projects. This marks the presented model as being the first probabilistic model for automated progress tracking and visualization of deviations that incorporates both as-planned models and unordered daily photographs in a principled way. [ABSTRACT FROM AUTHOR]

Efficient data collection of high-quantity and low-cost highway assets such as road signs, traffic signals, light poles, and guardrails is a critical element to the operation, maintenance, and preservation of transportation infrastructure systems. Despite its importance, current practice of highway asset data collection is time-consuming, subjective, and potentially unsafe. The high volume of the data that needs to be collected can also negatively impact the quality of the analysis. To address these limitations, this paper proposes a new algorithm for semantic segmentation and recognition of highway assets using video frames collected from a car-mounted camera. The proposed set of algorithms (1) takes the captured frames and using a pipeline of structure from motion and multiview stereo reconstructs a three-dimensional (3D) point cloud model of the highway and surrounding assets; (2) using a Semantic Texton Forest classifier, each geo-registered two-dimensional (2D) video frame at the pixel-level is segmented based on shape, texture, and color of the highway assets; and finally, (3) based on the results of the 2D segmentation and a new voting scheme, each reconstructed 3D point in the cloud is also categorized for one type of asset and is color coded accordingly. The resulting augmented reality environment that integrates the color-coded point clouds with the geo-registered video frames enables a user to conduct visual walk through and query different categories of assets. Experiments were performed on a challenging video data set containing sequences filmed from a moving car on a 2.2-mi-long, two-lane highway research facility. Experimental results with an average accuracy of 76.50 and 86.75% in segmentation and pixel-level recognition of 12 types of asset categories reflect the promise of the applicability of this approach for segmentation and recognition of highway assets from image-based 3D point clouds. It also enables future algorithmic developments for 3D localization of traffic signs and other assets that are detected using the state-of-the-art vision-based methods. [ABSTRACT FROM AUTHOR]

Data sharing between different organizations is critical in supporting decision making in civil infrastructure engineering projects (e.g., transportation projects). Understanding the characteristics of data-sharing-related factors in civil infrastructure engineering is crucial for the civil engineering community to identify the priorities in promoting data sharing. The ASCE Data Sensing and Analysis (DSA) Committee initiated an investigation on data-sharing barriers in civil infrastructure engineering. This work produced (1) a classification of data-collection and data-use contexts in the domain of civil infrastructure engineering; (2) a comprehensive classification of data-sharing barriers and data-sharing promotion strategies in this domain; and (3) a synthesis of the opinions about the current practices, future vision, barriers, and data-sharing promotion strategies from civil infrastructure professionals and researchers using an online survey. The survey results revealed that (1) more data-sharing activities are occurring within the organizations, such as academia, government agencies, and for-profit corporations, compared with data exchanges among these communities; (2) "if there is a cost sharing in the data sharing," is the condition that makes data providers more willing to share data with others; and (3) compared with technology improvements, the protocol of data sharing in the business domain has more potential in promoting data sharing. Meanwhile, this research identified the top five priorities among 20 data-sharing promotion strategies, which are protocol in industry and organization, data-standard regulation, data validity, data security protection, and large-scale data-sharing technology. These five priorities play critical roles in a roadmap showing how policy, business models, and technical solutions should cooperate to promote data sharing in civil infrastructure engineering. [ABSTRACT FROM AUTHOR]

The three-dimensional mapping of the built environment is of particular importance for engineering applications such as monitoring work-in-progress and energy performance simulation. The state-of-the-art methods for fitting primitives, non-uniform B-Spline surface (NURBS) and solid geometry to point clouds still fail to account for all the topological variations or struggle with mapping of physical space to parameter space given unordered, incomplete, and noisy point clouds. Assuming an input of points that can be described by a single non-self-intersecting NURBS, this article presents a new method that leverages segmented point clouds and outputs NURBS surfaces. It starts by successively fitting uniform B-Spline curves in two-dimensional as planar cross-sectional cuts on each surface. An intermediate B-Spline surface is then computed by globally optimizing and lofting over the cross-sections. This surface is used to parameterize the points and perform final refinement to a NURBS. For cylindrical segments such as pipes, a new supervised method is also introduced to string the fitted segments, identify connection types, standardize the connections, and then refine them using NURBS optimization. Experimental results show the applicability of the proposed methods for as-built modeling purposes. [ABSTRACT FROM AUTHOR]

An inexpensive and robust 3D localization system for tracking the position of a user in GPS- or WLAN-denied environments offers significant potential for improving decision-making tasks for civil and infrastructure engineering applications. To this end, an infrastructure-free approach for 3D event localization on commodity smartphones is presented. In the proposed method, the position of the user is continuously tracked based on the smartphone sensory data (the Forward approach) until the user reaches a certain event. Here, an event location refers to the 3D location of a user conducting value-added activities such as tasks involved in emergency response and field reporting of operational issues. Once an event is observed, the motion trajectory of the user is backtracked from the postevent landmark to reestimate the location of the event (the Backward approach). By integrating probability distributions of the Forward and Backward approaches together, the proposed method derives the most-likely location of the event. To validate the proposed approach, seven case studies are conducted in a multistory parking garage. The experimental results show that the probabilistic integration of the localization results from the Forward and Backward dead reckonings can produce more accurate 3D localization results when compared to a single best estimate from a one-way dead reckoning process. Lessons learned from several real-world case studies and open research challenges in improving localization accuracy are discussed in detail. [ABSTRACT FROM AUTHOR]

Mobile augmented reality is an emerging technique which allows users to use a mobile device's camera to capture real-world imagery and view real-world physical objects and their associated cyber-information overlaid on top of imagery of them. One key challenge for mobile augmented reality is the fast and precisely localization of a user in order to determine what is visible in their camera view. Recent advances in Structure-from-Motion (SfM) enable the creation of 3D point clouds of physical objects from an unordered set of photographs taken by commodity digital cameras. The generated 3D point cloud can be used to identify the location and orientation of the camera relative to the point cloud. While this SfM-based approach provides complete pixel-accurate camera pose estimation in 3D without relying on external GPS or geomagnetic sensors, the preparation of initial 3D point cloud typically takes from hours to a day, making it difficult to use in mobile augmented reality applications. Furthermore, creating 3D cyber-information and associating it with the 3D point cloud is also a challenge of using SfM-based approach for mobile augmented reality. To overcome these challenges in 3D point cloud creation and cyber-physical content authoring, the paper presents a new SfM framework that is optimized for mobile augmented reality and rapidly generates a complete 3D point cloud of a target scene up to 28 times faster than prior approaches. Key improvements in the proposed SfM framework stem from the use of (1) state-of-the-art binary feature descriptors, (2) new filtering approach for accurate 3D modeling, (3) optimized point cloud structure for augmented reality, and (4) hardware/software parallelism. The paper also provides a new image-based 3D content authoring method designed specifically for the limited user interfaces of mobile devices. The proposed content authoring method generates 3D cyber-information from a single 2D image and automatically associates it with the 3D point cloud. [ABSTRACT FROM AUTHOR]

Over the last few years, new methods that detect construction progress deviations by comparing laser scanning or image-based point clouds with 4D BIM are developed. To create complete as-built models, these methods require the visual sensors to have proper line-of-sight and field-of-view to building elements. For reporting progress deviations, they also require Building Information Modeling (BIM) and schedule Work-Breakdown-Structure (WBS) with high Level of Development (LoD). While certain logics behind sequences of construction activities can augment 4D BIM with lower LoDs to support making inferences about states of progress under limited visibility, their application in visual monitoring systems has not been explored. To address these limitations, this paper formalizes an ontology that models construction sequencing rationale such as physical relationships among components. It also presents a classification mechanism that integrates this ontology with BIM to infer states of progress for partially and fully occluded components. The ontology and classification mechanism are validated using a Charrette test and by presenting their application together with BIM and as-built data on real-world projects. The results demonstrate the effectiveness and generality of the proposed ontology. It also illustrates how the classification mechanism augments 4D BIM at lower LoDs and WBS to enable visual progress assessment for partially and fully occluded BIM elements and provide detailed operational-level progress information. [ABSTRACT FROM AUTHOR]

The visibility of a traffic sign at night depends on its retro-reflectivity, a property that needs to be frequently monitored to ensure transportation safety. In the U.S., Federal Highway Administration (FHWA) maintains regulations to ensure minimum retro-reflectivity levels. Current measurement techniques either (a) use vehicle mounted device during the night, or (b) use manual handheld devices during the day. The former is expensive due to nighttime labor cost. The latter is time-consuming and unsafe. To address these limitations, this paper presents a computer vision-based technique to measure retro-reflectivity during daytime using a vehicle mounted device. The presented algorithms simulate nighttime visibility of traffic signs from images taken during daytime and measure their retro-reflectivity. The technique is faster, cheaper, and safer as it neither requires nighttime operation nor requires manual sign inspection. It also satisfies FHWA measurement guidelines both in terms of granularity and accuracy. The performance of the presented technique is evaluated under various testing conditions. The results are promising and demonstrate a strong potential in lowering inspection cost and improving safety in practical applications on retro-reflectivity measurement. [ABSTRACT FROM AUTHOR]

Workface assessment – the process of determining the overall activity rates of onsite construction workers throughout a day – typically involves manual visual observations which are time-consuming and labor-intensive. To minimize subjectivity and the time required for conducting detailed assessments, and allowing managers to spend their time on the more important task of assessing and implementing improvements, we propose a new inexpensive vision-based method using RGB-D sensors that is applicable to interior construction operations. This is a particularly challenging task as construction activities have a large range of intra-class variability including varying sequences of body posture and time-spent on each individual activity. The skeleton extraction algorithms from RGB-D sequences produce noisy outputs when workers interact with tools or when there is a significant body occlusion within the camera's field-of-view. Existing vision-based methods are also limited as they can primarily classify “atomic” activities from RGB-D sequences involving one worker conducting a single activity. To address these limitations, our method includes three components: 1) an algorithm for detecting, tracking, and extracting body skeleton features from depth images; 2) a discriminative bag-of-poses activity classifier for classifying single visual activities from a given body skeleton sequence; and 3) a Hidden Markov Model to represent emission probabilities in the form of a statistical distribution of single activity classifiers. For training and testing purposes, we introduce a new dataset of eleven RGB-D sequences for interior drywall construction operations involving three actual construction workers conducting eight different activities in various interior locations. Our results with an average accuracy of 76% on the testing dataset show the promise of vision-based methods using RGB-D sequences for facilitating the activity analysis workface assessment. [ABSTRACT FROM AUTHOR]

[Display omitted] • A method to predict logical relationships between any pair of schedule activities with F1-Score of 88.3%. • A method to schedule a short list of input activities based on the learned logic. • Formalization of indirect logic dependencies between schedule activities. • A method for mining indirect dependencies from an input schedule. In construction, schedule mistakes causing delays beyond substantial completion dates cost contractors expensive liquidated damages. Hence, several industry guidelines, such as the DCMA's 14 point assessment, define schedule quality and offer systematic methods for ensuring it. These guidelines list "logic" as an essential control metric, and they require planners to ensure their schedules are free of missing or wrong logical dependencies. Checking the logic requires extensive construction domain knowledge, and planners perform it entirely manually as there are no available software solutions that support it. This paper offers a novel machine learning-based solution that learns construction scheduling domain knowledge from existing records completely automatically and applies it to validate the logic in input schedules achieving an F1 score of 88.3%. Furthermore, we tailor our method to use the learned knowledge to schedule a list of unordered activities. The details of the method, experimental results, benefits, and limitations are discussed. [ABSTRACT FROM AUTHOR]

[Display omitted] • An NLP transformer for relationship prediction between look-ahead tasks and master schedule activities with 82.1% Precision. • A generative model for creating look-ahead planning task descriptions based on a master schedule activity description. • A semi-automated human-in-the-loop method for linking look-ahead tasks to master schedule activities with 76.5% Precision. • A formalized representation for matching look-ahead planning tasks to master schedule activities. In construction, master schedules and look-ahead plans are created at different times (monthly vs. weekly), by different personas (planner vs. superintendent), with different software (scheduling solution vs. spreadsheet), and at different levels of granularity (milestones vs. production details). Their full-alignment is essential for project coordination, progress updating, and payment application reviews, and its absence may lead to costly litigation. This paper presents the first attempt to automate linking look-ahead planning tasks to master-schedule activities following an NLP-based multi-stage ranking formulation. Our model employs distance-based matching for candidate generation and a Transformer architecture for final matching. 1 1 Access to the code and a sample of the dataset can be granted upon direct request from the authors. Validation results from real-world projects demonstrate that the method helps planners match look-ahead planning tasks to master schedule activities by presenting a list of top-five matches with a precision of 76.5%. We also show that the method helps superintendents create look-ahead plans from a master schedule by generating lists of tasks based on activity descriptions. [ABSTRACT FROM AUTHOR]

This paper presents a mobile workstation chariot (MWC) that enables first responders and civil engineers to undertake initial disaster reconnaissance and damage assessment operations by quickly traversing hazardous terrain and through provisioning of a necessary computing infrastructure to support real time communication with the command center. The MWC comprises a personal transporter equipped with on board communication equipment and computational processing capabilities to collect, archive, analyze, and report large quantities of data from a disaster site to provide better situation awareness of an emerging disaster scenario. In the case study presented here, the MWC uses a commercially available Segway Personal Transporter modified with a framework designed to carry a payload of information gathering and communication equipment. The MWC supports both horizontal and vertical real time data capture and transmission flow from first responders and civil engineers on the scene up to the command center by means of multilevel wireless voice and data communication infrastructures. This on board computational and visualization infrastructure allows for collecting, aggregating and deaggregating semiautomatic data, analyzing, recording, and reporting critical response and recovery information easily and quickly. [ABSTRACT FROM AUTHOR]

The aging of bridges coupled with increased vehicular traffic requires timely and accurate inspections for elevated highway structures. Recent studies have leveraged the advent of drones and computer vision to automatically conduct quick, safe, and effective inspections for elevated highway structures. However, such studies rarely offer insight or recommendations for an end-to-end integrated system that streamlines data collection, analytics, and reporting. Toward this goal, we present an end-to-end robotic bridge inspection system consisting of five tightly coupled methods to: (1) create automatic data collection missions; (2) assure visual quality of such missions; (3) reconstruct three-dimensional (3D) models of elevated structures; (4) detect and localize surface distresses in 3D; and (5) generate reports complying with highway agencies' requirements. We validate each developed method and the whole system on two representative inspection projects. Results show that our system can objectively satisfy requirements for data collection and provide up to 85.3% average precision over five defect types. We finally share lessons learned while deploying our system to 30 bridge inspection projects in the US and Japan, particularly for documenting, communicating, and following-up with bridge inspectors' recommendations. [ABSTRACT FROM AUTHOR]

Effective progress monitoring is crucial for successful construction project delivery. Visual data, comprising images and videos of construction operations, has emerged as a valuable source for gaining comprehensive insights into project status. While various vision-based methods have been developed for automated progress monitoring at the element level (e.g., columns, beams, floors, walls), challenges persist in achieving accurate schedule activity-level (e.g., formwork, reinforcement, concrete placement) progress tracking. Existing methods often struggle to report progress status beyond binary form (built/not built). To address these limitations, this research proposes a novel framework for precisely measuring the percent completion of activities associated with under-construction building elements. The framework is implemented through an Activity-level progress monitoring system (ALPMS). ALPMS takes the input of construction site images and a four-dimensional building information model (BIM) and outputs the activity-wise completion percentage required for updating project schedules. It creates as-built point clouds from images, compares them with the as-planned BIM, generates orthographic views of under-construction elements through projective transformation or neural radiance fields (NeRF), applies deep learning-based semantic segmentation for progress reasoning, and estimates completion percentages. Finally, the activity-level rich semantic information is transferred to the as-built point cloud and BIM for three-dimensional visualization of the progress status. When the framework was applied to two building construction projects, on average, the ALPMS could report the activity-wise completion percentages with <6% mean absolute error. Future research shall focus on automating the mapping of project schedule activities and activity-level progress details for timely schedule updates and prediction of project completion dates. • Activity percent complete measurement framework for outdoor and indoor construction monitoring. • Percentage completion estimation of schedule activities through semantic segmentation of orthographic view images. • Methodology for orthographic view synthesis through projective transformation and neural radiance field (NeRF). • 3D progress visualization through semantic segmentation of the as-built point cloud and color-coded BIM. • Semantic image segmentation dataset for activity level progress monitoring. [ABSTRACT FROM AUTHOR]

Today, computer vision object detection methods are used for safety inspections from site videos and images. These methods detect bounding boxes and use hand-made rules to enable personal protective equipment compliance checks. This paper presents a new method to improve the breadth and depth of vision-based safety compliance checking by explicitly classifying worker-tool interactions. A detection model is trained on a newly constructed image dataset for construction sites, achieving 52.9% average mean precision for 10 object categories and 89.4% average precision for detecting workers. Using this detector and new dataset, the proposed human-object interaction recognition model achieved 79.78% precision and 77.64% recall for hard hat checking; 79.11% precision and 75.29% recall for safety coloring checking. The new model also verifies hand protection for workers when tools are being used with 66.2% precision and 64.86% recall. The proposed model is superior in these checking tasks when compared with post-processing detected objects with hand-made rules, or applying detected objects only. • A learned Human-Object Interaction (HOI) recognition model is proposed for personal protective equipment compliance checking. • Final HOI recognition model is validated by hard hat, safety coloring, and hand protection checking. • Final HOI recognition model achieves higher precision and recall than using hand-made rules or detected objects only. [ABSTRACT FROM AUTHOR]

Recent reports on America's infrastructure have emphasized the importance of structural health monitoring of civil infrastructures. System identification is a key component of many structural health monitoring strategies. Current system identification methods estimate models of a structure by measuring displacements, accelerations, and strains with wired or wireless sensors. However, these methods typically involve installation of a limited number of sensors at discrete locations and require additional data acquisition devices. To overcome these limitations, computer vision-based techniques have been introduced recently that employ high-speed and high-resolution cameras. Such cameras can be quite costly and require tedious installation of targets. This paper investigates the potential of using consumer-grade cameras for structural system identification without the need to install targets. The underlying methods for target-free displacement measurements are introduced, including region of interest selection, feature detection, point tracking, and outlier removal. A set of experiments are conducted to assess the efficacy of the proposed approach by comparing the accuracy of the identified model with one obtained using a conventional wired system. Careful comparison of the results demonstrates the significant potential of the proposed approach. Copyright © 2016 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]