Search

Autonomous navigation of Unmanned Aerial Vehicles (UAVs) is crucial for effective assessment of large-scale civil infrastructure, as manual UAV control is time consuming and prone to mishaps. Autonomous navigation outdoors typically employs GPS signals to enable accurate localization and reduce long-term drifts. However, in many civil engineering applications, GPS signals are either poor or unavailable due to interference and/or multi-path effects. Current approaches for UAV localization in GPS-denied environments, track geometric features such as corners; however, these natural features can be misrepresented due to the presence of occlusions and/or image processing errors, resulting in significant localization errors that can grow with time. AprilTags can reduce localization errors, but installation on large-scale civil infrastructure can be challenging. Therefore, this paper proposes a framework that leverages the wealth of visual and geometric information encoded in a Graphics-Based Digital Twin (GBDT) of a target infrastructure to provide accurate localization of a UAV. The GBDT is comprised of a computer graphics model that faithfully represents a target structure's geometry, structural features, and visual textures. The visual details of the GBDT are exploited to design an object recognition algorithm that detects GBDTtags, which are distinctive objects (or a collection of components) on the target structure in advance; GBDTtags provide functionality similar to AprilTags. When the camera attached to a UAV detects one or more GBDTtags, the vertices of the GBDTtags are mapped from the image plane into the GBDT coordinate system, allowing for the UAV to be localized. The framework, termed herein as GBDTpose, is first validated numerically using Blender, Gazebo, and Mavros software-in-the-loop (SITL). Subsequently, field validation is carried out using the Kavita and Lalit Bahl Smart Bridge at the University of Illinois Urbana-Champaign (UIUC). Results show that localization in GPS-denied environments can be achieved with 5-50 cm accuracy without the need for physical markers being placed on the structure. [ABSTRACT FROM AUTHOR]

Visual recognition of 3D point cloud data of bridge inspection scenes is a key step in automating the visual inspection process, which is currently largely manual and inefficient. To alleviate the lack of large-scale annotated point cloud datasets for training such 3D visual recognition algorithms, this research investigates an approach for developing large-scale synthetic point cloud datasets. The proposed approach proceeds in four steps: (1) random generation of different types of bridges in computer graphics environments; (2) sampling of camera trajectories that represent data collection scenarios during bridge inspection; (3) 3D reconstruction using Structure from Motion (SfM) applied to rendered synthetic images; (4) automated annotation of the reconstructed point cloud using ground truth masks obtained with synthetic images. Besides, this research proposes to store point uncertainty information defined by the error between the ground truth depth and the depth calculated from the SfM results. Prior to training, thresholds can be applied to this uncertainty information to control the levels of outliers in the dataset. This research demonstrates the proposed approach by generating point cloud datasets for two data collection scenarios. The effectiveness of the generated datasets is investigated by training 3D semantic segmentation algorithms and evaluating the performance on real and synthetic point cloud data. The proposed approach for point cloud dataset generation will facilitate the development of generalizable and high level-of-detail 3D recognition algorithms toward autonomous bridge inspection. [ABSTRACT FROM AUTHOR]

Summary: Traditional post‐earthquake inspection of civil infrastructure is conducted manually, taking a considerable amount of time and often putting inspectors in harm's way. This problem is exacerbated in modern cities, where millions of people can be left homeless until their residences are deemed safe to reinhabit. Image collection enabled by commercial unmanned aerial vehicles (UAVs) combined with computer vision‐based techniques has provided an alternative with high potential for rapid post‐earthquake inspection. However, the extracted images of the damage alone are inadequate to evaluate the system‐level safety condition of a structure. The quality of the visual information also heavily relies on the effectiveness of the UAV inspection scheme which is susceptible to environmental uncertainties. To this end, a graphics‐based digital twin (GBDT) framework is developed for UAV‐aided post‐earthquake inspection of high‐rise buildings and validated using a high‐rise building in Guangzhou, China. The GBDT is comprised of a finite element (FE) model and a photorealistic computer graphics (CG) model, with the latter being informed by the former, jointly providing as a comprehensive virtual representation of the structure so that every step of the post‐earthquake inspection procedure can be planned and evaluated virtually. First, to avoid the cumbersome nature of constructing the graphical representation of the numerous components in high‐rise buildings, the CG model in the GBDT is created by automatically importing structural components from the FE model and adding nonstructural components according to the dimensions of the as‐built structure. This fast modeling process as well as the accuracy of the virtual presentation are validated by point cloud comparisons between the CG model and the as‐built structure. Subsequently, the GBDT is used to showcase the evaluation of UAV flight schemes for post‐earthquake inspection of high‐rise buildings. To shorten flight time and place more emphasis on potential damage, FE analysis is conducted to determine the earthquake‐induced damage locations. Consistent damage hotspots are then marked on the CG model, along with restrictions from the real environment such as obstacles, weak satellite signal, wind speed, and lighting conditions considered in the synthetic environment. Finally, applying the synthetic environment as the testbed, three UAV‐aided inspection schemes are implemented virtually and the best UAV flight scheme is determined for the assumed field inspection. This example demonstrates the flexibility of the GBDT in representing the real‐world structure and environmental conditions and its efficacy in assisting decision making for rapid and effective structural inspection in the aftermath of an earthquake. [ABSTRACT FROM AUTHOR]

Summary: Cracks can develop in high‐rise buildings because of long‐term environmental changes and extreme loading events such as strong winds or earthquakes. Although deep learning‐based identification methods can efficiently identify cracks, the accuracy of crack identification in high‐rise buildings needs to be improved due to the lack of crack datasets specifically related to high‐rise structures. Moreover, the number of available images of cracks in high‐rise is limited. To this end, this paper establishes an intelligent crack identification method based on a photorealistic synthetic modeling technique. First, a computer graphics (CG) model of a high‐rise building with assumed damage is constructed. Subsequently, the CG model is utilized to generate a dataset that includes photorealistic images of the high‐rise building as well as corresponding labels for various components and types of damage. The generated dataset is then used to train a DeepLabv3 + neural network for structural component and damage identification, followed by validation by employing images of both synthetic and full‐scale high‐rise buildings. The trained network can accurately identify different components in images of the full‐scale, high‐rise building and identify cracks that are intentionally synthesized in those images. The results show that the synthetic dataset generated by the CG model not only allows for fast and efficient labeling for the purpose of neural network training but also outperforms methods that do not consider any application‐specific context in crack identification. [ABSTRACT FROM AUTHOR]

This research investigates a framework for the efficient development of vision-based dense three-dimensional displacement measurement algorithms to support reliable structural health monitoring of civil structures. The framework exploits the use of a photo-realistic synthetic model, termed a physics-based graphics model, to simulate the entire process of vision-based measurement. At the same time, the synthetic environment is used to evaluate the performance of different post-processing algorithms quantitatively for a given measurement scenario, such as camera selection and camera placement. The effectiveness of the framework is demonstrated by optimizing the algorithms for the three-dimensional displacement measurement of a 14-bay laboratory truss structure. The vision-based dense three-dimensional displacement estimation algorithms optimized in this study consist of four steps: (1) camera parameter estimation, (2) camera motion estimation and compensation, (3) vision-based two-dimensional tracking, and (4) projection of two-dimensional tracking results to three-dimensional space. The algorithms use the knowledge from the finite element model to facilitate the implementation and maximize the measurement outcome, that is, model-informed approach. To test and evaluate the model-informed approach, synthetic videos are rendered for two measurement scenarios, that is, using a Digital Single Lens Reflex camera mounted on a tripod and using an Unmanned Aerial Vehicle camera. Then, the performance of the model-informed approach is evaluated by comparing the estimated displacement with the ground truth values. Based on the performance evaluation, an algorithm with the highest expected performance is selected for each of the two measurement scenarios. Finally, the selected algorithm is tested in a laboratory experiment. In contrast to the existing literature that investigates fixed individual measurement scenarios, the proposed framework can be used to test different measurement scenarios and estimate the outcome of each scenario before performing actual tests, facilitating the implementation of vision-based measurement for the structural health monitoring of civil structures. [ABSTRACT FROM AUTHOR]

After a seismic event, buildings need to be inspected to confirm their safety prior to reoccupation. As such, the rapid evaluation of the condition of individual buildings is important for minimizing disruption to lives and business. However, traditional manual inspection by experts is laborious and time-consuming. Structural health monitoring provides the potential to accelerate the required evaluation. This article proposes a cost-effective approach for rapid safety evaluation of buildings after seismic events using sparse acceleration measurements. First, a damage-sensitive feature is defined that can be used to infer the condition of buildings. Herein, the maximum interstory drift angle is proposed as a reliable damage index to classify the safety of buildings after seismic events. A convolutional neural network is then employed to uncover the complex relationship between the damage-sensitive features and the building condition. A five-story steel building is considered to validate the proposed approach. First, a three-dimensional nonlinear model of the building is created. To generate the required training data, a simplified nonlinear model is developed, along with a corresponding linear model, as use of the three-dimensional model is too computationally expensive. The training data for the convolutional neural network incorporates uncertainties in both the analysis model and the ground motion. Initial evaluation is conducted using the simplified nonlinear model, while final validation of the proposed approach is performed using the results of the three-dimensional nonlinear analysis model subjected to historical earthquakes. The results demonstrate the ability of the proposed approach to accommodate differences between the in-situ structure and the analysis model, as well as the efficacy of this approach for rapid postearthquake safety evaluation of buildings. [ABSTRACT FROM AUTHOR]

This research investigates vision‐based automated bridge component recognition, which is critical for automating visual inspection of bridges during initial response after earthquakes. Semantic segmentation algorithms with up to 45 convolutional layers are applied to recognize bridge components from images of complex scenes. One of the challenges in such scenarios is to get the recognition results consistent with high‐level scene structure using limited amount of training data. To impose the high‐level scene consistency, this research combines 10‐class scene classification and 5‐class bridge component classification. Three approaches are investigated to combine scene classification results into bridge component classification: (a) naïve configuration, (b) parallel configuration, and (c) sequential configuration of classifiers. The proposed approaches, sequential configuration in particular, are demonstrated to be effective in recognizing bridge components in complex scenes, showing less than 1% of accuracy loss from the naïve/parallel configuration for bridge images, and less than 1% false positives for the nonbridge images. [ABSTRACT FROM AUTHOR]