Search

Visual inspection of civil infrastructures has traditionally been a crucial yet labor-intensive task. In contrast, unmanned robots equipped with deep learning-based visual defect detection methods offer a more comprehensive and efficient solution compared to conventional manual inspection techniques. However, the full potential of deep learning in defect detection has yet to be fully realized, primarily due to the scarcity of annotated, high-quality defect datasets. In this study, we introduce CUBIT-Det, a high-resolution defect detection dataset that includes over 5500 images captured under various scenarios using professional-grade equipment. Distinguishing itself from existing datasets, CUBIT-Det encompasses a wide array of practical situations, backgrounds, and defect categories. We perform extensive benchmarking experiments on the dataset with nearly 30 cutting-edge real-time detection methods, and analyze both the impact of the dataset's annotation methods and zero-shot transfer ability of it. This effort lays a robust foundation for future advancements in defect detection solutions. Additionally, the practicality and effectiveness of CUBIT-Det are confirmed through thorough inspections of real-world buildings. Finally, we detail the features and acknowledge the limitations of our dataset, thereby highlighting significant opportunities for future research. • Proposed and open-sourced a high-resolution infrastructure defect dataset: CUBIT-Det and analyzed the features of this dataset. • Evaluated the CUBIT-Det dataset with nearly 30 state-of-the-art deep learning-based real-time object detection models and conducted the comprehensive analysis. • Implemented the real-world experiment to verify the reliability and feasibility of CUBIT-Det dataset. [ABSTRACT FROM AUTHOR]

Efficiently monitoring the condition of civil infrastructure requires automating the structural condition assessment in visual inspection. This paper proposes an Attention-Enhanced Co-Interactive Fusion Network (AECIF-Net) for automatic structural condition assessment in visual bridge inspection. AECIF-Net can simultaneously parse structural elements and segment surface defects on the elements in inspection images. It integrates two task-specific relearning subnets to extract task-specific features from an overall feature embedding. A co-interactive feature fusion module further captures the spatial correlation and facilitates information sharing between tasks. Experimental results demonstrate that the proposed AECIF-Net outperforms the current state-of-the-art approaches, achieving promising performance with 92.11% mIoU for element segmentation and 87.16% mIoU for corrosion segmentation on the test set of the new benchmark dataset Steel Bridge Condition Inspection Visual (SBCIV). An ablation study verifies the merits of the designs for AECIF-Net, and a case study demonstrates its capability to automate structural condition assessment. • A deep learning model AECIF-Net is developed for automating the visual assessment of structural conditions. • SBCIV, an image dataset with annotations of structural elements and their surface defects, is created. • AECIF-Net outperforms current methods in segmenting structural elements and surface defects. • Merits of the designs for the AECIF-Net are verified by comprehensive experiments. [ABSTRACT FROM AUTHOR]

Abstract Routine inspection and maintenance are critical for the proper functioning of civil infrastructures such as bridges, pavements and underground structures. Civil infrastructures are being inspected less frequently because of the high cost and long duration of current inspection procedures. Furthermore, conventional inspection procedures often interrupt the routine functioning of the infrastructure, are inspector-dependent and expose the inspectors to complex and unsafe working environments. Visual inspection technologies play a crucial role in the inspection and maintenance of civil infrastructures. Automation-assisted technologies such as drones and underwater vehicles equipped with multiple imaging and sensing systems have been developed to address some of these issues with the conventional visual inspection processes. This paper reviews peer-reviewed research publications investigating automated visual inspection technologies following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Specifically, 53 publications satisfying a set of inclusion criteria were reviewed, its results highlighting the application domain, the level of autonomy of the automated systems, the sensor technologies used for the inspection process and navigation, the navigation and control technologies and the algorithms used. The review of the articles revealed that the data collected by automation is used to augment the qualitative assessment. Several types of algorithms such as target detection and image enhancing have been developed to reduce the inspector bias in these automated technologies. Path planning algorithms reduce the workload on the inspector by automating the navigation and control tasks. Remotely operated systems reduce the risk to the inspectors by minimizing their exposure to the inspection environment. However, only a limited number of studies investigated the human factors aspects of the automation-assisted inspection process. It is important to understand the cognitive, physical, and temporal demands these technologies place on inspectors to improve the design of systems assisting in the inspection process. Moreover, factors such as automation bias, trust in the system and communication between the automation and the operator need to be investigated. Furthermore, it is important to incorporate appropriate decision aids that support adequate situation awareness in the interface design. Based on these findings this review proposes directions for future research. This review concludes by highlighting the need for human-centered research to develop better solutions for infrastructure inspection problems. Highlights • Reviewed 53 articles on visual infrastructure inspection technologies • Thirty seven percentage of the articles focused on bridge inspection • Review found a lack of integration of human factors considerations in system design. • Calls for a multidisciplinary approach to design visual inspection automation [ABSTRACT FROM AUTHOR]

Audio commands are a preferred communication medium to keep inspectors in the loop of civil infrastructure inspection performed by a semi-autonomous drone. To understand job-specific commands from a group of heterogeneous and dynamic inspectors, a model must be developed cost-effectively for the group and easily adapted when the group changes. This paper is motivated to build a multi-tasking deep learning model that possesses a Share–Split–Collaborate architecture. This architecture allows the two classification tasks to share the feature extractor and then split subject-specific and keyword-specific features intertwined in the extracted features through feature projection and collaborative training. A base model for a group of five authorized subjects is trained and tested on the inspection keyword dataset collected by this study. The model achieved a 95.3% or higher mean accuracy in classifying the keywords of any authorized inspectors. Its mean accuracy in speaker classification is 99.2%. Due to the richer keyword representations that the model learns from the pooled training data Adapting the base model to a new inspector requires only a little training data from that inspector Like five utterances per keyword. Using the speaker classification scores for inspector verification can achieve a success rate of at least 93.9% in verifying authorized inspectors and 76.1% in detecting unauthorized ones. Further The paper demonstrates the applicability of the proposed model to larger-size groups on a public dataset. This paper provides a solution to addressing challenges facing AI-assisted human–robot interaction Including worker heterogeneity Worker dynamics And job heterogeneity. • The Share–Split–Collaborate multitask learning architecture is suitable for speaker-keyword classification. • Subject-specific and phonetic-specific features intertwined in audio data can be disentangled. • Rich keyword representations are learned from multi-subject spoken command data. • Small data of new speakers are sufficient for adding new classes to the speaker classifier. • Speaker classification scores are also effective for the speaker verification. [ABSTRACT FROM AUTHOR]

• A drone was used to fly over an airfield and collect image data for inspection purposes. • A convolutional neural network was trained for semantic segmentation on airfield pavement data to detect cracks and utility patches. • PCI values could be calculated from drone image data using a series of computer algorithms. Infrastructure inspection and maintenance is a necessary, and often costly, process required for civil engineering structures, which has led to a movement toward automated technologies within this domain. This paper describes a method for conducting a partially automated airfield pavement condition assessment, similar to the manual ASTM Pavement Condition Index (PCI), using drone mounted imaging technology. Aerial images were processed using a convolutional neural network trained for pavement distress detection to generate a condition index. Automated data collected on a single auxiliary airfield in Colorado Springs, CO produced a condition index value of 56.5, which strongly agreed with manual inspection results that calculated a PCI value of 54 for the same runway. These results indicate that automation of some time-intensive aspects of pavement evaluation is feasible using drone-captured images and machine learning. The results of this study represent a single test case; future work should focus on expanding this framework to include the effects of location-dependent variability. [ABSTRACT FROM AUTHOR]

We present a depth super-resolved 3D imaging method for inspection of defects in walls using terahertz swept-source optical coherence tomography. A depth profile analysis based on an annihilating filter method with noise reduction was applied for signal processing, and the results were compared with those of conventional inverse fast Fourier transform (IFFT) analysis. To evaluate the depth resolution, we measured a plastic step sample with 1 mm thickness, and its front and back surfaces were discriminated. This is corresponding to about 1/3 of the theoretical depth resolution of the conventional IFFT analysis. Moreover, we demonstrated 3D imaging of the void with a 10 mm width behind a ceramic tile and succeeded in reconstructing a significantly sharper cross-sectional void image than in the conventional analysis. [Display omitted] • THz-wave optical coherence tomography for nondestructive infrastructure inspection. • 3D imaging of voids behind a tile using a THz-wave interferometer. • Depth super-resolution analysis reconstructs sharper images than conventional method. [ABSTRACT FROM AUTHOR]