Search

In this paper, we introduce a novel approach for defect detection in hammering inspections using Unmanned Aerial Vehicles (UAVs). Despite the promising application of UAVs for inspecting hard-to-reach structures, like bridges, their efficiency is often compromised by the significant ego noise produced by their motor-propeller systems. This noise complicates the discrimination between healthy and defective hammering sounds. In previous research, methods to improve robustness through supervised learning have been proposed; however, these methods require the labeling of hammering sounds by skilled inspectors to train the discrimination model. To overcome this problem, we propose an ego-noise reduction method based on propeller vibrations. By reducing ego noise and thereby making the characteristics of hammering sound more dramatically clear, we enable unsupervised defect detection amidst ego noise. Our experiments with concrete specimens demonstrate that our technique achieves defect detection with an accuracy on par with the supervised method. The proposed method proves especially beneficial for hammering inspections, in which the domain gap–the variability in acoustic signatures of hammering sounds caused by differences in concrete mix ratios and curing conditions from one site to another–presents a significant challenge. Our approach effectively adapts to these variations, ensuring reliable defect detection across diverse construction environments. [ABSTRACT FROM AUTHOR]

In this study, a stable and high-speed vision-based self-position estimation method was proposed that improves upon the existing method of lane detection by recognizing the lighting facilities that are installed in tunnels on Japanese expressways where GNSS cannot be used. In addition, we proposed a method for inspecting multiple cracks at once by estimating the self-position with the successful rate 75% in the traveling direction by counting the lighting with the successful rate 99.85%. The effectiveness of the method was verified by capturing images of cracks in an actual tunnel. The proposed method will enable more frequent inspections for tunnel cracks that lead to flaking while maintaining infrastructure safety, reducing costs, and improving tunnel visual inspection flow efficiency. [ABSTRACT FROM AUTHOR]

Copyright of Electric Drive for Locomotives is the property of Electric Drive for Locomotives Editorial Office and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Abstract In recent years, the number of pipes that have exceeded their service life has increased. For this reason, earthworm-type robots equipped with cameras have been developed to perform regularly inspections of sewer pipes. However, inspection methods have not yet been established. This paper proposes a method for anomaly detection from images in pipes using Generative Adversarial Network (GAN). A model that combines f-AnoGAN and Lightweight GAN is used to detect anomalies by taking the difference between input images and generated images. Since the GANs are only trained with non-defective images, they are able to convert an image containing defects into one without them. Subtraction images is used to estimate the location of anomalies. Experiments were conducted using actual images of cast iron pipes to confirm the effectiveness of the proposed method. It was also validated using sewer-ml, a public dataset.

Many infrastructure facilities such as bridges, communication towers, and power transmission lines were constructed more than 30 years ago, and their inspection work is significant for disaster prevention and mitigation. Infrastructure inspections using unmanned aerial vehicles (UAVs) have been introduced and are expected to improve inspection efficiency significantly. However, infrastructure facilities are located in various places, and the cost of dispatching pilots and UAVs to each infrastructure is increasing costs. This paper proposes a migratory UAV (MiUAV) system in which a UAV automatically moves to infrastructure facilities and a pilot inspects remotely. The proposed MiUAV consists of multiple ports that can accommodate and charge UAVs, and the UAVs can travel to remote infrastructure by flying between the ports. Assuming an environment where multiple UAVs move simultaneously, setting up a route through a port with high utilization or a bad weather area will increase the travel time to the destination. The proposed route decision scheme uses weights considering port utilization ratio and inclement weather probability and autonomously sets a route to the destination for each UAV using the Dijkstra method. Computer simulations evaluate that the proposed scheme can reduce travel time by avoiding high-utilization ports and bad weather areas.

The structural condition of hydroelectric tunnels is important to the overall performance, safety, and longevity of generating stations. Significant effort is required to inspect, monitor, and maintain these tunnels. Photogrammetry is an effective method of collecting highly accurate visual and spatial data. However, it also presents the complex challenge of positioning a camera at thousands of difficult-to-reach locations throughout the large and varying-diameter tunnels. A semi-automated robotic camera positioning system was developed to enhance the collection of images within hydroelectric tunnels for photogrammetric inspections. A continuous spiral image network was developed to optimize the collection speed within the bounds of photography and capture-in-motion constraints. The positioning system and image network optimization reduce the time and effort required while providing the ability to adapt to different and varying tunnel diameters. To demonstrate, over 28,000 images were captured at a ground sampling distance of 0.4 mm in the 822 m long concrete-lined section of the Grand Falls Generating Station intake tunnel. [ABSTRACT FROM AUTHOR]

In recent years, the number of pipes that have exceeded their service life has increased. For this reason, earthworm-type robots equipped with cameras have been developed to perform regularly inspections of sewer pipes. However, inspection methods have not yet been established. This paper proposes a method for anomaly detection from images in pipes using Generative Adversarial Network (GAN). A model that combines f-AnoGAN and Lightweight GAN is used to detect anomalies by taking the difference between input images and generated images. Since the GANs are only trained with non-defective images, they are able to convert an image containing defects into one without them. Subtraction images is used to estimate the location of anomalies. Experiments were conducted using actual images of cast iron pipes to confirm the effectiveness of the proposed method. It was also validated using sewer-ml, a public dataset. [ABSTRACT FROM AUTHOR]

Inspection of critical infrastructure with drones is experiencing an increasing uptake in the industry driven by a demand for reduced cost, time, and risk for inspectors. Early deployments of drone inspection services involve manual drone operations with a pilot and do not obtain the technological benefits concerning autonomy, coordination, and cooperation. In this paper, we study the design needed to handle the complexity of an Unmanned Aerial System (UAS) to support autonomous inspection of safety-critical infrastructure. We apply a constructive research approach to link innovation needs with concepts, designs, and validations that include simulation and demonstration of key design parts. Our design approach addresses the complexity of the UAS and provides a selection of technology components for drone and ground control hardware and software including algorithms for autonomous operation and interaction with cloud services. The paper presents a drone perception system with accelerated onboard computing, communication technologies of the UAS, as well as algorithms for swarm membership, formation flying, object detection, and fault detection with artificial intelligence. We find that the design of a cooperative drone swarm and its integration into a custom-built UAS for infrastructure inspection is highly feasible given the current state of the art in electronic components, software, and communication technology. [ABSTRACT FROM AUTHOR]

In our previous works, we focused on the advantages of a passive mechanism in realizing autonomous robots, and we developed a multi-legged robot that climbs unknown vertical walls with uneven surfaces. In this study, we improve on our previous robot to broaden its workable area to include vertical pipes and corner walls. To demonstrate the effectiveness of the developed robot, experiments that included climbing uneven walls, corner walls, a large pipe, and parallel small pipes were conducted. Our results confirmed that the developed robot could climb in these environments. We conclude that the proposed mechanism is effective for climbing such unknown complex environments. The proposed robot is expected to be used in the inspection of large-scale infrastructures. [ABSTRACT FROM AUTHOR]

Efficient imaging is achieved under conditions of high relative velocity between the camera and the subject by using the following imaging system; two galvanometer mirrors are placed vertically in front of the camera, one for motion blur compensation and the other for switching the angle of view. The proposed system can overcome the shortcomings of conventional imaging systems with motion blur compensation, such as a small angle of view, and efficiently acquire high-resolution images. If the angle is changed for each capture while the mirrors are stationary for the exposure time, the natural frequency of the mirrors produces noise, leading to a poor resolution. However, this issue can be managed by generating and using an input that does not contain a natural frequency component. A target moving in one dimension can be captured and it is confirmed that the angle of view was extended from the obtained image. It is expected that the camera will be used for inspections under conditions where the relative speed between the camera and target is high, such as in highway tunnel inspections. [ABSTRACT FROM AUTHOR]

This article presents a constraint modeling approach to global coverage-path planning for linear-infrastructure inspection using multiple autonomous UAVs. The problem is mathematically formulated as a variant of the Min–Max K-Chinese Postman Problem (MM K-CPP) with multi-weight edges. A high-level constraint programming language is used to model the problem, which enables model execution with different third-party solvers. The optimal solutions are obtained in a reasonable time for most of the tested instances and different numbers of vehicles involved in the inspection. For some graphs with multi-weight edges, a time limit is applied, as the problem is NP-hard and the computation time increases exponentially. Despite that, the final total inspection cost proved to be lower when compared with the solution obtained for the unrestricted MM K-CPP with single-weight edges. This model can be applied to plan coverage paths for linear-infrastructure inspection, resulting in a minimal total inspection time for relatively simple graphs that resemble real transmission networks. For more extensive graphs, it is possible to obtain valid solutions in a reasonable time, but optimality cannot be guaranteed. For future improvements, further optimization could be considered, or different models could be developed, possibly involving artificial neural networks.

Inspection of critical infrastructure with drones is experiencing an increasing uptake in the industry driven by a demand for reduced cost, time, and risk for inspectors. Early deployments of drone inspection services involve manual drone operations with a pilot and do not obtain the technological benefits concerning autonomy, coordination, and cooperation. In this paper, we study the design needed to handle the complexity of an Unmanned Aerial System (UAS) to support autonomous inspection of safety-critical infrastructure. We apply a constructive research approach to link innovation needs with concepts, designs, and validations that include simulation and demonstration of key design parts. Our design approach addresses the complexity of the UAS and provides a selection of technology components for drone and ground control hardware and software including algorithms for autonomous operation and interaction with cloud services. The paper presents a drone perception system with accelerated onboard computing, communication technologies of the UAS, as well as algorithms for swarm membership, formation flying, object detection, and fault detection with artificial intelligence. We find that the design of a cooperative drone swarm and its integration into a custom-built UAS for infrastructure inspection is highly feasible given the current state of the art in electronic components, software, and communication technology.

The automation of inspection methods for concrete structures is a pressing issue worldwide. Weakly supervised approaches, i.e., approaches based on supervision in other forms than traditional class labels, offer a unique mix of automation and human involvement that is highly effective for critical tasks such as inspection work. Generating weak supervision is less tedious than generating training data for supervised learning approaches. However, since it is less informative, high amounts of weak supervision are often needed. In practice, it is often the case that only scarce amounts of weak supervision are available. In this article, we propose a novel approach for weakly supervised acoustic defect detection in concrete structures that augment human-provided weak supervision. Experiments in both laboratory and field conditions showed that the proposed method allows for considerable performance gains for low amounts of weak supervision. [ABSTRACT FROM AUTHOR]

Visual inspection of civil infrastructures has traditionally been a crucial yet labor-intensive task. In contrast, unmanned robots equipped with deep learning-based visual defect detection methods offer a more comprehensive and efficient solution compared to conventional manual inspection techniques. However, the full potential of deep learning in defect detection has yet to be fully realized, primarily due to the scarcity of annotated, high-quality defect datasets. In this study, we introduce CUBIT-Det, a high-resolution defect detection dataset that includes over 5500 images captured under various scenarios using professional-grade equipment. Distinguishing itself from existing datasets, CUBIT-Det encompasses a wide array of practical situations, backgrounds, and defect categories. We perform extensive benchmarking experiments on the dataset with nearly 30 cutting-edge real-time detection methods, and analyze both the impact of the dataset's annotation methods and zero-shot transfer ability of it. This effort lays a robust foundation for future advancements in defect detection solutions. Additionally, the practicality and effectiveness of CUBIT-Det are confirmed through thorough inspections of real-world buildings. Finally, we detail the features and acknowledge the limitations of our dataset, thereby highlighting significant opportunities for future research. • Proposed and open-sourced a high-resolution infrastructure defect dataset: CUBIT-Det and analyzed the features of this dataset. • Evaluated the CUBIT-Det dataset with nearly 30 state-of-the-art deep learning-based real-time object detection models and conducted the comprehensive analysis. • Implemented the real-world experiment to verify the reliability and feasibility of CUBIT-Det dataset. [ABSTRACT FROM AUTHOR]

We developed a new H-type climber for infrastructure inspection by applying the manufacturing technology for climbers used in space elevator experiments, which have been researched for many years. This inspection climber ascends and descends two belt tethers stretched in parallel, thereby reducing the degrees of freedom around each belt and making the climber more resistant to external disturbances such as wind and to twisting than when ascending and descending a single belt. The climber could move not only vertically but also horizontally and on inclines such as slopes. In addition, multiple climbers could be connected to each other by an aluminum frame, which has the advantage that various inspection units can be installed for versatile inspection. In this development work, we first conducted an experiment to investigate a drive system suitable for transporting heavy objects, and then developed the new climber for infrastructure inspection based on the results obtained. In addition, an optimal synchronous control system for position and velocity was developed as an alternative to the conventional optimal synchronous control system for position alone in order to keep the tilt of the inspection climber horizontal and to control its speed. The climber and control system were verified through simulations as well as actual experiments.

Decaying infrastructure maintenance cost allocation depends heavily on accurate and safe inspection in the field. New tools to conduct inspections can assist in prioritizing investments in maintenance and repairs. The industrial revolution termed as "Industry 4.0" is based on the intelligence of machines working with humans in a collaborative workspace. Contrarily, infrastructure management has relied on the human for making day-to-day decisions. New emerging technologies can assist during infrastructure inspections, to quantify structural condition with more objective data. However, today's owners agree in trusting the inspector's decision in the field over data collected with sensors. If data collected in the field is accessible during the inspections, the inspector decisions can be improved with sensors. New research opportunities in the human–infrastructure interface would allow researchers to improve the human awareness of their surrounding environment during inspections. This article studies the role of Augmented Reality (AR) technology as a tool to increase human awareness of infrastructure in their inspection work. The domains of interest of this research include both infrastructure inspections (emphasis on the collection of data of structures to inform management decisions) and emergency management (focus on the data collection of the environment to inform human actions). This article describes the use of a head-mounted device to access real-time data and information during their field inspection. The authors leverage the use of low-cost smart sensors and QR code scanners integrated with Augmented Reality applications for augmented human interface with the physical environment. This article presents a novel interface architecture for developing Augmented Reality–enabled inspection to assist the inspector's workflow in conducting infrastructure inspection works with two new applications and summarizes the results from various experiments. The main contributions of this work to computer-aided community are enabling inspectors to visualize data files from database and real-time data access using an Augmented Reality environment. [ABSTRACT FROM AUTHOR]

We are currently developing a mobile scaffolding device that can dodge the piers of bridges without dismantling the scaffold when moving between spans to improve efficiency and safety in bridge inspections. The mobile scaffolding device treated in this paper is split in the center of the device when it crosses the piers. In this process, the mobile scaffold is cantilevered and poses a danger to the workers on board. Therefore, we implemented horizontal holding control by wire reeling to keep the mobile scaffold horizontal. However, in this control method, the direction of motion of the mobile scaffold is only partially constrained by the wire. Thus, the device is greatly excited when an external force is applied. It is difficult to damp these oscillations in wire-reeling control. We designed a semi-active dynamic absorber for vibration control by applying the skyhook theory, a basic theory of the active suspension. When a random input that simulates the wind force was given to the mobile scaffolding device with the dynamic absorber, the vibration of the device was reduced by half compared with that of the mobile scaffolding device without the dynamic absorber. [ABSTRACT FROM AUTHOR]

Inspection of critical social infrastructures, which are mainly made of concrete, is a pressing issue. Weakly supervised methods are interesting for such critical tasks because they allow a unique mix of human involvement and automation. However, humans can make mistakes, resulting in the system being misled. In the present paper is proposed a framework for weakly supervised defect detection in concrete structures involving a consensus between several humans providing weak supervision. This allows to compensate for the shortcomings of each individual human and, therefore, yield better performance along with robustness to erroneous weak supervision. Experiments conducted with concrete test blocks in laboratory conditions showed the effectiveness of our proposed method. [ABSTRACT FROM AUTHOR]

Tracking and observing underwater structures at close range are important for gathering high-resolution information. This article proposes a method to track omnidirectional surfaces using an autonomous underwater vehicle (AUV). This method is suitable for surveying various kinds of underwater structures, such as rugged seafloors, ship hulls, pools, dams, and quay walls. It is based on stochastic means, and has two important features. First, it depends on only commercial off-the-shelf sensors (mechanical scanning imaging sonar, a water current sensor, and an attitude sensor). Second, it is applicable to fast moving vehicles. This method is implemented in the AUV highly agile terrain tracker for ocean research and investigation, and verification tests in the sea and in a tank are carried out to evaluate its performance. In the sea, the AUV succeeded in tracking the rugged seafloor at high speed. In the tank, the AUV succeeded in tracking vertical walls while keeping its roll angle at 90 $^\circ$. [ABSTRACT FROM AUTHOR]

This paper presents deterioration level estimation based on convolutional neural networks using a confidence-aware attention mechanism for infrastructure inspection. Spatial attention mechanisms try to highlight the important regions in feature maps for estimation by using an attention map. The attention mechanism using an effective attention map can improve feature maps. However, the conventional attention mechanisms have a problem as they fail to highlight important regions for estimation when an ineffective attention map is mistakenly used. To solve the above problem, this paper introduces the confidence-aware attention mechanism that reduces the effect of ineffective attention maps by considering the confidence corresponding to the attention map. The confidence is calculated from the entropy of the estimated class probabilities when generating the attention map. Because the proposed method can effectively utilize the attention map by considering the confidence, it can focus more on the important regions in the final estimation. This is the most significant contribution of this paper. The experimental results using images from actual infrastructure inspections confirm the performance improvement of the proposed method in estimating the deterioration level.

Manual inspection of infrastructure damages such as building cracks is difficult due to the objectivity and reliability of assessment and high demands of time and costs. This can be automated using unmanned aerial vehicles (UAVs) for aerial imagery of damages. Numerous computer vision-based approaches have been applied to address the limitations of crack detection but they have their limitations that can be overcome by using various hybrid approaches based on artificial intelligence (AI) and machine learning (ML) techniques. The convolutional neural networks (CNNs), an application of the deep learning (DL) method, display remarkable potential for automatically detecting image features such as damages and are less sensitive to image noise. A modified deep hierarchical CNN architecture has been used in this study for crack detection and damage assessment in civil infrastructures. The proposed architecture is based on 16 convolution layers and a cycle generative adversarial network (CycleGAN). For this study, the crack images were collected using UAVs and open-source images of mid to high rise buildings (five stories and above) constructed during 2000 in Sydney, Australia. Conventionally, a CNN network only utilizes the last layer of convolution. However, our proposed network is based on the utility of multiple layers. Another important component of the proposed CNN architecture is the application of guided filtering (GF) and conditional random fields (CRFs) to refine the predicted outputs to get reliable results. Benchmarking data (600 images) of Sydney-based buildings damages was used to test the proposed architecture. The proposed deep hierarchical CNN architecture produced superior performance when evaluated using five methods: GF method, Baseline (BN) method, Deep-Crack BN, Deep-Crack GF, and SegNet. Overall, the GF method outperformed all other methods as indicated by the global accuracy (0.990), class average accuracy (0.939), mean intersection of the union overall classes (IoU) (0.879), precision (0.838), recall (0.879), and F-score (0.8581) values. Overall, the proposed CNN architecture provides the advantages of reduced noise, highly integrated supervision of features, adequate learning, and aggregation of both multi-scale and multilevel features during the training procedure along with the refinement of the overall output predictions.

Collecting precise as-built data is essential for tracking construction progress. Three-dimensional models generated from such data capture the as-is conditions of the structures, providing valuable information for monitoring existing infrastructure over time. As-built data can be acquired using a wide range of remote sensing technologies, among which mobile LiDAR is gaining increasing attention due to its ability to collect high-resolution data over a relatively large area in a short time. The quality of mobile LiDAR data depends not only on the grade of onboard LiDAR scanners but also on the accuracy of direct georeferencing information and system calibration. Consequently, millimeter-level accuracy is difficult to achieve. In this study, the performance of mapping-grade and surveying-grade mobile LiDAR systems for bridge monitoring is evaluated against static laser scanners. Field surveys were conducted over a concrete bridge where grinding was required to achieve desired smoothness. A semi-automated, feature-based fine registration strategy is proposed to compensate for the impact of georeferencing and system calibration errors on mobile LiDAR data. Bridge deck thickness is evaluated using surface segments to minimize the impact of inherent noise in the point cloud. The results show that the two grades of mobile LiDAR delivered thickness estimates that are in agreement with those derived from static laser scanning in the 1 cm range. The mobile LiDAR data acquisition took roughly five minutes without having a significant impact on traffic, while the static laser scanning required more than three hours.

This paper presents a localization system for Unmanned Aerial Vehicles (UAVs) especially designed to be used in infrastructure inspection, where the UAVs have to fly in challenging conditions, such as relatively high altitude (e.g., 15 m), eventually with poor or absent GNSS (Global Navigation Satellite System) signal reception, or the need for a BVLOS (Beyond Visual Line of Sight) operation in some periods. In addition, these infrastructure inspection applications impose the following requirements for the localization system: defect traceability, accuracy, reliability, and fault tolerance. Our system proposes a lightweight solution combining multiple stereo cameras with a robotic total station to comply with these requirements, providing full-state estimation (i.e., position, orientation, and linear and angular velocities) in a fixed and time-persistent reference frame. Moreover, the system can align and fuse all sensor measurements in real-time at high frequency. We have integrated this localization system in our aerial platform, and we have tested its performance for inspection in a real-world viaduct scenario, where the UAV has to operate with poor or absent GNSS signal at high altitude.

This paper describes the development of a hanger-rope inspection robot for suspension bridges. The inspection robot inspects the hanger rope by going up and down the rope installed vertically downward from the main cable of a suspension bridge. The going-up-and-down mechanism of the robot consists of a drive roller driven by a motor and a non-excitation electromagnetic brake, and the robot can safely descend after climbing the rope at high speed. The developed robot is small in size and light in weight, and an inspection worker can easily install the robot on the rope. In addition, the robot can be wirelessly controlled with ease from the controller. First, this paper describes the hanger-rope inspection strategy of the suspension bridge. Then, the developed prototype 2 robot is introduced. Next, the result of the hanger-rope inspection in an actual suspension bridge and the problems are clearly revealed by experiment. Finally, the newly developed prototype 3 robot is introduced, and the result of the going-up-and-down experiment is described. [ABSTRACT FROM AUTHOR]

This paper presents a shadow-based operation assistant method for a pipeline-inspection robot using a variance value of the image histogram. By displacing the position of the head camera relative to that of the illuminator, a crescent-shaped shadow appears in the images captured in a bent pipe. The size, position, and orientation of the shadow depend on the robot's orientation around the pipe axis. By applying the pathway direction of the bent pipe obtained from the image processing to the rolling movement, the robot can automatically adjust its roll-orientation. Experiments were conducted in four types of pipeline environments to verify the autonomous navigation. These include a complex winding pipeline, a long straight pipe, and pipelines with replicated dust and dirt. [ABSTRACT FROM AUTHOR]

The purpose of this work is to establish a hammering echo inspection technology capable of detecting damage accurately irrespective of the skill of the inspector. To realize this technology, we have proposed and developed an "artificial intelligence (AI)-aided hammering test system" that automatically identifies the anomalous parts of a structure and the extent of the anomalies via the machine learning of the differences in hammering echoes. A laser range sensor is used to easily identify the hitting position of the hammer and integrate this information into the hammering echo analysis results to automatically generate an anomaly map. We performed hammering echo collection experiments using the AI-aided hammering test system and evaluated its performance. In the experiments, we inspected seven actual bridges in which internal defects (float) were detected by a detailed manual hammering test and compared the results with those obtained using our system. No defects were missed in a coarse block unit, and the accuracy for each hammering echo was determined to be 96.3% at maximum and 90.4% on average. [ABSTRACT FROM AUTHOR]

Most bridges, tunnels, dams, and other infrastructure facilities were constructed in the 1950s to 1960s. At present, aging in these facilities is a social problem. Most of the maintenance check-ups of these infrastructure facilities are manually conducted by close visual inspection, which has limitations. As a result, there have been many proposals on inspection systems for unmanned infrastructure facilities, such as drones and cable robots. However, there are also problems with the use of cable robots and drones, such as restrictions in places that they can be used and effects from the environment. Thus, this study proposes a robotics system for infrastructure inspection that solves such problems. Two robots move on parallel ropes stretching in an H shape, and one robot with a three-dimensional gimbal and cameramoves on a rope stretching between those robots. At this time, the three robots are capable of efficient, highly accurate wide-ranging observation by carrying out synchronous position control using wireless communication. The control system has optimum synchronous position capabilities and uses a disturbance observer. Regarding this infrastructure inspection robotics system, this study discusses the development of three mobile robotics systems and the synchronous position control of two robots. [ABSTRACT FROM AUTHOR]

Efficiently monitoring the condition of civil infrastructure requires automating the structural condition assessment in visual inspection. This paper proposes an Attention-Enhanced Co-Interactive Fusion Network (AECIF-Net) for automatic structural condition assessment in visual bridge inspection. AECIF-Net can simultaneously parse structural elements and segment surface defects on the elements in inspection images. It integrates two task-specific relearning subnets to extract task-specific features from an overall feature embedding. A co-interactive feature fusion module further captures the spatial correlation and facilitates information sharing between tasks. Experimental results demonstrate that the proposed AECIF-Net outperforms the current state-of-the-art approaches, achieving promising performance with 92.11% mIoU for element segmentation and 87.16% mIoU for corrosion segmentation on the test set of the new benchmark dataset Steel Bridge Condition Inspection Visual (SBCIV). An ablation study verifies the merits of the designs for AECIF-Net, and a case study demonstrates its capability to automate structural condition assessment. • A deep learning model AECIF-Net is developed for automating the visual assessment of structural conditions. • SBCIV, an image dataset with annotations of structural elements and their surface defects, is created. • AECIF-Net outperforms current methods in segmenting structural elements and surface defects. • Merits of the designs for the AECIF-Net are verified by comprehensive experiments. [ABSTRACT FROM AUTHOR]

Dams are a critical infrastructure system for many communities, but they are also one of the most challenging to inspect. Dams are typically very large and complex structures, and the result is that inspections are often time-intensive and require expensive, specialized equipment and training to provide inspectors with comprehensive access to the structure. The scale and nature of dam inspections also introduce additional safety risks to the inspectors. Unmanned aerial vehicles (UAV) have the potential to address many of these challenges, particularly when used as a data acquisition platform for photogrammetric three-dimensional (3D) reconstruction and analysis, though the nature of both UAV and modern photogrammetric methods necessitates careful planning and coordination for integration. This paper presents a case study on one such integration at the Brighton Dam, a large-scale concrete gravity dam in Maryland, USA. A combination of multiple UAV platforms and multi-scale photogrammetry was used to create two comprehensive and high-resolution 3D point clouds of the dam and surrounding environment at intervals. These models were then assessed for their overall quality, as well as their ability to resolve flaws and defects that were artificially applied to the structure between inspection intervals. The results indicate that the integrated process is capable of generating models that accurately render a variety of defect types with sub-millimeter accuracy. Recommendations for mission planning and imaging specifications are provided as well.

Unmanned aerial vehicles (UAVs) have gained significant attention in recent years. Low-cost platforms using inexpensive sensor payloads have been shown to provide satisfactory flight and navigation capabilities. In this report, we survey vision and control methods that can be applied to low-cost UAVs, and we list some popular inexpensive platforms and application fields where they are useful. We also highlight the sensor suites used where this information is available. We overview, among others, feature detection and tracking, optical flow and visual servoing, low-level stabilization and high-level planning methods. We then list popular low-cost UAVs, selecting mainly quadrotors. We discuss applications, restricting our focus to the field of infrastructure inspection. Finally, as an example, we formulate two use-cases for railway inspection, a less explored application field, and illustrate the usage of the vision and control techniques reviewed by selecting appropriate ones to tackle these use-cases. To select vision methods, we run a thorough set of experimental evaluations.

Abstract Routine inspection and maintenance are critical for the proper functioning of civil infrastructures such as bridges, pavements and underground structures. Civil infrastructures are being inspected less frequently because of the high cost and long duration of current inspection procedures. Furthermore, conventional inspection procedures often interrupt the routine functioning of the infrastructure, are inspector-dependent and expose the inspectors to complex and unsafe working environments. Visual inspection technologies play a crucial role in the inspection and maintenance of civil infrastructures. Automation-assisted technologies such as drones and underwater vehicles equipped with multiple imaging and sensing systems have been developed to address some of these issues with the conventional visual inspection processes. This paper reviews peer-reviewed research publications investigating automated visual inspection technologies following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Specifically, 53 publications satisfying a set of inclusion criteria were reviewed, its results highlighting the application domain, the level of autonomy of the automated systems, the sensor technologies used for the inspection process and navigation, the navigation and control technologies and the algorithms used. The review of the articles revealed that the data collected by automation is used to augment the qualitative assessment. Several types of algorithms such as target detection and image enhancing have been developed to reduce the inspector bias in these automated technologies. Path planning algorithms reduce the workload on the inspector by automating the navigation and control tasks. Remotely operated systems reduce the risk to the inspectors by minimizing their exposure to the inspection environment. However, only a limited number of studies investigated the human factors aspects of the automation-assisted inspection process. It is important to understand the cognitive, physical, and temporal demands these technologies place on inspectors to improve the design of systems assisting in the inspection process. Moreover, factors such as automation bias, trust in the system and communication between the automation and the operator need to be investigated. Furthermore, it is important to incorporate appropriate decision aids that support adequate situation awareness in the interface design. Based on these findings this review proposes directions for future research. This review concludes by highlighting the need for human-centered research to develop better solutions for infrastructure inspection problems. Highlights • Reviewed 53 articles on visual infrastructure inspection technologies • Thirty seven percentage of the articles focused on bridge inspection • Review found a lack of integration of human factors considerations in system design. • Calls for a multidisciplinary approach to design visual inspection automation [ABSTRACT FROM AUTHOR]

This paper describes a Mission Definition System and the automated flight process it enables to implement measurement plans for discrete infrastructure inspections using aerial platforms, and specifically multi-rotor drones. The mission definition aims at improving planning efficiency with respect to state-of-the-art waypoint-based techniques, using high-level mission definition primitives and linking them with realistic flight models to simulate the inspection in advance. It also provides flight scripts and measurement plans which can be executed by commercial drones. Its user interfaces facilitate mission definition, pre-flight 3D synthetic mission visualisation and flight evaluation. Results are delivered for a set of representative infrastructure inspection flights, showing the accuracy of the flight prediction tools in actual operations using automated flight control. [ABSTRACT FROM AUTHOR]

Accurate condition assessment of in-service infrastructure systems is critical for system-wide prioritization decisions. Current protocols require lengthy inspections and expensive equipment to examine large infrastructure systems. Furthermore, changes in inspection protocols over time can create discontinuities in recording and understanding the time history of a structure. To address these challenges, a systematic and adaptive technique for converting two-dimensional (2D) digital images into three-dimensional (3D) models has been developed, with the goal of creating high-resolution and scale-accurate inspection records. The developed reconstruction technique utilizes multiscale imaging to reconstruct a structure with varying levels of details and geometric complexity. The captured images are then converted into photorealistic, accurate, and dense 3D scene reconstructions by utilizing a hierarchical adaptation of a dense structure-from-motion (DSfM) algorithm. The result of this approach is a virtual 3D model of the structure with accurate geometry and high-fidelity representation of fine details. The accuracy, completeness, adaptability, and feasibility of the developed method were compared to both terrestrial laser scanner (TLS) and conventional DSfM methods. Experimental findings and direct comparison between generated point clouds indicate that the developed hierarchical technique consistently produces dense point clouds capable of resolving 0.1-mm details, an order of magnitude improvement over existing methods. The developed technique also allows for user-controlled point cloud density, a significant benefit over current reconstruction methods. [ABSTRACT FROM AUTHOR]

To avert potential crisis from Japan’s aging infrastructure and declining birth rate, the Japanese Government is planning to introduce robotic technology for the inspection of social infrastructure (such as pipes, dams, and bridges). Recording underwater positions is a difficult task for human divers who undertake conventional dam inspections. This study presents the Anchor Diver 5.2 system for efficient and effectual dam inspection. Anchor Diver 5.2 is based on an extended-tether-maneuvered remotely operated vehicle (ROV) equipped with cameras. The ROV is lowered into water by a hoist system from a boat and implements a visual survey of the concrete underwater structure. To improve the visibility of the ROV in murky and cloudy water, a novel concept named Water Loupe is proposed. In addition, a simple boat-fixing method is proposed to provide a stable base on the water surface, and the underwater position of the ROV, which cannot be accessed by global positioning systems, is recorded using a feasible localization method. Finally, the developed system was evaluated in field experiments conducted in the Amagase Dam, Japan, and its merits and problems are discussed. [ABSTRACT FROM AUTHOR]

We describe an automated system for detecting, localising, clustering and ranking visual changes on tunnel surfaces. The system is designed to provide assistance to expert human inspectors carrying out structural health monitoring and maintenance on ageing tunnel networks. A three-dimensional tunnel surface model is first recovered from a set of reference images using Structure from Motion techniques. New images are localised accurately within the model and changes are detected versus the reference images and model geometry. We formulate the problem of detecting changes probabilistically and evaluate the use of different feature maps and a novel geometric prior to achieve invariance to noise and nuisance sources such as parallax and lighting changes. A clustering and ranking method is proposed which efficiently presents detected changes and further improves the inspection efficiency. System performance is assessed on a real data set collected using a low-cost prototype capture device and labelled with ground truth. Results demonstrate that our system is a step towards higher frequency visual inspection at a reduced cost. [ABSTRACT FROM AUTHOR]

Audio commands are a preferred communication medium to keep inspectors in the loop of civil infrastructure inspection performed by a semi-autonomous drone. To understand job-specific commands from a group of heterogeneous and dynamic inspectors, a model must be developed cost-effectively for the group and easily adapted when the group changes. This paper is motivated to build a multi-tasking deep learning model that possesses a Share–Split–Collaborate architecture. This architecture allows the two classification tasks to share the feature extractor and then split subject-specific and keyword-specific features intertwined in the extracted features through feature projection and collaborative training. A base model for a group of five authorized subjects is trained and tested on the inspection keyword dataset collected by this study. The model achieved a 95.3% or higher mean accuracy in classifying the keywords of any authorized inspectors. Its mean accuracy in speaker classification is 99.2%. Due to the richer keyword representations that the model learns from the pooled training data Adapting the base model to a new inspector requires only a little training data from that inspector Like five utterances per keyword. Using the speaker classification scores for inspector verification can achieve a success rate of at least 93.9% in verifying authorized inspectors and 76.1% in detecting unauthorized ones. Further The paper demonstrates the applicability of the proposed model to larger-size groups on a public dataset. This paper provides a solution to addressing challenges facing AI-assisted human–robot interaction Including worker heterogeneity Worker dynamics And job heterogeneity. • The Share–Split–Collaborate multitask learning architecture is suitable for speaker-keyword classification. • Subject-specific and phonetic-specific features intertwined in audio data can be disentangled. • Rich keyword representations are learned from multi-subject spoken command data. • Small data of new speakers are sufficient for adding new classes to the speaker classifier. • Speaker classification scores are also effective for the speaker verification. [ABSTRACT FROM AUTHOR]