Search

This paper presents deterioration level estimation based on convolutional neural networks using a confidence-aware attention mechanism for infrastructure inspection. Spatial attention mechanisms try to highlight the important regions in feature maps for estimation by using an attention map. The attention mechanism using an effective attention map can improve feature maps. However, the conventional attention mechanisms have a problem as they fail to highlight important regions for estimation when an ineffective attention map is mistakenly used. To solve the above problem, this paper introduces the confidence-aware attention mechanism that reduces the effect of ineffective attention maps by considering the confidence corresponding to the attention map. The confidence is calculated from the entropy of the estimated class probabilities when generating the attention map. Because the proposed method can effectively utilize the attention map by considering the confidence, it can focus more on the important regions in the final estimation. This is the most significant contribution of this paper. The experimental results using images from actual infrastructure inspections confirm the performance improvement of the proposed method in estimating the deterioration level.

Manual inspection of infrastructure damages such as building cracks is difficult due to the objectivity and reliability of assessment and high demands of time and costs. This can be automated using unmanned aerial vehicles (UAVs) for aerial imagery of damages. Numerous computer vision-based approaches have been applied to address the limitations of crack detection but they have their limitations that can be overcome by using various hybrid approaches based on artificial intelligence (AI) and machine learning (ML) techniques. The convolutional neural networks (CNNs), an application of the deep learning (DL) method, display remarkable potential for automatically detecting image features such as damages and are less sensitive to image noise. A modified deep hierarchical CNN architecture has been used in this study for crack detection and damage assessment in civil infrastructures. The proposed architecture is based on 16 convolution layers and a cycle generative adversarial network (CycleGAN). For this study, the crack images were collected using UAVs and open-source images of mid to high rise buildings (five stories and above) constructed during 2000 in Sydney, Australia. Conventionally, a CNN network only utilizes the last layer of convolution. However, our proposed network is based on the utility of multiple layers. Another important component of the proposed CNN architecture is the application of guided filtering (GF) and conditional random fields (CRFs) to refine the predicted outputs to get reliable results. Benchmarking data (600 images) of Sydney-based buildings damages was used to test the proposed architecture. The proposed deep hierarchical CNN architecture produced superior performance when evaluated using five methods: GF method, Baseline (BN) method, Deep-Crack BN, Deep-Crack GF, and SegNet. Overall, the GF method outperformed all other methods as indicated by the global accuracy (0.990), class average accuracy (0.939), mean intersection of the un

This paper presents deterioration level estimation based on convolutional neural networks using a confidence-aware attention mechanism for infrastructure inspection. Spatial attention mechanisms try to highlight the important regions in feature maps for estimation by using an attention map. The attention mechanism using an effective attention map can improve feature maps. However, the conventional attention mechanisms have a problem as they fail to highlight important regions for estimation when an ineffective attention map is mistakenly used. To solve the above problem, this paper introduces the confidence-aware attention mechanism that reduces the effect of ineffective attention maps by considering the confidence corresponding to the attention map. The confidence is calculated from the entropy of the estimated class probabilities when generating the attention map. Because the proposed method can effectively utilize the attention map by considering the confidence, it can focus more on the important regions in the final estimation. This is the most significant contribution of this paper. The experimental results using images from actual infrastructure inspections confirm the performance improvement of the proposed method in estimating the deterioration level.

Objectius de Desenvolupament Sostenible::9 - Indústria, Innovació i Infraestructura

This paper presents deterioration level estimation based on convolutional neural networks using a confidence-aware attention mechanism for infrastructure inspection. Spatial attention mechanisms try to highlight the important regions in feature maps for estimation by using an attention map. The attention mechanism using an effective attention map can improve feature maps. However, the conventional attention mechanisms have a problem as they fail to highlight important regions for estimation when an ineffective attention map is mistakenly used. To solve the above problem, this paper introduces the confidence-aware attention mechanism that reduces the effect of ineffective attention maps by considering the confidence corresponding to the attention map. The confidence is calculated from the entropy of the estimated class probabilities when generating the attention map. Because the proposed method can effectively utilize the attention map by considering the confidence, it can focus more on the important regions in the final estimation. This is the most significant contribution of this paper. The experimental results using images from actual infrastructure inspections confirm the performance improvement of the proposed method in estimating the deterioration level.

This paper presents deterioration level estimation based on convolutional neural networks using a confidence-aware attention mechanism for infrastructure inspection. Spatial attention mechanisms try to highlight the important regions in feature maps for estimation by using an attention map. The attention mechanism using an effective attention map can improve feature maps. However, the conventional attention mechanisms have a problem as they fail to highlight important regions for estimation when an ineffective attention map is mistakenly used. To solve the above problem, this paper introduces the confidence-aware attention mechanism that reduces the effect of ineffective attention maps by considering the confidence corresponding to the attention map. The confidence is calculated from the entropy of the estimated class probabilities when generating the attention map. Because the proposed method can effectively utilize the attention map by considering the confidence, it can focus more on the important regions in the final estimation. This is the most significant contribution of this paper. The experimental results using images from actual infrastructure inspections confirm the performance improvement of the proposed method in estimating the deterioration level.

This paper presents a localization system for Unmanned Aerial Vehicles (UAVs) especially designed to be used in infrastructure inspection, where the UAVs have to fly in challenging conditions, such as relatively high altitude (e.g., 15 m), eventually with poor or absent GNSS (Global Navigation Satellite System) signal reception, or the need for a BVLOS (Beyond Visual Line of Sight) operation in some periods. In addition, these infrastructure inspection applications impose the following requirements for the localization system: defect traceability, accuracy, reliability, and fault tolerance. Our system proposes a lightweight solution combining multiple stereo cameras with a robotic total station to comply with these requirements, providing full-state estimation (i.e., position, orientation, and linear and angular velocities) in a fixed and time-persistent reference frame. Moreover, the system can align and fuse all sensor measurements in real-time at high frequency. We have integrated this localization system in our aerial platform, and we have tested its performance for inspection in a real-world viaduct scenario, where the UAV has to operate with poor or absent GNSS signal at high altitude.

This paper presents a localization system for Unmanned Aerial Vehicles (UAVs) especially designed to be used in infrastructure inspection, where the UAVs have to fly in challenging conditions, such as relatively high altitude (e.g., 15 m), eventually with poor or absent GNSS (Global Navigation Satellite System) signal reception, or the need for a BVLOS (Beyond Visual Line of Sight) operation in some periods. In addition, these infrastructure inspection applications impose the following requirements for the localization system: defect traceability, accuracy, reliability, and fault tolerance. Our system proposes a lightweight solution combining multiple stereo cameras with a robotic total station to comply with these requirements, providing full-state estimation (i.e., position, orientation, and linear and angular velocities) in a fixed and time-persistent reference frame. Moreover, the system can align and fuse all sensor measurements in real-time at high frequency. We have integrated this localization system in our aerial platform, and we have tested its performance for inspection in a real-world viaduct scenario, where the UAV has to operate with poor or absent GNSS signal at high altitude.

We are currently developing a mobile scaffolding device that can dodge the piers of bridges without dismantling the scaffold when moving between spans to improve efficiency and safety in bridge inspections. The mobile scaffolding device treated in this paper is split in the center of the device when it crosses the piers. In this process, the mobile scaffold is cantilevered and poses a danger to the workers on board. Therefore, we implemented horizontal holding control by wire reeling to keep the mobile scaffold horizontal. However, in this control method, the direction of motion of the mobile scaffold is only partially constrained by the wire. Thus, the device is greatly excited when an external force is applied. It is difficult to damp these oscillations in wire-reeling control. We designed a semi-active dynamic absorber for vibration control by applying the skyhook theory, a basic theory of the active suspension. When a random input that simulates the wind force was given to the mobile scaffolding device with the dynamic absorber, the vibration of the device was reduced by half compared with that of the mobile scaffolding device without the dynamic absorber., Sensors and Materials, 33(3), pp. 959-972; 2021

This paper presents a localization system for Unmanned Aerial Vehicles (UAVs) especially designed to be used in infrastructure inspection, where the UAVs have to fly in challenging conditions, such as relatively high altitude (e.g., 15 m), eventually with poor or absent GNSS (Global Navigation Satellite System) signal reception, or the need for a BVLOS (Beyond Visual Line of Sight) operation in some periods. In addition, these infrastructure inspection applications impose the following requirements for the localization system: defect traceability, accuracy, reliability, and fault tolerance. Our system proposes a lightweight solution combining multiple stereo cameras with a robotic total station to comply with these requirements, providing full-state estimation (i.e., position, orientation, and linear and angular velocities) in a fixed and time-persistent reference frame. Moreover, the system can align and fuse all sensor measurements in real-time at high frequency. We have integrated this localization system in our aerial platform, and we have tested its performance for inspection in a real-world viaduct scenario, where the UAV has to operate with poor or absent GNSS signal at high altitude.

This dissertation investigates how to plan paths for Unmanned Aerial Vehicles (UAV) for the task of covering an environment. Three increasingly complex coverage problems based on the environment that needs to be covered are studied. The dissertation starts with a 2D point coverage problem where the UAV needs to visit a set of sites on the ground plane by flying on a fixed altitude plane parallel to the ground. The UAV has limited battery capacity which may make it infeasible to visit all the points. A novel symbiotic UAV and Unmanned Ground Vehicle (UGV) system where the UGV acts as a mobile recharging station is proposed. A practical, efficient algorithm for solving this problem using Generalized Traveling Salesperson Problem (GTSP) solver is presented. Then the algorithm is extended to a coverage problem that covers 2D regions on the ground with a UAV that can operate in fixed-wing or multirotor mode. The algorithm is demonstrated through proof-of-concept experiments. Then this algorithm is applied to covering 2D regions, not all of which lie on the same plane. This is motivated by bridge inspection application, where the UAV is tasked with visually inspecting planar regions on the bridge. Finally, a general version of the problem where the UAV is allowed to fly in complete 3D space and the environment to be covered is in 3D as well is presented. An algorithm that clusters viewpoints on the surface of a 3D structure and has an UAV autonomously plan online paths to visit all viewpoints is presented. These online paths are re-planned in real time as the UAV obtains new information on the structure and strives to obtain an optimal 3D coverage path.

This paper presents a localization system for Unmanned Aerial Vehicles (UAVs) especially designed to be used in infrastructure inspection, where the UAVs have to fly in challenging conditions, such as relatively high altitude (e.g., 15 m), eventually with poor or absent GNSS (Global Navigation Satellite System) signal reception, or the need for a BVLOS (Beyond Visual Line of Sight) operation in some periods. In addition, these infrastructure inspection applications impose the following requirements for the localization system: defect traceability, accuracy, reliability, and fault tolerance. Our system proposes a lightweight solution combining multiple stereo cameras with a robotic total station to comply with these requirements, providing full-state estimation (i.e., position, orientation, and linear and angular velocities) in a fixed and time-persistent reference frame. Moreover, the system can align and fuse all sensor measurements in real-time at high frequency. We have integrated this localization system in our aerial platform, and we have tested its performance for inspection in a real-world viaduct scenario, where the UAV has to operate with poor or absent GNSS signal at high altitude.

Att underhålla infrastruktur med hög standard är viktigt för alla länder, och att misslyckas med detta innebär allvarliga logistiska och ekonomiska konsekvenser. Kraftledningsinspektion är en betydelsefull del i detta. Denna uppsats har sökt svar på vad för- och nackdelarna är med att använda satellitövervakning av kraftledningar, samt svar på ifall teknik för satellitövervakning av kraftledningar också kan tillämpas på volymberäkningar av skog. Metoden har utgått från intervjuer med experter och relevanta företag samt litteratur som underlag. Att undersöka vilka för- och nackdelar som finns med satellitövervakning av kraftledningar var viktigt eftersom satellitövervakning är ett snabbt växande fält, men inte särskilt väl undersökt. Att undersöka huruvida teknik för kraftledningsövervakning med satelliter är tillräcklig för att estimera skogsvolym bedömdes vara värdefullt eftersom skogsvolym idag estimeras med luftburen LiDAR, medan luftburen LiDAR påstods vara signifikant mycket dyrare överlag än satellitmätningar. Det fanns alltså en eventuell ekonomisk fördel med att estimera skogsvolym med satelliter istället för dagens luftburna mätningar. Det förväntade resultatet var att tekniken för satellitövervakning av kraftledningar är tillräcklig för att estimera skogsvolym. De största nackdelarna med satellitövervakning av kraftledningar berör problemen med att nå tillräckligt hög noggrannhet i processerna för trädidentifiering, samt att utveckla effektiva tillvägagångssätt för att utvärdera detta då underlaget gällande föreslagna och utvärderade metoder är glest. En annan nackdel visade sig vara att satellitmetoderna är svåra att göra konkurrenskraftiga i jämförelse med de etablerade luftburna LiDAR-metoderna i fråga om kostnader. Anledningen är att de högupplösta satellitbilder som ofta krävts för att nå hög noggrannhet fortfarande är dyra, även om en fördel som också identifierades var att ny och billigare satellitteknik just nu utvecklas i hög takt. Gällande denna, Maintaining infrastructure of high standard is important for all countries. Failing this means severe logistical and economic consequences. Power line inspection is an important part of this. This thesis has searched for an answer to what the advantages and disadvantages are of inspecting power lines by using satellites, as well as an answer to if the technology of satellite surveillance of power lines is sufficient to estimate forest volume. The methodology of the thesis has been to turn to companies and experts in the field and to use relevant literature. Examining what the advantages and disadvantages of satellite inspection of power lines are was important since satellite surveillance is a growing field, but not very well researched. To analyze whether technology of satellite surveillance of power lines is enough to estimate forest volume was thought to be valuable since forest volume today is estimated by airborne LiDAR, while airborne LiDAR was claimed to be significantly more expensive in general than 3 satellite measurements. Thus, there was a potential economic advantage to estimate forest volume with satellites instead of airborne measurements. The expected result was that the technology of satellite surveillance of power lines is sufficient to estimate forest volume. The biggest disadvantages of satellite surveillance of power lines involve the problems of achieving high enough accuracy in the processes of tree identification, as well as developing effective formulas to evaluate this when the research material of proposed methods is sparse. Another disadvantage turned out to be that the satellite methods are hard to compete with, in comparison to the established airborne LiDAR methods and in regard to cost. The reason is that the high-resolution satellite images that often are demanded still are expensive, even though an advantage that also was identified in this thesis is that new and cheaper satellite technology is being developed at a quick rate. The b

Att underhålla infrastruktur med hög standard är viktigt för alla länder, och att misslyckas med detta innebär allvarliga logistiska och ekonomiska konsekvenser. Kraftledningsinspektion är en betydelsefull del i detta. Denna uppsats har sökt svar på vad för- och nackdelarna är med att använda satellitövervakning av kraftledningar, samt svar på ifall teknik för satellitövervakning av kraftledningar också kan tillämpas på volymberäkningar av skog. Metoden har utgått från intervjuer med experter och relevanta företag samt litteratur som underlag. Att undersöka vilka för- och nackdelar som finns med satellitövervakning av kraftledningar var viktigt eftersom satellitövervakning är ett snabbt växande fält, men inte särskilt väl undersökt. Att undersöka huruvida teknik för kraftledningsövervakning med satelliter är tillräcklig för att estimera skogsvolym bedömdes vara värdefullt eftersom skogsvolym idag estimeras med luftburen LiDAR, medan luftburen LiDAR påstods vara signifikant mycket dyrare överlag än satellitmätningar. Det fanns alltså en eventuell ekonomisk fördel med att estimera skogsvolym med satelliter istället för dagens luftburna mätningar. Det förväntade resultatet var att tekniken för satellitövervakning av kraftledningar är tillräcklig för att estimera skogsvolym. De största nackdelarna med satellitövervakning av kraftledningar berör problemen med att nå tillräckligt hög noggrannhet i processerna för trädidentifiering, samt att utveckla effektiva tillvägagångssätt för att utvärdera detta då underlaget gällande föreslagna och utvärderade metoder är glest. En annan nackdel visade sig vara att satellitmetoderna är svåra att göra konkurrenskraftiga i jämförelse med de etablerade luftburna LiDAR-metoderna i fråga om kostnader. Anledningen är att de högupplösta satellitbilder som ofta krävts för att nå hög noggrannhet fortfarande är dyra, även om en fördel som också identifierades var att ny och billigare satellitteknik just nu utvecklas i hög takt. Gällande denna, Maintaining infrastructure of high standard is important for all countries. Failing this means severe logistical and economic consequences. Power line inspection is an important part of this. This thesis has searched for an answer to what the advantages and disadvantages are of inspecting power lines by using satellites, as well as an answer to if the technology of satellite surveillance of power lines is sufficient to estimate forest volume. The methodology of the thesis has been to turn to companies and experts in the field and to use relevant literature. Examining what the advantages and disadvantages of satellite inspection of power lines are was important since satellite surveillance is a growing field, but not very well researched. To analyze whether technology of satellite surveillance of power lines is enough to estimate forest volume was thought to be valuable since forest volume today is estimated by airborne LiDAR, while airborne LiDAR was claimed to be significantly more expensive in general than 3 satellite measurements. Thus, there was a potential economic advantage to estimate forest volume with satellites instead of airborne measurements. The expected result was that the technology of satellite surveillance of power lines is sufficient to estimate forest volume. The biggest disadvantages of satellite surveillance of power lines involve the problems of achieving high enough accuracy in the processes of tree identification, as well as developing effective formulas to evaluate this when the research material of proposed methods is sparse. Another disadvantage turned out to be that the satellite methods are hard to compete with, in comparison to the established airborne LiDAR methods and in regard to cost. The reason is that the high-resolution satellite images that often are demanded still are expensive, even though an advantage that also was identified in this thesis is that new and cheaper satellite technology is being developed at a quick rate. The b

The modern industrial society is built upon the productivity of advanced computing and agile machines assisting the human workforce to perform their tasks more effectively, accurately and efficiently. The industrial revolution termed as "Industry 4.0" is based on the intelligence of machines working with humans in a collaborative work-space. Contrarily, infrastructure management has relied on the human for making day to day decisions. New emerging technologies can assist during infrastructure inspections, to quantify structural condition with more objective data. However, today's’ owners agree in trusting the inspector's decision in the field over data collected with sensors. If data collected in the field would be accessible during the inspections, the inspector decisions would be improved with sensors. New research opportunities in the human-infrastructure interface would allow researchers to improve the human awareness of their surrounding environment during inspections. This MS thesis studies the role of Augmented Reality (AR) and sensor technology as tools to increase human awareness of infrastructure. The domains of interest of this research include both inspections and emergency scenarios where humans need fast information about their environment to save lives. The results of vi this MS thesis research are the design, programming, and validation of two applications using AR headsets: AR-QR code scanning and AR-sensor connection. The two new programs facilitate the interface of humans with infrastructure, laying out a new scenario for technology enabling new research, tools, and test-beds for human-infrastructure interface research. The results of this MS thesis lay the foundation for future growth in the area of machine-assisted human-centered structure inspections, emergency management and human-machine collaboration using AR.

The modern industrial society is built upon the productivity of advanced computing and agile machines assisting the human workforce to perform their tasks more effectively, accurately and efficiently. The industrial revolution termed as "Industry 4.0" is based on the intelligence of machines working with humans in a collaborative work-space. Contrarily, infrastructure management has relied on the human for making day to day decisions. New emerging technologies can assist during infrastructure inspections, to quantify structural condition with more objective data. However, today's’ owners agree in trusting the inspector's decision in the field over data collected with sensors. If data collected in the field would be accessible during the inspections, the inspector decisions would be improved with sensors. New research opportunities in the human-infrastructure interface would allow researchers to improve the human awareness of their surrounding environment during inspections. This MS thesis studies the role of Augmented Reality (AR) and sensor technology as tools to increase human awareness of infrastructure. The domains of interest of this research include both inspections and emergency scenarios where humans need fast information about their environment to save lives. The results of vi this MS thesis research are the design, programming, and validation of two applications using AR headsets: AR-QR code scanning and AR-sensor connection. The two new programs facilitate the interface of humans with infrastructure, laying out a new scenario for technology enabling new research, tools, and test-beds for human-infrastructure interface research. The results of this MS thesis lay the foundation for future growth in the area of machine-assisted human-centered structure inspections, emergency management and human-machine collaboration using AR.