Search

Wind turbine generator system plays a fundamental role in electricity generation in industry 4.0, and wind turbines are usually distributed separately and in poor locations. Unmanned Aerial Vehicles (UAV) which could overcome the above challenges are deployed to collect photographs of wind turbines, could be used for predictive maintenance of wind turbines and energy management. However, identifying meaningful information from huge amounts of photographs taken by drones is a challenging task due to various scales, different viewpoints, and tedious manual annotation. Besides, deep neural networks (DNN) are dominant in object detection, and training DNN requires large numbers of accurately labeled training data, and manual data annotation is tedious, inefficient, and error-prone. Considering these issues, we generate a synthetic UAV-taken dataset of wind turbines, which provides RGB images, target bounding boxes, and precise pixel annotations as well. But directly transferring the model trained on the synthetic dataset to the real dataset may lead to poor performance due to domain shifts (or domain gaps). The predominant approaches to alleviate the domain discrepancy are adversarial feature learning strategies, which focus on feature alignment for style (e.g., color, texture, illumination, etc.) gaps without considering the content (e.g., densities, backgrounds, and layout scenes) gaps. In this study, we scrutinize the real UAV-taken imagery of wind turbines and develop a synthetic generation method trying to simulate the real ones from the aspects of style and content. Besides, we propose a novel soft-masks guided faster region-based convolutional neural network (SMG Faster R-CNN) for domain adaptation in wind turbine detection, where the soft masks help to extract highly object-related features and suppress domain-specific features. We evaluate the accuracy of SMG Faster R-CNN on the wind turbine dataset and demonstrate the effectiveness of our approach compared with some prevalent object detection models and some adversarial DA models.

A bridge damage detector with preserving integrity based on modified Faster region-based convolutional neural network (R-CNN) is proposed for multiple damage types. The methodologies of dataset collection, damage annotation, and anchors generation are modified. The performance for bridge multiple-damage detectors with ResNet50 or ResNet101 as feature extraction network are compared. The results show that, with the modified Faster R-CNN, the mean average precision reaches 84.56% (76.43%) at the intersection-over-union metrics of 0.5 (0.75). We further demonstrate that the localization offset for Faster R-CNN is lower than that of YOLOv3. The modified bridge damage detector enables better detecting performance, and can preserve the damage integrity. [ABSTRACT FROM AUTHOR]

The productivity analysis of cable crane transportation in the construction field is of great significance to improve crane equipment management and reduce operation costs. However, the traditional manual recording method of analyzing cable crane productivity is time-consuming and tedious. The existing vision-based method requires significant amounts of time to collect extensive images at construction sites and does not achieve high-precision detection in complex scenes. Thus, an automated vision-based method for productivity analysis of cable crane transportation is proposed using a new synthetic image approach based on an augmented reality (AR) technique. The unmanned aerial vehicle-based three-dimensional (3D) reconstruction of a crane bucket model is superimposed on a realistic scene using AR to synthesize the images for vision-based model training without manual image acquisition at a construction site. The feature pyramid network and attention module are integrated into Faster region-based convolutional neural network (Faster R-CNN) to enhance the capability of feature extraction for the high-precision detection of a crane bucket and its ID number, which provides the logical basis for calculating productivity. The proposed vision-based productivity analysis method is evaluated on large-scale hydraulic engineering. The results demonstrate that the mean average precision (mAP) of detection performance is 98.01% using the model trained by AR-based synthetic images, which confirms the proposed AR-based synthetic image method could provide a new image generation mode for the construction industry. Additionally, the bias of productivity between the proposed method and ground truth is 0.03%, which confirms the effectiveness and accuracy of the proposed method. [ABSTRACT FROM AUTHOR]

Asbestos is a class 1 carcinogen, and it has become clear that it harms the human body. Its use has been banned in many countries, and now the investigation and removal of installed asbestos has become a very important social issue. Accordingly, many social costs are expected to occur, and an efficient asbestos investigation method is required. So far, the examination of asbestos slates was performed through visual inspection. With recent advances in deep learning technology, it is possible to distinguish objects by discovering patterns in numerous training data. In this study, we propose the use of drone images and a faster region-based convolutional neural network (Faster R-CNN) to identify asbestos slates in target sites. Furthermore, the locations of detected asbestos slates were estimated using orthoimages and compiled cadastral maps. A total of 91 asbestos slates were detected in the target sites, and 91 locations were estimated from a total of 45 addresses. To verify the estimated locations, an on-site survey was conducted, and the location estimation method obtained an accuracy of 98.9%. The study findings indicate that the proposed method could be a useful research method for identifying asbestos slate roofs.

Colon cancer is the third most common cancer in the world, and it is increasingly threatening people’s health. Early diagnosis is crucial to reducing the threat; however, the chance of missed polyps in today’s colonoscopy examination is still high (about 10%) due to limitations in diagnosis techniques and data analysis methods. The colonoscope is a kind of robot and on its tip there is a camera to acquire images. This paper presents a study aimed to improve the rate of successful diagnosis with a new image data analysis approach based on the faster regional convolutional neural network (faster R-CNN). This new approach has two steps for data analysis: (i) pre-processing of images to characterize polyps, and (ii) incorporating of the result of the pre-processing into the faster R-CNN. Specifically, the pre-processing of colonoscopy was expected to reduce the influence of specular reflections, resulting in an improved image, upon which the faster R-CNN algorithm was aplied. There are several improvements of the faster r-CNN tailoring to the task of colon polyps detection. To confirm the superiority of this new approach, the mean average precision (mAP) was used to compare the results obtained with the new approach and the faster R-CNN algorithm. The experimental result shows that the mAP of the new approach is 91.43%, as opposed to 90.57% with the faster R-CNN, which shows a significant improvement. [ABSTRACT FROM AUTHOR]

In this study, we propose a novel method incorporating faster region-based convolutional neural network and active shape model to automatically recognize, segment, and track the left ventricle in cardiac ultrasound image sequences, respectively. Ultrasound images typically contain noise and artifacts. The conventional filters cannot preserve the edges of image contours, and thus blurry images are often obtained. In this study, we propose an improved adaptive anisotropic diffusion filter to effectively reduce noise and reinforce image contours. In addition, because of the shape and appearance of the left ventricle vary considerably between adjacent images, conventional methods cannot automatically identify the position of the left ventricle or accurately segment them. A novel method that combines the faster region-based convolutional neural network with the active shape model is proposed to automatically recognize, segment, and track the left ventricle in cardiac ultrasound image sequences. Compared with four state-of-the-art approaches, the method proposed in this study can be applied to accurately segment and track the left ventricle in cardiac ultrasound image sequences. The proposed method produces the most satisfactory results in terms of visual presentation and segmentation quality based on four criteria.

Lesions of tea (Camellia sinensis) leaves are detrimental to the growth of tea crops. Their adverse effects include further disease of tea leaves and a direct reduction in yield and profit. Therefore, early detection and on‐site monitoring of tea leaf lesions are necessary for effective management to control infections and prevent further yield loss. In this study, 1,822 images of tea leaves with lesions caused by three diseases (brown blight, Colletotrichum camelliae; blister blight, Exobasidium vexans; and algal leaf spot, Cephaleuros virescens) and four pests (leaf miner, Tropicomyia theae; tea thrip, Scirtothrips dorsalis; tea leaf roller, Homona magnanima; and tea mosquito bug, Helopeltis fasciaticollis) were collected from northern and central Taiwan. A faster region‐based convolutional neural network (Faster R‐CNN) was then trained to detect the locations of the lesions on the leaves and to identify the causes of the lesions. The trained Faster R‐CNN detector achieved a precision of 77.5%, recall of 70.6%, an F1 score of 73.91%, and a mean average precision of 66.02%. An overall accuracy of 89.4% was obtained for identification of the seven classes of tea diseases and pests. The developed detector could assist tea farmers in identifying the causes of lesions in real time. [ABSTRACT FROM AUTHOR]

In order to solve the problem of small objects detection in unmanned aerial vehicle (UAV) aerial images with complex background, a general detection method for multi-scale small objects based on Faster region-based convolutional neural network (Faster R-CNN) is proposed. The bird's nest on the high-voltage tower is taken as the research object. Firstly, we use the improved convolutional neural network ResNetl01 to extract object features, and then use multi-scale sliding windows to obtain the object region proposals on the convolution feature maps with different resolutions. Finally, a deconvolution operation is added to further enhance the selected feature map with higher resolution, and then it taken as a feature mapping layer of the region proposals passing to the object detection sub-network. The detection results of the bird's nest in UAV aerial images show that the proposed method can precisely detect small objects in aerial images. [ABSTRACT FROM AUTHOR]

This paper attempts to replace the traditional manual slag offloading of magnesium smelting in Pidgeon process with robotic slag removal. Specifically, the high-temperature infrared dot matrix was used to measure the slag positions indirectly; the faster region-based convolutional neural network (Faster R-CNN) was trained with thermal image of the reduction jar as the dataset; the isothermal image of the reduction jar was plotted based on the slag centers, and adopted to detect the opening direction of the jar and the slag positions. The indirect measurement results show that the actual internal temperature of the jar can be detected accurately through repeated experiments, with an error of less than 10 °C. Finally, the proposed method was verified through a case study on 1,000 images. The results show that our model can correctly identify more than 90% of crude magnesium in the actual jar. [ABSTRACT FROM AUTHOR]

Abstract Sanitary sewer systems are designed to collect and transport sanitary wastewater and stormwater. Pipe inspection is important in identifying both the type and location of pipe defects to maintain the normal sewer operations. Closed-circuit television (CCTV) has been commonly utilized for sewer pipe inspection. Currently, interpretation of the CCTV images is mostly conducted manually to identify the defect type and location, which is time-consuming, labor-intensive and inaccurate. Conventional computer vision techniques are explored for automated interpretation of CCTV images, but such process requires large amount of image pre-processing and the design of complex feature extractor for certain cases. In this study, an automated approach is developed for detecting sewer pipe defects based on a deep learning technique namely faster region-based convolutional neural network (faster R-CNN). The detection model is trained using 3000 images collected from CCTV inspection videos of sewer pipes. After training, the model is evaluated in terms of detection accuracy and computation cost using mean average precision (mAP), missing rate, detection speed and training time. The proposed approach is demonstrated to be applicable for detecting sewer pipe defects accurately with high accuracy and fast speed. In addition, a new model is constructed and several hyper-parameters are adjusted to study the influential factors of the proposed approach. The experiment results demonstrate that dataset size, initialization network type and training mode, and network hyper-parameters have influence on model performance. Specifically, the increase of dataset size and convolutional layers can improve the model accuracy. The adjustment of hyper-parameters such as filter dimensions or stride values contributes to higher detection accuracy, achieving an mAP of 83%. The study lays the foundation for applying deep learning techniques in sewer pipe defect detection as well as addressing similar issues for construction and facility management. Highlights • A deep learning-based approach is proposed to identify and locate multiple defects. • The proposed approach detects cracks, deposits, infiltrations and roots accurately. • The influential factors of the proposed approach for defect detection are studied. • A new model with a modified neural network is built to improve detection accuracy. • The detection accuracy is improved greatly, with an mAP of 83% using the new model. [ABSTRACT FROM AUTHOR]