Search

The detection of surface defects on metals plays a pivotal role in the evaluation of aluminum profile product quality. To date, the most prevalent approach for detecting metal surface defects is via the use of conventional neural networks (CNNs). However, in recent years, transformer-based methods have achieved notable improvements in computer vision and exhibited superiority over traditional CNNs in most tasks. Regarding aluminum surface defect image characteristics, such as intricate textures, few defect samples, and large-scale differences in various defects, it is difficult to further improve the performance of traditional CNNs. Hence, this study proposes a novel variant transformer-based method for aluminum profile surface defect detection that combines a traditional CNN and transformer architecture with a deformable attention module (DAM). The DAM focuses on a subset of the critical sampling points around the reference point to alleviate the intractable complexity associated with high-resolution feature maps and enhances the recognition effect of small target defects. The proposed image-stitching method uses defect-free samples to generate new samples, which partially mitigates the issue of class imbalance. Moreover, we performed supplementary experiments to confirm the effectiveness of transfer learning in training small-scale datasets. Compared with a baseline, the mean average precision (mAP) improved by 3.32%. Extensive experimental results demonstrate the efficacy of our method in accurately detecting surface defects in aluminum profiles and significantly improving the detection of small target defects. [ABSTRACT FROM AUTHOR]

This study tackles a critical challenge in converting two-dimensional (2D) images into three-dimensional (3D) representations, focusing on the precise detection of flat surfaces. The research utilizes a triangulation method involving laser and camera systems, emphasizing the optimization of laser shooting angles and camera positioning to accurately determine z-coordinates. The methodology employs hue, saturation, and value (HSV) color masking, which has proven superior to traditional red, green, blue (RGB) methods for isolating red line objects. Key findings indicate that the optimal laser angle, ß1=70.65°, significantly minimizes root mean square (RMS) error, thereby enhancing the accuracy of 3D imaging. Additionally, the use of three laser lines at different angles enables a more comprehensive detection of z-coordinates by creating multiple reference points across the surface. This arrangement improves the robustness and precision of the 3D reconstruction process, as the intersecting laser lines generate detailed coordinate data that is critical for accurately mapping surface irregularities. These results not only support existing theories in digital feature extraction but also offer a robust framework for practical applications in manufacturing and quality control, particularly in surface defect detection. The study's innovative approach advances the field of computer vision, providing new insights and methodologies for optimizing image conversion techniques. [ABSTRACT FROM AUTHOR]

Image processing and object recognition technologies for detecting defects on the surfaces of a variety of materials is critical to ensuring the safety and durability of infrastructure. That is why this topic and problems of the field of defect detection were chosen in the study. Therefore, the development of effective image processing methods for defect identification, especially in low-light conditions and hard-to-reach places, is of great relevance. The research is a comparison of classical image processing methods with modern deep learning algorithms such as CNN (Convolutional Neural Networks) and YOLO (You Only Look Once). The study analyzes the effectiveness of these methods under specific defect inspection conditions, including diffuse lighting and device mobility. An important aspect is the use of microcomputers such as Raspberry Pi and Nvidia Jetson Nano, which ensures the mobility and autonomy of the system. The practical value of the research lies in the implementation of effective image processing methods for detecting defects on the surfaces of engineering structures. This makes it possible to significantly improve the accuracy of surface defect identification, which is confirmed by the IoU (Intersection over Union) and Dice metrics. In particular, using CNNs for surface defect identification showed 35% better results compared to existing implementations of similar networks and 12% more efficient compared to YOLO. On the other hand, YOLO proved to be more productive in terms of processing frames per second on microcomputers, which is important for real-time monitoring.

The steel manufacturing process is inherently continuous, meaning that if defects are not effectively detected in the initial stages, they may propagate through subsequent stages, resulting in high costs for corrections in the final product. Therefore, detecting surface defects and obtaining segmentation information is critical in the steel manufacturing industry to ensure product quality and enhance production efficiency. Specifically, segmentation information is essential for accurately understanding the shape and extent of defects, providing the necessary details for subsequent processes to address these defects. However, the time-consuming and costly process of generating segmentation annotations poses a significant barrier to practical industrial applications. This paper proposes a cost-efficient segmentation labeling framework that combines deep learning-based anomaly detection and label enhancement to address these challenges in the steel manufacturing process. Using ResNet-50, defects are classified, and faster region convolutional neural networks (faster R-CNNs) are employed to identify defect types and generate bounding boxes indicating the defect locations. Subsequently, recursive learning is performed using the GrabCut algorithm and the DeepLabv3+ model based on the generated bounding boxes, significantly reducing annotation costs by generating segmentation labels. The proposed framework effectively detects defects and accurately defines them, even in randomly collected images from the steel manufacturing process, contributing to both quality control and cost reduction. This study presents a novel approach for improving the quality of the steel manufacturing process and is expected to enhance overall efficiency in the steel manufacturing industry. [ABSTRACT FROM AUTHOR]

This paper presents Fix-YOLOX (Fixmatch-You Only Look Once X), a semisupervised target detection model that uses a small amount of annotated data for fully supervised training, and adds a semisupervised training module using both pseudolabelling and consistent regularization to prevent overfitting in fully supervised training by using unlabelled data. Additionally, the generalization of the model and its fault tolerance to labelled data are improved. The experimental results show that the proposed semisupervised visual detection algorithm, Fix-YOLOX, can substantially reduce the amount of data annotation required for the target detection task while effectively overcoming the problem caused by annotated data with uneven quality. The YOLOX model achieves 91.95% accuracy with 50% annotated data and an average detection time of 10.4 ms per image/frame, which is consistent with the detection time of original YOLOX. Therefore, the model has good real-time performance and generalizability. [ABSTRACT FROM AUTHOR]

In response to the scarcity of defect samples in industrial applications, this paper proposed a method for segmenting surface defects in metal workpieces with only a minimal number of required defect samples. The method combined image generation techniques with a semi-supervised learning strategy. It utilized small-sized defect patches, and extracted from a minimal number of defect images to train a defect generation model. Subsequently, the method integrated these generated defect images into normal images to facilitate data augmentation. Additionally, the method applied a semi-supervised learning strategy to train the segmentation network, aiming to mitigate the adverse effects of differences between generated and real data distributions. The experimental phase involved conducting tests on a real-world computer vision detection system for metal workpieces. The results demonstrate that the semi-supervised training strategy significantly enhances the segmentation model's generalization ability to real defects. The method achieves high segmentation accuracy using only five defect sample images. [ABSTRACT FROM AUTHOR]

Surface defect detection is a critical task in the manufacturing industry to ensure product quality and machining efficiency. Image-based precise defect detection faces significant challenges due to defects lacking fixed shapes and the detection being heavily influenced by lighting conditions. Addressing the efficiency demands of defect detection algorithms, often deployed on embedded devices, and the highly imbalanced pixel ratio between foreground and background images, this paper introduces a multi-scale fusion staged U-shaped convolutional neural network (DEU-Net). The network provides segmentation results for defect anomalies while indicating the probability of defect presence. It enables the model to train with fewer parameters, a crucial requirement for practical applications. The proposed model achieves an MIoU of 66.94 and an F1 score of 74.89 with lower Params (36.675) and Flops (19.714). Comparative analysis with FCN, U-Net, Deeplab v3+, U-Net++, Attention U-Net, and Trans U-Net demonstrates the superiority of the proposed approach in surface defect detection. [ABSTRACT FROM AUTHOR]

Detecting cracks in the pavement is a vital component of ensuring road safety. Since manual identification of these cracks can be time-consuming, an automated method is needed to speed up this process. However, creating such a system is challenging due to factors including crack variability, variations in pavement materials, and the occurrence of miscellaneous objects and anomalies on the pavement. Motivated by the latest progress in deep learning applied to computer vision, we propose an effective U-Net-shaped model named DepthCrackNet. Our model employs the Double Convolution Encoder (DCE), composed of a sequence of convolution layers, for robust feature extraction while keeping parameters optimally efficient. We have incorporated the TriInput Multi-Head Spatial Attention (TMSA) module into our model; in this module, each head operates independently, capturing various spatial relationships and boosting the extraction of rich contextual information. Furthermore, DepthCrackNet employs the Spatial Depth Enhancer (SDE) module, specifically designed to augment the feature extraction capabilities of our segmentation model. The performance of the DepthCrackNet was evaluated on two public crack datasets: Crack500 and DeepCrack. In our experimental studies, the network achieved mIoU scores of 77.0% and 83.9% with the Crack500 and DeepCrack datasets, respectively. [ABSTRACT FROM AUTHOR]

The detection of surface defects is crucial to industrial manufacturing. In recent years, numerous detection methods based on computer vision have been successfully applied in the industry. However, industrial defect detection is still full of challenges. In one aspect, most of the industrial defects are extremely small. In another aspect, even though the intra-class defects have numerous similar elements, their outward appearances differ significantly. In this paper, we propose a dual-structure attention-based multi-level feature fusion network (DaMFFN) to address these two issues. In the first attention-based multi-level feature extraction structure, we introduce novel attention pooling to preserve more detailed information about the defective features of the tiny defect by giving certain regions varying weights. In the second attention-based multi-level feature fusion structure, we propose channel attention to capture the defect feature with the greatest potential for discrimination rather than all possible defect features, which is employed to prevent the incorrect detection of intra-class defects. The experiments demonstrate that the detection performance of the DaMFFN is better than other methods in five surface defect datasets. [ABSTRACT FROM AUTHOR]

Surface defect detection of sawn timber is a critical task to ensure the quality of wooden products. Current methods have challenges in considering detection accuracy and speed simultaneously, due to the complexity of defects and the massive length of sawn timbers. Specifically, there are scale variation, large intraclass difference and high interclass similarity in the defects, which reduce the detection accuracy. To overcome these challenges, we propose an efficient multilevel-feature integration network (EMINet) based on YOLOv5s. To obtain discriminative features of defects, the cross fusion module (CFM) is proposed to fully integrate the multilevel features of backbone. In the CFM, the local information aggregation is designed to enrich the detailed information of high-level features, and the global information aggregation is designed to enhance the semantic information of low-level features. Experimental results demonstrate that the proposed EMINet achieves better accuracy with fast speed compared with the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

As an indispensable material in modern industry, steel requires real-time surface defect detection to ensure high-quality manufacturing. However, steel surface defects present significant challenges due to their tiny size, diverse morphology, and uneven feature distribution. To address these challenges and satisfy the balance between accuracy and detection speed, an efficient steel strip surface defect detection network, ACPP-Net, is proposed in this study. Firstly, Adaptive Ghost Convolution, the LM-block, is introduced to meet the need for rapid steel defect detection. By integrating Adaptive Ghost Convolution, this module increases efficiency by reducing redundant information acquisition and adaptively assigning weights to defect features. Secondly, a novel feature enhancement module, FEM-block, is proposed to address the complexity of steel defects and distinguish their subtle differences. This module excels at capturing complex defect textures, aiding in the accurate differentiation of various defects. Additionally, a channel spatial pyramid pooling (CSPP) module is incorporated into the final part of the backbone network. This module effectively helps the network understand the characteristics and distribution of steel defects. Extensive experiments on the NEU-DET and GC10-DET datasets demonstrate ACPP-Net’s superior performance, achieving 82.1% and 71.1% mAP respectively, while maintaining real-time detection capabilities. The detection performance of this model is superior to other methods. These results demonstrate the model’s high accuracy and real-time detection capabilities, thus contributing to efficient and high-quality steel detection processes.

The steel manufacturing process is inherently continuous, meaning that if defects are not effectively detected in the initial stages, they may propagate through subsequent stages, resulting in high costs for corrections in the final product. Therefore, detecting surface defects and obtaining segmentation information is critical in the steel manufacturing industry to ensure product quality and enhance production efficiency. Specifically, segmentation information is essential for accurately understanding the shape and extent of defects, providing the necessary details for subsequent processes to address these defects. However, the time-consuming and costly process of generating segmentation annotations poses a significant barrier to practical industrial applications. This paper proposes a cost-efficient segmentation labeling framework that combines deep learning-based anomaly detection and label enhancement to address these challenges in the steel manufacturing process. Using ResNet-50, defects are classified, and faster region convolutional neural networks (faster R-CNNs) are employed to identify defect types and generate bounding boxes indicating the defect locations. Subsequently, recursive learning is performed using the GrabCut algorithm and the DeepLabv3+ model based on the generated bounding boxes, significantly reducing annotation costs by generating segmentation labels. The proposed framework effectively detects defects and accurately defines them, even in randomly collected images from the steel manufacturing process, contributing to both quality control and cost reduction. This study presents a novel approach for improving the quality of the steel manufacturing process and is expected to enhance overall efficiency in the steel manufacturing industry.

Computer vision-based detection approaches have been widely used in defect inspection tasks. However, identifying small-sized defects is still a challenge for most existing methods. It is mainly because: (1) the existing methods fail to extract sufficient information from the small-sized defects; (2) the existing detectors cannot generate effective region proposals for small-sized defects, which results in a low recall rate. To address the above issues, an adaptive loss weighting multi-task model with attention-guide proposal generation is proposed. First, the proposed multi-task model can excavate contextual information to enrich the feature information of small-sized defect areas, enhancing the model's representation capability. Additionally, to improve the recall rate of small-sized defects, an object attention-guide proposal generation module is proposed by leveraging object attention to guide the confidence enhancement of small-sized defects, which can generate more high-quality region proposals for small-sized defects. Finally, to speed up the joint optimization of the proposed multi-task framework, an adaptive loss weighting algorithm is proposed to learn the optimal combination of multi-task loss functions by maintaining the gradient direction consistency and tuning each task's loss magnitude. The experimental results on the two public defect datasets demonstrate that the proposed method outperforms other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Aluminum Profiles (APs) are aluminum materials obtained by hot melting and extruding aluminum rods. It has the characteristics of low cost, strong plasticity, easy processing, and recyclability, and therefore plays an important role in industrial production. However, defects such as Non-Conductive (NC), Scratch, Orange Peel (OP), and Dirty Point (DP) often occur during the production and processing of APs, which can seriously affect the quality of APs. In addition, surface defects of APs also have problems such as fuzzy regional definition, large-scale variation, imbalance of aspect ratio, and high inter-class defect similarity, making defect detection more challenging. To solve these problems, this paper proposes an attention-guided object detection algorithm called MA-YOLO, specifically for Surface Defect Detection (SDD) of APs. The algorithm is based on YOLOv5s. Firstly, the K-Means++ clustering algorithm is used to optimize the anchor boxes, which alleviates the problem of aspect ratio imbalance. Secondly, by improving the multi-scale Feature Fusion Network (FFN), the detection performance of the model to detect the defects with unbalanced aspect ratio is improved, and the adaptability of the model to defects of different scales is enhanced. Finally, a novel Max Pooling Average Pooling (MA) attention module is proposed to improve the overall detection performance of the model, especially for small-scale defects. Experimental results on the aluminum profile surface defect dataset show that MA-YOLO has better detection performance and superiority than the current mainstream object detection algorithms, and compared with the baseline YOLOv5s, the mAP50 and F1 score are increased by 2.9% and 2.2%, respectively, while keeping the model lightweight. This indicates that MA-YOLO has broad application prospects in the surface defect detection of APs.

Bridge inspection and monitoring are usually used to evaluate the status and integrity of bridge structures to ensure their safety and reliability. Computer vision (CV)-based methods have the advantages of being low cost, simple to operate, remote, and non-contact, and have been widely used in bridge inspection and monitoring in recent years. Therefore, this paper reviews three significant aspects of CV-based methods, including surface defect detection, vibration measurement, and vehicle parameter identification. Firstly, the general procedure for CV-based surface defect detection is introduced, and its application for the detection of cracks, concrete spalling, steel corrosion, and multi-defects is reviewed, followed by the robot platforms for surface defect detection. Secondly, the basic principle of CV-based vibration measurement is introduced, followed by the application of displacement measurement, modal identification, and damage identification. Finally, the CV-based vehicle parameter identification methods are introduced and their application for the identification of temporal and spatial parameters, weight parameters, and multi-parameters are summarized. This comprehensive literature review aims to provide guidance for selecting appropriate CV-based methods for bridge inspection and monitoring. [ABSTRACT FROM AUTHOR]

Image segmentation is a significant step in image analysis and computer vision. Many entropy-based approaches have been presented on this topic. Among them, Tsallis entropy isone of the best-performing methods. In this paper, the surface defect images of galvanised steel sheets were studied. Atwo-dimensional asymmetric Tsallis cross-entropy image segmentation algorithm based on chaotic bee colony algorithm optimisation was used to investigate the segmentation of surface defects under complex texture backgrounds. On the basis of Tsallis entropy threshold segmentation, a more concise expression form was used to define the asymmetric Tsallis cross-entropy in order to reduce the calculation complexity of the algorithm. The chaotic algorithm was combined with the artificial bee colony (ABC) algorithm to construct the chaotic bee colony algorithm, so that the optimal threshold of Tsallis entropy could be quickly identified. The experimental results showed that compared with the maximum Shannon entropy algorithm, the calculation time of this algorithm decreased by about 58% and the threshold value increased by about (26%, 54%). Compared with the two-dimensional Tsallis cross-entropy algorithm, the calculation time of this algorithm decreased by about 35% and about 20% was improved in the g-axis direction only. Compared with the two-dimensional asymmetric Tsallis cross-entropy algorithm, the calculation time of this algorithm decreased by about 30% and the threshold values of the two algorithms were almost the same. The algorithm proposed in this paper can rapidly and effectively segment defect targets, making it a more suitable method for detecting surface defects in factories with a rapid production pace. [ABSTRACT FROM AUTHOR]

This paper proposes a lightweight detection model based on machine vision, YOLOv5-GC, to improve the efficiency and accuracy of detecting and classifying surface defects in preforming materials. During this process, clear images of the entire surface are difficult to obtain due to the stickiness, high reflectivity, and black resin of the thermosetting plain woven prepreg. To address this challenge, we built a machine vision platform equipped with a linescan camera and high-intensity linear light source that captures surface images of the material during the preforming process. To solve the problem of defect detection in the case of extremely small and imbalanced samples, we adopt a transfer learning approach based on the YOLOv5 neural network for defect recognition and introduce a coordinate attention and Ghost Bottleneck module to improve recognition accuracy and speed. Experimental results demonstrate that the proposed approach achieves rapid and high-precision identification of surface defects in preforming materials, outperforming other state-of-the-art methods. This work provides a promising solution for surface defect detection in preforming materials, contributing to the improvement of composite material quality. [ABSTRACT FROM AUTHOR]

Surface defect detection is a critical task in the manufacturing industry to ensure product quality and machining efficiency. Image-based precise defect detection faces significant challenges due to defects lacking fixed shapes and the detection being heavily influenced by lighting conditions. Addressing the efficiency demands of defect detection algorithms, often deployed on embedded devices, and the highly imbalanced pixel ratio between foreground and background images, this paper introduces a multi-scale fusion staged U-shaped convolutional neural network (DEU-Net). The network provides segmentation results for defect anomalies while indicating the probability of defect presence. It enables the model to train with fewer parameters, a crucial requirement for practical applications. The proposed model achieves an MIoU of 66.94 and an F1 score of 74.89 with lower Params (36.675) and Flops (19.714). Comparative analysis with FCN, U-Net, Deeplab v3+, U-Net++, Attention U-Net, and Trans U-Net demonstrates the superiority of the proposed approach in surface defect detection.

This paper presents an approach to automatic surface defect detection by a deep learning-based object detection method, particularly in challenging scenarios where defects are rare, i.e., with limited training data. We base our approach on an object detection model YOLOv8, preceded by a few steps: 1) filtering out irrelevant information, 2) enhancing the visibility of defects, namely brightness contrast, and 3) increasing the diversity of the training data through data augmentation. We evaluated the method in an industrial case study of crown wheel surface inspection in detecting Unclean Gear as well as Deburring defects, resulting in promising performances. With the combination of the three preprocessing steps, we improved the detection accuracy by 22.2% and 37.5% respectively while detecting those two defects. We believe that the proposed approach is also adaptable to various applications of surface defect detection in other industrial environments as the employed techniques, such as image segmentation, are available off the shelf. [ABSTRACT FROM AUTHOR]

Machine vision-based inspection technologies are gaining considerable importance for automated monitoring and quality control of manufactured products in recent years due to the advent of Industry 4.0. The involvement of advanced deep learning methods is a significant factor contributing to the advent of robust vision-based solutions for improving inspection accuracy at a significantly lower cost in manufacturing industries. The requirement of computational resources and large training datasets hinders the deployment of these solutions to manufacturing shop floors. The present research work develops an image-based framework considering pre-trained Convolutional Neural Network (CNN), ResNet-101 to detect surface defects with the minimum training datasets and computational requirements. The outcomes of the proposed framework are substantiated through a case study of detecting commonly observed surface defects during the centerless grinding of tapered rollers. The image datasets consisting of standard tapered rollers and three common defect classes are captured and enriched further with the help of the data augmentation technique. The present work employs ResNet-101 for feature extraction combined with and multi-class Support Vector Machine (SVM) as a classifier to detect defective images. The effects of the feature extraction layer (fc1000) and pooling layer (pool5) activation are explored to achieve the desired prediction abilities. The testing trials demonstrate that the proposed framework effectively performs image classification, achieving 100% precision for the 'Good' class components. The study showed that the proposed approach could overcome the requirements of large training datasets and higher computational power for deep learning models. The proposed system can be of significant importance for Micro, Small, and Medium Enterprises (MSMEs) and Small and Medium-sized Enterprises (SMEs) as an alternative to conventional labor-intensive manual inspection techniques. [ABSTRACT FROM AUTHOR]

Surface defect detection is an important part of the process of product quality control in the industry. Automatic detection of surface defects based on machine learning is an up-and-coming research field, and there have been many successful cases. Deep learning has become the most suitable detection method for this task. Most algorithms require a large number of defect samples to achieve good results. However, defect samples in actual production are very limited. Although some unsupervised or semi-supervised methods can reduce training costs, their accuracy is difficult to guarantee, so they are difficult to be applied in industrial inspection. In this paper, we propose a detail-semantic guide network (DSGNet), which can achieve better result with fewer training samples. It is a two-stage neural network framework. In the first stage, we design a new semantic branch based on the modified residual shrinkage network and the proposed joined atrous spatial pyramid pooling (JASPP) module. This is the first time that residual shrinkage network is applied to defect detection and achieves good results. Also, we design a clear and efficient detail branch based on dense connection network. Specially, we propose a new detail-semantic guide module (DSGM), which can better integrate the feature information of the two branches. In the training phase, we propose a weight mask based on defect area to improve the ability of extracting small defects. We did experiments on four datasets and our method achieved excellent detection results even with only a small number of training samples. [ABSTRACT FROM AUTHOR]

At present, the steel plate surface defect detection technology based on machine vision and convolutional neural network (CNN) has achieved good results. However, these models are mostly two-stage methods, extracting features first and then classifying them, which is slow and inaccurate. Therefore, this paper proposes a single-stage surface defect detection method of steel plate based on yolov3, which can classify defects, determine the location of defects, and greatly improve the detection speed. It is of great significance to realize the automation of cold rolling production line. The experiment shows that the detection speed of this model reaches 62 fps and the accuracy reaches 73 %, which has a good prospect in industry. [ABSTRACT FROM AUTHOR]

Recently, computer vision-based methods have been successfully applied in many industrial fields. Nevertheless, automated detection of steel surface defects remains a challenge due to the complexity of surface defects. To solve this problem, many models have been proposed, but these models are not good enough to detect all defects. After analyzing the previous research, we believe that the single-task network cannot fully meet the actual detection needs owing to its own characteristics. To address this problem, an end-to-end multi-task network has been proposed. It consists of one encoder and two decoders. The encoder is used for feature extraction, and the two decoders are used for object detection and semantic segmentation, respectively. In an effort to deal with the challenge of changing defect scales, we propose the Depthwise Separable Atrous Spatial Pyramid Pooling module. This module can obtain dense multi-scale features at a very low computational cost. After that, Residually Connected Depthwise Separable Atrous Convolutional Blocks are used to extract spatial information under low computation for better segmentation prediction. Furthermore, we investigate the impact of training strategies on network performance. The performance of the network can be optimized by adopting the strategy of training the segmentation task first and using the deep supervision training method. At length, the advantages of object detection and semantic segmentation are tactfully combined. Our model achieves mIOU 79.37% and mAP@0.5 78.38% on the NEU dataset. Comparative experiments demonstrate that this method has apparent advantages over other models. Meanwhile, the speed of detection amount to 85.6 FPS on a single GPU, which is acceptable in the practical detection process. [ABSTRACT FROM AUTHOR]

Machine vision systems commonly utilize Convolutional Neural Networks (CNNs) for in-line surface defect detection of manufactured components. The prediction abilities of vision-based inspection systems deteriorate with time as the defect detection model trained on fixed image datasets fails to accommodate deviations. This paper proposes a human-guided progressive learning approach that systematically imparts learning of new features to the CNN-powered vision-based defect detection system. The approach augments the surface defect detection model with human intelligence, using an intuitive user interface to address model drift. The human expert monitors the trained model performance under specific conditions leading to the change of characteristics during implementation, identifies misclassifications, and initiates re-training. The algorithm accumulates misclassified data till a pre-defined threshold level is reached or a human expert terminates inspection. The misclassified results merge with the original datasets for progressive re-training using a strategy similar to the base model development. The present work utilizes pre-trained CNN Efficientnet-b0 to develop the surface defect detection model for tapered roller inspection through transfer learning. It is concluded that the progressive re-training improves defect detection performance and reduces misclassifications. The Matthews Correlation Coefficient (MCC) score, derived from the confusion matrix, showed improvement from 0.6 to 0.82 after four iterations. A cross-model benchmarking study is also performed to show the versatility of the proposed approach. The present work demonstrated that the human-guided progressive learning approach can provide adaptability to vision-based surface defect detection utilizing deep learning algorithms and enhance system performance during real-time implementation. [ABSTRACT FROM AUTHOR]

Vision-based deep-learning methods are widely employed for Surface Defect Detection (SDD). However, as learning often needs extensive high-precision annotations, the practical application is always inconvenient and expensive. In this study, we alleviate the heavy requirement and further improve the accuracy. Specifically, a novel semi-supervised paradigm based on Diffusion Probabilistic Model (DPM) is proposed, namely SDDiff. It starts with unsupervised pre-training for DPM using low-cost image samples to learn the overall distribution. Next, we construct a detection pipeline that collects high-value features based on the trained model, which needs only a small amount of labeled data for supervised training. SDDiff combines the extensive data understanding of unsupervised learning and the precise pattern recognition of supervised learning, consequently reshaping the landscape of SDD. Sufficient experiments show that our method is effective, stable, and has low annotation dependence, which is better for practical visual measurement. • A novel semi-supervised paradigm with low annotation dependence. • Pioneering the application of DPM for visual measurement. • Effectively extracting rich features via unsupervised training. • Feature enhancement through frequency domain modulation. • Ensembling classifiers diminish the randomness in diffusion. [ABSTRACT FROM AUTHOR]

Metallic surface defect detection is critical to ensure the quality of industrial products. Recently, human-advanced surface defect detection algorithms have been proposed. Most of these algorithms rely on convolutional neural networks (CNN) and an anchoring scheme. However, a convolution unit only samples the input feature maps at fixed shapes and locations. Similarly, a set of anchors are uniformly predefined with fixed scales and shapes, which increases the difficulties of bounding box regression. Therefore, we propose an adaptive convolution and anchor network for metallic surface defect detection, named ACA-Net. Specifically, an adaptive convolution and anchor (ACA) module is proposed, which mainly consists of adaptive convolution and an adaptive anchor. Firstly, an adaptive convolution module (ACM) is designed, which adaptively determines the location and shape of each convolution unit. In addition, a multi-scale feature adaptive fusion (MFAF) is proposed, which is used in ACM to extract and integrate multi-scale features. Then, an adaptive anchor module (AAM) is proposed to yield more suitable anchor boxes by adaptively adjusting shapes. Extensive experiments on NEU-DET dataset and GC10 dataset validate the performance of the proposed approach. ACA-Net achieves 1.8% on NEU-DET dataset higher Average Precision (AP) than GA-RetinaNet. Furthermore, the proposed ACA module is also adopted in GA-Faster R-CNN, improving the AP by 1.2% on NEU-DET dataset. [ABSTRACT FROM AUTHOR]

Deep-learning-based approaches have proven to outperform other approaches in various computer vision tasks, making application-focused machine learning a promising area of research in automated visual inspection. In this work, we apply deep learning to the challenging real-world problem domain of automated visual inspection of pharmaceutical products. We focus on investigating whether compact network architectures, adhering to performance, resource, and accuracy requirements, are suitable for usage in the pharmaceutical visual inspection domain. We propose a compact and efficient convolutional neural network architecture design for segmentation and scoring of surface defects, which we evaluate on challenging real-world datasets from the pharmaceutical product-inspection domain. In comparison with other related segmentation approaches, we achieve state-of-the-art performance in terms of defect detection as well as real-time computational efficiency. Compared to the nearest best-performing architecture we achieve state-of-the-art performance with merely 3% of the parameter count, an approximately 8-fold increase in inference speed, and increased surface defect detection performance. [ABSTRACT FROM AUTHOR]

In the integrated circuit (IC) packaging, the surface defect detection of flexible printed circuit boards (FPCBs) is important to control the quality of IC. Although various computer vision (CV)-based object detection frameworks have been widely used in industrial surface defect detection scenarios, FPCB surface defect detection is still challenging due to non-salient defects and the similarities between diverse defects on FPCBs. To solve this problem, a decoupled two-stage object detection framework based on convolutional neural networks (CNNs) is proposed, wherein the localization task and the classification task are decoupled through two specific modules. Specifically, to effectively locate non-salient defects, a multi-hierarchical aggregation (MHA) block is proposed as a location feature (LF) enhancement module in the defect localization task. Meanwhile, to accurately classify similar defects, a locally non-local (LNL) block is presented as a SEF enhancement module in the defect classification task. What is more, an FPCB surface defect detection dataset (FPCB-DET) is built with corresponding defect category and defect location annotations. Evaluated on the FPCB-DET, the proposed framework achieves state-of-the-art (SOTA) accuracy to 94.15% mean average precision (mAP) compared with the existing surface defect detection networks. Soon, source code and dataset will be available at https://github.com/SCUTyzy/decoupled-two-stage-framework. [ABSTRACT FROM AUTHOR]

Metallic surface defect detection is critical to ensure the quality of industrial products. Recently, human-advanced surface defect detection algorithms have been proposed. Most of these algorithms rely on convolutional neural networks (CNN) and an anchoring scheme. However, a convolution unit only samples the input feature maps at fixed shapes and locations. Similarly, a set of anchors are uniformly predefined with fixed scales and shapes, which increases the difficulties of bounding box regression. Therefore, we propose an adaptive convolution and anchor network for metallic surface defect detection, named ACA-Net. Specifically, an adaptive convolution and anchor (ACA) module is proposed, which mainly consists of adaptive convolution and an adaptive anchor. Firstly, an adaptive convolution module (ACM) is designed, which adaptively determines the location and shape of each convolution unit. In addition, a multi-scale feature adaptive fusion (MFAF) is proposed, which is used in ACM to extract and integrate multi-scale features. Then, an adaptive anchor module (AAM) is proposed to yield more suitable anchor boxes by adaptively adjusting shapes. Extensive experiments on NEU-DET dataset and GC10 dataset validate the performance of the proposed approach. ACA-Net achieves 1.8% on NEU-DET dataset higher Average Precision (AP) than GA-RetinaNet. Furthermore, the proposed ACA module is also adopted in GA-Faster R-CNN, improving the AP by 1.2% on NEU-DET dataset.

The surface defects' region of strip steel is small, and has various defect types and, complex gray structures. There tend to be a large number of false defects and edge light interference, which lead traditional machine vision algorithms to be unable to detect defects for various types of strip steel. Image detection techniques based on deep learning require a large number of images to train a network. However, for a dataset with few samples with category imbalanced defects, common deep learning neural network training tasks cannot be carried out. Based on rapid image preprocessing algorithms (improved gray projection algorithm, ROI image augmentation algorithm) and transfer learning theory, this paper proposes a set of processes for complete strip steel defect detection. These methods achieved surface rapid screening, defect feature extraction, sample dataset's category balance, data augmentation, defect detection, and classification. Through verification of the mixed dataset, composed of the NEU surface dataset and dataset in this paper, the recognition accuracy of the improved VGG19 network in this paper reached 97.8%. The improved VGG19 network performs slightly better than the baseline VGG19 in six types of defects, but the improved VGG19 performs significantly better in the surface seams defects. The convergence speed and accuracy of the improved VGG19 network were taken into account, and the detection rate was greatly improved with few samples and imbalanced datasets. This paper also has practical value in terms of extending its method of strip steel defect detection to other products. [ABSTRACT FROM AUTHOR]

The appearance quality assessment based on defect inspection for painted car-body surfaces is an essential work to monitor and analyze the level of paint appearance quality. In the industrial application, there are some challenges, such as the huge and stereo skeleton of car bodies, a variety of irregular local surface areas, low visibility of defects due to tiny real size, and specular car-body surface. To overcome these problems, a lightweight online appearance quality assessment system (OAQAS) based on parallel deep learning is proposed, it includes two parts: 1) a vision inspection subsystem with distributed multi-camera image acquisition module and 2) an appearance quality evaluation subsystem (AQES) based on parallel TinyDefectRNet for evaluating the proposed painted surface grinding difficulty criteria. TinyDefectRNet is able to inspect relatively accurate defect size, although it is trained on a coarsely annotated data set. The OAQAS is implemented in an actual painting production line of a car factory, and the application results show that our OAQAS is far superior to the manual inspection in evaluation accuracy and time consumption. Moreover, our system is lightweight so that it is easy to be plugged into existing painting production lines without rebuilding or changing the inspection room. [ABSTRACT FROM AUTHOR]

Visual inspection plays a pivotal role in numerous industrial production processes, and the pursuit of automation has surged with the rise of deep learning and convolutional neural networks (CNNs). Therein, the deployment of visual inspection CNNs on resource-constrained edge devices stands as a critical problem as these devices are the most affordable and well-suited for many industrial applications, e.g., production chains. Nonetheless, it faces challenges in meeting the computational demands of deep CNN models. Consequently, optimizing these models for efficient operation in such settings is imperative. Visual inspection tasks are often highly specialized, differing significantly from general computer vision tasks. As a result, state-of-the-art CNNs can be excessively large for achieving high accuracy on these specific datasets. To address this challenge, this paper introduces a novel approach utilizing neural architecture search (NAS) and hyperparameter optimization. We present the generic toolkit for NAS (GT-NAS), an open-source toolkit available for public use on GitLab (https://gitlab.com/pmf5/open-source/generic-toolkit-for-neural-architecture-search). We showcase the results of applying our methodology to two established state-of-the-art CNN models designed for surface defect detection, a problem that encompasses binary classification and segmentation of images.Our approach yields significantly smaller models relative to baselines, but with accuracy in line with the current state-of-the-art results, demonstrating the potential for enhanced efficiency in industrial visual inspection systems. In one experimental setting (optimizing the Mixed Supervision model on the KolektorSDD2 dataset), GT-NAS produced an architecture that is 6.2 times faster than the baseline while sacrificing only 0.25% of its average precision for binary classification. In another batch of experiments (optimizing the TriNet model on the SensumSODF dataset), GT-NAS also achieved remarkable results. It found a TriNet architecture five times smaller than the baseline, at a small cost of a 0.25% drop in the ROC-AUC classification score on the capsule subset of the SensumSODF dataset. Furthermore, on the softgel subset of the same dataset, GT-NAS produced a model that was 2.7 times smaller than the baseline, yet 0.19% more precise. [ABSTRACT FROM AUTHOR]

The inspection of visual quality represents a crucial step in the production process of bottled products. Numerous machine vision methodologies have demonstrated proficient identification of significant defects on bottle surfaces in well-controlled imaging environments. However, the actual production scenario introduces a myriad of surface defect types on bottled products, exhibiting diverse shapes, with the majority of defect instances characterized by relatively small individual areas. Faced with such diverse and small-sized defects, traditional convolutional neural networks (CNNs) encounter limitations due to the utilization of fixed-size convolutional kernels. This choice results in a constrained receptive field, hindering the capture of sufficiently effective contextual information. Additionally, pooling operations within traditional CNNs lead to a substantial reduction in feature map dimensions, causing excessive blurring or outright neglect of small-sized defects. Consequently, this detrimentally impacts the accuracy of surface defect detection and recognition. This study proposes a novel multi-scale defect detection model incorporating a variable receptive field and Gather–Distribute feature fusion mechanism to overcome limitations of traditional CNNs. Nine different scale defect images of bottled products, including worn, wrinkles, joint markings and etc, were collected and enhanced to construct a surface defect detection dataset for the actual production of bottled products. Enhancements to convolutional layers and the C2f module improve feature extraction for different defect shapes and sizes. Integration of the Gather–Distribute feature fusion module (GDFFM) reduces feature information loss and enhances shallow-layer feature utilization. An efficient detection head (E-Detect) with "parameter sharing" is proposed to reduce computational complexity and improve detection speed without experiencing significant accuracy loss. Experimental results demonstrate that the model's superiority over advanced defect detection algorithms across various categories. Notably, it achieves accuracies of 89.9%, 55.6%, 89.4%, and 75% for challenging defects like small worn, subtle wrinkles, inclined external cover, and misaligned labels, with improvements of 3.2%, 7%, 2%, and 11.8% over the baseline model, respectively. The model's mean Average Precision (mAP) also increases by 2.7%. • Promoting deformable convolution and Gather–Distribute features enhances backbone flexibility, improving small defect extraction in shallow layers. • Leveraging "parameter sharing", we designed E-Detect module, reducing model complexity, computational load, memory consumption, and enhancing detection speed. • This model excels in industrial environments, yielding superior application results, offering a new solution for surface defect detection in manufacturing. [ABSTRACT FROM AUTHOR]

Detecting surface defects plays a crucial role in ensuring the quality, functionality, and security of the production process. Traditional image processing techniques and machine learning models rely on manual analysis and feature extraction for specific vision inspection tasks. Deep learning approaches, which can automatically extract features from images, have demonstrated outstanding performance in computer vision tasks, including detecting surface defects. Motivated by this consideration, a Systematic Literature Review (SLR) method is employed for the comprehensive analysis of studies published between 2020 and 2023 in the field of deep learning-based surface defect detection applications in industrial products. The study provides a technical taxonomy for deep learning models according to the content of current studies through the SLR process, including Convolutional Neural Networks (CNN), encoder–decoder models, pyramid network models, Generative Adversarial Networks (GAN), attention-based models, and other models for surface defect detection. Then, the commonly used datasets for surface defect detection are discussed, and a comparative analysis of deep learning models' performance is provided. Our comparative analysis reveals that pyramid network models and CNN models are the most frequently used deep learning models for surface defect detection. These models yield reasonable results in surface defect detection due to their exceptional feature extraction capabilities. Finally, some hints for addressing future research directions and identifying open issues in surface defect detection applications are presented. [ABSTRACT FROM AUTHOR]

Defects in product packaging are one of the key factors that affect product sales. Traditional defect detection depends primarily on artificial vision detection. With the rapid development of machine vision, image processing, pattern recognition, and other technologies, industrial automation detection has become an inevitable trend because machine vision technology can greatly improve accuracy and efficiency; therefore, it is of great practical value to study automatic detection technology of the surface defects encountered in packaging boxes. In this study, machine vision and machine learning were combined to examine a surface defect detection method based on support vector machine where defective products are eliminated by a sorting robot system. After testing, the support vector machine training model using radial basis function kernel detects three kinds of defects at the same time under the ideal condition of parameter selection, and the effective detection rate is 98.0296%. [ABSTRACT FROM AUTHOR]

The existing methods for detecting surface defects in electrolytic capacitors are typically based on conventional machine vision, with limited feature extraction capabilities, poor versatility, slow detection speed, and the inability to achieve accurate and real-time defect detection. In this study, a real-time object detection algorithm based on an improved single shot multibox detector (SSD) is proposed to achieve omnidirectional surface defect detection of electrolytic capacitors. First, an electrolytic capacitor surface image acquisition device was established to capture omnidirectional surface images of the capacitors, and an electrolytic capacitor surface defect dataset was created. Next, the visual geometry group (VGG)-16 network structure was replaced with the MobileNetv2 network structure, effectively reducing the model's parameter count and improving inference speed. Moreover, the Multibox Loss function was replaced with the Focal Loss function to increase the model's attention to difficult-to-classify samples and improve model accuracy. Additionally, a transfer learning network model was designed to apply the model to electrolytic capacitors of different colors using small sample learning. Finally, the performance of the improved network model was tested on a dataset of electrolytic capacitor surface defects. The experimental results demonstrate that the parameters quantity of improved model is 3.50 M, the mAP value reaches 92.67 %, which is improved by 2.54 %, and the Macro-F1 value reaches 92.15 %, which is 11.32 % higher than that before improvement. Thus, the proposed improved SSD model provides a theoretical basis and technical prerequisites for automated and intelligent surface defect detection in electrolytic capacitors. [ABSTRACT FROM AUTHOR]

• Development of versatile vision-based system for shop floor inspection. • Offers features such as image acquisition, labelling, storage, and processing. • Utilized for image dataset generation, and dimension measurement. Vision-based systems augmented with deep learning-based Convolutional Neural Networks (CNNs) can effectively capture process or product deviations and are vital for achieving smartness in manufacturing. Acquiring good-quality component images on the shop floor is challenging yet mandatory for labeled dataset generation while developing CNN models. The present work develops a vision-based system utilizing hardware and software to capture good-quality images of manufactured components. The system can perform onboard image pre-processing efficiently and generate labeled image datasets. The experiments are performed to capture component images with different lighting conditions. It is shown that training of CNN-based image classification algorithm using images acquired by the developed system achieves better prediction accuracy. The developed system can also perform dimension measurement tasks employing classical image processing modules. The study showed that the system could be effectively implemented for image-based dimensional metrology and labeled dataset generation tasks offering ease of operation, portability, robustness, and versatility. [ABSTRACT FROM AUTHOR]

Small and Medium Enterprises (SMEs) and Micro, Small, and Medium Enterprises (MSMEs) contemplate integrating machine vision with high throughput manufacturing lines to ensure a consistent quality of standardized components. The inspection productivity can improve considerably by substituting machine vision with manual activities. The pre-trained Convolutional Neural Networks (CNNs) can facilitate enhanced machine vision capabilities compared to the rule-based classical image processing algorithms. However, the non-availability of labeled datasets and lack of expertise in model development restricts their utilities for SMEs and MSMEs. The present work examines the practicality of utilizing publicly available labeled datasets while developing surface defect detection algorithms using pre-trained CNNs considering case studies of typical machined components - flat washers and tapered rollers. It is shown that the publicly available surface defect datasets are ineffective for specific-case such as machined surfaces of flat washers and tapered rollers. The explicitly labeled image datasets can offer better prediction abilities in such cases. A comparative assessment of common pre-trained CNNs is conducted to identify an appropriate network while developing a surface defect detection framework for machined components. The common pre-trained CNNs VGG-19, GoogLeNet, ResNet-50, EfficientNet-b0, and DenseNet-201 showing prediction abilities for similar classification tasks have been examined. The pre-trained CNNs developed using explicit image datasets were implemented to segregate defective flat washers and tapered rollers as sample components manufactured by SMEs and MSMEs. The performance assessment was accomplished using parameters estimated from the confusion matrix. It is observed that EfficientNet-b0 outperforms other networks on most parameters, and it can be preferred while developing a surface defect detection algorithm. The outcomes of the present study form the basis for developing an integrated vision-based expert system for surface defect detection tasks. [ABSTRACT FROM AUTHOR]

As a critical component of metal can containers, can end and its manufacture quality are closely relevant to product safety in food and beverage industry. To satisfy the requirements of quality control, a machine vision apparatus for real-time can-end inspection is presented in this paper. With a brief description of the apparatus system design and imaging system, our emphasis is put on the postprocessing image analysis. To detect defects and deformations across the imaged can-end surface, an entropy-rate clustering algorithm combined with prior shape constraint is proposed to locate the can-end object and divide it into multiple measuring regions. Then, a superpixel grouping and selection scheme is adopted to find defective areas inside the flat central panel. For the other three annular measuring regions, a multiscale ridge detection algorithm is introduced to seek defects and deformations along their projection profiles. According to in-line experiments and test, our apparatus can find out a majority of the can-end defects with a detection accuracy as high as 99.48% for various circular can ends. [ABSTRACT FROM AUTHOR]

Traditional machine vision detection methods have low efficiency and poor detection accuracy in the detection and recognition of surface defects of Si 3 N 4 bearing ceramic balls. In the paper, a nondestructive recognition and classification method of surface defects is proposed. It is based on the improved YOLOv5 algorithm. A novel attention mechanism for mobile networks, Coordinate Attention, is applied to the backbone of the YOLOv5 algorithm to accurately locate and identify defective regions on the surface of Si 3 N 4 ceramic bearing balls. In order to obtain a more effective multi-scale fusion method, fusing the defect feature information at different scales as a weighted bidirectional feature pyramid network, the BiFPN structure is fused in the neck of the YOLOv5 algorithm. The experimental results show that the proposed method achieves a performance with 98.8% mAP and 0.92 average F1-score for surface defect detection and classification. In addition, the detection speed of a signal defect image is 9.9 ms/img. • Coordinate Attention, a novel mobile network attention mechanism, is applied into the backbone of the YOLOv5 algorithm to accurately locate and identify defective regions on the surface of the ball. • BiFPN structure is fused in the neck part of the YOLOv5 algorithm to obtain a more efficient multi-scale fusion method and fuse the defect feature information at different scales. • The experimental results show that the proposed method achieves a performance with 98.8% mAP and 0.92 average F1-score of surface defect detection and classification. Besides, the detection speed of a signal defect image is 9.9 ms/img. [ABSTRACT FROM AUTHOR]

In order to realize automatic nondestructive testing for surface cable damage on a cable-stayed bridge, a distributed machine vision system was developed. It uses four cameras to acquire images around the cable surface. Surface defection may be distributed in several images. An improved scale invariant feature transform (SIFT) feature matching algorithm for image mosaicing is proposed to real time processing to obtain a whole defect effectively. First, feature points are extracted by a Harris operator. Second, according to defect images collected by the system, the steps of the SIFT operator such as the distribution of the main direction for the matching feature points and the matching image rotation is simplified. The simplified SIFT operator is employed to describe the feature points and match the images. Finally, image fusion is implemented and a complete image of a defect is obtained. Experimental results show that the algorithm complexity is greatly reduced and improves detection integrity for surface cable defects using our improved SIFT to automaticslly stitch the defect images together. [ABSTRACT FROM AUTHOR]

• Fabric textile defect detection approach based on completed local quartet patterns and majority decision algorithm. • A new local texture descriptor called completed local quartet patterns is presented to extract discriminative texture features. • An innovative majority voting algorithm is proposed to improve final detection accuracy. • The proposed approach has high accuracy at or above 97.2 percent in patterned fabrics with different pattern types. Visual inspection is a main stage of quality control process in many applications. Surface defect inspection plays an important role in fabric textile quality control systems. Each type of abnormality, that disturb the repetitive texture of fabric is called defect. The main aim of this paper is to propose a rotation-invariant fabric defect detection approach with high detection rate in comparison with well-known methods. In this paper, a new version of local binary pattern, that is called completed local quartet patterns, is proposed to extract fabric image local texture features. In the train step, first, the size of fabric repetitive patterns is computed using autocorrelation function on a defect-less fabric image, the completed local quartet patterns operator is applied to whole of this image and to a set of non-overlapping windows. The maximum value of non-similarity amount between feature vector extracted from the whole image and each window is considered as the defect-less threshold. Considering the threshold obtained in the training the fabric defects are detected. In order to evaluate performance, a benchmark dataset of patterned fabric images is used, that includes three groups of fabric patterns and six defect types. Experimental results indicated that the proposed method can detect defects with 97.66 percent detection rate. Simple process, rotation invariant, gray-scale invariant and high detection rate are main advantages of the proposed approach. Moreover, the proposed completed local quartet patterns descriptor operator is a general texture descriptor which can be used in many other computer vision applications to describe image contents. [ABSTRACT FROM AUTHOR]

Recent years have witnessed the widespread research of the surface defect detection technology based on machine vision, which has spawned various effective detection methods. In particular, the rise of deep learning has allowed the surface defect detection technology to develop further. However, these methods based on deep learning still have some drawbacks. For example, the size of the sample data is not large enough to support deep learning; the location and recognition of surface defects are not accurate enough; the real-time performance of segmentation and classification is not satisfactory. In the context, this paper proposes an end-to-end convolutional neural network model: the pixel-wise segmentation and image-wise classification network (PSIC-Net). With the innovative design of a three-stage network structure, improved loss function and a two-step training mode, PSIC-Net can accurately and quickly segment and classify surface defects with a small dataset of training data. This model was evaluated with three public datasets, and compared with the most advanced defect detection methods. All the performance metrics prove the effectiveness and advancement of PSIC-Net. [ABSTRACT FROM AUTHOR]

Machine-vision-based defect detection, instead of manual visual inspection, is becoming increasingly popular. In practice, images of the upper surface of cableway load sealing steel wire ropes are seriously affected by complex environments, including factors such as lubricants, adhering dust, natural light, reflections from metal or oil stains, and lack of defect samples. This makes it difficult to directly use traditional threshold-segmentation-based or supervised machine-learning-based defect detection methods for wire rope strand segmentation and fracture defect detection. In this study, we proposed a segmentation-template-based rope strand segmentation method with high detection accuracy, insensitivity to light, and insensitivity to oil stain interference. The method used the structural characteristics of steel wire rope to create a steel wire rope segmentation template, the best coincidence position of the steel wire rope segmentation template on the real-time edge image was obtained through multiple translations, and the steel wire rope strands were segmented. Aiming at the problem of steel wire rope fracture defect detection, inspired by the idea of dynamic background modeling, a steel wire rope surface defect detection method based on a steel wire rope segmentation template and a timely spatial gray sample set was proposed. The spatiotemporal gray sample set of each pixel in the image was designed by using the gray similarity of the same position in the time domain and the gray similarity of pixel neighborhood in the space domain, the dynamic gray background of wire rope surface image was constructed to realize the detection of wire rope surface defects. The method proposed in this paper was tested on the image set of Z-type double-layer load sealing steel wire rope of mine ropeway, and compared with the classic dynamic background modeling methods such as VIBE, KNN, and MOG2. The results show that the purposed method is more accurate, more effective, and has strong adaptability to complex environments. [ABSTRACT FROM AUTHOR]