Search

This paper delivers an exhaustive analysis of the fusion of multi-sensor technologies, including traditional sensors such as cameras, Light Detection and Ranging(LiDAR), Radio Detection and Ranging(RADAR), and ultrasonic sensors, with Artificial Intelligence(AI) powered methodologies in obstacle detection for Autonomous Vehicles(AVs). With the growing momentum in AVs adoption, a heightened need exists for versatile and resilient obstacle detection systems. Our research delves into study of literatures, where proposed approaches assimilate data from this diverse sensor suite, integrated through Deep Learning(DL) techniques, to refine AV performance. Recent advancements and prevailing challenges within the domain are thoroughly examined, with particular focus on the integration of sensor fusion techniques, the facilitation of real-time processing via edge and fog computing, and the implementation of advanced artificial intelligence architectures, including Convolutional Neural Networks(CNNs), Recurrent Neural Networks(RNNs), and Generative Adversarial Networks(GANs), to enhance data interpretation efficacy. In conclusion, the paper underscores the critical contribution of multi-sensor arrays and deep learning in enhancing the safety and reliability of autonomous vehicles, offering significant perspectives for future research and technological progress. [ABSTRACT FROM AUTHOR]

In recent years, decomposition-based combination models have been widely used in wind power prediction. This type of method decomposes the highly volatile wind power into some relatively smooth subsequences, which reduces the difficulty of modeling. However, this might use information from future data in advance, creating the illusion of high prediction accuracy. Therefore, this paper proposes a wind power ultra-short-term prediction model based on fixed scale dual mode decomposition (FSDMD) and deep learning networks. First, the wind power series after fixed scale blocking is decomposed using ensemble empirical mode decomposition (EEMD), and use the improved variational mode decomposition (VMD) based on Spearman rank order correlation coefficient (SROCC) to decompose the obtained high-frequency components twice. Then, the appropriate mode components were selected by calculating the SROCC and experimental analysis, and combined with the convolutional neural network (CNN) and the bidirectional long short-term memory (BiLSTM) network to train the model. Finally, the historical data of wind turbines in a wind farm in Northwest China is used for example verification, and the comparison with other models in the two scenarios of sufficient and insufficient features. The results show that the proposed FSDMD–CNN–BiLSTM model has high prediction accuracy in both scenarios. Especially in the scenario of insufficient features, compared with CNN-BiLSTM model, RMSE, MAE and MAPE are reduced by 8.20,14.24 and 0.15, respectively. In addition, this paper verifies that mode decomposition can improve the performance of prediction model without using future features, which provides ideas for solving similar problems. [ABSTRACT FROM AUTHOR]

Accurate detection of pavement markings at the pixel level is crucial for enhancing traffic safety. The majority of current advanced deep-learning networks predominantly focus on localized features, neglecting the global context of pavement image. Such networks often result in discontinuous segmentation outcomes and suboptimal recovery of local details. In this paper, a robust model named C-Transformer is proposed to provide an effective solution to this challenge. The contributions of this paper primarily involve two aspects. Firstly, the proposed C-Transformer is designed to succinctly integrate convolution operations and self-attention, facilitating a comprehensive understanding of essential features. Secondly, an efficient Feed-Forward Network called Inverse Residual Feed-Forward Network is also proposed in this paper and deployed in C-Transformer to improve latent representations. Experimental results demonstrate that, compared to other state-of-the-art networks, the proposed C-Transformer achieves a performance enhancement of 0.93% in F-measure and a 1.64% improvement in Intersection-Over-Union. In particular, the robustness and effectiveness of the C-Transformer in accurate pavement marking detection are proved through field test results. This paper illustrates the feasibility of employing a hybrid Convolutional neural network-Transformer-based network for automatic robust pavement marking detection under noisy conditions. [ABSTRACT FROM AUTHOR]

Gathering pipelines are one of the key upstream infrastructures in the gas industry that link production well to the processing plant. Leak detection is critical for ensuring the safety of pipeline transmission. The detection of small leakage in gathering pipelines consistently poses a formidable challenge. In this paper, a process model is built based on health data of supervisory control and data acquisition system from the actual operating pipeline. In the model structure, the convolutional neural network is used to extract the spatial features, the bi-directional long short-term memory is used to extract the temporal features, and the attention mechanism is employed to allocate the model's attention resources reasonably. Next, the residual between the entity pipeline's output data and the process model's output data is used as a monitoring indicator of the operating state of the pipeline. A clustering-based boundary determination method is proposed to recognize the centroid of normal and small leak conditions, and pipeline leak detection is performed by the Euclidean distance between the monitoring indicator and the centroid. This paper explores the feasibility of fast modeling and leak detection with limited hardware. Field tests for the validation of the proposed methods were implemented in two in-service natural gas gathering pipeline. The experimental results demonstrate that the proposed method significantly enhances the detection performance of small-size leak. The leak detection rates of 94.06% and 92.16% evinces the potency of the proposed method applied in the leak detection of gathering pipelines across diverse real-world scenarios. [ABSTRACT FROM AUTHOR]

The paper investigates the application of machine learning (ML) for voltage sag source localization (VSSL) in electrical power systems. To overcome feature-selection challenges for traditional ML methods and provide more meaningful sequential features for deep learning methods, the paper proposes three time-sample-based feature forms, and evaluates an existing feature form. The effectiveness of these feature forms is assessed using k-means clustering with k = 2 referred to as downstream and upstream classes, according to the direction of voltage sag origins. Through extensive voltage sag simulations, including noises in a regional electrical power network, k-means identifies a sequence involving the multiplication of positive-sequence current magnitude with the sine of its angle as the most prominent feature form. The study develops further traditional ML methods such as decision trees (DT), support vector machine (SVM), random forest (RF), k-nearest neighbor (KNN), an ensemble learning (EL), and a designed one-dimensional convolutional neural network (1D-CNN). The results found that the combination of 1D-CNN or SVM with the most prominent feature achieved the highest accuracies of 99.37% and 99.13%, respectively, with acceptable/fast prediction times, enhancing VSSL. The exceptional performance of the CNN was also approved by field measurements in a real power network. However, selecting the best ML methods for deployment requires a trade-off between accuracy and real-time implementation requirements. The research findings benefit network operators, large factory owners, and renewable energy park producers. They enable preventive maintenance, reduce equipment downtime/damage in industry and electrical power systems, mitigate financial losses, and facilitate the assignment of power-quality penalties to responsible parties. • Comprehensive study on enhanced voltage sag source localization assisted by ML. • Proposing three new time sample-based feature forms, effective for ML methods. • Identifying the most influential feature form (form 4) utilizing k-means clustering. • Developing diverse supervised models, including a designed one-dimensional CNN. • CNN-Feature form 4 achieved 99.37% accuracy with an acceptable prediction speed. [ABSTRACT FROM AUTHOR]

Pipe wall-thinning is a phenomenon whereby the thickness of pipes in a nuclear power plant decreases over time owing to extended operational years. This thickness reduction is caused by various long-term thermal aging mechanisms such as Flow-Accelerated Corrosion, Liquid Droplet Impingement Erosion, cavitation, and flashing. Reducing the thickness of the secondary system pipes to the point of rupture can lead to severe human casualties and significant economic losses. Consequently, domestic power plant operators regularly manage the power plant pipes. Ongoing research focuses on the identification and management of pipe wall-thinning. However, previous studies have encountered problems in accurately judging the decrease in pipe wall-thinning in the presence of noise. To overcome this, Convolutional Neural Network (CNN) models based on image feature analysis have been used. This approach allows for the differentiation of pipe wall-thinning feature from small-sized noise in a single image. However, there were difficulties in making accurate judgments for large-sized noise that resembled the feature of pipe wall-thinning. This paper aims to analyze the limitations of the current pipe wall-thinning evaluation methods and to achieve accurate pipe wall-thinning discrimination through time-series analysis of continuous pipe wall-thinning data. The proposed method employs a Convolutional Recurrent Neural Network (CRNN) model, integrating the Recurrent Neural Network(RNN) model with the CNN model. The image feature of the pipes, extracted using CNN, are utilized as inputs for the RNN. This enables the observation of how the image features of the pipes change over time. This feature differentiates from the time-series feature of noise that occurs suddenly. Through this method, the paper proposes a new approach for effectively identifying the gradual decrease in pipe wall-thinning, enabling precise assessment of pipe wall-thinning progression. [ABSTRACT FROM AUTHOR]

Over the past decades, deep learning techniques have attracted increased attention from various research and industrial domains aligned with the development of Industry Internet-of-Things(IIoT). Specifically, with the advantage of data-driven methods, industrial organizations are seeking novel proactive strategies supported by analytic models to guarantee the quality of their production by observing degradation or predicting failure ahead of the occurrence of the component or asset. Predictive strategies are expected to promise the influence of unnecessary maintenance interruptions and mitigate the consequence of that, hence, extending the remaining useful life of products. This paper conducts a survey of the utilization of deep learning technologies on engineering applications where they provide satisfactory solutions with respect to specific data types or input signals. 106 primary papers are reviewed on deep learning–driven approaches which mainly explore five of the most popular architectures in the application of predictive maintenance. The main content of this paper summarizes the common advantages of each architecture and, accordingly, points out their limitations, as well as describes the application scopes of fully connected deep neural networks, convolutional neural networks, stacked autoencoders, deep belief networks, and deep recurrent neural networks. Based on the technique discussion for each of them, we intend to provide a comprehensive understanding and guidance of the appropriate usage of deep learning architectures to devise an effective predictive maintenance strategy for the scientific and industrial developers whose expertise lies in the prior domain knowledge of multi-source isomerization data. Moreover, the main content demonstrated the summarization of the decisive factor by which the incremental stages of the approaches were determined, fundamentally including the dataset specification, feature extraction, and the integration of deep learning approaches. [ABSTRACT FROM AUTHOR]

Gathering pipelines are one of the key upstream infrastructures in the gas industry that link production well to the processing plant. Leak detection is critical for ensuring the safety of pipeline transmission. The detection of small leakage in gathering pipelines consistently poses a formidable challenge. In this paper, a process model is built based on health data of supervisory control and data acquisition system from the actual operating pipeline. In the model structure, the convolutional neural network is used to extract the spatial features, the bi-directional long short-term memory is used to extract the temporal features, and the attention mechanism is employed to allocate the model's attention resources reasonably. Next, the residual between the entity pipeline's output data and the process model's output data is used as a monitoring indicator of the operating state of the pipeline. A clustering-based boundary determination method is proposed to recognize the centroid of normal and small leak conditions, and pipeline leak detection is performed by the Euclidean distance between the monitoring indicator and the centroid. This paper explores the feasibility of fast modeling and leak detection with limited hardware. Field tests for the validation of the proposed methods were implemented in two in-service natural gas gathering pipeline. The experimental results demonstrate that the proposed method significantly enhances the detection performance of small-size leak. The leak detection rates of 94.06% and 92.16% evinces the potency of the proposed method applied in the leak detection of gathering pipelines across diverse real-world scenarios. [ABSTRACT FROM AUTHOR]

The paper investigates the application of machine learning (ML) for voltage sag source localization (VSSL) in electrical power systems. To overcome feature-selection challenges for traditional ML methods and provide more meaningful sequential features for deep learning methods, the paper proposes three time-sample-based feature forms, and evaluates an existing feature form. The effectiveness of these feature forms is assessed using k-means clustering with k = 2 referred to as downstream and upstream classes, according to the direction of voltage sag origins. Through extensive voltage sag simulations, including noises in a regional electrical power network, k-means identifies a sequence involving the multiplication of positive-sequence current magnitude with the sine of its angle as the most prominent feature form. The study develops further traditional ML methods such as decision trees (DT), support vector machine (SVM), random forest (RF), k-nearest neighbor (KNN), an ensemble learning (EL), and a designed one-dimensional convolutional neural network (1D-CNN). The results found that the combination of 1D-CNN or SVM with the most prominent feature achieved the highest accuracies of 99.37% and 99.13%, respectively, with acceptable/fast prediction times, enhancing VSSL. The exceptional performance of the CNN was also approved by field measurements in a real power network. However, selecting the best ML methods for deployment requires a trade-off between accuracy and real-time implementation requirements. The research findings benefit network operators, large factory owners, and renewable energy park producers. They enable preventive maintenance, reduce equipment downtime/damage in industry and electrical power systems, mitigate financial losses, and facilitate the assignment of power-quality penalties to responsible parties. • Comprehensive study on enhanced voltage sag source localization assisted by ML. • Proposing three new time sample-based feature forms, effective for ML methods. • Identifying the most influential feature form (form 4) utilizing k-means clustering. • Developing diverse supervised models, including a designed one-dimensional CNN. • CNN-Feature form 4 achieved 99.37% accuracy with an acceptable prediction speed. [ABSTRACT FROM AUTHOR]

Pipe wall-thinning is a phenomenon whereby the thickness of pipes in a nuclear power plant decreases over time owing to extended operational years. This thickness reduction is caused by various long-term thermal aging mechanisms such as Flow-Accelerated Corrosion, Liquid Droplet Impingement Erosion, cavitation, and flashing. Reducing the thickness of the secondary system pipes to the point of rupture can lead to severe human casualties and significant economic losses. Consequently, domestic power plant operators regularly manage the power plant pipes. Ongoing research focuses on the identification and management of pipe wall-thinning. However, previous studies have encountered problems in accurately judging the decrease in pipe wall-thinning in the presence of noise. To overcome this, Convolutional Neural Network (CNN) models based on image feature analysis have been used. This approach allows for the differentiation of pipe wall-thinning feature from small-sized noise in a single image. However, there were difficulties in making accurate judgments for large-sized noise that resembled the feature of pipe wall-thinning. This paper aims to analyze the limitations of the current pipe wall-thinning evaluation methods and to achieve accurate pipe wall-thinning discrimination through time-series analysis of continuous pipe wall-thinning data. The proposed method employs a Convolutional Recurrent Neural Network (CRNN) model, integrating the Recurrent Neural Network(RNN) model with the CNN model. The image feature of the pipes, extracted using CNN, are utilized as inputs for the RNN. This enables the observation of how the image features of the pipes change over time. This feature differentiates from the time-series feature of noise that occurs suddenly. Through this method, the paper proposes a new approach for effectively identifying the gradual decrease in pipe wall-thinning, enabling precise assessment of pipe wall-thinning progression. [ABSTRACT FROM AUTHOR]

Remote sensing scene (RSS) image classification plays a vital role in various fields such as urban planning and environmental protection. However, due to higher inter-class similarity and intra-class variability, achieving accurate classification for RSS images poses a considerable challenge for current convolutional neural networks (CNNs)-based and visual transformer (ViT)-based methods. To address these issues, this paper proposes a novel dual-encoding method named master–slave encoding network (MSE-Net) from two perspectives of feature extraction and fusion. The master encoder, based on ViT, extracts higher-level semantic features, while the slave encoder, based on CNN, captures relative lower-level spatial structure information. Secondly, to integrate feature information from the two encoders effectively, this paper further develop two fusion strategies. The first strategy involves the auxiliary enhancement units (AEUs), which eliminates semantic divergence between the two encoders, enhances spatial context awareness of the slave encoder and promotes effective feature learning. The interactive perception unit (IPU), as the second strategy, facilitates interaction and integration of the two encoders' representations to extract more discriminative feature information. In addition, we conducted comparative experiments on four widely-used RSS datasets, including RSSCN7, SIRI-WHU, the aerial image dataset (AID) and NWPU-RESISC45 (NWPU45), to verify the effectiveness of MSE-Net. The experimental results demonstrate that MSE-Net achieved state-of-the-art (SOTA) performance across all the datasets. [Display omitted] [ABSTRACT FROM AUTHOR]

The core of data series classification and clustering is how to evaluate the similarity between time series. However, for one-dimensional time series, the vast majority of similarity measurement methods currently relied on one-dimensional information. Although some classifications of time series were achieved by the images transformed from time series, they mostly relied on a single imaging mechanism (time domain or frequency domain). In order to give a new idea for time series similarity measurement, this paper proposed a similarity measurement method of time series based on the fused images. According to recurrence plots (RP) and wavelet scalogram (WS), the original time series was first transformed into images. Then, RP and WS were fused into an image. Next, the feature in the image was extracted by ResNet-18. Finally, the similarity of the time series can be obtained by the Euclidean distance between the features extracted by ResNet-18. The proposed method was verified by the time series from UCR Time Series Classification/Clustering Homepage. The results showed that the similarity measurement of time series can be effectively improved by fusing the images obtained by different imaging mechanisms in the time domain and frequency domain. • This paper proposes a new similarity measurement method for time series based on the image fusion of different forms. • The time domain and frequency domain information of time series are represented by recurrence plots and wavelet scalogram. • The characteristics of the image fused by recurrence plots and wavelet scalogram are extracted by Convolutional Neural Network to measure the similarity of time series. • 30 sets of time series show the validity and superiority of the proposed method. • The reasons for the effectiveness of the proposed method are analyzed in detail. [ABSTRACT FROM AUTHOR]

The fault management process is facilitated by equipping power distribution systems with automated devices, especially remote-controlled switches (RCSs). Although RCS plays a key role in improving the system's reliability, it imposes significant investment, installation, and maintenance costs. Hence, RCSs should be optimally located in distribution feeders for the highest profit. In previous works, reliability-oriented mathematical optimization models have been formulated to reach this goal. However, the number of test solutions exponentially grows with the problem size to find the globally optimal solution. This paper uses machine learning to propose a scalable and easy-to-implement model for optimal switch placement in real power distribution systems. At first, the features of candidate points for installing RCSs are introduced. Then, a learning model is applied to deeply explore the relationship between these features and optimal locations for installing RCSs. After training, the learning-based surrogate model directly determines the optimal RCS placement strategy in real power distribution systems by leveraging knowledge gained from past experiences. Simulation results demonstrate that the proposed surrogate model is approximately 29 times faster than the integer programming-based mathematical model, without a significant loss of accuracy, when implemented on a modified 11 kV network connected to Bus 4 of the Roy Billiton test system. This paper also employs explainable artificial intelligence (XAI) tools to select the most important features, where the Hamming loss is decreased by approximately 5%. [ABSTRACT FROM AUTHOR]

The aging of transformer oil–paper insulation (OPI) contributes significantly to transformer failure and reduced service life. Therefore, reliable and precise evaluation of the aging status of transformer insulation condition is crucial in averting transformer failure. This article proposes a novel deep learning-based aging condition assessment technique of transformers by combining FDS (Frequency domain spectroscopy) and squeeze-and-excitation-enabled convolutional neural network (SE-CNN). An intrinsic product-based transformation using a Gramian angular field is employed to dimensionally expand the 1-D FDS signals acquired from OPI samples into 2D images. Each feature extraction channel in the SE-CNN-based aging diagnosis framework is assigned a specific weight, enabling the model to concentrate on the most relevant features while discarding the rest. Furthermore, the effect of moisture variation on the model is also considered under three different conditions to ensure the consequence of compensation for the effect of moisture on FDS. A comparison with traditional machine learning models has also been made to ensure the viability of the proposed methodology. Additionally, the model is interpreted by creating explanation weight plots of the feature coefficients using a local interpretable model-agnostic explanations (LIME) method in Python. The findings demonstrate that the novel aging diagnosis framework visually assesses the transformer's OPI system with the highest accuracy of 99.12% in the test set. Furthermore, it facilitates the automatic extraction of deep aging features, thus providing better generalization and convergence capability. [ABSTRACT FROM AUTHOR]

Pavement crack detection continues to be a stubborn problem given the interference of various factors in the actual pavement and the complex topological structure of asphalt pavement. Among all the obstacles, the bottleneck of pavement crack detection lies in the difficulty of segmenting the cracks in the pavement images whose edges are blurred. This paper proposes a multi-scale context feature and cross-attention based on convolutional neural network for accurate and robust pavement crack segmentation. The multi-scale context feature module is built in different deep networks to extract rich crack feature information. Subsequently, in order to effectively promote the seamless integration of features at different levels, we deploy cross-attention modules to each branch. After that, we add deep supervision to each branch to accelerate training. Finally, we integrate the outputs of each branch to obtain the final output diagram. The comparative experiments on various pavement datasets show that the method has better robustness. At the same time, this paper designs a complete system of pavement crack detection (PCD) and develops corresponding engineering application software. The PCD system can record the real-time pavement image data to the edge server, and the client can also monitor the real-time pavement images from the edge server through the HTTP protocol. • A multi-scale context feature module is built in the deep layer for robustly describing the size of the cracks. • The cross-attention module enables each pixel to select individual contextual information from multi-layer features. • In-depth supervision is added to each branch to accelerate the optimization of the network model. • An intelligent detection application software is designed and integrated with multiple modules. [ABSTRACT FROM AUTHOR]

This paper proposes a real-time state of health (SOH) estimation model based on a deep learning (DL) framework. The proposed model is a combination of two different architectures; a one-dimensional convolutional neural network (1D-CNN) and a bidirectional gated recurrent unit (BiGRU). The hybrid CNN-BiGRU uses the 1D CNN layers to extract pertinent features from input data and then relies on the BiGRU layers for sequence learning in both directions. To account for all SOH indicators, the proposed approach uses the current, voltage, and temperature measurements, which are readily obtainable from the electric vehicle's battery management system (BMS). This prevents the complex and time-consuming feature extraction used in most related papers. Since the hyperparameters have a significant impact on the performance of neural network models, a Bayesian Optimization (BO) technique based on the Gaussian Process (GP) was considered to tune the CNN-BiGRU model hyperparameters. Accordingly, the objective function was able to converge to a low Mean Squared Error (MSE) of 1.2 × 10 − 5 in just 19 iterations. Afterward, to verify the accuracy of the optimized model, a Lithium-ion battery dataset with several discharge profiles provided by the National Aeronautics and Space Administration (NASA) was used. The obtained results demonstrated the accuracy and robustness of the proposed method compared to other commonly used models. The CNN-BiGRU model yielded a Mean Absolute Error (MAE) of 2.080% and a root-mean-square error (RMSE) of 2.516% in the case of the battery set #C, referring to a set with 70 cycles already used at 24 °C. Additionally, the End of life (EOL) indicator error of zero cycles for the same data. [ABSTRACT FROM AUTHOR]

Ensuring the safety and security of crowded places is a major concern for both the government and the public. Accurately and quickly estimating the number of people in a crowd is crucial for public safety, urban planning, and traffic monitoring. The existing methods have a minimum mean square error of 0.89 and require high storage space, making them inappropriate for crowd counting using low-computation and small-storage devices like single-board computers. Furthermore, these methods suffer from prediction time lag and are not suitable for live streaming. To tackle these challenges, this paper proposes a Deep Convolution Neural Network-based 'CrowdDCNN' model for crowd counting. This model reduces the value of mean square error by 0.29 and the size of the model by 80.01%. Additionally, the prediction time was decreased to 700 ms. Further, the 'NoLag' algorithm introduced in this paper is efficient in live crowd counting. Its O(1) time and space complexity make it appropriate for all devices, including single-board computers, laptops, and GPUs. The reported 0-p value during statistical analysis using Wilcoxon test, Friedman rank test, and paired t-test validates the superiority of the proposed model. [ABSTRACT FROM AUTHOR]

Structural topology optimization is a compute-intensive process due to several iterations of simulations required to evaluate the performance of the component during optimization. Deep learning (DL) based approaches can address this challenge, but these methods were demonstrated mainly using 2D shapes and, at best, in low-resolution 3D geometries (typically 3 2 3 ). Further, due to non-manufacturable geometric features, the predicted optimal geometries from DL may not be manufacturable, even using additive manufacturing. In this paper, we develop a DL framework using a multigrid convolutional neural network (CNN) to generate high-resolution topology-optimized 3D geometries with additional checks on the manufacturability of the predicted shapes. Our framework predicts the final optimal topology using the initial strain energy (objective function of structural topology optimization) and target volume fraction (material fraction to be preserved after optimization) as input. We train the network using a multigrid approach, which enables topology optimization at 12 8 3 resolution, which was previously computationally challenging. We first train the multigrid CNN at a lower resolution and then transfer the learned network to continue training at higher resolutions. We use a distributed deep learning framework on a GPU supercomputing cluster to further speed up the training time. Distributed DL significantly speeds up the training time by more than 4 × while achieving similar model performance. Finally, we check the optimal geometries for manufacturability using fused deposition modeling (FDM)-specific manufacturability constraints. The large training dataset (> 60,000 high-resolution topology optimization examples) will be released with the paper to enable further research on this topic. • Multigrid approach to train the neural network at fine resolution of 1283. • Demonstrate the manufacturability by 3D printing several sample models predicted our framework. • A data-parallel distributed deep learning framework to accelerate the training process. • A comprehensive high-resolution data set consisting of more than 60k optimal shapes. [ABSTRACT FROM AUTHOR]

Detecting faults in high-speed and high-power diesel engines under complex variable operating conditions is highly challenging. Online vibration monitoring systems have been used in such diesel engines in key fields, in which vibration sensors are installed on each cylinder to enable comprehensive monitoring. In this paper, a fault detection method for diesel engines under variable operating conditions is proposed based on multi-sensor signal multi-scale fusion. Firstly, a preprocessing framework is established for the raw vibration signals collected from each cylinder to eliminate random interference and system noise. Then, the resulting signals are phase-aligned based on the engine firing sequence and analyzed using a signal correlation algorithm to produce a multi-sensor multi-scale similarity matrix (MSMSSM). Finally, a multi-branch residual convolutional neural network (MBRCNN) model is constructed with the MSMSSM as the input to detect abnormal health states of the diesel engine. Fault simulation experiments are conducted on a 12-cylinder V-type high-speed and high-power diesel engine test rig. The comparative test results indicate that the proposed MSMSSM-MBRCNN method shows both the highest accuracy of 95.28% and the lowest standard deviation of 3.57% compared to other typical methods. The multi-sensor signals multi-scale fusion method proposed in this paper fully utilizes the key information that remains basically consistent in the synchronous acquisition signals of multiple sensors under different operating conditions. This can effectively reduce the interference of operating condition changes and improve the accuracy and robustness of fault detection. • A new fault detection method is proposed for high-speed and high-power diesel engines under variable operating conditions. • An angular domain multi-scale correlation analysis algorithm is developed for utilizing the consistency of operating condition information in multi-sensor signals. • A multi-branch residual convolutional neural network fault identification model is established for effective multi-scale similarity matrices fusion. • The proposed fault detection method achieved a well performance in experimental tests on a V12 diesel engine rig under complex conditions. [ABSTRACT FROM AUTHOR]

Sensors of an aircraft engine have the problems of multiple kinds of faults and difficulty to diagnose. Moreover, the diagnosis performance cannot meet practical demands. Therefore, this paper proposes a novel approach for sensor fault diagnosis integrating the advantages of selective ensemble learning, siamese convolutional neural network (SCNN), and Bayesian interval estimation (BIE). The Bayesian interval estimation with hypothesis testing is developed with real-time data update to reduce the confidence interval. The attention mechanism is introduced to enhance fault diagnosis performance. Moreover, ensemble selection learning with Akaike information criteria is proposed to obtain optimal SCNNs. The theorems of convergence and optimal confidence data in ensemble learning are proven. Furthermore, this paper develops a novel improved learning algorithm based on AdaMod (Adaptive and momental bound method) and SGD (stochastic gradient descent), and achieves a new fault diagnosis algorithm to improve the diagnosis performance. The sensor data of aircraft engines from China Eastern Airlines is adopted for validation and experimental results show that proposed approach reaches an excellent tradeoff between sensor diagnosis performance, confidence interval, and runtime. The ablation studies are performed to analyze the role of leaner's components. The proposed method improves training accuracy by 6.2%, fault diagnosis performance by 5.1%, fault prediction mean result by 4.6%, and fault prediction variance is reduced by 38.1% than the state-of-the-art methods. • A novel diagnosis approach combining selective ensemble SCNN and Bayesian interval estimation (BIE) is proposed. • The Bayesian interval estimation method with hypothesis testing is developed to support better fault diagnosis with data update. • The theorems of convergence and optimal confidence are proved. • The improved learning algorithm based on AdaMod and SGD is developed. • A novel algorithm is adopted for diagnosing sensor faults of aircraft engine to improve the diagnosis performance. [ABSTRACT FROM AUTHOR]

This paper investigates the problem of an effective and robust fusion representation method for foreground moving object detection. Many fusion representation learning approaches pay attention to the similarity measurement between the fusion results and source images in texture details and pixel intensity, ignoring the harmful information, e.g., noise, blur, and extreme illumination. Therefore, the aggregated features of infrared and visible images will introduce much harmful information, affecting the model performance of downstream visual tasks. This paper tackles these problems by proposing a contrastive fusion representation learning method for the foreground moving object detection task, which consists of two major modules: the upstream fusion representation module (FRM) and the downstream foreground moving object detection module (FODM). Unlike the traditional fusion optimization mechanism, the former aims to extract valuable features and reject harmful features via the maximum mutual information theory. The latter is a siamese convolutional neural network to detect foreground moving objects by aggregating the time-sequence images generated by FRM. Experimental results and comparisons with the state-of-the-art on three public datasets (i.e., TNO, MF, and cross-modal FOD dataset of infrared and visible images), validate the effectiveness, robustness, and overall superiority of the proposed contrast fusion representation learning method. Concisely, our contrastive fusion representation learning has gained 53. 9 % , 43. 2 % , 46. 4 % , 52. 3 % , 2. 2 % , 87. 1 % , 3. 5 % on EI, SF, DF, AG, MI, and Nabf metrics compared with the best competitors. [ABSTRACT FROM AUTHOR]

Surface defect detection remains a challenging task due to issues such as inconspicuous targets, significant variations among identical defects, and minimal differences between distinct defects. To address these challenges, a Fine Coordinate Attention (FCA) block is proposed in this paper, which encodes both average and salient information in two coordinate directions, so that the spatial dependence can be captured and the long-range interaction can be achieved. And such localization-friendly information is crucial for industrial surface defect images with subtle targets. Specifically, the FCA block can recalibrate feature maps of a surface defect image through three steps: coordinate information aggregation, cross-dimension interaction, and attention generation. It can be embedded into any convolutional neural network (CNN) structure to improve performance. Additionally, two resistance spot welding (RSW) surface defect datasets are published in this paper: an image classification dataset RSW-C and an object detection dataset RSW-D. Experimental results for image classification and object detection demonstrate that the FCA block outperforms existing attention mechanisms. The code is available at , while the two RSW datasets can be found at www.kaggle.com/datasets/alfredzimmer/rswdatasets. [ABSTRACT FROM AUTHOR]

With the rapid expansion of the electric vehicle market, the demand for lithium-ion batteries (LIBs) is exploding. The state of health (SOH) of LIBs is receiving more widespread attention, which is the key parameter for battery health management. This paper proposes a SOH estimation method for LIBs via feature optimization and convolutional neural network (CNN) to reduce the information redundancy of existing multiple features, aimed at leveraging multiple sources of features while optimizing their combination to minimize redundancy. Firstly, multiple features are extracted from different perspectives, including electrical, thermodynamic, and electrochemical properties, to comprehensively characterize the aging of batteries. Secondly, we construct a SOH estimator based on principal component analysis (PCA) with CNN (PCA-CNN). Finally, the dimension of features is optimized with a simulated annealing algorithm (SA) under the mean-variance objective function. Moreover, Comparative experiments are conducted on the Oxford dataset for validation. The results demonstrate the effectiveness of the proposed multi-feature description method in terms of accuracy and smoothness. Compared to traditional CNN methods and fixed-dimension PCA-CNN, this estimation approach significantly improves performance, showing more than 20% and 30% increases in the key metrics of MAE and RMSE, respectively. This study successfully optimized feature combinations to reduce redundancy within the feature set while enhancing the accuracy of SOH estimation. [ABSTRACT FROM AUTHOR]

This paper presents a novel deep learning approach for reconstruction of local through-the-thickness fiber orientation distribution (FOD) in discontinuous long-fiber, prepreg-platelet molded composites (PPMC). The thermal-residual strains on composite surfaces are used as inputs to train a fully convolutional neural network, named U-Net, to predict spatially varying, local FOD. High fidelity synthetic data was generated via computational simulation of PPMC with stochastic material orientation state and was used for training the deep learning model. The proposed U-Net model allowed for rapid recognition of PPMC morphology by solving the inverse structural mechanics problem of determining the average fiber orientation through the composite thickness based on the provided surface strain measurements. Upon training and validation, the U-Net deep learning model was deployed to rapidly predict complex distributions of the local through-the-thickness FOD in PPMC. [ABSTRACT FROM AUTHOR]

Audio classification and source separation are two important components in the music industry and for audio engineering as well. The Indian classical music industry is filled with a lot of similar-sounding instruments. The application of these in the field of music composition, audio signal processing and voice recognition can be a huge boon for the music industry. This research paper proposes an improvised method for audio source separation followed by a novel method for classification of the separated sources using a unique as well as novel dataset consisting of classical Indian musical instruments. The method entails decoupling the conglomerate of audio signals and categorising the resulting sources according to their unique traits using Indian Musical Instrument Classification models, CNN (Convolutional Neural Networks) and SVM (Support Vector Machine) models. The proposed novel dataset Indian Musical Instrument Dataset(IMID) is an important contribution to the area because there are not any unique organised datasets for classical Indian instruments. The IMIDataset includes ten Indian instruments — Flute, Veena, Sitar, Sarod, Tabla, Violin, Guitar, Piano, Bass and Drums. To evaluate and compare the efficacy of the proposed model, comparison was made with the different datasets based on their accuracy and loss metrics. Link to the dataset. [ABSTRACT FROM AUTHOR]

Stable fault self-healing ensures the smooth operation of intelligent industrial manufacturing processes. However, current research, while capable of inferring fault propagation paths or individually controlling manufacturing processes, still faces some challenges and limitations in addressing issues within manufacturing processes. In particular, it is easy to overlook the impact of fault variables on normal causal relationships in the manufacturing process and how to achieve adaptive fault repair from the source of faults. The biological immune system possesses sensitive nodes and a neuro-endocrine-immune network with interconnected functionalities, bearing remarkable resemblance to the sensor arrays, information networks, and intelligent controller systems in modern self-healing control systems. These problem domains have significant practical implications in industrial manufacturing processes and urgently require new methods and technologies to address them. To address the issue of root cause fault self-healing in intelligent industrial manufacturing processes, this paper proposes an innovative method that utilizes root cause inference and bio-inspired heuristic reinforcement learning models to achieve fault self-healing in industrial processes. First, innovatively introduce the Variational Autoencoder (VAE) into the Time Convolutional Network (TCN) to construct the TCN-VAE network. By performing feature reconstruction, the network's feature extraction capability is enhanced, further exploring the relationships between latent variables, and consequently building a causal graph of faults. A multi-head attention mechanism is introduced into the network, and the inference process is quantitatively evaluated, thereby improving the generalization and accuracy of root cause inference. Root cause determination rules are established to identify the fundamental reasons for fault occurrence, ensuring self-healing from the source of faults. Next, we developed for the first time a fault self-healing model based on reinforcement learning, utilizing the immune repair process of effector T cells to characterize the fault self-healing process. It effectively adapts to eliminate various levels of faults based on the state transition process of antigens in T helper cells. Additionally, stability control analysis is performed on the proposed model. The average reward function curve and Sobol sensitivity index demonstrates the model's strong robustness in practical applications. The validation results from case studies on the Tennessee Eastman (TE) and Continuous Stirred Tank Reactor (CSTR) show that the average improvement in fault repair F1 score was 0.0941 and 0.105 respectively, and the fastest improvement in fault repair response time was 0.266s and 0.258s respectively. It was confirmed that the root cause inference and fault repair results align with the actual mechanisms of the manufacturing processes. The proposed method has been verified to achieve stable and accurate root cause fault repair in real-world intelligent industrial manufacturing systems, making a significant contribution to achieving stable and accurate root cause fault repair in intelligent industrial manufacturing systems. • TCN-VAE enables feature reconstruction and latent variable exploration for root cause reasoning. • Multi-head attention enhances generality and accuracy in root cause reasoning. • Effector T-cell immune-inspired RL adapts to eliminate faults effectively. • The fault self-healing model exhibits strong reliability and robustness. • Method validated on Tennessee Eastman and continuous stirred-tank reactor datasets. [ABSTRACT FROM AUTHOR]

Pipelines are susceptible to oil and gas leaks during long-distance transportation due to factors such as damage from external forces and aging. However, existing pipeline leakage detection technologies that rely on physical inspections or sensors installed on pipelines are time-consuming and costly. In this paper, a percussion approach based on improved MobileNetV2 is proposed for pipeline leakage detection. Firstly, the influence of pipe leakage size on vibration characteristics was investigated by theoretical analysis and numerical simulation. Subsequently, experiments were conducted to assess the validity of the proposed method. The sounds produced by hammering the pipe under different damage conditions were recorded using a smartphone. The improved MobileNetV2 model was then used for classifying Mel spectrogram and Mel frequency cepstrum coefficient (MFCC) features extracted from the recorded sound signals. This model incorporates a multi-scale feature fusion module, which allows it to capture features at different scales and enhances its ability to differentiate between damage conditions. Experimental results show that using Mel spectrogram as input for the improved MobileNetV2 achieves a higher accuracy compared to using MFCC, with 100% accuracy for identifying leakage damage and 99.87% for classifying leak size. Compared to other methods, the improved MobileNetV2 exhibits superior classification performance while maintaining the lightweight characteristics of the original MobileNetV2. In conclusion, the improved model demonstrates significant enhancements in classification performance and operational efficiency, making it a promising approach for processing percussive signals. [ABSTRACT FROM AUTHOR]

The morphology of bone marrow cells is crucial for the diagnostics of blood disorders. However, the traditional approach of cell sorting and counting under a microscope is time-consuming and error-prone. To assist experts in diagnosis, there is considerable interest at present in using neural networks to develop automatic recognition algorithms. However, the results obtained from these approaches indicate that the achieved accuracy is not entirely convincing, and there remains a risk of misdiagnosis. In fact, a reliable collaborative recognition algorithm not only needs to have high recognition accuracy but also must be able to confidently reject low-confidence classification results and leave them for human experts to handle. Therefore, this paper proposes a method that combines high accuracy with a rejection recognition option for bone marrow cell recognition. This method incorporates Crossformer (Cross) and Class Centroid Learning (CCL), which integrate multi-scale information from cell images, enhance image feature differentiation, and effectively reject low-confidence predictions. In the experiment, the recognition accuracy of Cross-CCL reaches 94.41%, surpassing the performance of the current mainstream and advanced methods for bone marrow cell recognition. Simultaneously, CCL serves as a dependable collaborative algorithm by effectively rejecting 21.07% of cell images that represent low-confidence prediction results. Moreover, the accuracy of identification for the remaining portions exceeds 98%. This rejection mechanism not only enhances the efficiency of expert resource allocation but also ensures a higher level of confidence in the identified cell images. • Enhancing recognition accuracy by leveraging the characteristics of bone marrow cells. • Transformer architecture models outperform convolutional neural networks in bone marrow cell recognition tasks. • Excluding low-confidence results to enhance the reliability of predictions. • In high-risk medical domains, human-machine collaboration is essential. [ABSTRACT FROM AUTHOR]

The digitization of engineering drawings is crucial for efficient reuse, distribution, and archiving. Existing computer vision approaches for digitizing engineering drawings typically assume the input drawings have high quality. However, in reality, engineering drawings are often blurred and distorted due to improper scanning, storage, and transmission, which may jeopardize the effectiveness of existing approaches. This paper focuses on restoring and recognizing low-quality engineering drawings, where an end-to-end framework is proposed to improve the quality of the drawings and identify the graphical symbols on them. The framework uses K-means clustering to classify different engineering drawing patches into simple and complex texture patches based on their gray level co-occurrence matrix statistics. Computer vision operations and a modified Enhanced Super-Resolution Generative Adversarial Network (ESRGAN) model are then used to improve the quality of the two types of patches, respectively. A modified Faster Region-based Convolutional Neural Network (Faster R-CNN) model is used to recognize the quality-enhanced graphical symbols. Additionally, a multi-stage task-driven collaborative learning strategy is proposed to train the modified ESRGAN and Faster R-CNN models to improve the resolution of engineering drawings in the direction that facilitates graphical symbol recognition, rather than human visual perception. A synthetic data generation method is also proposed to construct quality-degraded samples for training the framework. Experiments on real-world electrical diagrams show that the proposed framework achieves an accuracy of 98.98% and a recall of 99.33%, demonstrating its superiority over previous approaches. Moreover, the framework is integrated into a widely-used power system software application to showcase its practicality. The reference codes and data can be found at https://github.com/Lattle-y/AI-recognition-for-lq-ed.git Future work will focus on improving the generalizability of the proposed framework to different quality degradation scenarios and extrapolating the application to different engineering domains. [ABSTRACT FROM AUTHOR]

Automatic ship detection from spaceborne systems such as satellites or aircrafts, raises considerable attention in sea surface monitoring because of the several applications in military and civilian field. In this context, processing satellite images on-board would reduce the latency time especially for emergency situations. In this paper, an hardware-oriented (HO) ship detection system based on a customized Convolutional Neural Network (CNN), here referred to as HO-ShipNet , is proposed and tested on a revised version of the "Ships in Satellite Imagery" (SSI) Kaggle dataset, reporting detection accuracy of up to 95%. Furthermore, the explainability of HO-ShipNet is investigated by means of explainable Artificial Intelligence (xAI) techniques (i.e., Local Interpretable Model-Agnostic Explanation (LIME) and Occlusion Sensitivuty Analysis (OSA)), in order to understand the reasoning behind the HO-ShipNet decisions by detecting the most important input features and consequently ensure the trustworthiness of the model itself. Finally, HO-ShipNet is also implemented on the heterogeneous Xilinx xc7z045ffg900-2 SoC Field Programmable Gate Array (FPGA) outperforming state-of-the-art FPGA-based accelerators dealing with high-resolution frames. The promising results encourage the potential deployment of the proposed system for on-board applications. [ABSTRACT FROM AUTHOR]

Longitudinal impulse, as a common dynamic problem of heavy haul trains, could induce the lateral instability of the coupler and the separation of the locomotive and wagon connection and thus seriously hinder the train running safety. The coupler force undergoes complex changes during longitudinal impulse transmission and exhibits strong nonlinear characteristics, which poses significant challenges to the identification of the coupler force. To address these issues, this paper reports a novel quantitative method for the coupler force identification of heavy haul locomotives. Firstly, the moving average filtering method is used to preprocess the signal of longitudinal relative displacement between the coupler and carbody (LRDCC), and the approximate relative velocity (ARV) feature is also extracted from the preprocessed signal. Then, a Convolutional Neural Network (CNN) model is constructed to extract high-level features from LRDCC and ARV. The model with randomly initialized weight parameters does not require additional training and thus reduces the computational cost. Finally, an Extreme Learning Machine (ELM) is used as the regression layer to process the extracted high-level features and further improve the identification performance. Additionally, the dung beetle algorithm is used to obtain the optimal model parameters to improve the coupler force identification performance. Not only a dynamics simulation but a field test is carried out to show that the proposed method could accurately detect the coupler force of heavy haul locomotives. Furthermore, the proposed method shows promising prospects in replacing the traditional monitoring method under the actual long-term working conditions of trains. • An indirect measurement scheme is proposed for coupler force dynamic identification. • The proposed network model presents high identification Accuracy and efficiency. • Method showcases strong generalization under complex, long-term train operations. [ABSTRACT FROM AUTHOR]

Object detection is one of the most important domains in computer vision tasks, which is an important branch of artificial intelligence. It aims at finding and locating the accurate position of objects in given pictures or videos. With the development of deep learning techniques, more powerful and robust algorithms have emerged to deal with multi-scale, high-level features to overcome the limitations of traditional pipeline of object detectors. The popularity of transformer framework enables larger capacity datasets by processing self-attention mechanism, and the object detection methods have evolved into a new era. This paper first reviews traditional object detection pipeline and brief history of deep learning, afterwards it focuses on the classification of deep learning-based object detection methods covering Convolution Neural Network based and transformer-based methods. Commonly used datasets and metrics are also covered in the next part. The Convolution Neural Network based methods mainly contain two-stage and one-stage detectors, Convolution Neural Network is the underlying structure of these methods convolutional stages are fundamental parts. Transformer-based models convert traditional object detection issues into end-to-end detection, which is widely used in dealing with images. Finally, the promising future of object detection areas are listed to show guidance on future work. [Display omitted] [ABSTRACT FROM AUTHOR]

Weakly-supervised learning approaches have gained significant attention due to their ability to reduce the effort required for human annotations in training neural networks. This paper investigates a framework for weakly-supervised object localization, which aims to train a neural network capable of predicting both the object class and its location using only images and their image-level class labels. The proposed framework consists of a shared feature extractor, a classifier, and a localizer. The localizer predicts pixel-level class probabilities, while the classifier predicts the object class at the image level. Since image-level class labels are insufficient for training the localizer, weakly-supervised object localization methods often encounter challenges in accurately localizing the entire object region. To address this issue, the proposed method incorporates adversarial erasing and pseudo labels to improve localization accuracy. Specifically, novel losses are designed to utilize adversarially erased foreground features and adversarially erased feature maps, reducing dependence on the most discriminative region. Additionally, the proposed method employs pseudo labels to suppress activation values in the background while increasing them in the foreground. The proposed method is applied to two backbone networks (MobileNetV1 and InceptionV3) and is evaluated on three publicly available datasets (ILSVRC-2012, CUB-200-2011, and PASCAL VOC 2012). The experimental results demonstrate that the proposed method outperforms previous state-of-the-art methods across all evaluated metrics. [ABSTRACT FROM AUTHOR]

Skin cancer is one of the most serious threats to human health among skin lesions. Computer-aided diagnosis methods can assist patients in identifying and detecting skin lesion types early, thereby enabling corresponding treatments. In this paper, we propose a dual-branch neural network model Conformer with Residual Cosine Similarity Attention and Bidirectional Convolutional fusion strategy, named RCSABC-Conformer. The core of this network structure comprises three parts: a Convolutional Neural Network (CNN) branch with Residual Cosine Similarity Attention (RCSA), a Transformer branch, and a Feature Couple Unit with Bidirectional Convolutional strategy (BC-FCU). The RCSA module calculates the cosine similarity value between the feature map generated by the convolutional operation and the feature map of the residual edge to assess whether their semantic information is similar. The semantic information of similar parts is weighted by exponential normalization to enhance the network's memory of similar features of the same type of skin lesion. The BC-FCU module interactively fuses local features and global representations of skin lesion images with different resolutions in the two branches. Specifically, when the global representations is integrated into local features, we introduce a new bidirectional convolution strategy to extract the feature map from both forward and backward directions, and then select the element with the smaller feature value from the two directions to fuse into local features. In this way, we can minimize the interference of the artifact features extracted by the Transformer branch on the CNN branch. In addition, taking advantage of the Transformer branch's capacity to construct global representations, our model can learn contextual semantic information of normal skin and lesion areas to enhance model robustness. We conducted experiments on three datasets, consisting of clinical and dermoscopic skin lesion images, as well as a hybrid of both. The experimental results show that RCSABC-Conformer outperforms both advanced and classical classification methods in terms of classification accuracy across all three datasets, without requiring an increase in the number of parameters and computational complexity. Compared with the baseline model, the classification accuracy of our proposed method improves by 2.40%, 5.39%, and 4.44% on the three datasets, respectively. To the best of our knowledge, this is the first study to apply an interactive fusion dual-branch network for multi-disease classification on different modalities of skin lesion databases. Code will be available at https://github.com/AlenLi817/RCSABC-Conformer. [ABSTRACT FROM AUTHOR]

Accurate positioning of curtain wall frames is crucial for the automated installation of curtain wall modules. However, the current robot-based installation methods overly depend on visual guidance from operators, resulting in high costs and limiting construction efficiency. The development of deep learning has introduced an image segmentation approach that offers a new solution for the visual positioning of curtain wall frames. This paper proposes a context collaboration pyramid network to automatically segment curtain wall frames by incorporating context interaction and channel guided pyramid structure. The model adopts an "encoder-decoder" architecture with a feature interaction block strategically inserted between the encoder and decoder. Specifically, the encoder utilizes the pyramid pooling Transformer as a backbone to extract multi-level features from original RGB images. The decoder employs a channel guided pyramid convolution module to integrate multi-scale features and achieve finer prediction. Meanwhile, a context interaction fusion module between the features of adjacent levels was designed carefully to enhance the collaboration of the architecture. In addition, a benchmark dataset for the curtain wall frame segmentation task, consisting of 1547 images, was established. The dataset incorporates challenging scenarios, including strong lights, low contrast, and cluttered backgrounds. This method is evaluated on the collected dataset, and achieves an impressive accuracy of 97.30% and an F1-Score of 88.95%, outperforming other segmentation networks. Overall, the proposed method can extract target information accurately and efficiently and provide critical visual guidance for the robot, so as to promote the automatic installation level of the curtain wall module. [ABSTRACT FROM AUTHOR]

In the field of prognostics and health management (PHM) for industrial equipment and systems, the estimation of remaining useful life (RUL) constitutes a fundamental task. A reliable and accurate method for estimating the RUL is therefore essential. This paper proposes a dynamic self-adaptive ensemble model, aimed at improving the rolling bearing RUL prediction. This model integrates an adaptive multi-scale feature extractor, a gated convolutional unit (GCU) and temporal convolutional networks (TCN). Through a redesign of the data flow, this model directly incorporates multi-scale comprehensive feature evaluation indicators into the neural network data flow, significantly enhancing the model's feature extraction capabilities. Subsequently, the study extends the traditional TCN by incorporating the GCU module and its gating mechanisms, further strengthening the model's capacity to capture long-term dependencies in sequence tasks. Experimental results demonstrate that, compared to existing state-of-the-art (SOTA) models, our method achieves at least a 10% increase in the prediction accuracy on two public run-to-failure bearing datasets. Beyond the tested datasets, the architecture that directly maps multi-scale evaluation indicators into the structure of neural network data flows also holds potential for broader application across diverse PHM tasks, promising significant improvements in the industrial safety and efficiency. • Multi-scale evaluation module for original features. • Direct mapping from multiple evaluation metrics to feature weights. • The feature selection process is incorporated into the RUL estimation model. • Better ability to capture inherent long-term and localized dependencies. [ABSTRACT FROM AUTHOR]

The harmful utilization of DeepFake technology poses a significant threat to public welfare, precipitating a crisis in public opinion. Existing detection methodologies, predominantly relying on convolutional neural networks and deep learning paradigms, focus on achieving high in-domain recognition accuracy amidst many forgery techniques. However, overseeing the intricate interplay between textures and artifacts results in compromised performance across diverse forgery scenarios. This paper introduces a groundbreaking framework, denoted as Texture and Artifact Detector (TAD), to mitigate the challenge posed by the limited generalization ability stemming from the mutual neglect of textures and artifacts. Specifically, our approach delves into the similarities among disparate forged datasets, discerning synthetic content based on the consistency of textures and the presence of artifacts. Furthermore, we use a model ensemble learning strategy to judiciously aggregate texture disparities and artifact patterns inherent in various forgery types, thereby enabling the model's generalization ability. Our comprehensive experimental analysis, encompassing extensive intra-dataset and cross-dataset validations along with evaluations on both video sequences and individual frames, confirms the effectiveness of TAD. The results from four benchmark datasets highlight the significant impact of the synergistic consideration of texture and artifact information, leading to a marked improvement in detection capabilities. • We build an artifact and texture inconsistency detector for deepfake detection. • Texture coherence is analyzed via a novel mask extraction method and an autoencoder. • The use of an ensemble training strategy improved the ability to generalize. • Extensive experiments verified the effectiveness of the model we built. [ABSTRACT FROM AUTHOR]

Automatic and accurate medical image segmentation is a crucial step for clinical diagnosis and treatment planning of diseases. The advanced convolutional neural network (CNN) approaches based on the encoder–decoder structure have achieved state-of-the-art performances in many different medical image segmentation tasks. However, existing networks have insufficient capability of extracting the context information in each encoding stage, so they cannot effectively perceive multi-scale objects in images. In addition, the continuous down-sampling and convolution operations in the encoding stage lead to much loss of the detailed information, resulting in poor segmentation performance. In this paper, we propose a Multi-path Multi-scale Context Feature MixUp and Aggregation Network (named MpMsCFMA-Net) which fuses and aggregates multi-path features with multi-scale context information to address these issues. Based on the encoder–decoder structure, we first design the encoder to encode the semantic and detailed information of input images and introduce multi-scale context extraction module in each encoding stage. Furthermore, we design multiple features mixup module between the encoder and the decoder, aiming at providing different levels of global context information for the decoder by reconstructing skip-connection. Finally, we introduce the decoder with deeper features aggregation to better fuse multi-scale context information across layers. Experimental results on four public medical image datasets confirm that our proposed network achieves promising results and outperforms other state-of-the-art methods in most of evaluation metrics. The source code will be publicly available at https://github.com/tricksterANDthug/MpMsCFMA-Net. [ABSTRACT FROM AUTHOR]

Semantic segmentation is a well-studied topic and one of the most challenging tasks in computer vision applications, such as autonomous driving. Deep learning approaches based on convolutional neural networks (CNN) have demonstrated exceptional success on this task in recent years. Despite this success, existing approaches are plagued by higher-order inconsistencies between the ground truth images and the ones predicted by the segmentation model. This paper proposes a novel post-processing scheme based on adversarial learning to counter these inconsistencies. Such a scheme can be combined with a variety of existing CNN-based semantic segmentation networks to improve their segmentation performances. The proposed scheme is a Two-Stream Conditional Generative Adversarial Network (TScGAN), with one stream having initial semantic segmentation masks predicted by an existing CNN, while the other stream utilizes scene images to retain high-level information under a supervised residual network structure. In addition, TScGAN incorporates a novel dynamic weighting mechanism, which leads to significant and consistent gains in segmentation performance. Several comparative tests on public benchmark driving databases, including Cityscapes, Mapillary, and Berkeley DeepDrive100K, demonstrate the effectiveness of the proposed method when used with state-of-the-art CNN-based semantic segmentation models. Furthermore, the ablation experiment proved the structural rationality of our two-stream structure. The code for TScGAN could be found at https://github.com/epan-utbm/TScGAN-for-Improving-Semantic-Predictions • Explores various post-processing techniques for enhancing semantic segmentation. • Novel Two-Stream Conditional GAN enhances urban drive scene segmentation. • Enhances feature extraction by implementing dynamic weighting within streams. • Evaluates the model against existing semantic segmentation methodologies. • Tests on Cityscapes, Mapillary, BDD100K reveal major IoU gains with proposed model. [ABSTRACT FROM AUTHOR]

The image noise generated by the welding process, such as arc light, splash, and smoke, brings significant challenges for the laser vision sensor-based welding robot to locate the weld seam and accurately conduct automatic welding. Currently, deep learning-based approaches surpass traditional methods in flexibility and robustness. However, their significant computational cost leads to a mismatch with the real-time requirement of automated welding. In this paper, we propose an efficient hybrid architecture of Convolutional Neural Network (CNN) and transformer, referred to as Dynamic Squeeze Network (DSNet), for real-time weld seam segmentation. More precisely, a lightweight segmentation framework is developed to fully leverage the advantages of the transformer structure without significantly increasing computational overhead. In this respect, an efficient encoder, which aims to increase its features diversity, has been designed and resulted in substantial improvement of encoding performance. Moreover, we propose a plug-and-play lightweight attention module that generates more effective attention weights by exploiting statistical information of weld seam data and introducing linear priors. Extensive experiments on weld seam images using NVIDIA GTX 1050Ti show that our approach reduces the number of parameters by 54x, decreases computational complexity by 34x, and improves inference speed by 33x compared to the baseline method TransUNet. DSNet achieves superior accuracy (78.01% IoU, 87.64% Dice) and speed performance (100 FPS) with lower model complexity and computational burden than most state-of-the-art methods. The code is available at https://github.com/hackerschen/DSNet. [ABSTRACT FROM AUTHOR]

Most existing Convolutional Neural Networks (CNNs), Transformers, and their variants have limitations in capturing relationships between hyperspectral image (HSI) data, leading to unclear descriptions of region boundaries and limited generalization abilities. While semi-supervised Graph Neural Networks (GNNs) come with higher computational costs. Therefore, this paper proposes a method interacting the frequency and topology information for HSI Classification to address the aforementioned shortcomings, which combines convolution and self-attention to capture both local and global contextual information, thereby enhancing feature representation. Additionally, this method focuses on exploring spectral and topological structure features and enhancing the information exchange and interaction to improve performance. Experimental results demonstrate that this method gains a competitive advantage in HSI classification by proving highly effective in handling spectral ambiguity and material heterogeneity. It also exhibits lower computational costs, making it more feasible and practical compared to most benchmark methods. Our code is available at https://github.com/youngboy03/FTINet. [Display omitted] • Use CIformer, AFLM and ECM to obtain features from different angles in the image. [ABSTRACT FROM AUTHOR]

In the face of uncontrolled offensive content on social media, automated detection emerges as a critical need. This paper tackles this challenge by proposing a novel approach for identifying offensive language in multilingual, code-mixed, and script-mixed settings. The study presents a novel multilingual hybrid dataset constructed by merging diverse monolingual and bilingual resources. Further, we systematically evaluate the impact of input representations (Word2Vec, Global Vectors for Word Representation (or GloVe), Bidirectional Encoder Representations from Transformers (or BERT), and uniform initialization) and deep learning models (Convolutional Neural Network (or CNN), Bidirectional Long Short Term Memory (or Bi-LSTM), Bi-LSTM-Attention, and fine-tuned BERT) on detection accuracy. Our comprehensive experiments on a dataset of 42,560 social media comments from five languages (English, Hindi, German, Tamil, and Malayalam) reveal the superiority of fine-tuned BERT. Notably, it achieves a macro average F 1 -score of 0.79 for monolingual tasks and an impressive 0.86 for code-mixed and script-mixed tasks. These findings significantly advance offensive language detection methodologies and shed light on the complex dynamics of multilingual social media, paving the way for more inclusive and safer online communities. [ABSTRACT FROM AUTHOR]

Remaining Useful Life (RUL) prediction is of great significance for maintaining the reliability and safety of industrial equipment. To address the challenges faced by existing methods in simultaneously extracting local and global degradation information from monitoring data. This paper proposes a Two-Stream Convolution Augmented Transformer (TACT) model based on L 2 regularization constraint. Specifically, we design the parallel multi-scale Convolution Neural Network (CNN) and Transformer module to combine the local modeling ability of CNN and the global modeling ability of Transformer to improve the overall architecture of RUL prediction model. Moreover, the two-stream network based on the parallel structure also realizes the synchronous extraction of different time steps and sensor features in the sequence. Then, in the process of model training, the prediction reliability constraint is fused, the delay prediction constraint term is introduced, and the L 2 regularization loss function is constructed. Finally, extensive experiments on the commercial modular aero-propulsion system simulation (C-MAPSS) show that our model provides competitive performance in terms of Root-Mean-Square Error (RMSE) and Score metrics. Compared to the state-of-the-art method based on Recurrent Neural Network (RNN) or CNN and its variants, Score is reduced by at least 2.71% and RMSE by at least 3.13%. Compared to the Transformer-based improved method, the Score is decreased by at least 4.54% and the RMSE is decreased by at least 2.78%. The effectiveness of the proposed method is demonstrated. • A hybrid end-to-end RUL prediction network based on a tandem combination of CNN and Transformer is proposed. • A two-stream structure is used to extract spatial and temporal features simultaneously. • The MSE loss function incorporating L 2 regularization constraint is proposed. [ABSTRACT FROM AUTHOR]

Skin cancer is one of the most serious threats to human health among skin lesions. Computer-aided diagnosis methods can assist patients in identifying and detecting skin lesion types early, thereby enabling corresponding treatments. In this paper, we propose a dual-branch neural network model Conformer with Residual Cosine Similarity Attention and Bidirectional Convolutional fusion strategy, named RCSABC-Conformer. The core of this network structure comprises three parts: a Convolutional Neural Network (CNN) branch with Residual Cosine Similarity Attention (RCSA), a Transformer branch, and a Feature Couple Unit with Bidirectional Convolutional strategy (BC-FCU). The RCSA module calculates the cosine similarity value between the feature map generated by the convolutional operation and the feature map of the residual edge to assess whether their semantic information is similar. The semantic information of similar parts is weighted by exponential normalization to enhance the network's memory of similar features of the same type of skin lesion. The BC-FCU module interactively fuses local features and global representations of skin lesion images with different resolutions in the two branches. Specifically, when the global representations is integrated into local features, we introduce a new bidirectional convolution strategy to extract the feature map from both forward and backward directions, and then select the element with the smaller feature value from the two directions to fuse into local features. In this way, we can minimize the interference of the artifact features extracted by the Transformer branch on the CNN branch. In addition, taking advantage of the Transformer branch's capacity to construct global representations, our model can learn contextual semantic information of normal skin and lesion areas to enhance model robustness. We conducted experiments on three datasets, consisting of clinical and dermoscopic skin lesion images, as well as a hybrid of both. The experimental results show that RCSABC-Conformer outperforms both advanced and classical classification methods in terms of classification accuracy across all three datasets, without requiring an increase in the number of parameters and computational complexity. Compared with the baseline model, the classification accuracy of our proposed method improves by 2.40%, 5.39%, and 4.44% on the three datasets, respectively. To the best of our knowledge, this is the first study to apply an interactive fusion dual-branch network for multi-disease classification on different modalities of skin lesion databases. Code will be available at https://github.com/AlenLi817/RCSABC-Conformer. [ABSTRACT FROM AUTHOR]

Accurate positioning of curtain wall frames is crucial for the automated installation of curtain wall modules. However, the current robot-based installation methods overly depend on visual guidance from operators, resulting in high costs and limiting construction efficiency. The development of deep learning has introduced an image segmentation approach that offers a new solution for the visual positioning of curtain wall frames. This paper proposes a context collaboration pyramid network to automatically segment curtain wall frames by incorporating context interaction and channel guided pyramid structure. The model adopts an "encoder-decoder" architecture with a feature interaction block strategically inserted between the encoder and decoder. Specifically, the encoder utilizes the pyramid pooling Transformer as a backbone to extract multi-level features from original RGB images. The decoder employs a channel guided pyramid convolution module to integrate multi-scale features and achieve finer prediction. Meanwhile, a context interaction fusion module between the features of adjacent levels was designed carefully to enhance the collaboration of the architecture. In addition, a benchmark dataset for the curtain wall frame segmentation task, consisting of 1547 images, was established. The dataset incorporates challenging scenarios, including strong lights, low contrast, and cluttered backgrounds. This method is evaluated on the collected dataset, and achieves an impressive accuracy of 97.30% and an F1-Score of 88.95%, outperforming other segmentation networks. Overall, the proposed method can extract target information accurately and efficiently and provide critical visual guidance for the robot, so as to promote the automatic installation level of the curtain wall module. [ABSTRACT FROM AUTHOR]

In the field of prognostics and health management (PHM) for industrial equipment and systems, the estimation of remaining useful life (RUL) constitutes a fundamental task. A reliable and accurate method for estimating the RUL is therefore essential. This paper proposes a dynamic self-adaptive ensemble model, aimed at improving the rolling bearing RUL prediction. This model integrates an adaptive multi-scale feature extractor, a gated convolutional unit (GCU) and temporal convolutional networks (TCN). Through a redesign of the data flow, this model directly incorporates multi-scale comprehensive feature evaluation indicators into the neural network data flow, significantly enhancing the model's feature extraction capabilities. Subsequently, the study extends the traditional TCN by incorporating the GCU module and its gating mechanisms, further strengthening the model's capacity to capture long-term dependencies in sequence tasks. Experimental results demonstrate that, compared to existing state-of-the-art (SOTA) models, our method achieves at least a 10% increase in the prediction accuracy on two public run-to-failure bearing datasets. Beyond the tested datasets, the architecture that directly maps multi-scale evaluation indicators into the structure of neural network data flows also holds potential for broader application across diverse PHM tasks, promising significant improvements in the industrial safety and efficiency. • Multi-scale evaluation module for original features. • Direct mapping from multiple evaluation metrics to feature weights. • The feature selection process is incorporated into the RUL estimation model. • Better ability to capture inherent long-term and localized dependencies. [ABSTRACT FROM AUTHOR]

In the face of uncontrolled offensive content on social media, automated detection emerges as a critical need. This paper tackles this challenge by proposing a novel approach for identifying offensive language in multilingual, code-mixed, and script-mixed settings. The study presents a novel multilingual hybrid dataset constructed by merging diverse monolingual and bilingual resources. Further, we systematically evaluate the impact of input representations (Word2Vec, Global Vectors for Word Representation (or GloVe), Bidirectional Encoder Representations from Transformers (or BERT), and uniform initialization) and deep learning models (Convolutional Neural Network (or CNN), Bidirectional Long Short Term Memory (or Bi-LSTM), Bi-LSTM-Attention, and fine-tuned BERT) on detection accuracy. Our comprehensive experiments on a dataset of 42,560 social media comments from five languages (English, Hindi, German, Tamil, and Malayalam) reveal the superiority of fine-tuned BERT. Notably, it achieves a macro average F 1 -score of 0.79 for monolingual tasks and an impressive 0.86 for code-mixed and script-mixed tasks. These findings significantly advance offensive language detection methodologies and shed light on the complex dynamics of multilingual social media, paving the way for more inclusive and safer online communities. [ABSTRACT FROM AUTHOR]

Remaining Useful Life (RUL) prediction is of great significance for maintaining the reliability and safety of industrial equipment. To address the challenges faced by existing methods in simultaneously extracting local and global degradation information from monitoring data. This paper proposes a Two-Stream Convolution Augmented Transformer (TACT) model based on L 2 regularization constraint. Specifically, we design the parallel multi-scale Convolution Neural Network (CNN) and Transformer module to combine the local modeling ability of CNN and the global modeling ability of Transformer to improve the overall architecture of RUL prediction model. Moreover, the two-stream network based on the parallel structure also realizes the synchronous extraction of different time steps and sensor features in the sequence. Then, in the process of model training, the prediction reliability constraint is fused, the delay prediction constraint term is introduced, and the L 2 regularization loss function is constructed. Finally, extensive experiments on the commercial modular aero-propulsion system simulation (C-MAPSS) show that our model provides competitive performance in terms of Root-Mean-Square Error (RMSE) and Score metrics. Compared to the state-of-the-art method based on Recurrent Neural Network (RNN) or CNN and its variants, Score is reduced by at least 2.71% and RMSE by at least 3.13%. Compared to the Transformer-based improved method, the Score is decreased by at least 4.54% and the RMSE is decreased by at least 2.78%. The effectiveness of the proposed method is demonstrated. • A hybrid end-to-end RUL prediction network based on a tandem combination of CNN and Transformer is proposed. • A two-stream structure is used to extract spatial and temporal features simultaneously. • The MSE loss function incorporating L 2 regularization constraint is proposed. [ABSTRACT FROM AUTHOR]

Crude oil prices are susceptible to a variety of elements, including military, political, financial, and diplomatic forces, resulting in price fluctuations that are random, abrupt, and chaotic. Based on the chaotic nature of crude oil price series, this paper proposes a model for predicting crude oil prices that incorporates variational modal decomposition algorithm (VMD), phase space reconstruction technique (PSR), convolutional neural network (CNN), and bidirectional long and short-term memory network (BiLSTM). Specifically, the noises in the original data are removed using VMD. Next, the crude oil price is reconstructed using PSR. Then the reconstructed and denoised phase space is fed into the CNN-BiLSTM model for multi-step predictions. The empirical results show that the proposed model achieves the lowest MAPEs as 0.662%, 2.283%, 3.829%, and 4.44% and the lowest MSEs as 0.335, 3.942, 10.047 and 13.681. The Diebold Mariano (DM) test also demonstrates the proposed model's superiority in the domain of crude oil price forecasting. [ABSTRACT FROM AUTHOR]

Fault testing in the production line of automotive traction machines is essential to ensure the desired lifetime. Since repetitive partial discharges (PDs) caused by anomalies in the insulation system lead to premature breakdowns of electrical machines, a reliable PD detection is of great importance. This paper proposes deep learning (DL) methods to improve the discrimination of PD from background noise in comparison with the state-of-the-art amplitude based PD detection in the production line. First, a systematic data extraction and labeling procedure is introduced to obtain correctly labeled datasets from arbitrary PD measurements. In addition, datasets are enhanced with low signal-to-noise ratio PD pulses by applying a special data augmentation approach. 13 different convolutional, recurrent and fully connected neural networks are compared for various time-frequency representations of the input signals. Hyperparameters for input transform, network topology and solver are optimized for all 13 combinations to ensure a fair case study. As a result, the two-dimensional convolutional neural network with continuous wavelet transform achieves the best accuracy of around 99.76% on a test dataset of PD signals originating from previously not utilized test objects. All DL models considered in this comparison outperform the state-of-the-art threshold-based PD classification. Even for PD events with an amplitude close to the noise level, the detection rate is still around 95% for the best network. Furthermore, without applying the proposed data augmentation procedure, the DL models investigated are not able to distinguish small PD pulses from noise. • Deep learning to improve partial discharge classification of electrical machines. • Rise of partial discharge detection sensitivity and decrease in false alarm rate. • Data extraction, labeling and augmentation procedure to generate reliable datasets. • Comprehensive comparison of time-frequency transforms and network architectures. [ABSTRACT FROM AUTHOR]