Search

The reliability of bolt connections significantly impacts the operational state and lifespan of industrial equipment. Vision-based noncontact methods exhibit high efficiency in bolt loosening detection. However, limited image features hinder measurement accuracy. To improve bolt loosening detection performance, this paper proposes a novel deep learning backbone, the high-resolution cross-scale transformer, to extract high precision keypoints for bolt three-dimensional model construction. Simultaneously, a monocular vision measurement model is established to get the bolt exposed length and evaluate the connection loosening state. The proposed backbone hybridizes the advantages of high-resolution architecture and transformer, realizing global information aggregation and fine-grained image details. A simplified module, dual-scale multi-head self-attention, is designed to reduce the computational redundancy caused by the implementation of high-resolution multi-branch architecture. In the experiment section, the high-resolution cross-scale transformer outperforms other keypoint detection baselines, achieving the top one performance with 91.6 average precision and 84.9 average recall. The monocular vision measurement model realizes a 0.053 mm error with a 0.028 mm standard deviation, satisfying the industrial implementation requirement. Additionally, the model is tested on different industrial situations and an additional outside dataset, indicating the model's robustness and actual environment adaptability. • A monocular vision measurement method for bolt loosening detection is proposed. • It is the first attempt to introduce the transformer mechanism in bolt keypoint detection. • A new keypoint detection backbone is proposed in this paper for bolt feature extraction. • The 3D exposed length of bolts is calculated with a monocular vision system. • The bolt loosening detection method is validated. [ABSTRACT FROM AUTHOR]

Over the past decades, deep learning techniques have attracted increased attention from various research and industrial domains aligned with the development of Industry Internet-of-Things(IIoT). Specifically, with the advantage of data-driven methods, industrial organizations are seeking novel proactive strategies supported by analytic models to guarantee the quality of their production by observing degradation or predicting failure ahead of the occurrence of the component or asset. Predictive strategies are expected to promise the influence of unnecessary maintenance interruptions and mitigate the consequence of that, hence, extending the remaining useful life of products. This paper conducts a survey of the utilization of deep learning technologies on engineering applications where they provide satisfactory solutions with respect to specific data types or input signals. 106 primary papers are reviewed on deep learning–driven approaches which mainly explore five of the most popular architectures in the application of predictive maintenance. The main content of this paper summarizes the common advantages of each architecture and, accordingly, points out their limitations, as well as describes the application scopes of fully connected deep neural networks, convolutional neural networks, stacked autoencoders, deep belief networks, and deep recurrent neural networks. Based on the technique discussion for each of them, we intend to provide a comprehensive understanding and guidance of the appropriate usage of deep learning architectures to devise an effective predictive maintenance strategy for the scientific and industrial developers whose expertise lies in the prior domain knowledge of multi-source isomerization data. Moreover, the main content demonstrated the summarization of the decisive factor by which the incremental stages of the approaches were determined, fundamentally including the dataset specification, feature extraction, and the integration of deep learning approaches. [ABSTRACT FROM AUTHOR]

Track geometry is one of the critical indicators of railway tracks' condition which requires continuous monitoring and maintenance over time. In this paper, a novel artificial intelligence (AI) based framework is proposed for railway track geometry inspection using vibration data collected from a dedicated measuring high-speed train. This AI-based anomaly track detection approach consists of two main stages. Firstly, a subset of features that best characterizes the track condition is defined. Several dynamic features from time domain data are extracted and importance scores are assigned to them, to determine the most effective subset for the purpose of track condition monitoring. Secondly, a data-driven based anomaly detection approach is developed to assess and identify track geometrical defects. In this stage, the acceleration responses collected from an in-service train traversing on a healthy track zone are employed as input into a One-Class Support Vector Machine (OCSVM) algorithm. The proposed algorithm defines the anomalies as relative changes to the historical behaviour. A comprehensive dataset from field measurements using a Société Nationale des Chemins de Fer Français (SNCF) Réseau IRIS320 highspeed train is used in this paper to implement the proposed approach. In addition, the impact of using different features and different locations/directions of the sensors on the accuracy of detecting geometrical defects is investigated. It is also shown that the OCSVM approach outperforms other algorithms based on Isolation Forest (IF), Local Outlier Factor (LOF), and Robust Mahalanobis Distance (RMD) in terms of recall, precision, and F1-score. The proposed anomaly detection approach has demonstrated a 12% increase in defect detection accuracy compared to the direct utilization of the raw acceleration response, which can facilitate track monitoring using in-service trains while providing cost-efficient maintenance in the future. [Display omitted] [ABSTRACT FROM AUTHOR]

Human activity recognition (HAR) is a crucial detection technique extensively employed in various contexts demanding accurate identification of human actions. Currently, mainstream HAR approaches frequently rely on sequential data generated by wearable sensors, with a primary focus on extracting feature representations that align with various human activities. Nevertheless, complex and diverse human activities encompass various kinds of features and substantial variations in the features emphasized across different activities, making it challenging for current artificial intelligence methods to achieve accurate recognition. To address this challenge, this paper introduces a network framework called the Multi-Feature Combining Attention Neural Network (MFCANN). This framework incorporates diverse feature extraction, along with both local and global feature attention mechanisms. It is composed of stacked Multi-Feature Combining Attention Blocks (MFCAB) designed by us. In contrast to traditional convolutional methods, MFCAB parallelly stacks various convolutional components, enabling the extraction of a broader range of features from human activity data. Additionally, we propose the Intra-Module Attention Block (Intra-MAB) and the Inter-Module Attention Block (Inter-MAB), which simultaneously focus on local fine-grained features within module feature maps and global distinguishing features across module feature maps. This aims to achieve more targeted feature learning, reinforcing the network's ability to distinguish between different human activities. Abundant experimental results demonstrate that the proposed MFCANN in this paper outperforms current mainstream deep learning algorithms in HAR tasks, achieving recognition accuracies of 0.9813, 0.9324, and 0.9930 on the UCI-HAR, USC-HAD, and RealWorld datasets respectively. We release our code at https://github.com/RangerHeart/MFCANN. [ABSTRACT FROM AUTHOR]

Track geometry is one of the critical indicators of railway tracks' condition which requires continuous monitoring and maintenance over time. In this paper, a novel artificial intelligence (AI) based framework is proposed for railway track geometry inspection using vibration data collected from a dedicated measuring high-speed train. This AI-based anomaly track detection approach consists of two main stages. Firstly, a subset of features that best characterizes the track condition is defined. Several dynamic features from time domain data are extracted and importance scores are assigned to them, to determine the most effective subset for the purpose of track condition monitoring. Secondly, a data-driven based anomaly detection approach is developed to assess and identify track geometrical defects. In this stage, the acceleration responses collected from an in-service train traversing on a healthy track zone are employed as input into a One-Class Support Vector Machine (OCSVM) algorithm. The proposed algorithm defines the anomalies as relative changes to the historical behaviour. A comprehensive dataset from field measurements using a Société Nationale des Chemins de Fer Français (SNCF) Réseau IRIS320 highspeed train is used in this paper to implement the proposed approach. In addition, the impact of using different features and different locations/directions of the sensors on the accuracy of detecting geometrical defects is investigated. It is also shown that the OCSVM approach outperforms other algorithms based on Isolation Forest (IF), Local Outlier Factor (LOF), and Robust Mahalanobis Distance (RMD) in terms of recall, precision, and F1-score. The proposed anomaly detection approach has demonstrated a 12% increase in defect detection accuracy compared to the direct utilization of the raw acceleration response, which can facilitate track monitoring using in-service trains while providing cost-efficient maintenance in the future. [Display omitted] [ABSTRACT FROM AUTHOR]

Human activity recognition (HAR) is a crucial detection technique extensively employed in various contexts demanding accurate identification of human actions. Currently, mainstream HAR approaches frequently rely on sequential data generated by wearable sensors, with a primary focus on extracting feature representations that align with various human activities. Nevertheless, complex and diverse human activities encompass various kinds of features and substantial variations in the features emphasized across different activities, making it challenging for current artificial intelligence methods to achieve accurate recognition. To address this challenge, this paper introduces a network framework called the Multi-Feature Combining Attention Neural Network (MFCANN). This framework incorporates diverse feature extraction, along with both local and global feature attention mechanisms. It is composed of stacked Multi-Feature Combining Attention Blocks (MFCAB) designed by us. In contrast to traditional convolutional methods, MFCAB parallelly stacks various convolutional components, enabling the extraction of a broader range of features from human activity data. Additionally, we propose the Intra-Module Attention Block (Intra-MAB) and the Inter-Module Attention Block (Inter-MAB), which simultaneously focus on local fine-grained features within module feature maps and global distinguishing features across module feature maps. This aims to achieve more targeted feature learning, reinforcing the network's ability to distinguish between different human activities. Abundant experimental results demonstrate that the proposed MFCANN in this paper outperforms current mainstream deep learning algorithms in HAR tasks, achieving recognition accuracies of 0.9813, 0.9324, and 0.9930 on the UCI-HAR, USC-HAD, and RealWorld datasets respectively. We release our code at https://github.com/RangerHeart/MFCANN. [ABSTRACT FROM AUTHOR]

Remote sensing scene (RSS) image classification plays a vital role in various fields such as urban planning and environmental protection. However, due to higher inter-class similarity and intra-class variability, achieving accurate classification for RSS images poses a considerable challenge for current convolutional neural networks (CNNs)-based and visual transformer (ViT)-based methods. To address these issues, this paper proposes a novel dual-encoding method named master–slave encoding network (MSE-Net) from two perspectives of feature extraction and fusion. The master encoder, based on ViT, extracts higher-level semantic features, while the slave encoder, based on CNN, captures relative lower-level spatial structure information. Secondly, to integrate feature information from the two encoders effectively, this paper further develop two fusion strategies. The first strategy involves the auxiliary enhancement units (AEUs), which eliminates semantic divergence between the two encoders, enhances spatial context awareness of the slave encoder and promotes effective feature learning. The interactive perception unit (IPU), as the second strategy, facilitates interaction and integration of the two encoders' representations to extract more discriminative feature information. In addition, we conducted comparative experiments on four widely-used RSS datasets, including RSSCN7, SIRI-WHU, the aerial image dataset (AID) and NWPU-RESISC45 (NWPU45), to verify the effectiveness of MSE-Net. The experimental results demonstrate that MSE-Net achieved state-of-the-art (SOTA) performance across all the datasets. [Display omitted] [ABSTRACT FROM AUTHOR]

Quick response codes are widely utilized in commodity anti-counterfeiting, and their traceability via smartphones is an effective authentication method. However, quick response codes are susceptible to illegal duplication, and the images captured by smartphones are easily affected by changes in light and environmental noise, leading to unsatisfactory verification results. To address these issues, this paper introduces a novel approach by combining the squeeze-excitation attention module with bottleneck residual block. It presents a squeeze-excitation bottleneck residual network for printer source identification of quick response codes. The squeeze-excitation attention module pays more attention to the features that represent the printer attributes, reduces the interference of useless information, and has low computational consumption; while the bottleneck residual block has the advantages of few parameters, strong feature extraction capability, and good expandability. Thus, the performance can be improved effectively only with a small increasing in parameters. The experimental results verify that the proposed method achieves an accuracy of 98.77% under smartphone capture conditions, it outperforms other convolutional neural network-based methods in terms of identification accuracy. The deep learning model proposed in this paper can be generalized and applied to the printer source identification of paper content in the civil, criminal investigation and judicial fields. [ABSTRACT FROM AUTHOR]

Applications in Engineering: In industry, surface defect detection is crucial for improving product quality. However, there are many challenges in industrial inspection scenarios, such as interference from background noise, complex small-target problems, significant variations in target objects, and the problem of finding a balance between inspection speed and accuracy. To address the above problems, this paper proposes an industrial defect-aware network based on cross-layer semantic information guidance and contextual attention enhancement (IDP-Net). Specifically, IDP-Net has four different new features. The contribution of artificial intelligence: Firstly, to solve the industrial surface context and defect similarity problem, this paper proposes a Lightweight Local Global Feature Extraction Network (LLG-Net), unlike other methods, the effective combination of self-attention blocks and convolution blocks ensures gradual integration of global and local features across multiple layers, to improve the detection ability of targets with significant changes in scale, this paper designs a Multiscale Perceptual Feature Aggregation Network (MPA-Net), adequately fuses the shallow fine-grained information and the deep semantic information. Then, to enhance the connection between multi-scale semantic information, an adaptive cross-layer feature fusion module (ACFF) is proposed, which is novel in integrating the characteristics of multiple adjacent levels to help the model better capture the different scale characterisation of the target. Finally, a Region Attention Module (RAM) is proposed and introduced in the detector to enhance the attention to the critical regions around the target object. In particular, this paper proposes a new localisation loss function (MEIoU) that enhances the network's attention to objects at different scales. The experimental results show that 94.3%, 98.7% and 99.5% of mAP@.5 are obtained on steel, PCB and aluminium surface defect datasets, respectively, and 50 FPS is achieved, which is better than the current mainstream detectors and meets the demand of practical industrial production. [ABSTRACT FROM AUTHOR]

The core of data series classification and clustering is how to evaluate the similarity between time series. However, for one-dimensional time series, the vast majority of similarity measurement methods currently relied on one-dimensional information. Although some classifications of time series were achieved by the images transformed from time series, they mostly relied on a single imaging mechanism (time domain or frequency domain). In order to give a new idea for time series similarity measurement, this paper proposed a similarity measurement method of time series based on the fused images. According to recurrence plots (RP) and wavelet scalogram (WS), the original time series was first transformed into images. Then, RP and WS were fused into an image. Next, the feature in the image was extracted by ResNet-18. Finally, the similarity of the time series can be obtained by the Euclidean distance between the features extracted by ResNet-18. The proposed method was verified by the time series from UCR Time Series Classification/Clustering Homepage. The results showed that the similarity measurement of time series can be effectively improved by fusing the images obtained by different imaging mechanisms in the time domain and frequency domain. • This paper proposes a new similarity measurement method for time series based on the image fusion of different forms. • The time domain and frequency domain information of time series are represented by recurrence plots and wavelet scalogram. • The characteristics of the image fused by recurrence plots and wavelet scalogram are extracted by Convolutional Neural Network to measure the similarity of time series. • 30 sets of time series show the validity and superiority of the proposed method. • The reasons for the effectiveness of the proposed method are analyzed in detail. [ABSTRACT FROM AUTHOR]

The aging of transformer oil–paper insulation (OPI) contributes significantly to transformer failure and reduced service life. Therefore, reliable and precise evaluation of the aging status of transformer insulation condition is crucial in averting transformer failure. This article proposes a novel deep learning-based aging condition assessment technique of transformers by combining FDS (Frequency domain spectroscopy) and squeeze-and-excitation-enabled convolutional neural network (SE-CNN). An intrinsic product-based transformation using a Gramian angular field is employed to dimensionally expand the 1-D FDS signals acquired from OPI samples into 2D images. Each feature extraction channel in the SE-CNN-based aging diagnosis framework is assigned a specific weight, enabling the model to concentrate on the most relevant features while discarding the rest. Furthermore, the effect of moisture variation on the model is also considered under three different conditions to ensure the consequence of compensation for the effect of moisture on FDS. A comparison with traditional machine learning models has also been made to ensure the viability of the proposed methodology. Additionally, the model is interpreted by creating explanation weight plots of the feature coefficients using a local interpretable model-agnostic explanations (LIME) method in Python. The findings demonstrate that the novel aging diagnosis framework visually assesses the transformer's OPI system with the highest accuracy of 99.12% in the test set. Furthermore, it facilitates the automatic extraction of deep aging features, thus providing better generalization and convergence capability. [ABSTRACT FROM AUTHOR]

With the rapid expansion of the electric vehicle market, the demand for lithium-ion batteries (LIBs) is exploding. The state of health (SOH) of LIBs is receiving more widespread attention, which is the key parameter for battery health management. This paper proposes a SOH estimation method for LIBs via feature optimization and convolutional neural network (CNN) to reduce the information redundancy of existing multiple features, aimed at leveraging multiple sources of features while optimizing their combination to minimize redundancy. Firstly, multiple features are extracted from different perspectives, including electrical, thermodynamic, and electrochemical properties, to comprehensively characterize the aging of batteries. Secondly, we construct a SOH estimator based on principal component analysis (PCA) with CNN (PCA-CNN). Finally, the dimension of features is optimized with a simulated annealing algorithm (SA) under the mean-variance objective function. Moreover, Comparative experiments are conducted on the Oxford dataset for validation. The results demonstrate the effectiveness of the proposed multi-feature description method in terms of accuracy and smoothness. Compared to traditional CNN methods and fixed-dimension PCA-CNN, this estimation approach significantly improves performance, showing more than 20% and 30% increases in the key metrics of MAE and RMSE, respectively. This study successfully optimized feature combinations to reduce redundancy within the feature set while enhancing the accuracy of SOH estimation. [ABSTRACT FROM AUTHOR]

Link prediction is a fundamental problem in complex network analysis, aimed at predicting missing or forthcoming connections. Recent research investigates the potential of Nonnegative Matrix Factorization (NMF) models in reconstructing sparse networks and using deep NMF to uncover the hierarchical structure. Deep models have demonstrated remarkable performance in various domains, but they are susceptible to overfitting, especially on the limited training data. This paper proposes a novel Link Prediction using Adversarial Deep NMF (LPADNMF) to enhance the generalization of network reconstruction in sparse graphs. The main contribution is the introduction of an adversarial training that incorporates a bounded attack on the input, leveraging the L 2 , 1 norm to generate diverse perturbations. This adversarial training aims to improve the model's robustness and prevent overfitting, particularly in scenarios with limited training data. Additionally, the proposed method incorporates first and second-order affinities as input to capture higher-order dependencies and encourage the extraction of informative features from the network structure. To further mitigate overfitting, a smooth L 2 regularization is applied to the model parameters. To optimize the proposed model effectively, we utilize a majorization-minimization algorithm that efficiently updates the perturbation and latent factors in an iterative manner. Our findings demonstrate that the proposed model not only has the ability to uncover and learn complex structures but also possesses generalization capabilities. This method demonstrates superior performance compared to state-of-the-art methods across eight networks. In comparison to the best-performing approach, LPADNMF exhibited improvements of 2.36% in AUC, 4.67% in Precision, 2.21% in Recall, and 3.73% in F-Measure. [ABSTRACT FROM AUTHOR]

Due to the inevitable light absorption and scattering, underwater images always suffer from severe quality degradation, leading to significant performance decline for various maritime engineering-related applications. In recent years, deep neural networks (DNNs) have been proven to achieve high-quality enhancement of underwater images, which aims to extract and learn abstract features to exhibit superior performance. However, existing solutions often stack multilayer processing units to enrich features, which not only significantly burdens computational load but also encounters difficulties in fully utilizing and interacting between features at different depths. To address the above problems, this paper presents a cost-effective underwater image enhancement network via cascaded feature extraction, termed CFENet. Specifically, we develop a cost-effective cascaded structure for sufficient feature extraction and multiscale establishment of pixel-level long-range dependencies. In addition, we construct a dual-branch structure for effective feature fusion, so that the detailed texture and semantic information in the image can be simultaneously enhanced. Extensive experiments reveal the superiority of our proposed CFENet in both underwater image enhancement effects and computational complexity. Sufficient ablation study is conducted to demonstrate the effectiveness of each component in the network. [ABSTRACT FROM AUTHOR]

Dynamic facial expression recognition (DFER) is of great significance in promoting empathetic machines and metaverse technology. However, dynamic facial expression recognition (DFER) in the wild remains a challenging task, often constrained by complex lighting changes, frequent key-points occlusion, uncertain emotional peaks and severe imbalanced dataset categories. To tackle these problems, this paper presents a depth neural network model based on spatial key-points optimized region feature fusion and temporal self-attention. The method includes three parts: spatial feature extraction module, temporal feature extraction module and region feature fusion module. The intra-frame spatial feature extraction module is composed of the key-points graph convolution network (GCN) and a convolution network (CNN) branch to obtain the global and local feature vectors. The newly proposed region fusion strategy based on face spatial structure is used to obtain the spatial fusion feature of each frame. The inter-frame temporal feature extraction module uses multi-head self-attention model to obtain the temporal information of inter-frames. The experimental results show that our method achieves accuracy of 68.73%, 55.00%, 47.80%, and 47.44% on the DFEW, AFEW, FERV39k, and MAFW datasets. Ablation experiments showed that the GCN module, fusion module, and temporal module improved the accuracy on DFEW by 0.68%, 1.66%, and 3.25%, respectively. The method also achieves competitive results in terms of parameter quantity and inference speed, which demonstrates the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

In practical engineering scenarios with limited labeled samples, conventional semi-supervised diagnostic methods face challenges in achieving satisfactory identification outcomes. To address the aforementioned issues, this paper introduces an incremental semi-supervised learning (ISL) approach based on boosting efficient attention (BEA) assisted cyclic adversarial auto-encoder (CAAE), referred to as BEA-CAAE. The CAAE enhances unsupervised feature representation by simultaneously constraining the distribution of encoded features and aligning elements of the reconstructed samples through a cyclic encoding strategy. The BEA improves classical attention weights' activation strength to better capture vital information, thereby boosting the feature extraction capabilities of both CAAE and the classifier. The ISL employs a stepwise pseudo-label propagation strategy to incrementally filter high-confidence samples, enhancing sample and label utilization, and improving diagnostic accuracy under low label rate conditions. Experiments conducted on multiple test rigs with simple structures as well as large-scale rotating components test rigs that mimic real-world engineering conditions have demonstrated that the proposed method exhibits a significant advantage over existing semi-supervised fault diagnosis approaches in terms of fault diagnosis accuracy and generalization capability, especially under low label rate conditions. [ABSTRACT FROM AUTHOR]

Longitudinal impulse, as a common dynamic problem of heavy haul trains, could induce the lateral instability of the coupler and the separation of the locomotive and wagon connection and thus seriously hinder the train running safety. The coupler force undergoes complex changes during longitudinal impulse transmission and exhibits strong nonlinear characteristics, which poses significant challenges to the identification of the coupler force. To address these issues, this paper reports a novel quantitative method for the coupler force identification of heavy haul locomotives. Firstly, the moving average filtering method is used to preprocess the signal of longitudinal relative displacement between the coupler and carbody (LRDCC), and the approximate relative velocity (ARV) feature is also extracted from the preprocessed signal. Then, a Convolutional Neural Network (CNN) model is constructed to extract high-level features from LRDCC and ARV. The model with randomly initialized weight parameters does not require additional training and thus reduces the computational cost. Finally, an Extreme Learning Machine (ELM) is used as the regression layer to process the extracted high-level features and further improve the identification performance. Additionally, the dung beetle algorithm is used to obtain the optimal model parameters to improve the coupler force identification performance. Not only a dynamics simulation but a field test is carried out to show that the proposed method could accurately detect the coupler force of heavy haul locomotives. Furthermore, the proposed method shows promising prospects in replacing the traditional monitoring method under the actual long-term working conditions of trains. • An indirect measurement scheme is proposed for coupler force dynamic identification. • The proposed network model presents high identification Accuracy and efficiency. • Method showcases strong generalization under complex, long-term train operations. [ABSTRACT FROM AUTHOR]

A data-driven approach for classifying defect patterns in wafer maps is crucial for automating the quality assurance process during semiconductor fabrication. Existing works have proposed machine learning-based methods to accomplish this important problem. However, many of these approaches assume model training from a supervised learning perspective, in which a substantial number of annotated wafer maps, each with its actual defect pattern, need to be available. Due to the cost and time required to generate such an annotated dataset, recently a few works started to explore the problem of identifying previously unseen defect patterns. However, the methods proposed in those works are able to only detect the presence of unseen defect patterns without identifying what those defect patterns are. To address this issue, this paper introduces a generalized zero-shot learning (GZSL)-based framework capable of classifying and identifying both seen and unseen defect patterns. It consists of a visual feature extraction phase based on semi-supervised contrastive learning (CL), a clustering phase using distance-based initialization, and an alignment phase based on the Wasserstein-Distance-based Visual Structure Constraint method. Our framework further reduces the need for expensive annotated datasets previously required to train supervised learning-based models, thereby improving the practicality of the data-driven approach for wafer map pattern classification. Furthermore, our framework introduces an effective way of incorporating additional semantic information in wafer map pattern classification that traditionally relied on training models solely from wafer maps. To the best of the authors' knowledge, our work is the first work to adopt GZSL in wafer map pattern classification. [ABSTRACT FROM AUTHOR]

The significant gap between the assumed and actual degradation model will undoubtedly lead to a serious performance decrease for deep learning based image super-resolution (SR) methods. To solve this shortcoming, this paper presents an enhanced blind image super-resolution (EBSR) network based on unsupervised degradation representation learning. The proposed EBSR mainly is made up of two branching subnetworks. The first subnetwork is a residual-based degradation encoder, which is responsible to learn a high-dimensional abstract degradation representation vector from the input low-resolution (LR) image patches using contrast learning. Another branch is the degradation-aware fusion SR (DAFSR) network that completes the image SR task with the aid of the degradation representation vector learned by the encoder. To obtain an improved SR performance under various degradation settings, the degradation-aware fusion block (DAFB), the core block of DAFSR, has been elaborated, which embeds the degradation model information learned by the encoder into every layer of the network. Additionally, a shallow feature extraction block (SFEB) is constructed for DAFSR to extract global information from the LR image. The proposed EBSR can flexibly adapt to various degradation models. Extensive experimental results on synthetic datasets and real images show that the proposed EBSR is able to achieve a leading reconstruction performance over other blind SR algorithms. • A boosted degradation representation learning is proposed for blind image SR. • A new degradation encoder is designed using residual connection. • A degradation-aware fusion block is designed to obtain improved SR performance. • A shallow feature extraction block is adopted to extract global informative features. [ABSTRACT FROM AUTHOR]

The performance of object detection networks is often limited by the depth of the feature extraction network. Increasing network parameters may yield limited improvements in the detection system's performance. Additional careful designs of network details are necessary, but they can significantly increase training difficulty. This paper introduces a novel object detection method that utilizes generative adversarial training. Our approach focuses on minimizing the EM distance (Wasserstein distance) of the feature distribution as the primary training objective. We enhance the image features so that training of GAN (Generative Adversarial Networks) yields a feature distribution that exceeds that of the original dataset, obtaining a better feature extraction network. A new loss function is also added to the adversarial training process to ensure stable improvement of the detector. A comparative experiment conducted with the original CenterNet network on MS COCO (Microsoft Common Objects in COntext) 2017 reveals that the generative adversarial training method significantly improves the average precision for most of the examined backbone networks. Among the four backbone networks employed in the experiments, the mean improvement in network AP (Average Precision) values ranged from 0.3 to 0.9, demonstrating their success with minimal training efforts. Moreover, none of the four backbone networks experienced an increase in network parameters during inference. Experimental results indicate that the proposed architecture effectively enhances the network's feature extraction capability without compromising speed during inference. [ABSTRACT FROM AUTHOR]

This paper delves into enhancing underwater images to improve human observation of underwater scenes. To this end, a reinforcement learning-based human visual perception-driven image enhancement paradigm for underwater scenes is proposed, referred to as INSPIRATION. This paradigm is so named because it constructs rewards for guiding model training by incorporating metrics inspired by human visual perception, eliminating the need for reference images. It overcomes the limitations of deep models, which often result in poor performance in dynamic and diverse underwater scenes due to the scarcity of underwater images with enhancement references. Specifically, a residual-enhancement network, composed of residual modules and a channel-attention module, is utilized to extract features as a state. An extensible collection encompassing diverse image enhancement algorithms is utilized to provide an image enhancement algorithm as an action. A multi-non-reference human visual perception metric increment is utilized as a reward, and proximal policy optimization (PPO) is utilized to conduct reinforcement learning. This paradigm treats underwater images as sole inputs during both training and implementation, learning and organizing a sequence of image enhancement algorithms to explicitly achieve a step-wise image enhancement process, aligning the enhanced images with human visual perception. Extensive qualitative and quantitative evaluations conclusively demonstrate that our paradigm outperforms nine state-of-the-art underwater image enhancement methods in terms of visual quality and achieves better performance on five underwater image quality assessment measures across three underwater image datasets. Additionally, the potential advantages of our paradigm as a pre-processing step in other underwater computer vision applications are demonstrated. The code has been released publicly for reproducibility and evaluations at https://gitee.com/wanghaoupc/uie_inspiration. [ABSTRACT FROM AUTHOR]

Group activity recognition aims to recognize holistic activity in multi-person scene, necessitating consideration of the interactions between actors and their surroundings. It has various applications, such as public surveillance, video analysis. Nonetheless, existing works merely extract scene features as a supplementary component of activity features, failing to adequately explore the interplay between the scene and actors. To address this limitation, this paper proposes a Local–Global Context-Aware Graph Reasoning Model (LG-CAGR), which leverages and reasons through local and global context features to gain deeper insights into group activity within the scene. In particular, we present an innovative feature extraction strategy to harness local location features and global scene attributes, effectively complementing actor features by capturing spatial group topology and determining relative positions between actors. Subsequently, we delve into these features by devising a local–global group reasoning module that deduces pair-wise interactions between actors and scenes within Graph Convolutional Network, comprehensively elucidating correlations between overall scene and local individuals to construct group-level features. Multi-graphs are constructed considering actor's features, scene features as nodes, and interactions as edges. A self-attention graph pooling network is introduced to automatically integrate key actor features and form rich group-level features to recognize group activity. The results on Collective Activity Dataset, Collective Activity Extended Dataset, Volleyball Dataset and Public Life in Public Space dataset have reached 94.0%, 97.7%, 92.7% and 56.1%. Compared with existing methods using the same backbone, we exceeded 1%, 2.1%, 0.3%, and 14.9% respectively, affirming the superiority of the proposed method compared with state-of-the-art methods. • A novel progressive network structure for group activity recognition. • Context feature extraction strategy for the exploration of group activity. • Capturing temporal and spatial dynamics by reasoning local–global GCN. [ABSTRACT FROM AUTHOR]

Semantic segmentation is a well-studied topic and one of the most challenging tasks in computer vision applications, such as autonomous driving. Deep learning approaches based on convolutional neural networks (CNN) have demonstrated exceptional success on this task in recent years. Despite this success, existing approaches are plagued by higher-order inconsistencies between the ground truth images and the ones predicted by the segmentation model. This paper proposes a novel post-processing scheme based on adversarial learning to counter these inconsistencies. Such a scheme can be combined with a variety of existing CNN-based semantic segmentation networks to improve their segmentation performances. The proposed scheme is a Two-Stream Conditional Generative Adversarial Network (TScGAN), with one stream having initial semantic segmentation masks predicted by an existing CNN, while the other stream utilizes scene images to retain high-level information under a supervised residual network structure. In addition, TScGAN incorporates a novel dynamic weighting mechanism, which leads to significant and consistent gains in segmentation performance. Several comparative tests on public benchmark driving databases, including Cityscapes, Mapillary, and Berkeley DeepDrive100K, demonstrate the effectiveness of the proposed method when used with state-of-the-art CNN-based semantic segmentation models. Furthermore, the ablation experiment proved the structural rationality of our two-stream structure. The code for TScGAN could be found at https://github.com/epan-utbm/TScGAN-for-Improving-Semantic-Predictions • Explores various post-processing techniques for enhancing semantic segmentation. • Novel Two-Stream Conditional GAN enhances urban drive scene segmentation. • Enhances feature extraction by implementing dynamic weighting within streams. • Evaluates the model against existing semantic segmentation methodologies. • Tests on Cityscapes, Mapillary, BDD100K reveal major IoU gains with proposed model. [ABSTRACT FROM AUTHOR]

Owing to the development of deep learning, indoor personnel detection methods based on deep neural networks have been extensively investigated in recent years. However, more complex and deeper network structures may consume more computational resources, which seriously limits the deployment of large-scale deep neural networks on lightweight devices. In view of this, a lightweight anchor-free one-level feature indoor personnel detection method based on transformer (LAOF-IPDT) is proposed in this paper, which is deployed on two embedded devices and achieves good detection accuracy. In the feature extraction backbone, an enhanced cross-stage partial ghost convolution block with ghost convolution and channel shuffle is designed to extract shallow features. Additionally, to obtain more comprehensive global features in the high-level semantic structure, an embedded vision transformer cross-stage partial block is constructed by embedding a lightweight mobile-friendly vision transformer. For the path-aggregation neck, a lightweight feature pyramid network is proposed, which integrates multi-scale feature maps to obtain richer one-level feature representations. Subsequently, a dilated convolution group block is applied to expand the one-level feature receptive field and detection is accomplished using a one-level feature map. For the detection head, an anchor-free mechanism is applied to reduce the hyper-parameter interference of anchor boxes. Extensive experiments on four datasets indicated that the LAOF-IPDT outperforms other lightweight networks in terms of accuracy, speed, model parameters, and model size. For example, the frames per second of the LAOF-IPDT are 8.39 and 14.67 on CPU devices and Jetson Nano devices, respectively, and the mean average precision is 84.49%. [ABSTRACT FROM AUTHOR]

Remaining Useful Life (RUL) prediction is of great significance for maintaining the reliability and safety of industrial equipment. To address the challenges faced by existing methods in simultaneously extracting local and global degradation information from monitoring data. This paper proposes a Two-Stream Convolution Augmented Transformer (TACT) model based on L 2 regularization constraint. Specifically, we design the parallel multi-scale Convolution Neural Network (CNN) and Transformer module to combine the local modeling ability of CNN and the global modeling ability of Transformer to improve the overall architecture of RUL prediction model. Moreover, the two-stream network based on the parallel structure also realizes the synchronous extraction of different time steps and sensor features in the sequence. Then, in the process of model training, the prediction reliability constraint is fused, the delay prediction constraint term is introduced, and the L 2 regularization loss function is constructed. Finally, extensive experiments on the commercial modular aero-propulsion system simulation (C-MAPSS) show that our model provides competitive performance in terms of Root-Mean-Square Error (RMSE) and Score metrics. Compared to the state-of-the-art method based on Recurrent Neural Network (RNN) or CNN and its variants, Score is reduced by at least 2.71% and RMSE by at least 3.13%. Compared to the Transformer-based improved method, the Score is decreased by at least 4.54% and the RMSE is decreased by at least 2.78%. The effectiveness of the proposed method is demonstrated. • A hybrid end-to-end RUL prediction network based on a tandem combination of CNN and Transformer is proposed. • A two-stream structure is used to extract spatial and temporal features simultaneously. • The MSE loss function incorporating L 2 regularization constraint is proposed. [ABSTRACT FROM AUTHOR]

Efficient forecasting of ship order books holds immense significance in the maritime industry, enabling companies to optimize their operations, allocate resources effectively, and make informed decisions. However, volatile characteristics within historical order books pose challenges in achieving reliable, intelligent, and precise forecasts. This paper presents a novel ensemble deep random vector functional link (edRVFL) algorithm to anticipate future ship order book dynamics. The edRVFL leverages deep feature extraction and ensemble learning to enhance forecasting performance. To further elevate its capabilities, we introduce a discontinuous and parsimonious embedding strategy, which deviates from the conventional dense collection of continuous time steps used in vanilla edRVFL. This parsimonious embedding approach limits the model's complexity and boosts its generalization ability. We extensively evaluate the proposed method using ship order book data, and comparative studies demonstrate its superiority over alternative approaches. Our proposed edRVFL offers a promising solution for accurate and efficient ship order book forecasting, making it a valuable asset in the maritime industry's decision-making processes. The source codes utilized in this research are openly available on GitHub at the following link: https://github.com/crkkkaa/Ship-order-book-forecasting-by-an-ensemble-deep-parsimonious-random-vector-functional-link-network-. • A novel ensemble deep RVFL to anticipate ship order book dynamics is proposed. • We propose a parsimonious embedding algorithm to optimize temporal patterns. • The direct links ablation study is conducted for comparison. [ABSTRACT FROM AUTHOR]

The pivotal factor to regulate and enhance within an operating batch is the quality process, a challenging variable to monitor online. Soft sensors offer an immediate alternative for providing real-time insights into process quality, yet persisting issues include the imbalance between process and quality measurements, noisy measurements, and the intricate 3-dimensional dynamic batch data structure. This paper introduces the Robust Semi-Supervised Dual-Attention Latent Dynamic Conditional State-Space Model (RS2DA-LDCSSM) to address these challenges for within-batch quality prediction. Given the frequent absence of quality data due to measurement inconveniences, an imputation network is embedded within the RS2DA-LDCSSM to facilitate the prediction of future quality. To prevent information distortion during data unfolding, the Attentional Sequence-to-Sequence RNN Encoder-Decoder (AS2S-RNNED) is employed to process the 3-dimensional batch data. The proposed method integrates AS2S-RNNED with a probability state-space model to filter out noisy process data and stabilize the probability prediction of quality data, extracting spatial and temporal latent data from past process and quality data while minimizing the loss in multi-step prediction. This work represents a novel approach to dynamic nonlinear batch processes through probabilistic semi-supervised learning. The RS2DA-LDCSSM is adaptable to any batch process for within-batch quality prediction, as evidenced by numerical cases demonstrating its reliability, achieving an R2 value of 0.87, surpassing comparison methods. In an industrial penicillin fermentation batch process, the RS2DA-LDCSSM exhibits remarkable robustness with a high R2 index value of 0.99 in practical quality prediction scenarios, showcasing its efficacy. • Semi-supervised model training utilizes unlabeled process data. • Separate networks are crafted for process and quality feature extraction. • The state-space model with the encoder-decoder is used to extract latent features. • A multi-step-ahead prediction learning scheme yields favorable prediction results. [ABSTRACT FROM AUTHOR]

Group activity recognition aims to recognize holistic activity in multi-person scene, necessitating consideration of the interactions between actors and their surroundings. It has various applications, such as public surveillance, video analysis. Nonetheless, existing works merely extract scene features as a supplementary component of activity features, failing to adequately explore the interplay between the scene and actors. To address this limitation, this paper proposes a Local–Global Context-Aware Graph Reasoning Model (LG-CAGR), which leverages and reasons through local and global context features to gain deeper insights into group activity within the scene. In particular, we present an innovative feature extraction strategy to harness local location features and global scene attributes, effectively complementing actor features by capturing spatial group topology and determining relative positions between actors. Subsequently, we delve into these features by devising a local–global group reasoning module that deduces pair-wise interactions between actors and scenes within Graph Convolutional Network, comprehensively elucidating correlations between overall scene and local individuals to construct group-level features. Multi-graphs are constructed considering actor's features, scene features as nodes, and interactions as edges. A self-attention graph pooling network is introduced to automatically integrate key actor features and form rich group-level features to recognize group activity. The results on Collective Activity Dataset, Collective Activity Extended Dataset, Volleyball Dataset and Public Life in Public Space dataset have reached 94.0%, 97.7%, 92.7% and 56.1%. Compared with existing methods using the same backbone, we exceeded 1%, 2.1%, 0.3%, and 14.9% respectively, affirming the superiority of the proposed method compared with state-of-the-art methods. • A novel progressive network structure for group activity recognition. • Context feature extraction strategy for the exploration of group activity. • Capturing temporal and spatial dynamics by reasoning local–global GCN. [ABSTRACT FROM AUTHOR]

This paper delves into enhancing underwater images to improve human observation of underwater scenes. To this end, a reinforcement learning-based human visual perception-driven image enhancement paradigm for underwater scenes is proposed, referred to as INSPIRATION. This paradigm is so named because it constructs rewards for guiding model training by incorporating metrics inspired by human visual perception, eliminating the need for reference images. It overcomes the limitations of deep models, which often result in poor performance in dynamic and diverse underwater scenes due to the scarcity of underwater images with enhancement references. Specifically, a residual-enhancement network, composed of residual modules and a channel-attention module, is utilized to extract features as a state. An extensible collection encompassing diverse image enhancement algorithms is utilized to provide an image enhancement algorithm as an action. A multi-non-reference human visual perception metric increment is utilized as a reward, and proximal policy optimization (PPO) is utilized to conduct reinforcement learning. This paradigm treats underwater images as sole inputs during both training and implementation, learning and organizing a sequence of image enhancement algorithms to explicitly achieve a step-wise image enhancement process, aligning the enhanced images with human visual perception. Extensive qualitative and quantitative evaluations conclusively demonstrate that our paradigm outperforms nine state-of-the-art underwater image enhancement methods in terms of visual quality and achieves better performance on five underwater image quality assessment measures across three underwater image datasets. Additionally, the potential advantages of our paradigm as a pre-processing step in other underwater computer vision applications are demonstrated. The code has been released publicly for reproducibility and evaluations at https://gitee.com/wanghaoupc/uie_inspiration. [ABSTRACT FROM AUTHOR]

Remaining Useful Life (RUL) prediction is of great significance for maintaining the reliability and safety of industrial equipment. To address the challenges faced by existing methods in simultaneously extracting local and global degradation information from monitoring data. This paper proposes a Two-Stream Convolution Augmented Transformer (TACT) model based on L 2 regularization constraint. Specifically, we design the parallel multi-scale Convolution Neural Network (CNN) and Transformer module to combine the local modeling ability of CNN and the global modeling ability of Transformer to improve the overall architecture of RUL prediction model. Moreover, the two-stream network based on the parallel structure also realizes the synchronous extraction of different time steps and sensor features in the sequence. Then, in the process of model training, the prediction reliability constraint is fused, the delay prediction constraint term is introduced, and the L 2 regularization loss function is constructed. Finally, extensive experiments on the commercial modular aero-propulsion system simulation (C-MAPSS) show that our model provides competitive performance in terms of Root-Mean-Square Error (RMSE) and Score metrics. Compared to the state-of-the-art method based on Recurrent Neural Network (RNN) or CNN and its variants, Score is reduced by at least 2.71% and RMSE by at least 3.13%. Compared to the Transformer-based improved method, the Score is decreased by at least 4.54% and the RMSE is decreased by at least 2.78%. The effectiveness of the proposed method is demonstrated. • A hybrid end-to-end RUL prediction network based on a tandem combination of CNN and Transformer is proposed. • A two-stream structure is used to extract spatial and temporal features simultaneously. • The MSE loss function incorporating L 2 regularization constraint is proposed. [ABSTRACT FROM AUTHOR]

Efficient forecasting of ship order books holds immense significance in the maritime industry, enabling companies to optimize their operations, allocate resources effectively, and make informed decisions. However, volatile characteristics within historical order books pose challenges in achieving reliable, intelligent, and precise forecasts. This paper presents a novel ensemble deep random vector functional link (edRVFL) algorithm to anticipate future ship order book dynamics. The edRVFL leverages deep feature extraction and ensemble learning to enhance forecasting performance. To further elevate its capabilities, we introduce a discontinuous and parsimonious embedding strategy, which deviates from the conventional dense collection of continuous time steps used in vanilla edRVFL. This parsimonious embedding approach limits the model's complexity and boosts its generalization ability. We extensively evaluate the proposed method using ship order book data, and comparative studies demonstrate its superiority over alternative approaches. Our proposed edRVFL offers a promising solution for accurate and efficient ship order book forecasting, making it a valuable asset in the maritime industry's decision-making processes. The source codes utilized in this research are openly available on GitHub at the following link: https://github.com/crkkkaa/Ship-order-book-forecasting-by-an-ensemble-deep-parsimonious-random-vector-functional-link-network-. • A novel ensemble deep RVFL to anticipate ship order book dynamics is proposed. • We propose a parsimonious embedding algorithm to optimize temporal patterns. • The direct links ablation study is conducted for comparison. [ABSTRACT FROM AUTHOR]

The increasing popularity of mobile applications has catalyzed advancements in object tracking algorithms for mobile platforms. This paper introduces a lightweight object tracking algorithm, designed for real-time and accurate tracking on mobile devices. We propose a feature extraction network with a quantization-friendly architecture and a novel mask-based padding method to enhance tracking precision. By refining MobileNetV3's structure, we mitigate accuracy degradation during quantization and reduce computational demands. The proposed padding technique addresses the outlier generation issue common in traditional padding methods, thereby enhancing positional accuracy. Additionally, an extent adjustment module is incorporated to facilitate prompt error correction during suboptimal tracking. Comprehensive evaluations on OTB50, OTB100, VOT2018, UAV123, and LaSOT demonstrate the superiority of our method over several state-of-the-art trackers in both speed and accuracy. Notably, our feature extraction model is only 3.4 MB, running at 243fps on an RTX 3090 GPU, and can adapt to various sizes of video inputs. • A quantization-friendly structure is designed to reduce the network model size without compromising accuracy. • A mask-based padding method is introduced to better adapt to tracking tasks. • An extent adjustment module is proposed to detect the tracking quality and adjust the size of the search area. • The proposed method achieves excellent tracking accuracy and surpasses similar algorithms in tracking speed. [ABSTRACT FROM AUTHOR]

The pivotal factor to regulate and enhance within an operating batch is the quality process, a challenging variable to monitor online. Soft sensors offer an immediate alternative for providing real-time insights into process quality, yet persisting issues include the imbalance between process and quality measurements, noisy measurements, and the intricate 3-dimensional dynamic batch data structure. This paper introduces the Robust Semi-Supervised Dual-Attention Latent Dynamic Conditional State-Space Model (RS2DA-LDCSSM) to address these challenges for within-batch quality prediction. Given the frequent absence of quality data due to measurement inconveniences, an imputation network is embedded within the RS2DA-LDCSSM to facilitate the prediction of future quality. To prevent information distortion during data unfolding, the Attentional Sequence-to-Sequence RNN Encoder-Decoder (AS2S-RNNED) is employed to process the 3-dimensional batch data. The proposed method integrates AS2S-RNNED with a probability state-space model to filter out noisy process data and stabilize the probability prediction of quality data, extracting spatial and temporal latent data from past process and quality data while minimizing the loss in multi-step prediction. This work represents a novel approach to dynamic nonlinear batch processes through probabilistic semi-supervised learning. The RS2DA-LDCSSM is adaptable to any batch process for within-batch quality prediction, as evidenced by numerical cases demonstrating its reliability, achieving an R2 value of 0.87, surpassing comparison methods. In an industrial penicillin fermentation batch process, the RS2DA-LDCSSM exhibits remarkable robustness with a high R2 index value of 0.99 in practical quality prediction scenarios, showcasing its efficacy. • Semi-supervised model training utilizes unlabeled process data. • Separate networks are crafted for process and quality feature extraction. • The state-space model with the encoder-decoder is used to extract latent features. • A multi-step-ahead prediction learning scheme yields favorable prediction results. [ABSTRACT FROM AUTHOR]

Fault diagnosis of rotating machines is essential for the safe and efficient operation of maritime vessels. It prevents potential failures in rotating machines in maritime systems. Simultaneously developed faults severely damage the machines, leading to unprecedented incidents and higher maintenance costs. Data availability for developing multi-fault diagnosis solutions is limited, creating maintainability problems. The unavailability of data also causes further obstacles in the accurate diagnosis of machines. The efficiency of maritime machinery can be improved with intelligent diagnosis by advanced machine learning algorithms. This paper proposes a methodology to diagnose multi-faults with balanced and imbalanced acoustic, vibration and current signals datasets. After acquiring the data from an experimental setup under various speeds, a time-frequency-based Stepping Frame Synchrosqueezed Fourier Transform (SFSFT) is deployed in each segmented signal prior to feature extraction. The extracted features are used for training the Radial Basis Function Neural Network (RBFNN). Different balanced and imbalanced datasets are created to check the multi-fault classification ability of the proposed methodology. The results show that SFSFT-RBFNN is a generalised methodology, which performed tremendously well with a maximum classification accuracy of 100 % for balanced and imbalanced datasets in all three modality datasets. • Synchrosqueezed Fourier Transform is improved with stepping frame extraction method for multi-fault diagnosis. • Minute details of multi-fault signatures are extracted using SFSFT, and RBFNN is used for fault detection. • Performance of the proposed method is tested with balanced and different imbalanced datasets. • Methodology validation is carried out with acoustic, vibration and current datasets. [ABSTRACT FROM AUTHOR]

Inter-turn Short Circuit (ITSC) faults in Permanent Magnet Synchronous Motor (PMSM) have gained significant attention due to the growing demand for enhanced reliability and safety in actuation systems. This paper presents a adaptive fault diagnosis approach specifically designed for ITSC faults in PMSM used in oil-drilling applications, where sparse actual data and varying speed conditions pose considerable sparse training data and ITSC feature extraction challenges respectively. To address these challenges, we first construct a physical fault model of PMSM-ITSC and establish a simulation experiment platform to replicate downhole environments with high temperatures and rapidly changing PMSM speeds, ensuring a reliable data source for analysis. Subsequently, we propose a novel adaptive peak-to-peak self-finding method (APPS) that leverages frequency-domain prior knowledge to adaptively extract ITSC fault characteristics, even amidst drastic changes in PMSM speed. Furthermore, we introduce the time-sequence efficient moving window self-attention network (EMWSAN) data model for inferring the stator phase winding state, incorporating Half-sandwich and Cascaded windows group attention operations. This approach significantly reduces computational complexity and network parameters compared to traditional self-attention mechanisms. To expedite the fitting process, we apply transfer learning (TL) theory, transferring knowledge from the physical knowledge to the data model, enabling EMWSAN to be trained more efficiently. As a result, our proposed ITSC fault diagnosis scheme achieves an impressive 96.72% classification accuracy, achieving existing state-of-the-art methods. [ABSTRACT FROM AUTHOR]

With the rise of the digital comic industry, pirated comic works have also emerged. Textual plagiarism, mainly in terms of character language, is gradually increasing, mainly in terms of plot and character description. In this paper, we propose a novel copyright recognition framework based on the language features of comic characters to identify and certify the textual infringement of digital comic works. The framework consists of internal and external copyright recognition methods. The internal method analyzes the writing styles of a comic and extracts text style features to detect abnormal chapters and determines whether it is a suspicious comic. The external method matches the suspicious comic text and other original comic texts with semantic and document features. Comic copyright recognition is achieved based on the similarities between comics. We also collect comic works from different domains and construct an original comic corpus and a pirated comic corpus in terms of chapters. The experimental results show that the proposed framework can detect abnormal chapters and plagiarized documents in comic text. After integrating several types of plagiarisms, the recognition accuracy of the proposed method is about 98%, higher than those of the state-of-the-art models. [Display omitted] • A novel two-stage framework of comic plagiarism detection is proposed. • Comic corpus are constructed for internal and external copyright recognition. • Comic style feature is proposed by lexical, structural and syntactic features. • The proposed method is robust against various types of plagiarisms. [ABSTRACT FROM AUTHOR]

Implementing key engineering solutions to optimise the operation of energy industries requires daily electricity demand forecasting and including uncertainty, to promote markets insight analysis as part of their strategic planning, regulating and supplying electricity to consumers. This paper proposes hybrid artificial intelligence models combining convolutional neural networks (CNN) as a feature extraction algorithm with extreme learning machines (ELM) as a framework to predict electricity demand with confidence intervals generated by Kernel Density Estimation (KDE) approaches. In order to develop CELM-KDE model, time-lagged series of daily electricity demand with local climate variables based on the air temperature, atmospheric vapour pressure, evaporation, solar radiation, humidity and sea level pressures are used to train the proposed CELM-KDE hybrid model. In order to fully evaluate the newly developed model from a point-based, as well as a probabilistic prediction strategy, the observed and predicted electricity demand as well as the probability distribution of errors are analysed using KDE method that operates without prior data distribution assumptions. Based on observed and predicted electricity demand and the relevant probabilistic confidence intervals generated by the CELM-KDE model, the final results show that the proposed method attains significantly better probability interval predictions than traditionally-used point-based models. The proposed CELM-KDE model is demonstrated to be highly effective in providing a comprehensive coverage of predicted errors, as well as providing greater insights into the average bandwidth and detailed predicted electricity demand in the testing stage. The results also indicate that the proposed hybrid model is a reliable decision support tool to develop engineering solutions in area of energy modelling, monitoring and forecasting, which could potentially be useful to the industry policymakers. We show that the point-and probabilistic-based electricity demand predictive models can be employed as an effective tool to improve accuracy of forecasting and provision of insights for national electricity markets and key energy industry stakeholder application tools. [Display omitted] • A hybrid deep learning model for daily electricity demand prediction is proposed. • A Kernel Density Estimation is applied to derive probabilistic prediction intervals. • The algorithm quantifies uncertainties in point-based electricity demand predictions. • The proposed model outperforms deep learning models using statistical uncertainty indicators. [ABSTRACT FROM AUTHOR]

In the era of We-media, articles are written by independently individuals that are not officially registered with the authorities. Propaganda hidden in the We-media articles have the potential to polarize public opinion and influence the mindset of the target audience. Currently, graph neural networks (GNNs) have been remarkably successful in Natural Language Processing (NLP). However, there are still some challenges to apply existing GNN-based for propaganda detection due to the limitation of extracting diverse word dependencies and capturing non-consecutive and long-range context. In this paper, we have proposed a Graph-based Hierarchical Feature Integration Network (abbreviated as G-HFIN) for Propaganda Detection. Specifically, semantic, syntactic, and sequential features are extracted to construct three heterogeneous graphs. Then, the Residual-connected Dual-layer Coarsening and Refining procedures (abbreviated as RDCR) are proposed to promote information interactions between distant nodes that are not directly connected, preserving both local and global node information during the intra-graph information propagation. Subsequently, an Attention-based Three-channel Feature Integration (abbreviated as ATFI) is proposed to harmonize sequence, semantic and syntactic information from three channels during the inter-graph information enhancement. Intra-graph and Inter-graph Joint Information Propagation is to implement homogeneous and heterogeneous information interaction respectively. Finally, these news representations are pooled and fed into the propaganda detection classifier. The experiments on three public datasets demonstrate that our model has outperformed state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Few-Shot Classification (FSC) is an innovative application in machine learning. FSC consists of two main components: (1) Pre-training, where a feature extraction model (FEM) is trained using base data, and (2) Meta-testing, where the FEM is utilized to extract features from novel data (with categories different from the base data) for classification. Implementing FSC presents several challenges. For example, due to the cross-domain limitation (base data → novel data), the FEM may generate inappropriate features for new classes, leading to a Sample-Feature-Mismatch problem, and the corresponding feature is dubbed as Original-Shift-Feature (OSF). This paper proposes a generative method to construct Multi-View Discriminative Semantic Feature (MVDSF) to address the fundamental problem from the perspective of improving the discriminability of OSF. Typically, a linear projection is designed to transform OSF into a semantic space, generating Discriminative-Semantic-Features (DSF). By incorporating a reconstructive representation that is not solely reliant on the FEM, the influence of the Sample-Feature-Mismatch problem is reduced. Furthermore, considering that descriptions based on a single DSF tend to be one-sided, an attention mechanism is devised to fuse multi-view DSF, thereby improving the robustness of the proposed method. The effectiveness of MVDSF is evaluated on five benchmark few-shot learning datasets, where it achieves outstanding performance. This evaluation demonstrates the efficiency and performance of the proposed approach. [ABSTRACT FROM AUTHOR]

Most of the current tunnel defect detection methods do not perform repeated inspection of the defects taken at different times at the same location, which is a major deficiency for detecting the development trend of the defects. To solve the problem of repeated inspection of tunnel lining defects, this paper proposes a new method based on the combination of enhanced visual odometer and a Sequence-Siamese network(Seq-Siamese-Net) to identify and locate tunnel lining defects. We first detect the defects in the tunnel lining surface video by You Only Look Once network, then estimate the general location of the defects by using the proposed enhanced visual odometer, and finally obtain the most matching defects in the vicinity of the defects by Seq-Siamese-Net to achieve the repeated inspection of the tunnel lining surface defects. Experiments show that our proposed enhanced visual odometer method outperforms the original visual odometer method in terms of localization accuracy, while the feature extracted form Seq-Siamese-Network is more effective than the average-pooled feature in the tunnel defect image sequence matching task. Furthermore, experiments in a real tunnel demonstrate the practical value of the proposed method. • A network-based method for repeated detection of tunnel lining defects is proposed. • Average error of localization method reached about 1% at real tunnel experiment. • Sequence-Siamese-Network surpasses averaged feature in image sequence matching task. [ABSTRACT FROM AUTHOR]

Photovoltaic defect detection is an essential aspect of research on building-distributed photovoltaic systems. Existing photovoltaic defect detection models based on deep learning, such as YOLOv5 and YOLOv8, have significantly improved the accuracy of photovoltaic defect detection. However, these models are too large, and their feature extraction ability is insufficient, leading to low detection efficiency and inability to cope with the continuous evolution of defects. Therefore, this study proposes an accurate and lightweight YOLOv8 (You Only Look Once v8) GD algorithm. The algorithm is an improved version of YOLOv8, wherein DW-Conv (DepthWise-Conv) is applied to the YOLOv8 backbone network. Moreover, convolution is replaced with the GSConv (Group-shuffle Conv) and the BiFPN (bidirectional feature pyramid network) structure is added to the architecture. Several electroluminescent photovoltaic defect datasets are used to verify the effectiveness of the proposed method. The final experimental results show that the map@0.5 and map@0.5 ∼ 0.95 of YOLOv8-GD are 92.8% and 63.1%, respectively, which are 4.2% and 5.7% higher than those of the original algorithm, respectively, and the model volume is reduced by 16.7%. Thus, the proposed algorithm shows considerable potential in the field of photovoltaic defect detection. • This paper proposes an improved method based on YOLOv8 for building distributed PV defects, named YOLOv8-GD. • YOLOv8-GD includes improved methods such as GSConv, BiFPN and DW-Conv, which shows better accuracy and speed. • YOLOv8-GD can be used for defect detection and health monitoring in building distributed PV. [ABSTRACT FROM AUTHOR]

3D face reconstruction and dense alignment play a key role in digital humans. Learning-based approaches have been developed by employing cascaded convolutional neural networks for feature extraction and spatial-semantic relationship construction. However, these networks often struggle with accurately capturing the complex relationships among various facial components and attributes, particularly in unconstrained environments. Furthermore, the mappings modeled by convolutional weights are at low levels, which are usually implicit and local, and lack global discrimination. In this paper, we propose a global and instance-level perceived relationship matrices-based network (PRMNet) to recover high-fidelity 3D faces and perform accurate dense alignment in unconstrained environments. Specifically, a Key Information Extraction Module (KIEM) extracts crucial features from global and local feature banks. This reduces reasoning costs while maintaining high perceptual quality. Learnable global and instance-level perceived feature relationship matrices are then integrated into the Feature Relationship Reasoning Module (FRRM). This calibration combines key information from both the global macroscopic and sample-specific microscopic views, allowing accurate construction of spatial-semantic relationships to harvest both global discrimination and local relevance. Finally, we introduce a Spatial Reasoning and Guidance Module (SRGM), designed to recalibrate the joint weight responses of feature extraction and its enhancement paths using various attention mechanisms, thereby further enhancing global discrimination ability. Extensive quantitative and qualitative experiments on the benchmark datasets show that our PRMNet outperforms the state-of-the-art. Codes and all resources will be publicly available at https://github.com/Ray-tju/PRMNet. [ABSTRACT FROM AUTHOR]

Multi-view stereo (MVS) reconstruction is a key task of image-based 3D reconstruction, and deep learning-based methods can achieve better results than traditional algorithms. However, most of the current deep learning-based MVS methods use convolutional neural networks (CNNs) to extract image features, which cannot achieve the aggregation of long-distance context information and capture robust global information. In addition, in the process of fusing depth maps into point clouds, the confidence filters will filter out the depth values with low confidence in weak texture areas. These problems will lead to the low completeness of 3D reconstruction of weak texture and texture-less areas. To address the above problems, this paper proposes SA-MVSNet based on the PatchmatchNet with a self-attentive mechanism. First, we design a coarse-to-fine network framework to advance depth map estimation. In the feature extraction network, a module with a pyramid structure based on Swin Transformer Block is used to replace the original Feature Pyramid Network (FPN), and the self-correlation between weak texture areas is enhanced by applying a global self-attention mechanism. Then, we also propose a self-attention-based adaptive propagation module (SA-AP), which applies a self-attention calculation within depth value propagation window to obtain the relative weight values of current pixel and others, and then adaptively samples the depth values of neighbors on the same surface for propagation. Experiments show that SA-MVSNet has significantly improved the completeness of 3D reconstruction for the images with weak texture on DTU (provided by Danish Technical University), BlendedMVS, and Tanks and Temple datasets. Our code is available at https://github.com/miaowang525/SA-MVSNet. • SA-MVSNet proposed a multi-scale feature extraction module based on Swin Transformer. • A global self-attention mechanism is applied to enhance the self-correlation between weak textured areas. • SA-MVSNet proposed a self-attention-based adaptive propagation module (SA-AP). • Apply window self-attention mechanism to enhance the confidence of depth value. • SA-MVSNet has significantly improved the completeness of 3D reconstruction on three datasets. [ABSTRACT FROM AUTHOR]

Currently, the fusion of surface Electromyography (EMG) and deep learning is gradually showing immense potential in the research of Lower Limb Motion Intention Recognition (LLMIR). Nevertheless, most deep learning algorithms have poor interpretability without special design or the help of other post-hoc analysis tools, as well as unsatisfactory performance in cross-subject prediction. Hence, this paper presents a novel Interpretable Dual-Branch EMG Network (IDB-EMGNet), in which one branch is dedicated to lower limb motion recognition, and the other is able to predict knee joint angles in advance. The shallow feature extraction module of IDB-EMGNet is constructed using an ante-hoc interpretable SincNet technique, which enables the detection of the spectral range of EMG used for the LLMIR task. An improved bottleneck block with shuffle attention is designed for deep feature extraction, which enhances model performance with only a little increase in complexity. The performance of IDB-EMGNet in both intra-subject and inter-subject scenarios is investigated, where the latter integrates the transfer learning technique. Specifically, by conducting model pre-training on source-domain subjects and transferring the learned knowledge to target-domain subjects, satisfactory performance can be achieved even with less computing resource. Experimental results on two publicly available datasets indicate that the proposed approach exhibits superior applicability to both normal subjects and knee-pathology patients, showing a promising prospect in the controller design of human-robot collaborative exoskeletons. • A novel IDB-EMGNet is proposed for simultaneous discrete and continuous lower limb motion intention recognition. • Interpretability of IDB-EMGNet is enhanced via an ante-hoc SincConv technique. • An IB-Neck block is designed to improve model prediction performance. • Transfer learning technique is utilized to improve model generalization in inter-subject prediction. • The proposed approach performs well for both normal subjects and knee-pathology patients. [ABSTRACT FROM AUTHOR]

The aim of this paper is to investigate emotion recognition using a multimodal approach that exploits convolutional neural networks (CNNs) with multiple input. Multimodal approaches allow different modalities to cooperate in order to achieve generally better performances because different features are extracted from different pieces of information. In this work, the facial frames, the optical flow computed from consecutive facial frames, and the Mel Spectrograms (from the word melody) are extracted from videos and combined together in different ways to understand which modality combination works better. Several experiments are run on the models by first considering one modality at a time so that good accuracy results are found on each modality. Afterward, the models are concatenated to create a final model that allows multiple inputs. For the experiments the datasets used are BAUM-1 ((Bahçeşehir University Multimodal Affective Database - 1) and RAVDESS (Ryerson Audio–Visual Database of Emotional Speech and Song), which both collect two distinguished sets of videos based on the different intensity of the expression, that is acted/strong or spontaneous/normal, providing the representations of the following emotional states that will be taken into consideration: angry, disgust, fearful, happy and sad. The performances of the proposed models are shown through accuracy results and some confusion matrices, demonstrating better accuracy than the compared proposals in the literature. The best accuracy achieved on BAUM-1 dataset is about 95%, while on RAVDESS it is about 95.5%. [Display omitted] • Emotion recognition through multimodal architectures. • Comparison of early to late fusion of 1-input models. • Synchronization of video and audio channels. [ABSTRACT FROM AUTHOR]

In this paper, we propose a lightweight approach for industrial defect detection that is modeled based on anomaly detection using point pattern data. Most recent works use global features for feature extraction to summarize image content. However, global features are not robust against lighting and viewpoint changes and do not describe images' geometrical information which, thus, cannot be fully utilized in manufacturing industry. To the best of our knowledge, we are the first to propose using transfer learning of local/point pattern features to overcome these limitations and capture geometrical information of the interested image regions. We model these local/point pattern features as a random finite set (RFS). In addition, we propose RFS energy, in contrast to RFS likelihood as an anomaly score. The similarity distribution of point pattern features of the normal sample has been modeled as a multivariate Gaussian. Parameters learning of the proposed RFS energy does not require any heavy computation. We evaluate the proposed approach on the MVTec AD dataset, a multi-object defect detection dataset. Experimental results show the outstanding performance of our proposed approach compared to the state-of-the-art methods, and the proposed RFS energy outperforms the state-of-the-art in the few shot learning settings. Also, we evaluated the proposed approach on needle dataset, a real life application, to detect defective needle samples given three different views. The results show superior performance compared to DiffeNet. [Display omitted] • A lightweight approach for defect detection based on anomaly detection is proposed. • Transfer learning of deep local features is used instead of global features. • We model these local/point pattern features as a random finite set (RFS). • We propose RFS energy, in contrast to RFS likelihood as an anomaly score. • Results show the outstanding performance of our proposed approach compared to the state-of-the-art methods in few-shot learning. [ABSTRACT FROM AUTHOR]

Induction motors are widely used in various industrial applications due to their simplicity, robustness, and high efficiency. In recent years, deep learning approaches have shown great potential for detecting and diagnosing faults in induction motors. However, it was observed that the convolutional neural network (CNN) might miss the crucial long-range temporal dependencies of the significant features. At the same time, the recursive neural network (RNN) may face issues such as vanishing gradient and less computational efficiency if the data is too long with complex patterns. This paper proposes a novel deep-learning model that combines a one-dimensional CNN and an RNN into two independent pipelines. The spatial, temporal, long-term, and short-term features can be extracted from the raw input signal. A multi-head mechanism, a state-of-the-art deep learning technique in natural language processing (NLP), enables the model to focus more on relevant features and prevent overfitting. Specifically, our method simplifies the conventional fault diagnosis workflow while maintaining competitive accuracy compared to several state-of-the-art methods. By leveraging the ability to detect faults directly from raw, unprocessed electrical signals, our approach offers a practical, cost-effective solution for real-time fault detection in industrial applications. This innovation has the potential to significantly reduce downtime and the associated operational costs, thereby providing immediate practical value in various industrial settings. [ABSTRACT FROM AUTHOR]

Intelligent fault diagnosis plays an important role in maintaining the safe and reliable operation of rotating machinery. However, the data collected in real engineering scenarios may be severely insufficient, which presents challenges to the intelligent fault diagnosis methods. To address this problem, this paper introduces a metric-based meta learning approach for gear fault diagnosis under zero shot conditions. Firstly, a gear-rotor dynamics model is established to simulate the vibration signals under different fault conditions. And the signals are converted into energy maps through wavelet transformation to provide frequency domain fault features. Secondly, a deep convolutional network is employed as the feature extraction module to construct the prototype representations by calculating the average embedding within each fault class. Then, the distances between the actual signals collected from the gear test rig and the class prototypes are computed. Finally, the softmax is applied to convert these distances into probability distributions for outputting the predicted fault classes. Furthermore, label smoothing technology is introduced to mitigate the probability distribution differences between simulated signals and real signals. The experimental results demonstrate that the average diagnostic accuracy of the proposed model reaches 98.9%, which is better than other models. [ABSTRACT FROM AUTHOR]

Immunohistochemical (IHC) detection is of critical importance in the pathological diagnosis of lung cancer. Interleukin-24 (IL-24) is a significant predictive and prognostic marker in IHC detection, which can help to characterize the tumor and predict the clinical course. To determine the score of the IL-24 expression, pathologists will assess the positive area and staining intensity through a microscope and then use a semi-quantitative assessment method to assign a score for each IHC image of lung cancer. However, this process is a time-consuming, imprecise, and subjective process, which can result in inter- and intra-observer discrepancies. Meanwhile, The performance of computer-aided diagnosis (CAD) systems for IHC scoring depends on the quality of manually extracted features. For example, many advanced methods require precise extraction of features of cell nuclei and membranes. However, the complex background and overlapping cells in lung cancer IHC images present significant challenges for segmentation, which can affect the accuracy of the final system's predicted scoring results. In this paper, an automatic Darknet-based IHC scoring system for IL-24 in lung cancer is proposed. Firstly, the original IHC images of lung cancer are blocked and the features are concentrated by a "block attention mechanism, which can reduce the computational burden of the analysis of millions of pixels in IHC images. The blocked images are then inputted into a Darknet-based scoring network, which incorporates a novel feature extraction backbone and loss function to obtain the final scores. To the best of our knowledge, this is the first end-to-end system that directly outputs a clinical score using lung cancer IL-24 IHC images as input. We have constructed a dataset of 5000 manually annotated IL-24 IHC images of lung cancer obtained from the Institute of Life Science and Bioengineering at Beijing Jiaotong University. We will present experimental results to demonstrate the feasibility of our proposed method, which can greatly assist with the clinical diagnosis and treatment of lung cancer. • This is the first work that attempts to end-to-end scoring of IL-24 immunohistochemistry for the detection of lung cancer. • An automatic Darknet-based network is proposed for immunohistochemical score of IL-24 in lung cancer. • A "block attention" mechanismas is proposed as preprocessing operation to segment the region of interest. [ABSTRACT FROM AUTHOR]

Simultaneously extracting junctions and their corresponding line segments from images presents a promising approach to structural environment cognition. However, conventional methods employ square convolution for line feature extraction, resulting in the exclusion of long-range dependencies and the generation of suboptimal wireframe predictions. In this paper, we introduce an efficient and concise parsing method named Structural Asymmetric Convolution-based Wireframe Parser (SACWP). Taking advantage of the inherent similarities between structural asymmetric convolution and the predominant distribution of line segments in man-made environments, we propose a Structural Asymmetric Convolution module (SAC) that captures long-range contextual features while efficiently filtering out irrelevant information from neighboring pixels. Additionally, we introduce a feature aggregation module based on dilated convolution (DCFA) to seamlessly integrate contextual information from multiple receptive fields. We thoroughly evaluate our approach on the Wireframe and YorkUrban datasets, achieving preferable results of 69.3% and 29.7% msAP respectively. On the other hand, the promising results adequately demonstrate the effectiveness of SACWP to Wireframe Parsing task. [ABSTRACT FROM AUTHOR]