Search

The reliability of bolt connections significantly impacts the operational state and lifespan of industrial equipment. Vision-based noncontact methods exhibit high efficiency in bolt loosening detection. However, limited image features hinder measurement accuracy. To improve bolt loosening detection performance, this paper proposes a novel deep learning backbone, the high-resolution cross-scale transformer, to extract high precision keypoints for bolt three-dimensional model construction. Simultaneously, a monocular vision measurement model is established to get the bolt exposed length and evaluate the connection loosening state. The proposed backbone hybridizes the advantages of high-resolution architecture and transformer, realizing global information aggregation and fine-grained image details. A simplified module, dual-scale multi-head self-attention, is designed to reduce the computational redundancy caused by the implementation of high-resolution multi-branch architecture. In the experiment section, the high-resolution cross-scale transformer outperforms other keypoint detection baselines, achieving the top one performance with 91.6 average precision and 84.9 average recall. The monocular vision measurement model realizes a 0.053 mm error with a 0.028 mm standard deviation, satisfying the industrial implementation requirement. Additionally, the model is tested on different industrial situations and an additional outside dataset, indicating the model's robustness and actual environment adaptability. • A monocular vision measurement method for bolt loosening detection is proposed. • It is the first attempt to introduce the transformer mechanism in bolt keypoint detection. • A new keypoint detection backbone is proposed in this paper for bolt feature extraction. • The 3D exposed length of bolts is calculated with a monocular vision system. • The bolt loosening detection method is validated. [ABSTRACT FROM AUTHOR]

Accurate detection of pavement markings at the pixel level is crucial for enhancing traffic safety. The majority of current advanced deep-learning networks predominantly focus on localized features, neglecting the global context of pavement image. Such networks often result in discontinuous segmentation outcomes and suboptimal recovery of local details. In this paper, a robust model named C-Transformer is proposed to provide an effective solution to this challenge. The contributions of this paper primarily involve two aspects. Firstly, the proposed C-Transformer is designed to succinctly integrate convolution operations and self-attention, facilitating a comprehensive understanding of essential features. Secondly, an efficient Feed-Forward Network called Inverse Residual Feed-Forward Network is also proposed in this paper and deployed in C-Transformer to improve latent representations. Experimental results demonstrate that, compared to other state-of-the-art networks, the proposed C-Transformer achieves a performance enhancement of 0.93% in F-measure and a 1.64% improvement in Intersection-Over-Union. In particular, the robustness and effectiveness of the C-Transformer in accurate pavement marking detection are proved through field test results. This paper illustrates the feasibility of employing a hybrid Convolutional neural network-Transformer-based network for automatic robust pavement marking detection under noisy conditions. [ABSTRACT FROM AUTHOR]

Emotion recognition is an aspect of human-computer interaction, affective computing, and social robotics. Conventional unimodal approaches for emotion recognition, depending on single data sources such as facial expressions or speech signals often fall short in capturing the complexity and context-dependent nature of emotions. Multimodal Emotion Recognition (MER), which integrates information from multiple modalities, has emerged as a promising solution to overcome these limitations. In recent years, Transformers-based approaches have gathered significant attention in the fields of natural language processing and computer vision, highlighting their ability to capture long-range dependencies and semantic representations. These models have rapidly achieved the MER state-of-the-art. However, current survey papers that cover MER lack a specific focus on Transformer-based techniques. To bridge this research gap, this review paper provides a comprehensive investigation of Transformers-based approaches for MER. It explores various Transformer architectures and proposes several scenarios for using Transformers at different stages of MER process. In addition, it examines datasets suitable for MER, discusses fusion mechanisms, and introduces novel taxonomies in both MER and Transformer technologies. The review also addresses challenges and future research directions. Through this review, we aim to provide researchers with an inclusive understanding of the current state-of-the-art in Transformers-based approaches for MER, paving the way for further advancements in this rapidly developing field. • First specialized survey in transformer-based Multimodal Emotion Recognition. • Organized taxonomy of fusion techniques based on discerning criteria. • Transformers taxonomy categorized by structural and operational distinctions. • Diverse scenarios of applying transformers at different stages of MER process. • Meticulous analysis identifies trends and challenges across multimodal datasets. [ABSTRACT FROM AUTHOR]

In visual tracking, the Transformer architecture is widely used because it can capture the global dependencies of sequence data without inductive bias. However, the attention mechanism of Transformer will bring ultra-high computational complexity and space occupancy, so that the tracking task cannot meet the real-time requirements. In this paper, we explore a sparsity region-aware attention mechanism. The sparse attention mechanism retains the regions with semantic relevance, and performs fine-grained attention calculation in this region. In the region-aware attention mechanism, a DropKey technique is introduced to reduce model over-fitting and improve the generalization ability of the model. Using region-aware attention as the basic building block, we design a dynamic region-aware Transformer backbone for visual tracking. This backbone network can effectively reduce the computational complexity while exploring global context dependencies. Based on the region-aware Transformer backbone network, this paper proposes a dynamic region-aware Transformer backbone visual tracking algorithm, which uses an optimization based model predictor to fully fuse object appearance and background information, so as to achieve more robust object tracking. The proposed tracker is trained in an end-to-end manner and experimentally evaluated on eight tracking benchmarks. Experimental results show that the algorithm has good tracking performance, especially in the application of unmanned aerial vehicle (UAV) tracking, our proposed tracker achieves an area under curve (AUC) score of 66.5% on the UAV123 dataset. Code is available at https://github.com/YSGFF/RTDiMP. [ABSTRACT FROM AUTHOR]

In recent years, transformer-based models have emerged as powerful tools for natural language processing tasks, demonstrating remarkable performance in several domains. However, they still present significant limitations. These shortcomings become more noticeable when dealing with highly specific and complex concepts, particularly within the scientific domain. For example, transformer models have particular difficulties when processing scientific articles due to the domain-specific terminologies and sophisticated ideas often encountered in scientific literature. To overcome these challenges and further enhance the effectiveness of transformers in specific fields, researchers have turned their attention to the concept of knowledge injection. Knowledge injection is the process of incorporating outside knowledge into transformer models to improve their performance on certain tasks. In this paper, we present a comprehensive study of knowledge injection strategies for transformers within the scientific domain. Specifically, we provide a detailed overview and comparative assessment of four primary methodologies, evaluating their efficacy in the task of classifying scientific articles. For this purpose, we constructed a new benchmark including both 24K labelled papers and a knowledge graph of 9.2K triples describing pertinent research topics. We also developed a full codebase to easily re-implement all knowledge injection strategies in different domains. A formal evaluation indicates that the majority of the proposed knowledge injection methodologies significantly outperform the baseline established by Bidirectional Encoder Representations from Transformers. [ABSTRACT FROM AUTHOR]

Emotion recognition is an aspect of human-computer interaction, affective computing, and social robotics. Conventional unimodal approaches for emotion recognition, depending on single data sources such as facial expressions or speech signals often fall short in capturing the complexity and context-dependent nature of emotions. Multimodal Emotion Recognition (MER), which integrates information from multiple modalities, has emerged as a promising solution to overcome these limitations. In recent years, Transformers-based approaches have gathered significant attention in the fields of natural language processing and computer vision, highlighting their ability to capture long-range dependencies and semantic representations. These models have rapidly achieved the MER state-of-the-art. However, current survey papers that cover MER lack a specific focus on Transformer-based techniques. To bridge this research gap, this review paper provides a comprehensive investigation of Transformers-based approaches for MER. It explores various Transformer architectures and proposes several scenarios for using Transformers at different stages of MER process. In addition, it examines datasets suitable for MER, discusses fusion mechanisms, and introduces novel taxonomies in both MER and Transformer technologies. The review also addresses challenges and future research directions. Through this review, we aim to provide researchers with an inclusive understanding of the current state-of-the-art in Transformers-based approaches for MER, paving the way for further advancements in this rapidly developing field. • First specialized survey in transformer-based Multimodal Emotion Recognition. • Organized taxonomy of fusion techniques based on discerning criteria. • Transformers taxonomy categorized by structural and operational distinctions. • Diverse scenarios of applying transformers at different stages of MER process. • Meticulous analysis identifies trends and challenges across multimodal datasets. [ABSTRACT FROM AUTHOR]

In visual tracking, the Transformer architecture is widely used because it can capture the global dependencies of sequence data without inductive bias. However, the attention mechanism of Transformer will bring ultra-high computational complexity and space occupancy, so that the tracking task cannot meet the real-time requirements. In this paper, we explore a sparsity region-aware attention mechanism. The sparse attention mechanism retains the regions with semantic relevance, and performs fine-grained attention calculation in this region. In the region-aware attention mechanism, a DropKey technique is introduced to reduce model over-fitting and improve the generalization ability of the model. Using region-aware attention as the basic building block, we design a dynamic region-aware Transformer backbone for visual tracking. This backbone network can effectively reduce the computational complexity while exploring global context dependencies. Based on the region-aware Transformer backbone network, this paper proposes a dynamic region-aware Transformer backbone visual tracking algorithm, which uses an optimization based model predictor to fully fuse object appearance and background information, so as to achieve more robust object tracking. The proposed tracker is trained in an end-to-end manner and experimentally evaluated on eight tracking benchmarks. Experimental results show that the algorithm has good tracking performance, especially in the application of unmanned aerial vehicle (UAV) tracking, our proposed tracker achieves an area under curve (AUC) score of 66.5% on the UAV123 dataset. Code is available at https://github.com/YSGFF/RTDiMP. [ABSTRACT FROM AUTHOR]

In recent years, transformer-based models have emerged as powerful tools for natural language processing tasks, demonstrating remarkable performance in several domains. However, they still present significant limitations. These shortcomings become more noticeable when dealing with highly specific and complex concepts, particularly within the scientific domain. For example, transformer models have particular difficulties when processing scientific articles due to the domain-specific terminologies and sophisticated ideas often encountered in scientific literature. To overcome these challenges and further enhance the effectiveness of transformers in specific fields, researchers have turned their attention to the concept of knowledge injection. Knowledge injection is the process of incorporating outside knowledge into transformer models to improve their performance on certain tasks. In this paper, we present a comprehensive study of knowledge injection strategies for transformers within the scientific domain. Specifically, we provide a detailed overview and comparative assessment of four primary methodologies, evaluating their efficacy in the task of classifying scientific articles. For this purpose, we constructed a new benchmark including both 24K labelled papers and a knowledge graph of 9.2K triples describing pertinent research topics. We also developed a full codebase to easily re-implement all knowledge injection strategies in different domains. A formal evaluation indicates that the majority of the proposed knowledge injection methodologies significantly outperform the baseline established by Bidirectional Encoder Representations from Transformers. [ABSTRACT FROM AUTHOR]

Remote sensing scene (RSS) image classification plays a vital role in various fields such as urban planning and environmental protection. However, due to higher inter-class similarity and intra-class variability, achieving accurate classification for RSS images poses a considerable challenge for current convolutional neural networks (CNNs)-based and visual transformer (ViT)-based methods. To address these issues, this paper proposes a novel dual-encoding method named master–slave encoding network (MSE-Net) from two perspectives of feature extraction and fusion. The master encoder, based on ViT, extracts higher-level semantic features, while the slave encoder, based on CNN, captures relative lower-level spatial structure information. Secondly, to integrate feature information from the two encoders effectively, this paper further develop two fusion strategies. The first strategy involves the auxiliary enhancement units (AEUs), which eliminates semantic divergence between the two encoders, enhances spatial context awareness of the slave encoder and promotes effective feature learning. The interactive perception unit (IPU), as the second strategy, facilitates interaction and integration of the two encoders' representations to extract more discriminative feature information. In addition, we conducted comparative experiments on four widely-used RSS datasets, including RSSCN7, SIRI-WHU, the aerial image dataset (AID) and NWPU-RESISC45 (NWPU45), to verify the effectiveness of MSE-Net. The experimental results demonstrate that MSE-Net achieved state-of-the-art (SOTA) performance across all the datasets. [Display omitted] [ABSTRACT FROM AUTHOR]

The aging of transformer oil–paper insulation (OPI) contributes significantly to transformer failure and reduced service life. Therefore, reliable and precise evaluation of the aging status of transformer insulation condition is crucial in averting transformer failure. This article proposes a novel deep learning-based aging condition assessment technique of transformers by combining FDS (Frequency domain spectroscopy) and squeeze-and-excitation-enabled convolutional neural network (SE-CNN). An intrinsic product-based transformation using a Gramian angular field is employed to dimensionally expand the 1-D FDS signals acquired from OPI samples into 2D images. Each feature extraction channel in the SE-CNN-based aging diagnosis framework is assigned a specific weight, enabling the model to concentrate on the most relevant features while discarding the rest. Furthermore, the effect of moisture variation on the model is also considered under three different conditions to ensure the consequence of compensation for the effect of moisture on FDS. A comparison with traditional machine learning models has also been made to ensure the viability of the proposed methodology. Additionally, the model is interpreted by creating explanation weight plots of the feature coefficients using a local interpretable model-agnostic explanations (LIME) method in Python. The findings demonstrate that the novel aging diagnosis framework visually assesses the transformer's OPI system with the highest accuracy of 99.12% in the test set. Furthermore, it facilitates the automatic extraction of deep aging features, thus providing better generalization and convergence capability. [ABSTRACT FROM AUTHOR]

Water temperature affects many physical, chemical and biological processes in rivers and plays a crucial role in determining the quality of aquatic ecosystems. Due to the complexity and nonlinear characteristics of most factors affecting river water temperature, it is difficult for traditional models to accurately predict river water temperature. In this context, accurate prediction of river water temperature calls for new and innovative machine learning techniques. This paper presents a new hybrid model, called LSTM-Encoder, that combines long short-term memory (LSTM) with transformer encoder. To improve the accuracy of the hybrid LSTM-Encoder model, wavelet threshold denoising (WTD) method was used to denoise the data. The proposed model was compared with the Air2water model as well as eight other artificial intelligence (AI) models. The results show that the mean absolute percentage error (MAPE), mean absolute error (MAE), root mean squared error (RMSE) and coefficient of determination (R2) values of the WTD-LSTM-Encoder model are 2.9, 0.279 °C, 0.567 °C and 0.867, respectively, for the training datasets and 3.2, 0.312 °C, 0.625 °C and 0.714, respectively, for the testing datasets, indicating that the proposed WTD-LSTM-Encoder model has the best comprehensive performance in predicting the river water temperature. • A novel hybrid model coupling long short-term memory (LSTM) and transformer encoder is proposed for predicting river water temperature. • Utilization of wavelet threshold denoising (WTD) technology to reduce data noise and improve prediction accuracy. • Integration of self-attention and multi-head mechanisms enhances the prediction ability of the model. • A comparison among the proposed model, the Air2water model and eight other artificial intelligence (AI) models is conducted. [ABSTRACT FROM AUTHOR]

Salient object detection involves identifying visually distinctive objects within an image. However, the effective integration of long-range dependencies and local feature information in search for detailed segmentation remains a challenging task for established methods. In this paper, a novel dual-path multi-branch feature residual network (DMFRN) is proposed for salient object detection. First, we design a dual-path encoder consisting of a convolutional network and a lightweight vision transformer, which ensures the capture of local features and long-range dependencies while balancing efficiency. Secondly, a multi-scale feature interaction module (MFIM) is designed to further extract global feature information. It utilizes skip connections to facilitate interaction between multi-scale information, thereby enhancing feature representation. Finally, a multi-branch feature residual module (MFRM) is designed in the decoder to more effectively fuse the features provided by skip connections with the features in the encoder path. The residual structure is employed to refine the features. Our method is compared with competitors on five benchmark datasets, with average MAE and F m a x of 0.042 and 0.893 respectively, both outperforming other competing methods. • CNNs-ViT hybrid architecture encoder. • Multi-scale feature interaction module. • Multi-branch feature residuals module. [ABSTRACT FROM AUTHOR]

Object detection is one of the most important domains in computer vision tasks, which is an important branch of artificial intelligence. It aims at finding and locating the accurate position of objects in given pictures or videos. With the development of deep learning techniques, more powerful and robust algorithms have emerged to deal with multi-scale, high-level features to overcome the limitations of traditional pipeline of object detectors. The popularity of transformer framework enables larger capacity datasets by processing self-attention mechanism, and the object detection methods have evolved into a new era. This paper first reviews traditional object detection pipeline and brief history of deep learning, afterwards it focuses on the classification of deep learning-based object detection methods covering Convolution Neural Network based and transformer-based methods. Commonly used datasets and metrics are also covered in the next part. The Convolution Neural Network based methods mainly contain two-stage and one-stage detectors, Convolution Neural Network is the underlying structure of these methods convolutional stages are fundamental parts. Transformer-based models convert traditional object detection issues into end-to-end detection, which is widely used in dealing with images. Finally, the promising future of object detection areas are listed to show guidance on future work. [Display omitted] [ABSTRACT FROM AUTHOR]

Skin cancer is one of the most serious threats to human health among skin lesions. Computer-aided diagnosis methods can assist patients in identifying and detecting skin lesion types early, thereby enabling corresponding treatments. In this paper, we propose a dual-branch neural network model Conformer with Residual Cosine Similarity Attention and Bidirectional Convolutional fusion strategy, named RCSABC-Conformer. The core of this network structure comprises three parts: a Convolutional Neural Network (CNN) branch with Residual Cosine Similarity Attention (RCSA), a Transformer branch, and a Feature Couple Unit with Bidirectional Convolutional strategy (BC-FCU). The RCSA module calculates the cosine similarity value between the feature map generated by the convolutional operation and the feature map of the residual edge to assess whether their semantic information is similar. The semantic information of similar parts is weighted by exponential normalization to enhance the network's memory of similar features of the same type of skin lesion. The BC-FCU module interactively fuses local features and global representations of skin lesion images with different resolutions in the two branches. Specifically, when the global representations is integrated into local features, we introduce a new bidirectional convolution strategy to extract the feature map from both forward and backward directions, and then select the element with the smaller feature value from the two directions to fuse into local features. In this way, we can minimize the interference of the artifact features extracted by the Transformer branch on the CNN branch. In addition, taking advantage of the Transformer branch's capacity to construct global representations, our model can learn contextual semantic information of normal skin and lesion areas to enhance model robustness. We conducted experiments on three datasets, consisting of clinical and dermoscopic skin lesion images, as well as a hybrid of both. The experimental results show that RCSABC-Conformer outperforms both advanced and classical classification methods in terms of classification accuracy across all three datasets, without requiring an increase in the number of parameters and computational complexity. Compared with the baseline model, the classification accuracy of our proposed method improves by 2.40%, 5.39%, and 4.44% on the three datasets, respectively. To the best of our knowledge, this is the first study to apply an interactive fusion dual-branch network for multi-disease classification on different modalities of skin lesion databases. Code will be available at https://github.com/AlenLi817/RCSABC-Conformer. [ABSTRACT FROM AUTHOR]

Executing various production tasks is critical to the safe operation and efficient production of industrial processes. As one of them, the detection task of key quality variables directly affects the operation optimization and decision-making of industrial processes, but it is severely limited by the harsh environment and detection instruments. Therefore, the real-time prediction task of key quality variables becomes the basis for optimal control of industrial processes. To address this issue, this paper proposes a task-oriented deep learning framework based on a target-related transformer (TR-Former) network for industrial quality prediction tasks. Specifically, a new target-related self-attention (TR-SA) mechanism is developed to guide feature learning by adding attention scores between task-related target variables and other variables. As a result, the learned features in this instance will be guaranteed to be relevant to the target variable and useful for the quality prediction task. Moreover, the long-range dynamics of industrial process data can also be captured, which can further improve the prediction performance of the model. Finally, extensive experiments were conducted on two industrial processes to validate the superiority of the proposed method in terms of quality prediction tasks. The experimental results demonstrate that the proposed TR-Former method exhibits an improvement ranging from 3% to 13% in the mean absolute error indicator compared to the traditional transformer and other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Accurate positioning of curtain wall frames is crucial for the automated installation of curtain wall modules. However, the current robot-based installation methods overly depend on visual guidance from operators, resulting in high costs and limiting construction efficiency. The development of deep learning has introduced an image segmentation approach that offers a new solution for the visual positioning of curtain wall frames. This paper proposes a context collaboration pyramid network to automatically segment curtain wall frames by incorporating context interaction and channel guided pyramid structure. The model adopts an "encoder-decoder" architecture with a feature interaction block strategically inserted between the encoder and decoder. Specifically, the encoder utilizes the pyramid pooling Transformer as a backbone to extract multi-level features from original RGB images. The decoder employs a channel guided pyramid convolution module to integrate multi-scale features and achieve finer prediction. Meanwhile, a context interaction fusion module between the features of adjacent levels was designed carefully to enhance the collaboration of the architecture. In addition, a benchmark dataset for the curtain wall frame segmentation task, consisting of 1547 images, was established. The dataset incorporates challenging scenarios, including strong lights, low contrast, and cluttered backgrounds. This method is evaluated on the collected dataset, and achieves an impressive accuracy of 97.30% and an F1-Score of 88.95%, outperforming other segmentation networks. Overall, the proposed method can extract target information accurately and efficiently and provide critical visual guidance for the robot, so as to promote the automatic installation level of the curtain wall module. [ABSTRACT FROM AUTHOR]

Most existing Convolutional Neural Networks (CNNs), Transformers, and their variants have limitations in capturing relationships between hyperspectral image (HSI) data, leading to unclear descriptions of region boundaries and limited generalization abilities. While semi-supervised Graph Neural Networks (GNNs) come with higher computational costs. Therefore, this paper proposes a method interacting the frequency and topology information for HSI Classification to address the aforementioned shortcomings, which combines convolution and self-attention to capture both local and global contextual information, thereby enhancing feature representation. Additionally, this method focuses on exploring spectral and topological structure features and enhancing the information exchange and interaction to improve performance. Experimental results demonstrate that this method gains a competitive advantage in HSI classification by proving highly effective in handling spectral ambiguity and material heterogeneity. It also exhibits lower computational costs, making it more feasible and practical compared to most benchmark methods. Our code is available at https://github.com/youngboy03/FTINet. [Display omitted] • Use CIformer, AFLM and ECM to obtain features from different angles in the image. [ABSTRACT FROM AUTHOR]

Owing to the development of deep learning, indoor personnel detection methods based on deep neural networks have been extensively investigated in recent years. However, more complex and deeper network structures may consume more computational resources, which seriously limits the deployment of large-scale deep neural networks on lightweight devices. In view of this, a lightweight anchor-free one-level feature indoor personnel detection method based on transformer (LAOF-IPDT) is proposed in this paper, which is deployed on two embedded devices and achieves good detection accuracy. In the feature extraction backbone, an enhanced cross-stage partial ghost convolution block with ghost convolution and channel shuffle is designed to extract shallow features. Additionally, to obtain more comprehensive global features in the high-level semantic structure, an embedded vision transformer cross-stage partial block is constructed by embedding a lightweight mobile-friendly vision transformer. For the path-aggregation neck, a lightweight feature pyramid network is proposed, which integrates multi-scale feature maps to obtain richer one-level feature representations. Subsequently, a dilated convolution group block is applied to expand the one-level feature receptive field and detection is accomplished using a one-level feature map. For the detection head, an anchor-free mechanism is applied to reduce the hyper-parameter interference of anchor boxes. Extensive experiments on four datasets indicated that the LAOF-IPDT outperforms other lightweight networks in terms of accuracy, speed, model parameters, and model size. For example, the frames per second of the LAOF-IPDT are 8.39 and 14.67 on CPU devices and Jetson Nano devices, respectively, and the mean average precision is 84.49%. [ABSTRACT FROM AUTHOR]

Remaining Useful Life (RUL) prediction is of great significance for maintaining the reliability and safety of industrial equipment. To address the challenges faced by existing methods in simultaneously extracting local and global degradation information from monitoring data. This paper proposes a Two-Stream Convolution Augmented Transformer (TACT) model based on L 2 regularization constraint. Specifically, we design the parallel multi-scale Convolution Neural Network (CNN) and Transformer module to combine the local modeling ability of CNN and the global modeling ability of Transformer to improve the overall architecture of RUL prediction model. Moreover, the two-stream network based on the parallel structure also realizes the synchronous extraction of different time steps and sensor features in the sequence. Then, in the process of model training, the prediction reliability constraint is fused, the delay prediction constraint term is introduced, and the L 2 regularization loss function is constructed. Finally, extensive experiments on the commercial modular aero-propulsion system simulation (C-MAPSS) show that our model provides competitive performance in terms of Root-Mean-Square Error (RMSE) and Score metrics. Compared to the state-of-the-art method based on Recurrent Neural Network (RNN) or CNN and its variants, Score is reduced by at least 2.71% and RMSE by at least 3.13%. Compared to the Transformer-based improved method, the Score is decreased by at least 4.54% and the RMSE is decreased by at least 2.78%. The effectiveness of the proposed method is demonstrated. • A hybrid end-to-end RUL prediction network based on a tandem combination of CNN and Transformer is proposed. • A two-stream structure is used to extract spatial and temporal features simultaneously. • The MSE loss function incorporating L 2 regularization constraint is proposed. [ABSTRACT FROM AUTHOR]

Stain normalization is a key preprocessing step that has been shown to significantly improve the segmentation and classification performance of computer-aided diagnosis (CAD) systems. In recent advancements, numerous approaches have demonstrated significant progress in the domain of stain normalization; however, the most of these approaches are based on Generative Adversarial Networks. In this paper, we propose a novel vision transformer-based model, termed as StainSWIN, that combines the strengths of swin transformer and the architecture of super resolution to achieve improved performance in stain normalization task. The key concept behind the StainSWIN is the utilization of swin transformer blocks that exploit content-based interactions to capture long-range dependencies. The proposed model is equipped with two key blocks, including residual stain swin block (ResStainSWIN) and swin transformer block (STB). The StainSWIN has a residual super resolution architecture, in which the high-level features, extracted by STB, are combined to ResStainSWIN block. The performance of the StainSWIN model was compared with other state-of-the-art methods on a commonly used MITOS-ATYPIA14 histopathology dataset. The StainSWIN outperformed other methods in stain normalization with a large margin in terms of PSNR, SSIM, and RMSE metrics. The StainSWIN model achieved PSNR of 26.667 ± 3.492, SSIM of 0.943 ± 0.037, and RMSE of 6.206 ± 1.973. Additionally, we evaluated the model's impact to the segmentation performance of the MICCAI GlaS'16 dataset. The results demonstrates a 4.3% improvement in segmentation accuracy, attributed to a reduction in stain color variation. The proposed method has the ability to greatly assist CAD systems in maintaining consistent performance despite color variations. [ABSTRACT FROM AUTHOR]

Skin cancer is one of the most serious threats to human health among skin lesions. Computer-aided diagnosis methods can assist patients in identifying and detecting skin lesion types early, thereby enabling corresponding treatments. In this paper, we propose a dual-branch neural network model Conformer with Residual Cosine Similarity Attention and Bidirectional Convolutional fusion strategy, named RCSABC-Conformer. The core of this network structure comprises three parts: a Convolutional Neural Network (CNN) branch with Residual Cosine Similarity Attention (RCSA), a Transformer branch, and a Feature Couple Unit with Bidirectional Convolutional strategy (BC-FCU). The RCSA module calculates the cosine similarity value between the feature map generated by the convolutional operation and the feature map of the residual edge to assess whether their semantic information is similar. The semantic information of similar parts is weighted by exponential normalization to enhance the network's memory of similar features of the same type of skin lesion. The BC-FCU module interactively fuses local features and global representations of skin lesion images with different resolutions in the two branches. Specifically, when the global representations is integrated into local features, we introduce a new bidirectional convolution strategy to extract the feature map from both forward and backward directions, and then select the element with the smaller feature value from the two directions to fuse into local features. In this way, we can minimize the interference of the artifact features extracted by the Transformer branch on the CNN branch. In addition, taking advantage of the Transformer branch's capacity to construct global representations, our model can learn contextual semantic information of normal skin and lesion areas to enhance model robustness. We conducted experiments on three datasets, consisting of clinical and dermoscopic skin lesion images, as well as a hybrid of both. The experimental results show that RCSABC-Conformer outperforms both advanced and classical classification methods in terms of classification accuracy across all three datasets, without requiring an increase in the number of parameters and computational complexity. Compared with the baseline model, the classification accuracy of our proposed method improves by 2.40%, 5.39%, and 4.44% on the three datasets, respectively. To the best of our knowledge, this is the first study to apply an interactive fusion dual-branch network for multi-disease classification on different modalities of skin lesion databases. Code will be available at https://github.com/AlenLi817/RCSABC-Conformer. [ABSTRACT FROM AUTHOR]

Executing various production tasks is critical to the safe operation and efficient production of industrial processes. As one of them, the detection task of key quality variables directly affects the operation optimization and decision-making of industrial processes, but it is severely limited by the harsh environment and detection instruments. Therefore, the real-time prediction task of key quality variables becomes the basis for optimal control of industrial processes. To address this issue, this paper proposes a task-oriented deep learning framework based on a target-related transformer (TR-Former) network for industrial quality prediction tasks. Specifically, a new target-related self-attention (TR-SA) mechanism is developed to guide feature learning by adding attention scores between task-related target variables and other variables. As a result, the learned features in this instance will be guaranteed to be relevant to the target variable and useful for the quality prediction task. Moreover, the long-range dynamics of industrial process data can also be captured, which can further improve the prediction performance of the model. Finally, extensive experiments were conducted on two industrial processes to validate the superiority of the proposed method in terms of quality prediction tasks. The experimental results demonstrate that the proposed TR-Former method exhibits an improvement ranging from 3% to 13% in the mean absolute error indicator compared to the traditional transformer and other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Accurate positioning of curtain wall frames is crucial for the automated installation of curtain wall modules. However, the current robot-based installation methods overly depend on visual guidance from operators, resulting in high costs and limiting construction efficiency. The development of deep learning has introduced an image segmentation approach that offers a new solution for the visual positioning of curtain wall frames. This paper proposes a context collaboration pyramid network to automatically segment curtain wall frames by incorporating context interaction and channel guided pyramid structure. The model adopts an "encoder-decoder" architecture with a feature interaction block strategically inserted between the encoder and decoder. Specifically, the encoder utilizes the pyramid pooling Transformer as a backbone to extract multi-level features from original RGB images. The decoder employs a channel guided pyramid convolution module to integrate multi-scale features and achieve finer prediction. Meanwhile, a context interaction fusion module between the features of adjacent levels was designed carefully to enhance the collaboration of the architecture. In addition, a benchmark dataset for the curtain wall frame segmentation task, consisting of 1547 images, was established. The dataset incorporates challenging scenarios, including strong lights, low contrast, and cluttered backgrounds. This method is evaluated on the collected dataset, and achieves an impressive accuracy of 97.30% and an F1-Score of 88.95%, outperforming other segmentation networks. Overall, the proposed method can extract target information accurately and efficiently and provide critical visual guidance for the robot, so as to promote the automatic installation level of the curtain wall module. [ABSTRACT FROM AUTHOR]

Remaining Useful Life (RUL) prediction is of great significance for maintaining the reliability and safety of industrial equipment. To address the challenges faced by existing methods in simultaneously extracting local and global degradation information from monitoring data. This paper proposes a Two-Stream Convolution Augmented Transformer (TACT) model based on L 2 regularization constraint. Specifically, we design the parallel multi-scale Convolution Neural Network (CNN) and Transformer module to combine the local modeling ability of CNN and the global modeling ability of Transformer to improve the overall architecture of RUL prediction model. Moreover, the two-stream network based on the parallel structure also realizes the synchronous extraction of different time steps and sensor features in the sequence. Then, in the process of model training, the prediction reliability constraint is fused, the delay prediction constraint term is introduced, and the L 2 regularization loss function is constructed. Finally, extensive experiments on the commercial modular aero-propulsion system simulation (C-MAPSS) show that our model provides competitive performance in terms of Root-Mean-Square Error (RMSE) and Score metrics. Compared to the state-of-the-art method based on Recurrent Neural Network (RNN) or CNN and its variants, Score is reduced by at least 2.71% and RMSE by at least 3.13%. Compared to the Transformer-based improved method, the Score is decreased by at least 4.54% and the RMSE is decreased by at least 2.78%. The effectiveness of the proposed method is demonstrated. • A hybrid end-to-end RUL prediction network based on a tandem combination of CNN and Transformer is proposed. • A two-stream structure is used to extract spatial and temporal features simultaneously. • The MSE loss function incorporating L 2 regularization constraint is proposed. [ABSTRACT FROM AUTHOR]

Stain normalization is a key preprocessing step that has been shown to significantly improve the segmentation and classification performance of computer-aided diagnosis (CAD) systems. In recent advancements, numerous approaches have demonstrated significant progress in the domain of stain normalization; however, the most of these approaches are based on Generative Adversarial Networks. In this paper, we propose a novel vision transformer-based model, termed as StainSWIN, that combines the strengths of swin transformer and the architecture of super resolution to achieve improved performance in stain normalization task. The key concept behind the StainSWIN is the utilization of swin transformer blocks that exploit content-based interactions to capture long-range dependencies. The proposed model is equipped with two key blocks, including residual stain swin block (ResStainSWIN) and swin transformer block (STB). The StainSWIN has a residual super resolution architecture, in which the high-level features, extracted by STB, are combined to ResStainSWIN block. The performance of the StainSWIN model was compared with other state-of-the-art methods on a commonly used MITOS-ATYPIA14 histopathology dataset. The StainSWIN outperformed other methods in stain normalization with a large margin in terms of PSNR, SSIM, and RMSE metrics. The StainSWIN model achieved PSNR of 26.667 ± 3.492, SSIM of 0.943 ± 0.037, and RMSE of 6.206 ± 1.973. Additionally, we evaluated the model's impact to the segmentation performance of the MICCAI GlaS'16 dataset. The results demonstrates a 4.3% improvement in segmentation accuracy, attributed to a reduction in stain color variation. The proposed method has the ability to greatly assist CAD systems in maintaining consistent performance despite color variations. [ABSTRACT FROM AUTHOR]

This paper introduces a novel parallel deep neural network for fault diagnosis of drilling pumps. It integrates the Convolutional Block Attention Module with the AlexNet and synchronizes with the Anomaly Transformer model to delve meticulously into both the time and time-frequency domains of signals. The method prioritizes the singular extraction and subsequent amalgamation of features, facilitating a detailed view of diagnostic data and mitigating the risk of interference and overfitting. The integration of the anomaly attention of the Anomaly Transformer with the features of the Convolutional Block Attention Module results in a distinctive dual attention mechanism that is critical to the methodology. This mechanism emphasizes essential features in both the time domain and the time-frequency domain, improving the accuracy of fault diagnosis. Verification with on-site data underscores the preeminence of the approach over existing models, signaling improved reliability and accuracy in diagnosing faults in drilling pumps. This meticulous approach offers promising advances in the study and application of fault diagnosis in energy equipment, demonstrating increased efficiency and refined accuracy. • A novel Parallel Deep Neural Network-Based Fault Diagnosis Framework is proposed. • Time and time-frequency domains of signals are extracted and fused for Fault Diagnosis. • Integrating the Transformer with CBAM to consider the essential features of signals. [ABSTRACT FROM AUTHOR]

In light of the pandemic, the identification and processing of COVID-19-related text have emerged as critical research areas within the field of Natural Language Processing (NLP). With a growing reliance on online portals and social media for information exchange and interaction, a surge in online textual content, comprising disinformation, misinformation, fake news , and rumors has led to the phenomenon of an infodemic on the World Wide Web. Arabic, spoken by over 420 million people worldwide, stands as a significant low-resource language, lacking efficient tools or applications for the detection of COVID-19-related text. Additionally, the identification of COVID-19 text is an essential prerequisite task for detecting fake and toxic content associated with COVID-19. This gap hampers crucial COVID information retrieval and processing necessary for policymakers and health authorities. Addressing this issue, this paper introduces an intelligent Arabic COVID-19 text identification system named 'AraCovTexFinder,' leveraging a fine-tuned fusion-based transformer model. Recognizing the challenges posed by a scarcity of related text corpora, substantial morphological variations in the language, and a deficiency of well-tuned hyperparameters, the proposed system aims to mitigate these hurdles. To support the proposed method, two corpora are developed: an Arabic embedding corpus (AraEC) and an Arabic COVID-19 text identification corpus (AraCoV). The study evaluates the performance of six transformer-based language models (mBERT, XML-RoBERTa, mDeBERTa-V3, mDistilBERT, BERT-Arabic, and AraBERT), 12 deep learning models (combining Word2Vec, GloVe, and FastText embedding with CNN, LSTM, VDCNN, and BiLSTM), and the newly introduced model AraCovTexFinder. Through extensive evaluation, AraCovTexFinder achieves a high accuracy of 98.89 ± 0.001%, outperforming other baseline models, including transformer-based language and deep learning models. This research highlights the importance of specialized tools in low-resource languages to combat the infodemic relating to COVID-19, which can assist policymakers and health authorities in making informed decisions. [ABSTRACT FROM AUTHOR]

In this paper, we present a novel method for point cloud registration in large-scale 3D scenes. Our approach is accurate and robust, and does not rely on unrealistic assumptions. We address the challenges posed by scanning equipment like LiDAR, which often produce point clouds with dense properties. Additionally, our method is effective even in scenes with low overlap rates, specifically less than 30%. Our approach begins by computing overlap region-based correspondences. This involves extracting deep geometric features from point cloud pairs, which is especially beneficial in enhancing registration performance in cases with low overlap ratios. We then construct efficient triplets that vote in the 6D Hough space, representing the transformation parameters. This process involves creating a quartet from overlap region-based correspondences and then forming a final triplet following a sampling process. To mitigate ambiguity during training, we use similarity values of the triplet as features of each vote when configuring votes for network input. Our framework incorporates the architecture of the Fully Convolutional Geometric Features (FCGF) network, augmented with a transformer's attention mechanism, to reduce noise in the voting process. The final stage involves identifying the consensus of correspondence in the Hough space using a binning approach, which enables us to predict the final transformation parameters. Our method has demonstrated state-of-the-art performance on indoor datasets, including high overlap ratio data like 3DMatch and low overlap ratio data like 3DLoMatch. It has also shown comparable performance to leading methods on outdoor datasets like KITTI. [Display omitted] The proposed method employs a local feature-based matching technique between pairs of point clouds, utilizing a set of triplets. Initially, we assign each triplet to a specific bin in a sparsely divided Hough space. Following this initial step, we employ a fully sparse convolutional network to refine the Hough space, effectively removing noise from the voting process. The final stage involves selecting the bin that receives the maximum number of votes, which we then use to determine the final transformation parameter. This process ensures both accuracy and precision in our point cloud matching methodology. [ABSTRACT FROM AUTHOR]

Multi-focus image fusion technology is wildly used in the digital photography, and also be considered as a pre-task of other high level vision tasks. The main purpose is producing an all-in-focus image from multiple partly focused sources accurately, naturally and efficiently. Nowadays, transformer models have achieved great success in numerous vision-related tasks, and its powerful attention modeling ability brings significant possibilities for focus property detection. To make multi-focus image fusion technology even further, in this paper, a novel multi-focus image fusion method using interactive transformer and asymmetric soft sharing is proposed. First, for taking the transformer's advantages in global context modeling and improve its limitations in diversity and efficiency, a locally-enhanced interactive approach is devised. More specifically, it used cross-scale and cross-domain computation strategy to overcome the transformer's certain limitations when comes to domain-specific task, and simultaneously alleviate the insufficient local feature perception and redundant computational cost of existing approach. Second, to cope with the problems of fusion image distortion and artifacts, the proposed method adopts a multi-task learning strategy with asymmetric soft sharing. It predicts the fusion images and decision maps at the same time for avoiding image distortion and further obtaining natural fusion effect. Experimental results reveal that, the proposed method reaches promising focus property detection performance and high-fidelity fusion results. Moreover, when compared with other state-of-the-art methods on five multi-focus image datasets, it is more prominent in both qualitative and quantitative analysis as well as efficiency. The source code is available at https://github.com/zwy0913/MFFT. • An interactive transformer architecture is proposed for both cross-scale feature integration and cross-domain information sharing. • A strategy simultaneously execute generative and discriminative task is exploited for both high fidelity and natural fusion result. • A more objective and comprehensive rank-based efficiency evaluation approach is devised for better efficiency comparison. • Experimental result reveal that proposed method reaches state-of-the-art in fusion performance as well as efficiency. [ABSTRACT FROM AUTHOR]

Falls pose a major threat for the elderly as they result in severe consequences for their physical and mental health or even death in the worst-case scenario. Nonetheless, the impact of falls can be alleviated with appropriate technological solutions. Fall detection is the task of recognising a fall, i.e. detecting when a person has fallen in a video. Such an algorithm can be implemented in lightweight devices which can then cater to the users' needs, e.g. alerting emergency services or caregivers. At the core of those systems, a model capable of promptly recognising falls is crucial for reducing the time until help comes. In this paper we propose a fall detection solution based on transformers, i.e. state-of-the-art neural networks for computer vision tasks. Our model takes a video clip and decides if a fall has occurred or not. In a video stream, it would be applied in a sliding-window fashion to trigger an alarm as soon as it detects a fall. We evaluate our fall detection backbone model on the large UP-Fall dataset, as well as on the UR fall dataset, and compare our results with existing literature using the former dataset. • Fall detection task tackled using only RGB data. • The network directly learns from RGB data, no extra features are computed. • UP-Fall and UR Fall dataset are used for the evaluation. [ABSTRACT FROM AUTHOR]

The increased penetration of renewable energy sources (RESs) as an effective mechanism to reduce carbon emissions leads to an increased weather dependency for power and energy systems. This has created dynamic operation and degradation phenomena, which affect the lifetime estimation of the assets operated with RESs. For the reliable and efficient operation of RES it is crucial to monitor the health of its constituent components and feature selection is a crucial step for building robust and accurate health monitoring approaches. In this context, this paper presents a probabilistic feature selection approach, which probabilistically weights and selects features through a heuristic and iterative process for an improved asset lifetime estimation. Power transformers are key power grid assets and they are used to demonstrate the validity and impact of the proposed approach. The approach is tested on two different photovoltaic power plants operated in Spain and Australia. Results consistently show that the proposed feature-selection approach reduces the prediction error and consistently selects relevant features. The approach has been applied to transformer lifetime estimation, but it can be generally applied to assist in the lifetime estimation of other components operated in RESs. Part of the studies presented here as well as source codes are all open-source under the GitHub repository https://github.com/iramirezg/FeatureSelection. • Probabilistic feature selection approach for improved asset lifetime estimation. • Integration of environmental features for improved renewable-operated asset lifetime. • Systematic and robust feature weighting methodology. • Improved transformer lifetime estimation including sensor and environmental data. • Validated on two real photovoltaic power plant case studies. [ABSTRACT FROM AUTHOR]

The global generation of renewable energy has rapidly increased, primarily due to the installation of large-scale renewable energy power plants. However, monitoring renewable energy assets in these large plants remains challenging due to environmental factors that could result in reduced power generation, malfunctioning, and degradation of asset life. Therefore, the detection of surface defects on renewable energy assets is crucial for maintaining the performance and efficiency of these plants. This paper proposes an innovative detection framework to achieve an economical surface monitoring system for renewable energy assets. High-resolution images of the assets are captured regularly and inspected to identify surface or structural damages on solar panels and wind turbine blades. Vision transformer (ViT), one of the latest attention-based deep learning (DL) models in computer vision, is proposed in this work to classify surface defects. The ViT model outperforms other DL models, including MobileNet, VGG16, Xception, EfficientNetB7, and ResNet50, achieving high accuracy scores above 97% for both wind and solar plant assets. From the results, our proposed model demonstrates its potential for monitoring and detecting damages in renewable energy assets for efficient and reliable operation of renewable power plants. • For the first, vision transformer-based model is proposed for electrical asset management. • Framework for autonomous inspection of large-scale renewable power plant's asset damages. • Low cost with less human intervention and less processing time. • Model achieves high accuracy >97% for damage detection on solar panels and wind blades. • ViT outperforms in classification compared to other deep learning models. [ABSTRACT FROM AUTHOR]

Multi-view stereo (MVS) reconstruction is a key task of image-based 3D reconstruction, and deep learning-based methods can achieve better results than traditional algorithms. However, most of the current deep learning-based MVS methods use convolutional neural networks (CNNs) to extract image features, which cannot achieve the aggregation of long-distance context information and capture robust global information. In addition, in the process of fusing depth maps into point clouds, the confidence filters will filter out the depth values with low confidence in weak texture areas. These problems will lead to the low completeness of 3D reconstruction of weak texture and texture-less areas. To address the above problems, this paper proposes SA-MVSNet based on the PatchmatchNet with a self-attentive mechanism. First, we design a coarse-to-fine network framework to advance depth map estimation. In the feature extraction network, a module with a pyramid structure based on Swin Transformer Block is used to replace the original Feature Pyramid Network (FPN), and the self-correlation between weak texture areas is enhanced by applying a global self-attention mechanism. Then, we also propose a self-attention-based adaptive propagation module (SA-AP), which applies a self-attention calculation within depth value propagation window to obtain the relative weight values of current pixel and others, and then adaptively samples the depth values of neighbors on the same surface for propagation. Experiments show that SA-MVSNet has significantly improved the completeness of 3D reconstruction for the images with weak texture on DTU (provided by Danish Technical University), BlendedMVS, and Tanks and Temple datasets. Our code is available at https://github.com/miaowang525/SA-MVSNet. • SA-MVSNet proposed a multi-scale feature extraction module based on Swin Transformer. • A global self-attention mechanism is applied to enhance the self-correlation between weak textured areas. • SA-MVSNet proposed a self-attention-based adaptive propagation module (SA-AP). • Apply window self-attention mechanism to enhance the confidence of depth value. • SA-MVSNet has significantly improved the completeness of 3D reconstruction on three datasets. [ABSTRACT FROM AUTHOR]

Urban scene image segmentation is an important research area in high-resolution remote sensing image processing. However, due to its complex three-dimensional structure, interference factors such as occlusion, shadow, intra-class inconsistency, and inter-class indistinction affect segmentation performance. Many methods have combined local and global information using CNNs and Transformers to achieve high performance in remote sensing image segmentation tasks. However, these methods are not stable when dealing with these interference factors. Recent studies have found that semantic segmentation is highly sensitive to frequency information, so we introduced frequency information to make the model learn more comprehensively about different categories of targets from multiple dimensions. By modeling the target with local features, global information, and frequency information, the target features can be learned in multiple dimensions to reduce the impact of interference factors on the model and improve its robustness. In this paper, we consider frequency information in addition to combining CNNs and Transformers for modeling and propose a Multidimensional Information Fusion Network (MIFNet) for high-resolution remote sensing image segmentation of urban scenes. Specifically, we design an information fusion Transformer module that can adaptively associate local features, global semantic information, and frequency information and a relevant semantic aggregation module for aggregating features at different scales to construct the decoder. By aggregating image features at different depths, the specific representation of the target and the correlation between targets can be modeled in multiple dimensions, allowing the network to better recognize and understand the features of each class of targets to resist various interference factors that affect segmentation performance. We conducted extensive ablation experiments and comparative experiments on the ISPRS Vaihingen and ISPRS Potsdam benchmarks to verify our proposed method. In a large number of experiments, our method achieved the best results, with 84.53% and 87.3% mIoU scores on the Vaihingen and Potsdam datasets, respectively, proving the superiority of our method. The source code will be available at https://github.com/JunyuFan/MIFNet. [ABSTRACT FROM AUTHOR]

Remaining useful life (RUL) prediction of aircraft engines is significant in the health monitoring, operation, and maintenance of aircraft. Capturing more comprehensive device degradation trends at different time scales and extracting long-term dependencies effectively among elements in long time series are two challenges in the field of aircraft engine RUL estimation. To address the aforementioned challenges, this paper proposes a novel multiscale Hourglass-Transformer (MHT) aircraft engine RUL prognostics. Specifically, an hourglass-shaped multiscale feature extractor (HME) is designed based on one-dimensional convolutional neural network, which can scale the time sequence into multi-time scales for feature fusion. Then, a transformer network is employed to further extract features from the fused feature map and output the RUL. To enhance inter-scale data attention, a pyramid self-attention mechanism is employed in both the encoder and decoder. Finally, the superiority and effectiveness of this approach are verified on the Commercial Modular Aero-Propulsion System Simulation (C-MAPSS) dataset. Furthermore, the robustness and generalization capability of this method are further validated on New Commercial Modular Aero-Propulsion System Simulation (N-CMAPSS) dataset. • A novel multiscale Hourglass-Transformer is developed for aircraft engine RUL prognostics. • An hourglass-shaped multiscale feature extractor is designed based on one-dimensional convolutional neural network. • A transformer network is employed to further extract features. • Pyramid self-attention mechanism is adopted to reduce space complexity. [ABSTRACT FROM AUTHOR]

Artworks are treasures of valuable cultural and historical heritage. Artworks get damaged due to environmental and other factors. The artificial intelligence-based restoration of digitized artwork images can guide the artists in physically restoring the damaged artworks. Previous methods have not been able to restore artwork images well. This paper proposes a dual (spatial and channel) attention and channel transformer-based generative adversarial network to restore damaged artwork images digitally. The proposed generative adversarial network has spatial and channel attention layers in the encoder part of the generator and a channel transformer between skip connections from the encoder to the decoder part of the generator. Spatial and channel attention helps learn inter-spatial and inter-channel global relationships among image features. Channel transformer ensures multiscale feature fusion and reduces the semantic gap between encoder and decoder layer features. Moreover, the proposed network has been trained using a linear combination of perceptual, adversarial, and structured similarity index measure loss, which helps better train the network. Further, the proposed network has been validated on two different datasets, and the results indicate that the proposed method outperforms state-of-the-art artwork restoration methods. [Display omitted] • Spatial and channel attention guided generator architecture for artwork restoration. • Channel transformer between skip connections for multiscale feature fusion. • A combination of perceptual, adversarial, and SSIM loss as objective function. [ABSTRACT FROM AUTHOR]

Deep learning-based X-ray detection of hazardous materials is crucial for public safety, as it can automatically detect them in baggage. However, most existing algorithms for X-ray detection rely on supervised learning, necessitating large amounts of labeled data for training. Obtaining such data is challenging due to the unique nature of X-ray safety images, requiring significant time and money investment from trained staff for accurate labeling. In this paper, we propose a new X-ray dangerous goods detection framework called EslaXDET, whose backbone is trained by a hybrid Self-Supervised Learning (SSL) strategy and whose detection head is designed for a non-multistage backbone. First, the hybrid strategy named ESLA is an abbreviation for E nhanced S elf-supervised L earning with masked A utoencoders, which does not require the involvement of labels as it is derived from a hybrid architecture of Contrastive Learning (CL) and Masked Image Model (MIM). Then, the detection head, called Head-Tail Feature Pyramid (HTFP), creates multi-level feature maps comparable to that produced by a multi-level backbone structure by downsampling the output feature of the last stage of the plain Vision Transformers (ViT) multiple times. Finally, the experimental results show that with the backbone of ViT-B, the top-1 accuracy of ESLA on the ImageNet-1K dataset is 77.2%, which is 3% higher than BYOL. Moreover, the average precision (AP, the same evaluation metrics as COCO) of EslaXDET on the PIDray dataset is 69.2%, nearly 8% higher than the original method SDANet, providing a new idea for the subsequent SSL-based X-ray baggage security detection method. [Display omitted] • A new hybrid SSL framework named ESLA. • A new SSL detection framework for X-ray baggage screening named EslaXDET. • A new detector head called HTFP is adapted to a plain ViT. [ABSTRACT FROM AUTHOR]

The Tarim Basin region harbors abundant carbonate reservoirs in which fractures throughout the structures play a crucial role in facilitating the transportation of oil and gas. The degree and distribution of fracture development play a crucial role in achieving a high yield and stable production. Seismic attribute analysis is a widely used and efficacious approach for examining fractures in reservoirs. However, different seismic attributes characterize fractures at different scales, and any single seismic attribute may be of poor quality, making it difficult to describe reservoir channel media comprehensively. In the present study, the seismic attribute volume was sliced from three perspectives to obtain complete geological spatial structural information of the seismic volume. This was accomplished using a shifted window (Swin) transformer module guided by a t-distribution before extracting and capturing the prior information and global contextual cues of the seismic attributes. Additionally, a dual-branch fusion (DBFusion) network that combined a convolutional neural network and a Swin transformer to extract the spatial geological structural features of seismic attributes was used to facilitate a comprehensive integration of local and global contextual information. Through the DBFusion network modeling, the network comprehensively extracted local information and integrated complementary global information. Taking three-dimensional seismic data from a unit in the Tarim Basin as an example, this study explores the fusion of three attributes: coherence, ant tracking, and curvature. When compared to individual attributes, the fused attributes can comprehensively leverage the advantages of all three seismic attributes in reflecting fractures. This results in a more precise representation of fractures of varying sizes in the fused output. It was discovered through comparative experiments that the DBFusion network proposed in this paper exhibits the best performance compared to other models. • The DBFusion network is proposed for fracture attribute fusion. • The geological spatial structure information is extracted from multiple angles. • The T-distribution prior is proposed to extract and capture the prior information. • CNN and Swin Transformer are combined to extract the spatial geological structural features. [ABSTRACT FROM AUTHOR]

Negative emotion detection is challenging during the peak time of the COVID-19 period. Most earlier studies contain individuals' physical health recognition rather than mental health detection, which is a significant concern in facing the COVID-19 situation. Identifying mental health in advance is essential to understand individuals' psychological condition. This paper considers the texts from social media during the pandemic of COVID-19. We propose a novel context-based auto-regressive transformer with bidirectional long short-term memory and a convolutional neural network (Context-ABT-BiLSTM-CNN) model to detect emotions such as abuse, anger, anxiety, depression, disgust, fear, guilt, sadness, and shame on social media. The existing works do not suggest relevant terms to detect suitable context; as a result, there is no scope for detecting emotions. We introduce a novel topic-based text (TBT) with a rule-based permutation (RBP) procedure to extract the relevant text from social media to identify emotions. Random search is suggested to store each input's correlated information and the order of each sequence. We recommend various transformer components to maintain the text sequence, avoid discrepancies during model training, capture the long-distance semantics in bidirectional contexts, and adopt both the permutation and factorization processes to build the model. Moreover, a comparative study is introduced to detect the most dominant emotions on social media during the pandemic and non-pandemic periods. The proposed model with XLNet embeddings surpasses state-of-the-art models for detecting text emotions. The ablation study is conducted to understand the essential components needed for the proposed model. [ABSTRACT FROM AUTHOR]

Ship classification based on machine learning (ML) has proven to be a significant underwater acoustic research direction. One of the critical challenges rests with how to embed domain signal knowledge into ML models to obtain suitable features that highly correlate with the classification and create better predictors. In this paper, a novel ML-based ship classification model, Hierarchical Underwater Acoustic Transformer (HUAT), is proposed to improve the classification performance. Firstly, the Detection of Envelope Modulation on Noise (DEMON) spectra of ship-radiated noise signals are estimated by cyclostationary analysis. The motivation for using a DEMON-based preprocessing scheme is that valuable propeller information can be revealed by exploiting the second-order cyclostationarity of ship-radiated noise signals. Secondly, the useful features of DEMON spectra are enhanced using a multi-head self-attention module, and the potential features of the Mel spectrograms are extracted employing a Convolutional Neural Network (CNN) module. The two kinds of features are fused to provide ship classification patterns. The challenge of feature learning in the deep classification model is reduced by leveraging domain-related classification knowledge. Finally, the Swin Transformer, based on shifted window self-attention mechanism, is used to learn high-level feature representations and conduct ship classification. Experimental results show that the HUAT model achieves excellent classification performance on ship-radiated noise datasets, ShipsEar and DeepShip. And its classification efficiency is better than the model based on traditional Transformer architecture. In addition, the proposed method provides technical support for the underwater intelligent system capable of automatically sensing sailing vessels and recognizing vessel types. • The Detection of Envelope Modulation on Noise (DEMON) spectra of ship-radiated noise signals are estimated by cyclostationary analysis. • A novel feature fusion strategy is proposed for embedding domain signal knowledge into ML models to obtain suitable features highly correlated with the classification. • The valuable features of DEMON spectra and the Mel spectrogram of ship signals are extracted using two different feature extractors. • The Swin Transformer is used to learn high-level feature representations and conduct ship classification. [ABSTRACT FROM AUTHOR]

Transformers with long-range dependency and data specificity act as an effective means of classifying insect pests in agricultural engineering. Although many methods have been proposed to confine the range of self-attention within a local region to reduce the computation complexity, none of them can reduce the number of model parameters. Moreover, the self-attention mechanism usually causes query tokens to focus excessively on image patches, which limits the effective receptive field and the long-range dependence. To address these issues, this paper establishes a novel D ilated- W indows-based Vi sion T ransformer with E fficient- S uppressive-self-attention (DWViT-ES) architecture, which includes efficient-self-attention (ESA), dilated window (DW), and suppressive-self-attention (SSA) as its core components. The ESA simplifies the successive linear Transformations to reduce the number of model parameters and computational costs. Meanwhile, the DW and SSA expand the effective receptive field of self-attention mechanism to prevent query tokens from focusing on similar and close regions, thereby preventing the loss of useful information. Finally, experiments show that the DWViT-ES only has 19.6 M parameters and 3.5G FLOPs (over 20% reductions vs. 19.6 M and 4.5G of Swin-T). Meanwhile, the DWViT-ES training from scratch has 71.6% top-1 accuracy on the IP102 dataset (2.4% absolute improvement of Swin-T); after fine-tuning on Imagenet-1K, the DWViT-ES achieves 76.0% and 78.7% top-1 accuracy on IP102 and CPB (0.1% and 0.9% absolute improvement of Swin-T), respectively. Meanwhile, practical deployment on a mobile-embedded device is presented, which validates the feasibility of the DWViT-ES. [ABSTRACT FROM AUTHOR]

The query-based object detection models are improvement of transformer. Using the learnable query, they complete the transition from traditional dense detection to sparse detection. However, such models have the weaknesses of slow convergence speed, poor adaptability to target changes, and additional networks. The CycMixer proposed in this paper uses multi-scale granule cluster sampling for encoding, avoiding a large number of parameters brought by the explicit structure. It adopts the cycle mixing module to improve the adaptability and increase the receptive field to cope with the change in detection targets by means of adaptive channel sampling and cycle spatial sampling. Firstly, the query extracted by backbone is decoupled into content vector and positional vector. Then the multi-head attention mechanism generates offsets. Secondly, the multi-scale feature map obtained by backbone and the transformed position enter the multi-scale granule cluster sampling stage. The stage consists of multi-scale feature space generation and granule cluster sampling. In addition, offsets and original positional vectors are converted to a new positional vector, which is decoded to get the bounding box. Finally, the content vector and the sampled matrix enter the cycle mixing. Cycle mixing consists of adaptive channel mixing and cycle spatial mixing. The mixed content vector is updated by the post feed-forward network (FFN) to receive a new content vector. Another FFN outputs the predicted categories. Compared with the existing detectors, the AP of our method on MS COCO dataset can reach 44.0 in 12 epochs and 46.0 in 36 epochs of training, which has obvious advantages in the convergence speed and the simplicity of the model. Experimental results on the cityscapes dataset also demonstrated the superiority of the method. • Cycle spatial mixing modules • Step sampling • Multi-scale particle cluster sampling [ABSTRACT FROM AUTHOR]