Search

Abstract Mine flooding accidents have occurred frequently in recent years, and the predicting of mine water inflow is one of the most crucial flood warning indicators. Further, the mine water inflow is characterized by non-linearity and instability, making it difficult to predict. Accordingly, we propose a time series prediction model based on the fusion of the Transformer algorithm, which relies on self-attention, and the LSTM algorithm, which captures long-term dependencies. In this paper, Baotailong mine water inflow in Heilongjiang Province is used as sample data, and the sample data is divided into different ratios of the training set and test set in order to obtain optimal prediction results. In this study, we demonstrate that the LSTM-Transformer model exhibits the highest training accuracy when the ratio is 7:3. To improve the efficiency of search, the combination of random search and Bayesian optimization is used to determine the network model parameters and regularization parameters. Finally, in order to verify the accuracy of the LSTM-Transformer model, the LSTM-Transformer model is compared with LSTM, CNN, Transformer and CNN–LSTM models. The results prove that LSTM-Transformer has the highest prediction accuracy, and all the indicators of its model are well improved.

Abstract The influence maximization problem that has drawn a great deal of attention from researchers aims to identify a subset of influential spreaders that can maximize the expected influence spread in social networks. Existing works on the problem primarily concentrate on developing non-adaptive policies, where all seeds will be ignited at the very beginning of the diffusion after the identification. However, in non-adaptive policies, budget redundancy could occur as a result of some seeds being naturally infected by other active seeds during the diffusion process. In this paper, the adaptive seeding policies are investigated for the intractable adaptive influence maximization problem. Based on deep learning model, a novel approach named graph convolutional networks with self-attention mechanism (ATGCN) is proposed to address the adaptive influence maximization as a regression task. A controlling parameter is introduced for the adaptive seeding model to make a tradeoff between the spreading delay and influence coverage. The proposed approach leverages the self-attention mechanism to dynamically assign importance weight to node representations efficiently to capture the node influence feature information relevant to the adaptive influence maximization problem. Finally, intensive experimental findings on six real-world social networks demonstrate the superiorities of the adaptive seeding policy over the state-of-the-art baseline methods to the conventional influence maximization problem. Meanwhile, the proposed adaptive seeding policy ATGCN improves the influence spread rate by up to 7% in comparison to the existing state-of-the-art greedy-based adaptive seeding policy.

Copyright of Telecommunication Engineering is the property of Telecommunication Engineering and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

The neurological condition epilepsy is demanding and even fatal. Electroencephalogram (EEG)-based epilepsy detection still faces various difficulties. EEG readings fluctuate, and different patients have various seizure activity patterns. EEG signal detection is time-consuming and labour-intensive, which puts a strain on medical staff and raises the possibility of erroneous detections. Usually, electrodes are inserted into the scalp or inside the brain for a brief period of time in order to obtain EEG data. It is essential to research efficient cross-patient automatic epilepsy detection techniques. The multi-head self-attention mechanism recognises long-distance dependencies with the same proficiency as it does temporal dynamic correlations between short-term temporal pattern characteristics and sequential relationships. The contextual representations are inputted into a bidirectional long short-term memory network (BiLSTM) so that information can be extracted in both directions. Classification and training are carried out utilising the log SoftMax algorithm. The experiments utilised scalp EEG data from the CHB-MIT database. Sensitivity, specificity, F1-score, and accuracy were computed to be 96.5 percent, 97.04 percent, 96.6 percent, and 96.2 percent, respectively. The results of the experiment show how well the technique works for detecting seizures in several patients utilising multi-channel EEG recordings. The results also demonstrate the method's improved generalisation capabilities and resilience and consistency in collecting seizure patterns. This is particularly critical for the tertiary diagnosis of epilepsy, and the findings indicate that the proposed method significantly improves the accuracy of detection. [ABSTRACT FROM AUTHOR]

Aiming at the problem that existing models cannot adequately fully consider the latent intent in user interaction sequences and the overfit of models, the Stochastic Shared Embeddings and Latent Intent Aware Self-Attention for Sequential Recommendation (SSELISR) are proposed. Temporal convolutional networks are used to convolve the sequence of user interactions deeply, for obtaining a representation of the user's latent intent for the project. The absolute position and time interval of the project is modeled by using the time interval perception self-attention layer. The output of temporal convolution layer and time-aware self-attention layer is used as the intention time-aware attention layer to predict the query, key and value of the next item and find the correlation of items with latent intent. Stochastic shared embedding techniques are used to reduce model overfitting caused by over-parameterization and improve model recommendation accuracy through randomly transformations between embeddings. The experimental results show the SSELISR model, on the microblog and movie datasets NDCG@10 and Hit@10. Compared with the baseline models of GRU4Rec+, Caser, MaRank, and TiSASRec, the two aspects improved by 2.21%, 0.871%, 1.43%, and 3.99%, respectively. Ablation experiments also verified the interpretability and effectiveness of random shared embedding an underlying intent modules. Code is available at: Link Text [ABSTRACT FROM AUTHOR]

Traditional object counting systems use object detection methods to count objects. However, when objects are small, crowded, and dense, object detection may fail, leading to inaccuracies in counting. To address this issue, we propose a crowded object counting system based on density map estimation. While most density map estimation models employ encoder–decoder or multi-branch approaches to generate feature maps at different scales for obtaining an accurate density map, improving the accuracy of crowded object counting remains a challenge. In this paper, we propose a novel model that can generate more accurate density maps, utilizing the context-aware network as the primary structure and integrating the self-attention mechanism. There are three main contributions in this paper. Firstly, the self-attention mechanism is employed to improve the accuracy of density map estimation. Secondly, the missing vehicle labels in the TRANCOS database are relabeled, ensuring that the ground truth data are more complete than the original TRANCOS database, thus enabling the proposed novel model to have higher crowded object counting accuracy. Thirdly, the parameters of the self-attention mechanism are analyzed to obtain the optimum parameter combination. The experimental results demonstrate that the accuracy of crowded object counting can reach 85.9%, 90.0%, 83.4%, and 92.6% for the TRANCOS, relabeled TRANCOS, ShanghaiTech Part A, and Part B datasets, respectively. Furthermore, the ablation study for the context-aware network with self-attention mechanism analyzes the optimum parameter combination. [ABSTRACT FROM AUTHOR]

Liver cancer is one of the leading causes of cancer death in the world, and early diagnosis is important. However, the similarity in shape, texture, and intensity values between the liver, tumors, and other neighboring organs such as the heart, spleen, stomach, and kidneys often complicates visual differentiation. Manual identification of tumors in the liver is time‐consuming, intricate, and susceptible to errors with potential repercussions for patient care. While machine learning–based approaches have emerged for liver organ recognition and segmenting the tumor, they continue to face challenges related to recognition accuracy and the inability to distinguish tumors of varied sizes. To solve the problems, a multiattention network made up of cascaded ResUNet and U‐Net with attention mechanisms was proposed in this study. We investigated liver tumor segmentation with various configurations of U‐Net, ResUNet, U‐Net with attention mechanisms, and ResUNet with attention mechanisms on augmented and nonaugmented data. We used the 3Dircadb dataset for training and validation purposes, and the proposed method was evaluated on dice score, intersection of union (IoU), recall, and precision. The performance metrics achieved with this method on the dataset are as follows: approximately 0.89 for the dice coefficient, 0.90 for IoU, 0.93 for recall, and 0.96 for precision in the case of liver segmentation without data augmentation and 0.92, 0.90, 0.92, and 0.94, respectively, for dice score, IoU, recall, and precision with data augmentation. For tumor segmentation, the metrics include 0.70 for dice coefficient, 0.61 for IoU, 0.91 for recall, and 0.94 for precision when the data were augmented but 0.83 for dice score, 0.78 for IoU, and 0.89 and 0.90, respectively, for recall and precision. [ABSTRACT FROM AUTHOR]

Point cloud completion network often encodes points into a global feature vector, then predicts the complete point cloud through the vector generation process. However, this method may not accurately capture complex shapes, as global feature vectors struggle to recover their detailed structure. In this paper, we present a novel shape completion network, namely RD-Net, that innovatively focuses on the interaction of information between points to provide both local and global information for generating fine-grained complete shape. Specifically, we propose a stored iteration-based method for point cloud sampling that quickly captures representative points within the point cloud. Subsequently, in order to better predict the shape and structure of the missing part, we design an iterative edge-convolution module. It uses a CNN-like hierarchy for feature extraction and learning context information. Moreover, we design a two-stage reconstruction process for latent vector decoding. We first employ a feature-points-based multi-scale generating decoder to estimate the missing point cloud hierarchically. This is followed by a self-attention mechanism that refines the generated shape and effectively generates structural details. By combining these innovations, RD-Net achieves a 2% reduction in CD error compared to the state-of-the-art method on the ShapeNet-part dataset. • We propose a novel method for high-quality 3D point cloud shape completion. • We design a pipeline to sample points, reducing computation and boosting efficiency. • We use iterative edge convolution to capture local and global point relationships. • We propose a multi-scale structure to predict and refine missing feature points. • We propose a module that aggregates local features to generate point cloud details. [ABSTRACT FROM AUTHOR]

The Transformer architecture has shown to be a powerful tool for a wide range of tasks. It is based on the self-attention mechanism, which is an inherently computationally expensive operation with quadratic computational complexity: memory usage and compute time increase quadratically with the length of the input sequences, thus limiting the application of Transformers. In this work, we propose a novel Clustering self-Attention mechanism using Surrogate Tokens (CAST), to optimize the attention computation and achieve efficient transformers. CAST utilizes learnable surrogate tokens to construct a cluster affinity matrix, used to cluster the input sequence and generate novel cluster summaries. The self-attention from within each cluster is then combined with the cluster summaries of other clusters, enabling information flow across the entire input sequence. CAST improves efficiency by reducing the complexity from O (N 2) to O (α N) where N is the sequence length, and α is constant according to the number of clusters and samples per cluster. We show that CAST performs better than or comparable to the baseline Transformers on long-range sequence modeling tasks, while also achieving higher results on time and memory efficiency than other efficient transformers. • Computation of self-attention Transformers is limited by the input sequence length. • We propose CAST, an efficient self-attention mechanism with clustered attention. • We propose the use of surrogate tokens to optimize self-attention in transformers. • We observe that cluster summaries enhance training efficiency and results. • CAST reduces complexity of self-attention computation from O (N 2) to O (α N). [ABSTRACT FROM AUTHOR]

The need for sentiment analysis in the mental health field is increasing, and electroencephalogram (EEG) signals and music therapy have attracted extensive attention from researchers as breakthrough ideas. However, the existing methods still face the challenge of integrating temporal and spatial features when combining these two types, especially when considering the volume conduction differences among multichannel EEG signals and the different response speeds of subjects; moreover, the precision and accuracy of emotion analysis have yet to be improved. To solve this problem, we integrate the idea of top-k selection into the classic transformer model and construct a novel top-k sparse transformer model. This model captures emotion-related information in a finer way by selecting k data segments from an EEG signal with distinct signal features. However, this optimization process is not without its challenges, and we need to balance the selected k values to ensure that the important features are preserved while avoiding excessive information loss. Experiments conducted on the DEAP dataset demonstrate that our approach achieves significant improvements over other models. By enhancing the sensitivity of the model to the emotion-related information contained in EEG signals, our method achieves an overall emotion classification accuracy improvement and obtains satisfactory results when classifying different emotion dimensions. This study fills a research gap in the field of sentiment analysis involving EEG signals and music therapy, provides a novel and effective method, and is expected to lead to new ideas regarding the application of deep learning in sentiment analysis. [ABSTRACT FROM AUTHOR]

A key study area throughout the medical sector for image processing and analysis is medical image segmentation. The diagnosis and treatment strategies of doctors may have a solid foundation owing to accurate and effective medical image segmentation. Conventional approaches in this field rely on manual feature extraction, which makes segmentation complex, costs doctors' time and energy, and involves a subjective evaluation that is readily susceptible to diagnostic errors. Researchers have applied convolutional neural network-based deep learning techniques to the segmentation of medical images as a result of their impressive advancements and successes in the field of computer vision. The research described here uses the U-Net network's outstanding feature learning capabilities and end-to-end processing mode for lung CT image segmentation via fully convolutional network (FCN) research. However, focusing on valuable, crucial information aspects in the U-Net network is challenging. This study employed multilevel attention mechanisms on the basis of U-Net networks to enhance the model's accuracy in lung CT image segmentation. These mechanisms were inspired by attention mechanisms. By improving the segmentation accuracy and optimizing the segmentation effect, the new model embeds a self-attention module in front of each upsampling layer in the U-Net model. This module provides more detailed information by stitching the self-attention module of the original image and then suppresses irrelevant and redundant information by using the effect of feature extraction of the upsampling layer. Several additional comparative experiments were conducted on the 2019nCoVR dataset. The outcomes demonstrate the efficacy of the optimized model described in this paper and its application results in improved segmentation effects in lung CT images. Additionally, the new model has distinct advantages over existing approaches that are typical of medical image segmentation, which represent its higher level of lung CT image segmentation. [ABSTRACT FROM AUTHOR]

Addressing the privacy protection and data sharing issues in Chinese medical texts, this paper introduces a federated learning approach named FLCMC for Chinese medical text classification. The paper first discusses the data heterogeneity issue in federated language modeling. Then, it proposes two perturbed federated learning algorithms, FedPA and FedPAP, based on the self-attention mechanism. In these algorithms, the self-attention mechanism is incorporated within the model aggregation module, while a perturbation term, which measures the differences between the client and the server, is added to the local update module along with a customized PAdam optimizer. Secondly, to enable a fair comparison of algorithms' performance, existing federated algorithms are improved by integrating a customized Adam optimizer. Through experiments, this paper first conducts experimental analyses on hyperparameters, data heterogeneity, and validity on synthetic datasets, which proves that the proposed federated learning algorithm has significant advantages in classification performance and convergence stability when dealing with heterogeneous data. Then, the algorithm is applied to Chinese medical text datasets to verify its effectiveness on real datasets. The comparative analysis of algorithm performance and communication efficiency shows that the algorithm exhibits strong generalization ability on deep learning models for Chinese medical texts. As for the synthetic dataset, upon comparing with comparison algorithms FedAvg, FedProx, FedAtt, and their improved versions, the experimental results show that for data with general heterogeneity, both FedPA and FedPAP show significantly more accurate and stable convergence behavior. On the real Chinese medical dataset of doctor–patient conversations, IMCS-V2, with logistic regression and long short-term memory network as training models, the experiment results show that in comparison to the above three comparison algorithms and their improved versions, FedPA and FedPAP both possess the best accuracy performance and display significantly more stable and accurate convergence behavior, proving that the method in this paper has better classification effects for Chinese medical texts. [ABSTRACT FROM AUTHOR]

The operational stability of the power transformer is essential for maintaining the symmetry, balance, and security of power systems. Once the power transformer fails, it will lead to heightened instability within grid operations. Accurate prediction of oil temperature is crucial for efficient transformer operation. To address challenges such as the difficulty in selecting model hyperparameters and incomplete consideration of temporal information in transformer oil temperature prediction, a novel model is constructed based on the improved whale optimization algorithm (IWOA) and long short-term memory (LSTM) neural network with self-attention (SA) mechanism. To incorporate holistic and local information, the SA is integrated with the LSTM model. Furthermore, the IWOA is employed in the optimization of the hyper-parameters for the LSTM-SA model. The standard IWOA is improved by incorporating adaptive parameters, thresholds, and a Latin hypercube sampling initialization strategy. The proposed method was applied and tested using real operational data from two transformers within a practical power grid. The results of the single-step prediction experiments demonstrate that the proposed method significantly improves the accuracy of oil temperature prediction for power transformers, with enhancements ranging from 1.06% to 18.85% compared to benchmark models. Additionally, the proposed model performs effectively across various prediction steps, consistently outperforming benchmark models. [ABSTRACT FROM AUTHOR]

Interactive image segmentation extremely accelerates the generation of high-quality annotation image datasets, which are the pillars of the applications of deep learning. However, these methods suffer from the insignificance of interaction information and excessively high optimization costs, resulting in unexpected segmentation outcomes and increased computational burden. To address these issues, this paper focuses on interactive information mining from the network architecture and optimization procedure. In terms of network architecture, the issue mentioned above arises from two perspectives: the less representative feature of interactive regions in each layer and the interactive information weakened by the network hierarchy structure. Therefore, the paper proposes a network called EnNet. The network addresses the two aforementioned issues by employing attention mechanisms to integrate user interaction information across the entire image and incorporating interaction information twice in a design that progresses from coarse to fine. In terms of optimization, this paper proposes a method of using zero-order optimization during the first four iterations of training. This approach can reduce computational overhead with only a minimal reduction in accuracy. The experimental results on GrabCut, Berkeley, DAVIS, and SBD datasets validate the effectiveness of the proposed method, with our approach achieving an average NOC@90 that surpasses RITM by 0.35. [ABSTRACT FROM AUTHOR]

Thermal errors affect machining accuracy in high-speed precision machining. The variability of machine tool operating conditions poses a challenge to the modeling of thermal errors. In this paper, a thermal error model based on transfer temperature feature fusion is proposed. Firstly, the temperature information fusion features are built as inputs to the model, which is based on a self-attention mechanism to assign weights to the temperature information and fuse the features. Secondly, an improved direct normalization-based adaptive matrix approach is proposed, updating the background matrix using an autoencoder and reconstructing the adaptive matrix to realize domain self-adaptation. In addition, for the improved adaptive matrix, a criterion for determining whether the working conditions are transferrable to each other is proposed. The proposed method shows high prediction accuracy while ensuring training efficiency. Finally, thermal error experiments are performed on a VCM850 CNC machine tool. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Graphics is the property of Journal of Graphics Editorial Office and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Copyright of Journal of Graphics is the property of Journal of Graphics Editorial Office and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

In interior interaction design, achieving intelligent user-interior interaction is contingent upon understanding the user's emotional responses. Precise identification of the user's visual emotions holds paramount importance. Current visual emotion recognition methods rely solely on singular features, predominantly facial expressions, resulting in inadequate coverage of visual characteristics and low recognition rates. This study introduces a deep learning-based multimodal weighting network model to address this challenge. The model initiates with a convolutional attention module, employing a self-attention mechanism within a convolutional neural network (CNN). As a result, the multimodal weighting network model is integrated to optimize weights during training. Finally, a weight network classifier is derived from these optimized weights to facilitate visual emotion recognition. Experimental outcomes reveal a 77.057% correctness rate and a 74.75% accuracy rate in visual emotion recognition. Comparative analysis against existing models demonstrates the superiority of the multimodal weight network model, showcasing its potential to enhance human-centric and intelligent indoor interaction design. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Computer Engineering & Applications is the property of Beijing Journal of Computer Engineering & Applications Journal Co Ltd. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Copyright of Journal of Chongqing University of Posts & Telecommunications (Natural Science Edition) is the property of Chongqing University of Posts & Telecommunications and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

2D magnetotelluric (MT) imaging detects underground structures by measuring electromagnetic fields. This study tackles two issues in the field: traditional methods' limitations due to insufficient forward modeling data, and the challenge of multiple solutions in complex scenarios. We introduce an enhanced 2D MT imaging approach with a novel self-attention mechanism, involving: 1. Generating diverse geophysical models and responses to increase data variety and volume. 2. Creating a Swin–Unet-based 2D MT Imaging network with self-attention for better modeling and relation capture, incorporating a MT sample generator using real data to lessen large-scale supervised training dependence, and refining the loss function for optimal validation. This method also includes eliminating MT background response to boost training efficiency and reduce training time. 3. Applying a transverse electric/transverse magnetic method for comprehensive 2D MT data response. Tests show that our method greatly improves 2D MT imaging's accuracy and efficiency, with excellent generalization. [ABSTRACT FROM AUTHOR]

In the structural health monitoring system, a large amount of testing data from sensors has been used to detect the health status of a structure. However, due to factors such as sensor failure and electromagnetic interference, the data collected by the structural health monitoring system may exhibit various abnormal states, which will affect the accuracy of structural damage identification and state assessment. How to process large amounts of data with high accuracy and recognize those abnormal data with high efficiency poses significant challenges. This paper proposes an innovative deep-learning method for data abnormality diagnosis by designing and establishing a self-attention residual network (SARN) model. In the method, all the monitoring data, including the abnormal data, are first converted into a time–frequency diagram by continuous wavelet transform, which are then used as the input of the SARN model for training. Then the convolutional layer, batch normalization, max pooling layer, the set self-attention residual building unit-1, and unit-2 are established successively to process the data, and finally the abnormal data can be identified by deep-learning technique. The proposed method broadens the receptive field of convolutional neural networks limited by the size of convolution kernels, allowing for the extraction of richer global features of data, thereby enabling high-precision automatic diagnosis and classification of a large amount of abnormal monitoring data. As an application case, a long-span cable-stayed bridge being monitored is introduced, and its acceleration monitoring data during one month are used as a dataset, and the developed SARN model is used for data anomaly diagnosis. The test accuracy of the model is found to be 97.80%, indicating its efficiency and accuracy in identifying abnormal data in the structural health monitoring dataset. [ABSTRACT FROM AUTHOR]

The self-attention mechanism has been the most frequent and efficient way for processing and learning sequences in numerous domains of artificial intelligence, including natural language processing, automatic speech recognition, and computer vision in recent years. It has a strong ability to learn the dependencies between the points of the input sequence, particularly those that are separated by a distance, and it also allows for parallel processing of the sequence. As a result, when used in processing sequences, this mechanism has a stronger ability to extract an appropriate representation from the input sequence at a faster rate than other approaches such as recurrent neural networks. Despite the benefits of the self-attention mechanism, recurrent neural networks along with feature engineering have been the most commonly employed approaches to online handwriting recognition. This study introduces an end-to-end online handwriting recognition system that utilizes the self-attention mechanism into three different modeling methods: CTC-based, RNN-T, and encoder–decoder. The proposed system demonstrates the capacity to recognize handwritten scripts without the need for feature engineering. The system's performance was evaluated using the Arabic Online-KHATT dataset and the English IAM-OnDB dataset. On the former, it achieved character error rate (CER) of 4.78% and word error rate (WER) of 20.63%, and on the latter, the CER of 4.10% and the WER of 14.31%, both of which were noticeably better than the results previously reported. Additionally, the Persian Online Handwriting Database was utilized for experimental validation, resulting in a CER 8.03% and a WER of 28.39%. [ABSTRACT FROM AUTHOR]

In recent years, DNA microarray is a recent research technique aimed at classifying gene expression. This paper is to develop Optimal Bidirectional Long Short-Term Memory with Self Attention Mechanism (OBiLSTM-SAM) for gene expression data classification. In the initial phase, the database is gathered from online sources. The projected technique is proceeding with three stages pre-processing, Gene extraction, and classification. In the pre-processing phase, the MIN–MAX normalization techniques are considered. After that, gene extraction is achieved with the consideration of Mutual Information Maximization (MIM). The extracted data will be utilized aimed at gene expression data classification. In the classification stage, the OBiLSTM-SAM is utilized. The OBiLSTM-SAM is a grouping of Bidirectional Long Short-Term Memory with Self Attention Mechanism (BiLSTM-SAM) and Remora Optimization Algorithm (ROA). In the BiLSTM-SAM, the weight parameters are selected with the assistance of ROA. Based on the ODSNN, gene expression data classification is achieved. The presentation of the proposed methodology is evaluated in f-measure, recall, precision, specificity, sensitivity, and accuracy. To validate the projected technique, it is compared with optimized long short-term memory, algorithm (OLSTM) and Adaptive Salp Swarm Optimization algorithm (ASSA) respectively. [ABSTRACT FROM AUTHOR]

Building change detection (BCD) from remote sensing images is an essential field for urban studies. In this well-developed field, Convolutional Neural Networks (CNNs) and Transformer have been leveraged to empower BCD models in handling multi-scale information. However, it is still challenging to accurately detect subtle changes using current models, which has been the main bottleneck to improving detection accuracy. In this paper, a multi-scale differential feature self-attention network (MDFA-Net) is proposed to effectively integrate CNN and Transformer by balancing the global receptive field from the self-attention mechanism and the local receptive field from convolutions. In MDFA-Net, two innovative modules were designed. Particularly, a hierarchical multi-scale dilated convolution (HMDConv) module was proposed to extract local features with hybrid dilation convolutions, which can ameliorate the effect of CNN's local bias. In addition, a differential feature self-attention (DFA) module was developed to implement the self-attention mechanism at multi-scale difference feature maps to overcome the problem that local details may be lost in the global receptive field in Transformer. The proposed MDFA-Net achieves state-of-the-art accuracy performance in comparison with related works, e.g., USSFC-Net, in three open datasets: WHU-CD, CDD-CD, and LEVIR-CD. Based on the experimental results, MDFA-Net significantly exceeds other models in F1 score, IoU, and overall accuracy; the F1 score is 93.81%, 95.52%, and 91.21% in WHU-CD, CDD-CD, and LEVIR-CD datasets, respectively. Furthermore, MDFA-Net achieved first or second place in precision and recall in the test in all three datasets, which indicates its better balance in precision and recall than other models. We also found that subtle changes, i.e., small-sized building changes and irregular boundary changes, are better detected thanks to the introduction of HMDConv and DFA. To this end, with its better ability to leverage multi-scale differential information than traditional methods, MDFA-Net provides a novel and effective avenue to integrate CNN and Transformer in BCD. Further studies could focus on improving the model's insensitivity to hyper-parameters and the model's generalizability in practical applications. [ABSTRACT FROM AUTHOR]

Effective semantic segmentation of Airborne Laser Scanning (ALS) point clouds is a crucial field of study and influences subsequent point cloud application tasks. Transformer networks have made significant progress in 2D/3D computer vision tasks, exhibiting superior performance. We propose a multilevel geometric feature embedding transformer network (MGFE-T), which aims to fully utilize the three-dimensional structural information carried by point clouds and enhance transformer performance in ALS point cloud semantic segmentation. In the encoding stage, compute the geometric features surrounding tee sampling points at each layer and embed them into the transformer workflow. To ensure that the receptive field of the self-attention mechanism and the geometric computation domain can maintain a consistent scale at each layer, we propose a fixed-radius dilated KNN (FR-DKNN) search method to address the limitation of traditional KNN search methods in considering domain radius. In the decoding stage, we aggregate prediction deviations at each level into a unified loss value, enabling multilevel supervision to improve the network's feature learning ability at different levels. The MGFE-T network can predict the class label of each point in an end-to-end manner. Experiments were conducted on three widely used benchmark datasets. The results indicate that the MGFE-T network achieves superior OA and mF1 scores on the LASDU and DFC2019 datasets and performs well on the ISPRS dataset with imbalanced classes. [ABSTRACT FROM AUTHOR]

To address the issue of missing near-surface air temperature data caused by the uneven distribution of ground meteorological observation stations, we propose a method for near-surface air temperature estimation based on an improved conditional generative adversarial network (CGAN) framework. Leveraging the all-weather coverage advantage of Fengyun meteorological satellites, Fengyun-4A (FY-4A) satellite remote sensing data are utilized as conditional guiding information for the CGAN, helping to direct and constrain the near-surface air temperature estimation process. In the proposed network model of the method based on the conditional generative adversarial network structure, the generator combining a self-attention mechanism and cascaded residual blocks is designed with U-Net as the backbone, which extracts implicit feature information and suppresses the irrelevant information in the Fengyun satellite data. Furthermore, a discriminator with multi-level and multi-scale spatial feature fusion is constructed to enhance the network's perception of details and the global structure, enabling accurate air temperature estimation. The experimental results demonstrate that, compared with Attention U-Net, Pix2pix, and other deep learning models, the method presents significant improvements of 68.75% and 10.53%, respectively in the root mean square error (RMSE) and Pearson's correlation coefficient (CC). These results indicate the superior performance of the proposed model for near-surface air temperature estimation. [ABSTRACT FROM AUTHOR]

The prediction of fuel consumption and Carbon Intensity Index (CII) of ships is crucial for optimizing decarbonization strategies in the maritime industry. This study proposes a ship fuel consumption prediction model based on the Long Short-Term Memory with Self-Attention Mechanism (SA-LSTM). The model is applied to a container ship of 2400 TEU to predict its hourly fuel consumption, hourly CII, and annual CII rating. Four different feature sets are selected from these data sources and are used as inputs for SA-LSTM and another ten models. The results demonstrate that the SA-LSTM model outperforms the other models in prediction accuracy. Specifically, the Mean Absolute Percentage Error (MAPE) for fuel consumption predictions using the SA-LSTM model is reduced by up to 20% compared to the XGBoost and by up to 12% compared to the LSTM model. Additionally, the SA-LSTM model achieves the highest accuracy in annual CII predictions. [ABSTRACT FROM AUTHOR]

The state of health (SOH) of lithium-ion batteries is a critical parameter of the battery management system. An accurate estimation of it can improve the battery's overall life. The data-driven method has the problems of low long-term prediction accuracy and difficult feature recognition. Therefore, this paper proposes a SOH estimation method based on the convolutional gated recurrent neural network with the self-attention mechanism. Firstly, the convolutional neural network (CNN) is applied to the input data, and its convolution operation is used to extract important local features. Then, the self-attention mechanism is added later to give greater weight to the more important features in the training process. Finally, the gated recurrent unit (GRU) recurrent neural network extracts the long-term correlation from the weighted output data. The experimental data of lithium-ion batteries from three sources are used to verify the correctness and effectiveness of this method. Experiments were based on whether the self-attention mechanism is present, different prediction starting points and ambient temperature, and analyzed and compared with the common machine learning methods. The experimental results show that the method exhibits good estimation accuracy, with a maximum mean absolute error below 1.78% and a maximum root mean square error below 2.3%. [ABSTRACT FROM AUTHOR]

Point of interest (POI) recommendation is one of the most important applications of location-based social networking (LBSN). Existing studies have proposed methods that utilize POI information and spatio-temporal information for recommendation, but the relevant auxiliary information has not been fully utilized, so it can not alleviate the problems of insufficient information and less travel data caused by short track check-in of users. To solve these problems, a friends relationship and selfattention recommendation model ATFR is proposed. The prediction model consists of three parts: Firstly, the representation vector of friend relationship is obtained by graph embedding method and input into the neural network. Secondly, the user interest preference vector is obtained by GRU. Then, the self-attention mechanism is used to model the sequential and social effects of the user check-in sequence and selectively focus on the historical check-in records associated with the check-in sequence. Finally, the future interest points are recommended according to the interest points sorting list. Experimental results on two public datasets show that this model performs better than other algorithms and can be used to improve the quality of personalized interest recommendation services for websites and applications. [ABSTRACT FROM AUTHOR]

The global dilemma of informal settlements persists alongside the fast process of urbanization. Various methods for analyzing remotely sensed images to identify informal settlements using semantic segmentation have been extensively researched, resulting in the development of numerous supervised and unsupervised algorithms. Texture-based analysis is a topic extensively studied in the literature. However, it is important to note that approaches that do not utilize a multiresolution strategy are unable to take advantage of the fact that texture exists at different spatial scales. The capacity to do online mapping and precise segmentation on a vast scale while considering the diverse characteristics present in remotely sensed images carries significant consequences. This research presents a novel approach for identifying informal settlements using multiresolution analysis and self-attention techniques. The technique shows potential for being resilient in the presence of inherent variability in remotely sensed images due to its capacity to extract characteristics at many scales and prioritize areas that contain significant information. Segmented pictures underwent an accuracy assessment, where a comparison analysis was conducted based on metrics such as mean intersection over union, precision, recall, F-score, and overall accuracy. The proposed method's robustness is demonstrated by comparing it to various state-of-the-art techniques. This comparison is conducted using remotely sensed images that have different spatial resolutions and informal settlement characteristics. The proposed method achieves a higher accuracy of approximately 95%, even when dealing with significantly different image characteristics. [ABSTRACT FROM AUTHOR]

Hyperspectral unmixing (HU) aims to decompose mixed pixels into a set of endmembers and corresponding abundances. Deep learning-based HU methods are currently a hot research topic, but most existing unmixing methods still rely on per-pixel training or employ convolutional neural networks (CNNs), which overlook the non-local correlations of materials and spectral characteristics. Furthermore, current research mainly focuses on linear mixing models, which limits the feature extraction capability of deep encoders and further improvement in unmixing accuracy. In this paper, we propose a nonlinear unmixing network capable of extracting global spatial-spectral features. The network is designed based on an autoencoder architecture, where a dual-stream CNNs is employed in the encoder to separately extract spectral and local spatial information. The extracted features are then fused together to form a more complete representation of the input data. Subsequently, a linear projection-based multi-head self-attention mechanism is applied to capture global contextual information, allowing for comprehensive spatial information extraction while maintaining lightweight computation. To achieve better reconstruction performance, a model-free nonlinear mixing approach is adopted to enhance the model's universality, with the mixing model learned entirely from the data. Additionally, an initialization method based on endmember bundles is utilized to reduce interference from outliers and noise. Comparative results on real datasets against several state-of-the-art unmixing methods demonstrate the superior of the proposed approach. [ABSTRACT FROM AUTHOR]

Deep learning and the Internet of Things (IoT) are rapidly advancing technologies with wide-ranging applications, notably in areas such as face detection. There is a growing need for sophisticated touchless authentication systems within this domain. While numerous security methods are available, many of them suffer from various shortcomings, including issues like forgotten passwords and the potential transmission of diseases through touch-based authentication techniques. This paper proposes IoT-based intelligent control face detection systems that leverage deep learning (DL) models. These systems are designed to significantly enhance security without relying on manually crafted rules. Face detection plays a pivotal role in safeguarding society, identifying wrongdoers, and bolstering community safety. However, challenges persist, particularly in achieving high accuracy with face recognition control systems in uncontrolled environments and real-time scenarios, such as intersections. In response to these challenges, our paper introduces a novel model that harnesses a combination of deep convolutional neural networks (CNNs) with self-attention mechanisms. Our experimental results demonstrate that this model can rapidly detect whole-scale images in a single forward pass. Notably, our proposed method achieved an outstanding accuracy rate of 99.7%. In comparison to existing state-of-the-art methods, our approach exhibits superior efficiency. This work showcases how the integration of IoT and DL, particularly with the use of a CNN with self-attention, outperforms other CNN-based approaches in terms of both speed and accuracy. [ABSTRACT FROM AUTHOR]

As the foundation of circuits and electronic systems, the quality of printed circuit boards (PCBs) is directly related to the reliability of the whole system. At present, deep learning-based target detection technology has a broad application prospect in PCB defect detection. However, there are still some shortcomings in the process of detection. For example, the detection speed is slow, and the detection accuracy is not ideal for defects with small size. Therefore, this paper proposes an improved algorithm based on the YOLOv5s model to improve the detection of PCB defects. The algorithm enhances the connection between the shallow and high layers of the network by improving the backbone network structure. Meanwhile, a self-attention mechanism is introduced into YoloHead to enhance the feature extraction and the ability to focus on the global information of the feature map, and the self-attention mechanism is optimized by combining the computational characteristics of the SENet, the CBAM and the residual unit. While improving the accuracy of PCB defect detection, better computational efficiency is ensured. In subsequent experiments, this paper trains and tests the improved YOLOv5s algorithm using publicly available PCB defect datasets. The results show that the algorithm achieves a detection rate of 39.7 frames per second and a maximum average precision of 96.14%, which is a 4.68% improvement over the original YOLOv5s. [ABSTRACT FROM AUTHOR]

Prediction of Remaining Useful Life (RUL) for Rolling Element Bearings (REB) has attracted widespread attention from academia and industry. However, there are still several bottlenecks, including the effective utilization of multi-sensor data, the interpretability of prediction models, and the prediction across the entire life cycle, which limit prediction accuracy. In view of that, we propose a knowledge-based explainable life-cycle RUL prediction framework. First, considering the feature fusion of fast-changing signals, the Pearson correlation coefficient matrix and feature transformation objective function are incorporated to an Improved Graph Convolutional Autoencoder. Furthermore, to integrate the multi-source signals, a Cascaded Multi-head Self-attention Autoencoder with Characteristic Guidance is proposed to construct health indicators. Then, the whole life cycle of REB is divided into different stages based on the Continuous Gradient Recognition with Outlier Detection. With the development of Measurement-based Correction Life Formula and Bidirectional Recursive Gated Dual Attention Unit, accurate life-cycle RUL prediction is achieved. Data from self-designed test rig and PHM 2012 Prognostic challenge datasets are analyzed with the proposed framework and five existing prediction models. Compared with the strongest prediction model among the five, the proposed framework demonstrates significant improvements. For the data from self-designed test rig, there is a 1.66 % enhancement in Corrected Cumulative Relative Accuracy (CCRA) and a 49.00 % improvement in Coefficient of Determination (R2). For the PHM 2012 datasets, there is a 4.04 % increase in CCRA and a 120.72 % boost in R2. [Display omitted] • Advocate explainable knowledge-data integration for RUL prediction throughout the life cycle, leveraging multi-sensor data. • Introduce a novel unsupervised learning framework that integrates IGCA and CMSACG to construct HI. • A two-stage prediction framework, fusing MCLF and BR-GDAU, is proposed for predicting the life cycle RUL. • The effectiveness of the proposed framework is validated using both Self-Designed Experiment-Derived and PHM 2012 Prognostic Challenge Bearing Datasets. [ABSTRACT FROM AUTHOR]

The accurate estimation of battery state of health (SOH) is critical for ensuring the safety and reliability of devices. Considering the variation in health degradation across different types of lithium-ion battery materials, this paper proposes an SOH estimation method based on a graph perceptual neural network, designed to adapt to multiple battery materials. This method adapts to various battery materials by extracting crucial features from current, voltage, voltage–capacity, and temperature data, and it constructs a graph structure to encapsulate these features. This approach effectively captures the complex interactions and dependencies among different battery types. The novel technique of randomly removing features addresses feature redundancy. Initially, a mutual information graph structure is defined to illustrate the interdependencies among battery features. Moreover, a graph perceptual self-attention mechanism is implemented, integrating the adjacency matrix and edge features into the self-attention calculations. This enhancement aids the model's understanding of battery behaviors, thereby improving the transparency and interpretability of predictions. The experimental results demonstrate that this method outperforms traditional models in both accuracy and generalizability across various battery types, particularly those with significant chemical and degradation discrepancies. The model achieves a minimum mean absolute error of 0.357, a root mean square error of 0.560, and a maximum error of 0.941. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Guangxi Normal University - Natural Science Edition is the property of Gai Kan Bian Wei Hui and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Copyright of Journal of Shenzhen University Science & Engineering is the property of Editorial Department of Journal of Shenzhen University Science & Engineering and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Copyright of Journal of Computer Engineering & Applications is the property of Beijing Journal of Computer Engineering & Applications Journal Co Ltd. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Conducting the remaining useful life (RUL) prediction for an aircraft engines is of significant importance in enhancing aircraft operation safety and formulating reasonable maintenance plans. Addressing the issue of low prediction model accuracy due to traditional neural networks' inability to fully extract key features, this paper proposes an engine RUL prediction model based on the adaptive moment estimation (Adam) optimized self-attention mechanism–temporal convolutional network (SAM-TCN) neural network. Firstly, the raw data monitored by sensors are normalized, and RUL labels are set. A sliding window is utilized for overlapping sampling of the data, capturing more temporal features while eliminating data dimensionality. Secondly, the SAM-TCN neural network prediction model is constructed. The temporal convolutional network (TCN) neural network is used to capture the temporal dependency between data, solving the mapping relationship of engine degradation characteristics. A self-attention mechanism (SAM) is employed to adaptively assign different weight contributions to different input features. In the experiments, the root mean square error (RMSE) values on four datasets are 11.50, 16.45, 11.62, and 15.47 respectively. These values indicate further reduction in errors compared to methods reported in other literature. Finally, the SAM-TCN prediction model is optimized using the Adam optimizer to improve the training effectiveness and convergence speed of the model. Experimental results demonstrate that the proposed method can effectively learn feature data, with prediction accuracy superior to other models. [ABSTRACT FROM AUTHOR]

Cameras and LiDAR are important sensors in autonomous driving systems that can provide complementary information to each other. However, most LiDAR-only methods outperform the fusion method on the main benchmark datasets. Current studies attribute the reasons for this to misalignment of views and difficulty in matching heterogeneous features. Specially, using the single-stage fusion method, it is difficult to fully fuse the features of the image and point cloud. In this work, we propose a 3D object detection network based on the multi-layer and multi-modal fusion (3DMMF) method. 3DMMF works by painting and encoding the point cloud in the frustum proposed by the 2D object detection network. Then, the painted point cloud is fed to the LiDAR-only object detection network, which has expanded channels and a self-attention mechanism module. Finally, the camera-LiDAR object candidates fusion for 3D object detection(CLOCs) method is used to match the geometric direction features and category semantic features of the 2D and 3D detection results. Experiments on the KITTI dataset (a public dataset) show that this fusion method has a significant improvement over the baseline of the LiDAR-only method, with an average mAP improvement of 6.3%. [ABSTRACT FROM AUTHOR]

Deep learning (DL) has brought new perspectives and methods to automatic modulation recognition (AMR), enabling AMR systems to operate more efficiently and reliably in modern wireless communication environments through its powerful feature learning and complex pattern recognition capabilities. However, convolutional neural networks (CNNs) and recurrent neural networks (RNNs), which are used for sequence recognition tasks, face two main challenges, respectively: the ineffective utilization of global information and slow processing speeds due to sequential operations. To address these issues, this paper introduces CTRNet, a novel automatic modulation recognition network that combines a CNN with Transformer. This combination leverages Transformer's ability to adequately capture the long-distance dependencies between global sequences and its advantages in sequence modeling, along with the CNN's capability to extract features from local feature regions of signals. During the data preprocessing stage, the original IQ-modulated signals undergo sliding-window processing. By selecting the appropriate window sizes and strides, multiple subsequences are formed, enabling the network to effectively handle complex modulation patterns. In the embedding module, token vectors are designed to integrate information from multiple samples within each window, enhancing the model's understanding and modeling ability of global information. In the feedforward neural network, a more effective Bilinear layer is employed for processing to capture the higher-order relationship between input features, thereby enhancing the ability of the model to capture complex patterns. Experiments conducted on the RML2016.10A public dataset demonstrate that compared with the existing algorithms, the proposed algorithm not only exhibits significant advantages in terms of parameter efficiency but also achieves higher recognition accuracy under various signal-to-noise ratio (SNR) conditions. In particular, it performs relatively well in terms of accuracy, precision, recall, and F1-score, with clearer classification of higher-order modulations and notable overall accuracy improvement. [ABSTRACT FROM AUTHOR]

The traditional lane detection networks mainly use independent single frame images to extract features first and then detect them, which cannot deal with the scene with complex background well. Therefore, this paper proposes a lane parallel detection network based on multi-frame network feature fusion model and self-attention mechanism according to the scene characteristics that vehicles can obtain continuous images during normal driving. Firstly, a parallel feature extraction structure is designed. On the one hand, a single frame network with high precision is used to extract the features of the current frame. On the other hand, a lightweight multi-frame network is designed to extract features of low-resolution multi-frame temporal images. And the recurrent neural network module is used to fuse the extracted temporal features and obtain multi-frame features. Self-attention mechanism can effectively capture the relevant information of internal features. Then the fusion module of single frame feature, multi-frame feature and self-attention feature is designed. The feature map of lane line is output by up-sampling network. The experimental results show that the network in this paper has significant improvement in both objective detection accuracy and subjective effect compared with other methods. [ABSTRACT FROM AUTHOR]

The proportion of small objects in remote sensing images is relatively small, which is prone to feature loss or interference from the surrounding complex background in the detection process. To solve this problem, a remote sensing small object detection network (CD-YOLOX) based on enhanceds' feature pyramid network (CS-PANet) and global local combine module (DSAN) based on YOLOX is proposed. Firstly, to solve the problem of small object feature loss and surrounding complex background interference caused by multiple convolution and feature stacking operations in PANet, CS-PANet is proposed. This method improves the network's focus on effective feature channels for small objects by using channel attention at the input of PANet, meanwhile, the input that passes the channel attention is connected to the output of PANet across layers, which makes the PANet retain the richer original information of small objects. Secondly, to further reduce the interference of the surrounding complex background on the small objects, a DSAN module consisting of self-attention mechanism, dilated convolution and residual connection is proposed before network prediction. This module combines self-attention mechanism with dilated convolution so that the network focuses on the global feature region of the small object in the feature map while effectively complementing the local context of this region, and preserving the original information through residual connection. Finally, the effectiveness of the method is verified using the remote sensing dataset NWPU VHR-10 and the general datasets KITTI and PASCAL VOC. The experiment shows that the method improves the detection accuracy by 5.56% in the NWPU VHR-10 dataset and by 1.93% and 2.51% in the KITTI and PASCAL VOC datasets respectively compared to the original network, which fully verifies the effectiveness of the method for detecting the small objects of remote sensing and the ability of the method to detect general purpose objects. [ABSTRACT FROM AUTHOR]

In infrared detection scenarios, detecting and recognizing low-contrast and small-sized targets has always been a challenge in the field of computer vision, particularly in complex road traffic environments. Traditional target detection methods usually perform poorly when processing infrared small targets, mainly due to their inability to effectively extract key features and the significant feature loss that occurs during feature transmission. To address these issues, this paper proposes a fast detection and recognition model based on a multi-scale self-attention mechanism, specifically for small road targets in infrared detection scenarios. We first introduce and improve the DyHead structure based on the YOLOv8 algorithm, which employs a multi-head self-attention mechanism to capture target features at various scales and enhance the model's perception of small targets. Additionally, to prevent information loss during the feature transmission process via the FPN structure in traditional YOLO algorithms, this paper introduces and enhances the Gather-and-Distribute Mechanism. By computing dependencies between features using self-attention, it reallocates attention weights in the feature maps to highlight important features and suppress irrelevant information. These improvements significantly enhance the model's capability to detect small targets. Moreover, to further increase detection speed, we pruned the network architecture to reduce computational complexity and parameter count, making the model suitable for real-time processing scenarios. Experiments on our self built infrared road traffic dataset (mainly including two types of targets: vehicles and people) show that compared with the baseline, our method achieves a 3.1% improvement in AP and a 2.5% increase in mAP on the VisDrone2019 dataset, showing significant enhancements in both detection accuracy and processing speed for small targets, with improved robustness and adaptability. [ABSTRACT FROM AUTHOR]

Mine flooding accidents have occurred frequently in recent years, and the predicting of mine water inflow is one of the most crucial flood warning indicators. Further, the mine water inflow is characterized by non-linearity and instability, making it difficult to predict. Accordingly, we propose a time series prediction model based on the fusion of the Transformer algorithm, which relies on self-attention, and the LSTM algorithm, which captures long-term dependencies. In this paper, Baotailong mine water inflow in Heilongjiang Province is used as sample data, and the sample data is divided into different ratios of the training set and test set in order to obtain optimal prediction results. In this study, we demonstrate that the LSTM-Transformer model exhibits the highest training accuracy when the ratio is 7:3. To improve the efficiency of search, the combination of random search and Bayesian optimization is used to determine the network model parameters and regularization parameters. Finally, in order to verify the accuracy of the LSTM-Transformer model, the LSTM-Transformer model is compared with LSTM, CNN, Transformer and CNN–LSTM models. The results prove that LSTM-Transformer has the highest prediction accuracy, and all the indicators of its model are well improved. [ABSTRACT FROM AUTHOR]

The ability to extract high‐quality features from data is critical for machine learning applications. With the development of deep learning, various methods have been developed for image feature extraction, and unsupervised techniques have gained popularity due to their ability to operate without response variables. Autoencoders with encoder–decoder architectures are a common example of such techniques, but they are limited by a lack of proportional relationship between model reconstruction and encoder feature extraction performance. If the decoder is composed of multiple layers and mapping to a higher dimension is easier, the feature extraction performance of the encoder is likely to decrease. However, previous research has not adequately addressed this limitation. This study identifies the limitations of conventional unsupervised feature extraction techniques that utilize the encoder–decoder architecture, and proposes a novel feature extraction technique called SAFE, which utilizes a self‐attention mechanism to eliminate decoder effects and improve the performance of encoder. To validate the effectiveness of the proposed model, we conducted experiments using diverse datasets (MNIST, Fashion MNIST, SVHN, and WM811K). The results of the experiments demonstrated that our proposed method exhibited, on average, 2%–10% higher performance in terms of accuracy and F‐measure compared to the existing feature extraction techniques in the classification problem. While our research has limitations, specifically in its applicability only to the selection of image features, future studies should be undertaken to explore its potential application in various fields. [ABSTRACT FROM AUTHOR]

Sequence recommendation is a prominent research area within recommender systems, focused on predicting items that users may be interested in by modeling their historical interaction sequences. However, due to data sparsity, user interaction sequences in sequence recommendation are typically short. A common approach to address this issue is filling sequences with zero values, significantly reducing the effective utilization of input space. Furthermore, traditional sequence recommendation methods based on self-attention mechanisms exhibit quadratic complexity with respect to sequence length. These issues affect the performance of recommendation algorithms. To tackle these challenges, we propose a multi-task sequence recommendation model, Blin, which integrates bidirectional KL divergence and linear attention. Blin abandons the conventional zero-padding strategy, opting instead for random repeat padding to enhance sequence data. Additionally, bidirectional KL divergence loss is introduced as an auxiliary task to regularize the probability distributions obtained from different sequence representations. To improve the computational efficiency compared to traditional attention mechanisms, a linear attention mechanism is employed during sequence encoding, significantly reducing the computational complexity while preserving the learning capacity of traditional attention. Experimental results on multiple public datasets demonstrate the effectiveness of the proposed model. [ABSTRACT FROM AUTHOR]

In the unsupervised domain adaptation (UDA) (Akada et al. Self-supervised learning of domain invariant features for depth estimation, in: 2022 IEEE/CVF winter conference on applications of computer vision (WACV), pp 3377–3387 (2022). ) depth estimation task, a new adaptive approach is to use the bidirectional transformation network to transfer the style between the target and source domain inputs, and then train the depth estimation network in their respective domains. However, the domain adaptation process and the style transfer may result in defects and biases, often leading to depth holes and instance edge depth missing in the target domain’s depth output. To address these issues, We propose a training network that has been improved in terms of model structure and supervision constraints. First, we introduce a edge-guided self-attention mechanism in the task network of each domain to enhance the network’s attention to high-frequency edge features, maintain clear boundaries and fill in missing areas of depth. Furthermore, we utilize an edge detection algorithm to extract edge features from the input of the target domain. Then we establish edge consistency constraints between inter-domain entities in order to narrow the gap between domains and make domain-to-domain transfers easier. Our experimental demonstrate that our proposed method effectively solve the aforementioned problem, resulting in a higher quality depth map and outperforming existing state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Computer-Aided Design & Computer Graphics / Jisuanji Fuzhu Sheji Yu Tuxingxue Xuebao is the property of Gai Kan Bian Wei Hui and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)