Search

Copyright of Electric Power is the property of Electric Power Editorial Office and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Knowledge graphs (KGs) play a vital role in enhancing search results and recommendation systems. With the rapid increase in the size of KGs, they are becoming inaccurate and incomplete. This problem can be solved by the KG completion methods, of which graph attention network (GAT)-based methods stand out because of their superior performance. However, existing GAT-based KG completion methods often suffer from overfitting issues when dealing with heterogeneous KGs, primarily due to the unbalanced number of samples. Additionally, these methods demonstrate poor performance in predicting the tail (head) entity that shares the same relation and head (tail) entity with others. To solve these problems, we propose GATH, a novel GAT-based method designed for Heterogeneous KGs. GATH incorporates two separate attention network modules that work synergistically to predict the missing entities. We also introduce novel encoding and feature transformation approaches, enabling the robust performance of GATH in scenarios with imbalanced samples. Comprehensive experiments are conducted to evaluate GATH's performance. Compared with the existing state-of-the-art GAT-based model on Hits@10 and MRR metrics, our model improves performance by 5.2% and 5.2% on the FB15K-237 dataset and by 4.5% and 14.6% on the WN18RR dataset, respectively. [ABSTRACT FROM AUTHOR]

Background: Motif finding in Assay for Transposase-Accessible Chromatin using sequencing (ATAC-seq) data is essential to reveal the intricacies of transcription factor binding sites (TFBSs) and their pivotal roles in gene regulation. Deep learning technologies including convolutional neural networks (CNNs) and graph neural networks (GNNs), have achieved success in finding ATAC-seq motifs. However, CNN-based methods are limited by the fixed width of the convolutional kernel, which makes it difficult to find multiple transcription factor binding sites with different lengths. GNN-based methods has the limitation of using the edge weight information directly, makes it difficult to aggregate the neighboring nodes' information more efficiently when representing node embedding. Results: To address this challenge, we developed a novel graph attention network framework named MMGAT, which employs an attention mechanism to adjust the attention coefficients among different nodes. And then MMGAT finds multiple ATAC-seq motifs based on the attention coefficients of sequence nodes and k-mer nodes as well as the coexisting probability of k-mers. Our approach achieved better performance on the human ATAC-seq datasets compared to existing tools, as evidenced the highest scores on the precision, recall, F1_score, ACC, AUC, and PRC metrics, as well as finding 389 higher quality motifs. To validate the performance of MMGAT in predicting TFBSs and finding motifs on more datasets, we enlarged the number of the human ATAC-seq datasets to 180 and newly integrated 80 mouse ATAC-seq datasets for multi-species experimental validation. Specifically on the mouse ATAC-seq dataset, MMGAT also achieved the highest scores on six metrics and found 356 higher-quality motifs. To facilitate researchers in utilizing MMGAT, we have also developed a user-friendly web server named MMGAT-S that hosts the MMGAT method and ATAC-seq motif finding results. Conclusions: The advanced methodology MMGAT provides a robust tool for finding ATAC-seq motifs, and the comprehensive server MMGAT-S makes a significant contribution to genomics research. The open-source code of MMGAT can be found at https://github.com/xiaotianr/MMGAT, and MMGAT-S is freely available at https://www.mmgraphws.com/MMGAT-S/. [ABSTRACT FROM AUTHOR]

In the realm of aspect-based sentiment analysis (ABSA), a paramount task is the extraction of triplets, which define aspect terms, opinion terms, and their respective sentiment orientations within text. This study introduces a novel extraction model, BiLSTM-BGAT-GCN, which seamlessly integrates graph neural networks with an enhanced biaffine attention mechanism. This model amalgamates the sophisticated capabilities of both graph attention and convolutional networks to process graph-structured data, substantially enhancing the interpretation and extraction of textual features. By optimizing the biaffine attention mechanism, the model adeptly uncovers the subtle interplay between aspect terms and emotional expressions, offering enhanced flexibility and superior contextual analysis through dynamic weight distribution. A series of comparative experiments confirm the model's significant performance improvements across various metrics, underscoring its efficacy and refined effectiveness in ABSA tasks. [ABSTRACT FROM AUTHOR]

The frequent occurrence of extreme heavy rainfall in cities has posed a severe threat to the personal and property safety of residents due to urban flooding. Accurate and efficient prediction of flooding areas within cities plays a crucial role in enhancing urban disaster emergency response capabilities. In order to improve the accuracy and intuitiveness of urban flooding area predictions, this article proposed a combination prediction model called GATLSTM, based on GAT (Graph Attention Network) and LSTM (Long Short-Time Memory). The GAT was used to extract local spatial features of flooding information, and it enhanced the memory of key information sequences by assigning weights to nodes. Subsequently, LSTM was employed to extract temporal features of flooding area sequences and predicted the flooding areas at inundation points for the next 10 minutes. The model was built and evaluated by using inundation data from a specific point in Kaifeng City. It was compared with LSTM, GAT and GCNLSTM models. The results indicate that the GATLSTM model outperforms the other three models in terms of prediction accuracy. It can accurately forecast flooding areas at inundation points in the short term, providing a scientific basis for flood prevention efforts and emergency response measures. [ABSTRACT FROM AUTHOR]

Graph-structured data is ubiquitous in real-world applications, such as social networks, citation networks, and communication networks. Graph neural network (GNN) is the key to process them. In recent years, graph attention networks (GATs) have been proposed for node classification and achieved encouraging performance. It focuses on the content associated on nodes to evaluate the attention weights, and the rich structure information in the graph is almost ignored. Therefore, we propose a multi-head attention mechanism to fully employ node content and graph structure information. The core idea is to introduce the interactions in the topological structure into the existing GATs. This method can more accurately estimate the attention weights among nodes, thereby improving the convergence of GATs. Second, the mechanism is lightweight and efficient, requires no training to learn, can accurately analyze higher-order structural information, and can be strongly interpreted through heatmaps. We name the proposed model content- and structure-based graph attention network (CSGAT). Furthermore, our proposed model achieves state-of-the-art performance on a number of datasets in node classification. The code and data are available at https://github.com/CroakerShark/CSGAT. [ABSTRACT FROM AUTHOR]

The online calibration method for current transformers is an important research direction in the field of smart grids. This article constructs a transformer error prediction model based on spatiotemporal integration. This model draws inspiration from the structure of forgetting gates in gated loop units and combines it with a graph convolutional network (GCN) that is good at capturing the spatial relationships within the graph attention network to construct an adaptive GCN. The spatial module formed by this adaptive GCN is used to model the spatial relationships in the circuit network, and the attention mechanism and gated time convolutional network are combined to form a time module to learn the temporal relationships in the circuit network. The layer that combines the time and space modules is used, which consists of a gating mechanism for spatiotemporal fusion, and a transformer error prediction model based on a spatiotemporal correlation analysis is constructed. Finally, it is verified on a real power grid operation dataset, and compared with the existing prediction methods to analyze its performance. [ABSTRACT FROM AUTHOR]

In order to solve the problem that the existing log anomaly detection methods tend to focus only on the single feature of the quantitative relationship mode or the sequential mode, ignoring the relationship of the log time structure and the interrelation between different features, resulting in a high error detection rate and false positive rate, this paper proposed a log anomaly detection method based on the log time graph attention network. Firstly, this paper constructed a log time graph by designing a joint feature extraction module of log semantics and time structure, which effectively integrated the time structure relationship and semantic information of log. Secondly, it constructed the time relationship graph attention network, and used the graph structure to describe the time structure relationship between logs, which could adaptively learn the importance of different logs and carry out anomaly detection. Finally, it used three public datasets to verify the effectiveness of the model. Extensive experiments results indicate that the proposed method is able to effectively capture the temporal structure relationships in the logs, thereby improving the accuracy of anomaly detection. [ABSTRACT FROM AUTHOR]

Sentiment classification and emotion recognition are two close related tasks in NLP. However, most of the recent studies have treated them as two separate tasks, where the shared knowledge are neglected. In this paper, we propose a multi-task interactive graph attention network with position encodings, termed MIP-GAT, to improve the performance of each task by simultaneously leveraging similarities and differences. The main proposal is a multi-interactive graph interaction layer where a syntactic dependency connection , a cross-task connection and position encodings are constructed and incorporated into a unified graphical structure. Empirical evaluation on two benchmarking datasets, i.e., CMU-MOSEI and GoEmotions, shows the effectiveness of the proposed model over state-of-the-art baselines with the margin of 0.18%, 0.67% for sentiment analysis, 1.77%, 0.89% for emotion recognition. In addition, we also explore the superiority and limitations of the proposed model. • Sentiment classification and emotion recognition are two correlative tasks in NLP. • But they are often treated as separative tasks. • In contrast to differences, commonalities of sentiment and emotion are often ignored. • Keys in the task: syntactic knowledge, cross-task interaction, position information. • Simultaneously incorporating the three key factors into a graph structure. • Performance boosts by the interaction of two tasks in a unified graphical structure. [ABSTRACT FROM AUTHOR]

In response to the problem of aspect-level sentiment analysis uses attention mechanism and traditional deep learning methods to extract the relations between aspect words and contextual words at present, which do not fully considers syntactic dependency information and relational labels, resulting the worse the predicted effects, an relational graph attention network of aspect-level sentiment analysis was proposed. Firstly, pre-training model DeBERTa was used to get word embedding and initial word vector was used for multi-head attention to enhance the relationship between aspect words and contextual words. Then, relational labels were learned through graph attention networks. The relationship between aspects and contextual words can be further extracted with these relational label features, which improve ability of the model to extract sentimental features. Experimental results on the SemEval-2014 dataset show that the model outperforms the comparison model in both accuracy and Macro-F1. [ABSTRACT FROM AUTHOR]

Drug combination therapy is generally more effective than monotherapy in the field of cancer treatment. However, screening for effective synergistic combinations from a wide range of drug combinations is particularly important given the increase in the number of available drug classes and potential drug-drug interactions. Existing methods for predicting the synergistic effects of drug combinations primarily focus on extracting structural features of drug molecules and cell lines, but neglect the interaction mechanisms between cell lines and drug combinations. Consequently, there is a deficiency in comprehensive understanding of the synergistic effects of drug combinations. To address this issue, we propose a drug combination synergy prediction model based on multi-source feature interaction learning, named MFSynDCP, aiming to predict the synergistic effects of anti-tumor drug combinations. This model includes a graph aggregation module with an adaptive attention mechanism for learning drug interactions and a multi-source feature interaction learning controller for managing information transfer between different data sources, accommodating both drug and cell line features. Comparative studies with benchmark datasets demonstrate MFSynDCP's superiority over existing methods. Additionally, its adaptive attention mechanism graph aggregation module identifies drug chemical substructures crucial to the synergy mechanism. Overall, MFSynDCP is a robust tool for predicting synergistic drug combinations. The source code is available from GitHub at https://github.com/kkioplkg/MFSynDCP. [ABSTRACT FROM AUTHOR]

Recently, large-scale artificial intelligence models with billions of parameters have achieved good results in experiments, but their practical deployment on edge computing platforms is often subject to many constraints because of their resource requirements. These models require powerful computing platforms with a high memory capacity to store and process the numerous parameters and activations, which makes it challenging to deploy these large-scale models directly. Therefore, model compression techniques are crucial role in making these models more practical and accessible. In this article, a progressive channel pruning strategy combining graph attention network and transformer, namely GAT TransPruning, is proposed, which uses the graph attention networks (GAT) and the attention of transformer mechanism to determine the channel-to-channel relationship in large networks. This approach ensures that the network maintains its critical functional connections and optimizes the trade-off between model size and performance. In this study, VGG-16, VGG-19, ResNet-18, ResNet-34, and ResNet-50 are used as large-scale network models with the CIFAR-10 and CIFAR-100 datasets for verification and quantitative analysis of the proposed progressive channel pruning strategy. The experimental results reveal that the accuracy rate only drops by 6.58% when the channel pruning rate is 89% for VGG-19/CIFAR-100. In addition, the lightweight model inference speed is 9.10 times faster than that of the original large model. In comparison with the traditional channel pruning schemes, the proposed progressive channel pruning strategy based on the GAT and Transformer cannot only cut out the insignificant weight channels and effectively reduce the model size, but also ensure that the performance drop rate of its lightweight model is still the smallest even under high pruning ratio. [ABSTRACT FROM AUTHOR]

Point cloud super-resolution plays a central role in the mesh's quality in 3D reconstruction, while the feature extractor is vital for the learning-based point cloud upsampling pipelines. In this paper, we propose an arbitrary 3D point cloud upsampling network (PSR-GAT), which comprises the feature extraction module, GAT module, and upsampling module. For the input point cloud, the feature extraction module locates k nearest points of each point in 3D space by k-NN algorithm, then converts the local geometry information into high dimensional feature space through a multi-layer point-wise convolution. The GAT module converts the local geometry feature of each point into the semantic feature through a multi-layer graph attention network. The module dynamically adjusts the neighbor space of the point in each layer to increase the receptive field range and effectively fuses the semantic information of different levels through residual connection. This makes the local geometric in- formation extraction efficient. The upsampling module adds the number of points and maps them from feature space to 3D space. Extensive experimental results show that PSR-GAT exhibits a better performance than the existing state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Single-cell ribonucleic acid sequencing (scRNA-seq) allows researchers to study cell heterogeneity and diversity at the individual cell level. Cell clustering is an essential component of scRNA-seq data processing. However, the high dimensionality and high noise characteristics of scRNA-seq data may pose problems during data processing. Although many methods are available for scRNA-seq clustering analysis, most of them ignore the topological relationships of scRNA-seq data and do not fully utilize the potential associations between cells. In this study, we present scGAD, a graph attention autoencoder model with a dual decoder structure for clustering scRNA-seq data. We utilize a graph attention autoencoder with two decoders to learn feature representations of cells in latent space. To ensure that the learned latent feature representation maintains node properties and graph structure, we use an inner product decoder and a learnable graph attention decoder to reconstruct graph structure and node properties, respectively. On the 12 real scRNA-seq datasets, the average NMI and ARI scores of scGAD are 0.762 and 0.695, respectively, outperforming state-of-the-art single-cell clustering approaches. Biological analysis shows that the cell labels predicted by scGAD can assist in the downstream analysis of scRNA-seq data. [ABSTRACT FROM AUTHOR]

In wireless networks, the effectiveness of beamforming and power allocation strategies is crucial in meeting the increasing data demands of users and ensuring rapid data transmission. Graph learning approaches have been developed to tackle complex challenges in wireless communications and have shown promising results. However, most existing graph learning methods primarily focus on node features, neglecting the potential benefits of leveraging rich information from edge features. This study addresses this limitation and proposes a novel framework called Heterogeneous Node and Edge Graph Neural Network (HNENN). Specifically designed for heterogeneous networks, HNENN incorporates node-level and edge-level attention layers to learn and aggregate node and edge embeddings. The alternating stacking of these two layers facilitates the mutual enhancement of node and edge embeddings. Simulations show that the proposed framework works better than state-of-the-art approaches, getting a higher sum rate in different scenarios with different numbers of D2D pairs, training samples, interference levels, and transmit power budgets. [ABSTRACT FROM AUTHOR]

The construction of medical knowledge graphs (MKGs) is steadily progressing from manual to automatic methods, which inevitably introduce noise, which could impair the performance of downstream healthcare applications. Existing error detection approaches depend on the topological structure and external labels of entities in MKGs to improve their quality. Nevertheless, due to the cost of manual annotation and imperfect automatic algorithms, precise entity labels in MKGs cannot be readily obtained. To address these issues, we propose an approach named Enhancing error detection on Medical knowledge graphs via intrinsic labEL (EMKGEL). Considering the absence of hyper-view KG, we establish a hyper-view KG and a triplet-level KG for implicit label information and neighborhood information, respectively. Inspired by the success of graph attention networks (GATs), we introduce the hyper-view GAT to incorporate label messages and neighborhood information into representation learning. We leverage a confidence score that combines local and global trustworthiness to estimate the triplets. To validate the effectiveness of our approach, we conducted experiments on three publicly available MKGs, namely PharmKG-8k, DiseaseKG, and DiaKG. Compared with the baseline models, the Precision@K value improved by 0.7%, 6.1%, and 3.6%, respectively, on these datasets. Furthermore, our method empirically showed that it significantly outperformed the baseline on a general knowledge graph, Nell-995. [ABSTRACT FROM AUTHOR]

Time series anomaly detection is very important to ensure the security of industrial control systems (ICSs). Many algorithms have performed well in anomaly detection. However, the performance of most of these algorithms decreases sharply with the increase in feature dimension. This paper proposes an anomaly detection scheme based on Graph Attention Network (GAT) and Informer. GAT learns sequential characteristics effectively, and Informer performs excellently in long time series prediction. In addition, long-time forecasting loss and short-time forecasting loss are used to detect multivariate time series anomalies. Short-time forecasting is used to predict the next time value, and long-time forecasting is employed to assist the short-time prediction. We conduct a large number of experiments on industrial control system datasets SWaT and WADI. Compared with most advanced methods, we achieve competitive results, especially on higher-dimensional datasets. Moreover, the proposed method can accurately locate anomalies and realize interpretability. [ABSTRACT FROM AUTHOR]

Accurate cancer subtype prediction is crucial for personalized medicine. Integrating multi-omics data represents a viable approach to comprehending the intricate pathophysiology of complex diseases like cancer. Conventional machine learning techniques are not ideal for analyzing the complex interrelationships among different categories of omics data. Numerous models have been suggested using graph-based learning to uncover veiled representations and network formations unique to distinct types of omics data to heighten predictions regarding cancers and characterize patients' profiles, amongst other applications aimed at improving disease management in medical research. The existing graph-based state-of-the-art multi-omics integration approaches for cancer subtype prediction, MOGONET, and SUPREME, use a graph convolutional network (GCN), which fails to consider the level of importance of neighboring nodes on a particular node. To address this gap, we hypothesize that paying attention to each neighbor or providing appropriate weights to neighbors based on their importance might improve the cancer subtype prediction. The natural choice to determine the importance of each neighbor of a node in a graph is to explore the graph attention network (GAT). Here, we propose MOGAT, a novel multi-omics integration approach, leveraging GAT models that incorporate graph-based learning with an attention mechanism. MOGAT utilizes a multi-head attention mechanism to extract appropriate information for a specific sample by assigning unique attention coefficients to neighboring samples. Based on our knowledge, our group is the first to explore GAT in multi-omics integration for cancer subtype prediction. To evaluate the performance of MOGAT in predicting cancer subtypes, we explored two sets of breast cancer data from TCGA and METABRIC. Our proposed approach, MOGAT, outperforms MOGONET by 32% to 46% and SUPREME by 2% to 16% in cancer subtype prediction in different scenarios, supporting our hypothesis. Our results also showed that GAT embeddings provide a better prognosis in differentiating the high-risk group from the low-risk group than raw features. [ABSTRACT FROM AUTHOR]

This article addresses the pervasive issue of fraud in financial transactions by introducing the Graph Attention Network (GAN) into graph neural networks. The article integrates Node Attention Networks and Semantic Attention Networks to construct a Dual-Head Attention Network module, enabling a comprehensive analysis of complex relationships in user transaction data. This approach adeptly handles non-linear features and intricate data interaction relationships. The article incorporates a Gradient-Boosting Decision Tree (GBDT) to enhance fraud identification to create the GBDT–Dual-channel Graph Attention Network (GBDT-DGAN). In a bid to ensure user privacy, this article introduces blockchain technology, culminating in the development of a financial anti-fraud model that fuses blockchain with the GBDT-DGAN algorithm. Experimental verification demonstrates the model's accuracy, reaching 93.82%, a notable improvement of at least 5.76% compared to baseline algorithms such as Convolutional Neural Networks. The recall and F1 values stand at 89.5% and 81.66%, respectively. Additionally, the model exhibits superior network data transmission security, maintaining a packet loss rate below 7%. Consequently, the proposed model significantly outperforms traditional approaches in financial fraud detection accuracy and ensures excellent network data transmission security, offering an efficient and secure solution for fraud detection in the financial domain. [ABSTRACT FROM AUTHOR]

Traditional two-dimensional dynamic fault detection methods describe nonlinear dynamics by constructing a two-dimensional sliding window in the batch and time directions. However, determining the shape of a two-dimensional sliding window for different phases can be challenging. Samples in the two-dimensional sliding windows are assigned equal importance before being utilized for feature engineering and statistical control. This will inevitably lead to redundancy in the input, complicating fault detection. This paper proposes a novel method named attention-based two-dimensional dynamic-scale graph autoencoder (2D-ADSGAE). Firstly, a new approach is introduced to construct a graph based on a predefined sliding window, taking into account the differences in importance and redundancy. Secondly, to address the training difficulties and adapt to the inherent heterogeneity typically present in the dynamics of a batch across both its time and batch directions, we devise a method to determine the shape of the sliding window using the Pearson correlation coefficient and a high-density gridding policy. The method is advantageous in determining the shape of the sliding windows at different phases, extracting nonlinear dynamics from batch process data, and reducing redundant information in the sliding windows. Two case studies demonstrate the superiority of 2D-ADSGAE. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Jilin University (Science Edition) / Jilin Daxue Xuebao (Lixue Ban) is the property of Zhongguo Xue shu qi Kan (Guang Pan Ban) Dian zi Za zhi She and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

As the cornerstone of intelligent transportation systems, accurate traffic prediction can reduce the pressure of urban traffic, reduce the cost of residents' travel time, and provide a reference basis for urban construction planning. Existing traffic prediction methods focus on spatio-temporal dependence modeling, ignoring the influence of weather factors on spatio-temporal characteristics, and the prediction task has complexity and an uneven distribution in different spatio-temporal scenarios and weather changes. In view of this, we propose a weather interaction-aware spatio-temporal attention network (WST-ANet), in which we integrate feature models and dynamic graph modules in the encoder and decoder, and use a spatio-temporal weather interaction perception module for prediction. Firstly, the contextual semantics of the traffic flows are fused using a feature embedding module to improve the adaptability to weather drivers; then, an encoder–decoder is constructed by combining the Dynamic Graph Module and the WSTA Block, to extract spatio-temporal aggregated correlations in the roadway network; finally, the feature information of the encoder was weighted and aggregated using the cross-focusing mechanism, and attention was paid to the hidden state of the encoding. Traffic flow was predicted using the PeMS04 and PeMS08 datasets and compared with multiple typical baseline models. It was learned through extensive experiments that the accuracy evaluation result is the smallest in WST-ANet, which demonstrated the superiority of the proposed model. This can more accurately predict future changes in traffic in different weather conditions, providing decision makers with a basis for optimizing scenarios. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Computer-Aided Design & Computer Graphics / Jisuanji Fuzhu Sheji Yu Tuxingxue Xuebao is the property of Gai Kan Bian Wei Hui and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

This article addresses the pervasive issue of fraud in financial transactions by introducing the Graph Attention Network (GAN) into graph neural networks. The article integrates Node Attention Networks and Semantic Attention Networks to construct a Dual-Head Attention Network module, enabling a comprehensive analysis of complex relationships in user transaction data. This approach adeptly handles non-linear features and intricate data interaction relationships. The article incorporates a Gradient-Boosting Decision Tree (GBDT) to enhance fraud identification to create the GBDT–Dual-channel Graph Attention Network (GBDT-DGAN). In a bid to ensure user privacy, this article introduces blockchain technology, culminating in the development of a financial anti-fraud model that fuses blockchain with the GBDT-DGAN algorithm. Experimental verification demonstrates the model’s accuracy, reaching 93.82%, a notable improvement of at least 5.76% compared to baseline algorithms such as Convolutional Neural Networks. The recall and F1 values stand at 89.5% and 81.66%, respectively. Additionally, the model exhibits superior network data transmission security, maintaining a packet loss rate below 7%. Consequently, the proposed model significantly outperforms traditional approaches in financial fraud detection accuracy and ensures excellent network data transmission security, offering an efficient and secure solution for fraud detection in the financial domain.

Hierarchical Text Classification (HTC) is a challenging task where labels are structured in a tree or Directed Acyclic Graph (DAG) format. Current approaches often struggle with data imbalance and fail to fully capture rich semantic information. This paper proposes an Adaptive Hierarchical Text Classification method, EDTPA (ERNIE and Dynamic Threshold Pruning-based Adaptive classification), which leverages Large Language Models (LLMs) for data augmentation to mitigate imbalanced datasets. The model first uses Graph Attention Networks (GAT) to capture hierarchical dependencies among labels, effectively modeling structured relationships. ERNIE enhances the semantic representation of both the text and hierarchical labels, optimizing the model’s ability to process Chinese text. An attention mechanism strengthens the alignment between text and labels, improving accuracy. The model combines global and local information flows, while dynamic threshold pruning prunes low-probability branches, improving interpretability. Results on the Chinese Scientific Literature (CSL) dataset show EDTPA significantly outperforms baseline models in both Micro-F1 and Macro-F1 scores, effectively addressing data imbalance and improving classification performance.

Drug-drug interactions (DDIs) can severely affect patient health and safety. Predicting potential DDIs before administering medication to patients is crucial for drug development as it helps to prevent adverse drug reactions. Many effective DDI prediction methods have been proposed using graph neural networks; however, these methods only aggregate information from directly connected nodes restricted to a drug-related manner and fail to capture long-range dependencies in heterogeneous networks. To address this issue, we propose an enhanced multiple-feature hybrid graph method to predict DDIs (MFHG-DDI). Specifically, we constructed a heterogeneous network incorporating multiple drug features and DDI information. We employed an enhanced hybrid graph module that integrates a graph convolutional network, graph attention network, and global average pooling to learn latent features, ultimately applying a prediction function to predict DDIs. MFHG-DDI reframes the heterogeneous network as a graph classification task, capturing DDI information efficiently through an enhanced hybrid graph. Known DDI datasets were used to train and evaluate the proposed model. The experimental results indicate that integrating multiple drug features into the hybrid graph method can enhance the DDI prediction accuracy, increases the success rate of combination therapy, and has the potential to enhance drug safety.

The high penetration of renewables into the active distribution network (ADN) brings voltage deviation and difficulties to the optimal power flow (OPF) problem. The optimal operation of the distribution grid aims to efficiently manage the flow of electricity from sources to end-users, ensuring a resilient and sustainable grid. To perform the optimal operation, OPF plays a pivotal role in solving a complex optimization problem due to the system’s operational constraints and the provided OPF solutions. Implementing traditional OPF algorithms can be challenging for large-scale networks with complex topologies and constraints. The most recent advancement in learning-based models has shifted the paradigm towards data-driven approaches. This paper proposes a high-fidelity graph attention networks (GAT) model that leverages the attention mechanism and graph convolution feature mapping property to learn neighbor informative node representations for OPF solutions. We validated the proposed model on the IEEE-33, 69, and modified 123-bus power distribution networks. The proposed GAT model outperformed the state-of-the-art MPGCN and DNN models, achieving improvement of 86.33% and 62.71%, respectively, under 60% DG penetration condition. The robustness assessment of DNN, MPGCN, and GAT models are also compared in all three test cases. The GAT model exhibited less variability in its median error 0.22, 0.21, and 0.038, respectively, in each case. For computational efficiency analysis, the GAT model was processed on an IEEE-123 bus with 13,810 samples in 782 seconds, which remains within the steady-state OPF calculation time limit of 15 minutes (900 seconds). The proposed GAT model showcases its effectiveness and promises results for addressing the OPF problem in the distribution network, as evidenced by performance evaluation metrics.

The importance of an efficient network resource allocation strategy has grown significantly with the rapid advancement of cellular network technology and the widespread use of mobile devices. Efficient resource allocation is crucial for enhancing user services and optimizing network performance. The primary objective is to optimize the power distribution method to maximize the total aggregate rate for all customers within the network. In recent years, graph-based deep learning approaches have shown great promise in addressing the challenge of network resource allocation. Graph neural networks (GNNs) have particularly excelled in handling graph-structured data, benefiting from the inherent topological characteristics of mobile networks. However, many of these methodologies tend to focus predominantly on node characteristics during the learning phase, occasionally overlooking or oversimplifying the importance of edge attributes, which are equally vital as nodes in network modeling. To tackle this limitation, we introduce a novel framework known as the Heterogeneous Edge Feature Enhanced Graph Attention Network (HEGAT). This framework establishes a direct connection between the evolving network topology and the optimal power distribution strategy throughout the learning process. Our proposed HEGAT approach exhibits improved performance and demonstrates significant generalization capabilities, as evidenced by extensive simulation results.

This study aims to investigate the dissemination of traditional Chinese music in the digital era and the application of deep learning in predicting the influence of Weibo content propagation. Firstly, the impact of digitization on the dissemination of traditional Chinese music is analyzed. Secondly, based on the Bidirectional Gated Recurrent Unit (Bi-GRU), traditional content feature modeling methods are optimized by introducing content text features. Simultaneously, the Graph Attention Network (GAN) is divided into three steps, allowing it to consider the edge properties of input sequences. The improved content feature modeling, GAN, and multilayer perceptron are integrated to construct a Context-dependent Dynamic Graph Attention Network (C-DGAN). In order to validate the performance of the C-DGAN model, Mean Square Logarithmic Error (MSLE) is used as the evaluation metric in comparative experiments at observation times T=1, 2, 3, and 4 hours. The results indicate that at T=4 hours, C-DGAN achieves an MSLE of 1.854, reducing by at least 0.134 compared to the baseline model, demonstrating superior performance in predicting the scale of Weibo content propagation. Additionally, in comparison with models using different recurrent neural networks, the model employing the Bi-GRU network performs the best. Thus, the proposed C-DGAN model exhibits excellent performance in predicting Weibo content propagation influence. The study findings provide robust support for the study and practice of Weibo content propagation.

The fractured-vuggy carbonate reservoir comprises various types of storage and seepage spaces, and is composed of multi-scale dissolution pores and fractures. The frequent changes to working systems make the characteristics of water breakthrough complex, and the production data nonlinear and non-stationary, resulting in great difficulty in real-time prediction. Traditional production forecasting methods only consider temporal correlations, neglecting the spatial correlations between production wells and local geological features. In this paper, adopts a modular design approach that comprehensively considers the spatiotemporal characteristics by abstracting each production well in the unit as a directed graph network node. We establish a graph attention network module based on the connectivity between wells to simulate fluid motion patterns and extract spatial features. To address the autocorrelation characteristics of the production sequences, we use a self-attention mechanism module to capture the temporal dependency relationships between production sequences. Finally, considering the fusion of spatiotemporal features, a gating mechanism is designed to adaptively aggregate spatiotemporal characteristics produced by the previous two modules, enabling dynamic production forecasting. We validate our proposed model using real-world production data from the Tarim Basin in China. Our experimental results demonstrate the superiority of the new model over existing production prediction models in fractured-vuggy carbonate reservoirs.

Extended use of social media, leads to personality detection from online content shared by users. While it has numerous applications in different areas such as recommendation systems, most of existing studies focus on superficial, statistical, and explicit user contents, ignoring the knowledge hidden in semantic features. In this study, we proposed a method to explore psycholinguistic knowledge hidden in deep levels of user data for the task of personality prediction. We proposed a method utilizing a fine-tuned domain-specific BERT model to extract features from a sentence and enrich the outputs by leveraging emotional information to highlight the important words. Furthermore, by conducting a double-way-attention mechanism we reflected the information from highlighted words into the whole knowledge extracted from inputs. Then, created a graph by considering extracted embeddings from the last step as node features and developing a dynamic and task-related learning approach to specify edges to connect different pairs of nodes based on a neural network, and leveraged graph attention network to predict personality traits. Finally, experimental results confirmed the effectiveness of our proposed model with 80.63% accuracy, compared to other state-of-the-art studies for the essays dataset. Also, several ablations are conducted to illustrate and verify the impact of different sections and parameters of the proposed architecture.

Healthcare fraud increases healthcare expenses for insurers, premiums for policyholders, and dissatisfaction of legitimate patients and causes severe damage to the health system. Therefore, it is critically important to address healthcare fraud detection. Most fraud detection models consider only claims data for analysis. Since committing healthcare fraud can include more than one party, i.e., healthcare providers, physicians, and patients, it is crucial to consider the relationship among them. In this paper, we propose a healthcare provider fraud detection model that applies the effect of the parties’ interdependencies on the claims data. It leverages a graph attention network for embedding the relationships and classifies samples using a feed-forward neural network. Our explicit contribution is using the latent information of the interdependency of claims’ parties to detect healthcare provider fraud more accurately. The information, along with other features, can identify complex patterns of fraud. We tested our approach on the healthcare provider fraud detection dataset and reached 0.56 recall compared to best available approaches such as GTN with 0.5 and XGBoost with 0.46 recall.

Emotion recognition is an important part of human-computer interaction and human communication information is multi-model. Despite advancements in emotion recognition models, certain challenges persist. The first problem (Problem 1) pertains to the predominant focus in existing research on mining the interaction information between modes and the context information in the dialogue process but neglects to mine the role information between multi-model states and context information in the dialogue process. The second problem (Problem 2) is in the context information of the dialogue where the information is not completely transmitted in a temporal structure. Aiming at these two problems, we propose a multi-model fusion dialogue graph attention network (MM DialogueGAT). To solve the problem 1, the bidirectional GRU mechanism is used to extract the information from each model. In the multi-model information fusion problem, different model configurations and different combinations use the cross-model multi-head attention mechanism to establish a multi-head attention layer. Text, video and audio information are used as the main and auxiliary modes for information fusion. To solve the problem 2, in the temporal context information extraction problem, the GAT graph structure is used to capture the context information in the mode. The results show that our model achieves good results using the IMEOCAP datasets.

We present GeoGlue, a novel method using high-resolution UAV imagery for accurate feature matching, which is normally challenging due to the complicated scenes. Current feature detection methods are performed without guidance of geometric priors (e.g., geometric lines), lacking enough attention given to salient geometric features which are indispensable for accurate matching due to their stable existence across views. In this work, geometric lines are firstly detected by a CNN-based geometry detector (GD) which is pre-trained in a self-supervised manner through automatically generated images. Then, geometric lines are naturally vectorized based on GD and thus non-significant features can be disregarded as judged by their disordered geometric morphology. A graph attention network (GAT) is utilized for final feature matching, spanning across the image pair with geometric priors informed by GD. Comprehensive experiments show that GeoGlue outperforms other state-of-the-art methods in feature-matching accuracy and performance stability, achieving pose estimation with maximum rotation and translation errors under 1% in challenging scenes from benchmark datasets, Tanks & Temples and ETH3D. This study also proposes the first self-supervised deep-learning model for curved line detection, generating geometric priors for matching so that more attention is put on prominent features and improving the visual effect of 3D reconstruction.

Recent advancements in crime prediction have increasingly focused on street networks, which offer finer granularity and a closer reflection of real-world urban dynamics. However, existing studies on street-level graph representation learning often overlook the variability in node features when aggregating information from neighboring nodes. This limitation reduces the model’s capacity to fully capture the diverse street attributes and their influence on crime patterns. To address this issue, we introduce an end-to-end deep spatio-temporal learning model that employs a graph attention mechanism (GAT) to analyze the spatio-temporal features of 110 call incidents. Experimental results show that our proposed model outperforms existing methods across multiple prediction metrics. Additionally, ablation studies confirm that the GAT’s capacity to capture spatial dependencies within the street network significantly enhances the model’s overall predictive performance.

Trajectory prediction is a key component in the development of applications such as mixed urban traffic management and public safety. Traditional models have struggled with the complexity of modeling dynamic crowd interactions, the intricacies of spatiotemporal dependencies, and environmental constraints. Addressing these challenges, this paper introduces the innovative Social Attention Graph Neural Network (SA-GAT) framework. Utilizing Long Short-Term Memory (LSTM) networks, SA-GAT encodes pedestrian trajectory data to extract temporal correlations, while Graph Attention Networks (GAT) are employed to precisely capture the subtle interactions among pedestrians. The SA-GAT framework boosts its predictive accuracy with two key innovations. First, it features a Scene Potential Module that utilizes a Scene Tensor to dynamically capture the interplay between crowds and their environment. Second, it incorporates a Transition Intention Module with a Transition Tensor, which interprets latent transfer probabilities from trajectory data to reveal pedestrians’ implicit intentions at specific locations. Based on AnyLogic modeling of the metro station on Line 10 of Chengdu Shuangliu Airport, China, numerical studies reveal that the SA-GAT model achieves a substantial reduction in ADE and FDE metrics by 34.22% and 38.04% compared to baseline models.

Pluvial floods are destructive natural disasters in cities. With high computational efficiency, machine learning models are increasingly used for flood susceptibility mapping. However, limited flooded or nonflooded samples constrain models’ predictive capability and introduce uncertainty in feature engineering. This study introduces a semi-supervised graph-structured model, Graph Attention Network (GAT), to address data scarcity and enable the use of only basic flood conditioning factors as inputs. GAT uses nodes and edges to represent spatial units and their relative spatial relationships. Based on its graph structure and attention mechanism, GAT automatically extracts high-order features from inputs of labeled and unlabeled units for modeling. In the metropolitan area of Dalian, China, GAT outperformed other models in flooded-nonflooded sample classification and exhibited a rational flood susceptibility distribution pattern, with only four basic flood conditioning factors and less than 1.2% of units for training. GAT can be an effective tool for practical urban flood management.

The development of deep learning techniques and support of big data computing power have revolutionized graph representation research by facilitating the implementation of the learning of different graph neural network structures. Existing methods, such as graph attention networks, mainly focus on global information propagation in graph neural networks, which have theoretically proven their strong representation capability. However, these general methods lack flexible representation mechanisms when facing graph data with local topology involving specific semantics, such as functional groups in the chemical reaction. Accordingly, it is of great importance to further exploit the local structure representations for graph-based tasks. Several existing methods either use domain expert knowledge or conduct subgraph isomorphism counting to learn local topology representations of graphs. However, there is no guarantee that these methods can easily be generalized to different domains without specific knowledge or complex substructure preprocessing. In this study, we propose a simple and automatic local topology inference method that uses variational convolutions to improve the local representation ability of graph attention networks. The proposed method not only considers the relationship reasoning and message passing on the global graph structure but also adaptively learns the graph’s local structure representations with the guidance of statistical priors that can be readily accessible. To be more specific, the variational inference is used to adaptively learn the convolutional template size, and the inference is conducted layer-by-layer with the guidance of the statistical priors to make the convolutional template size adaptable to multiple subgraphs with different structures in a self-supervised way. The variational convolution module is easily pluggable and can be concatenated with arbitrary hidden layers of any graph neural network. In contrast, due to the locality of the convolution operations, the relations between graph nodes can be further sparse to alleviate the over-squeezing problem in the global information propagation of the graph neural network. As a result, the proposed method can significantly improve the overall representation ability of the graph attention network using the variational inference of the convolutional operations for local topology representation. Experiments are conducted on three large-scale and publicly available datasets, i.e., the OGBG-MolHIV, USPTO, and Buchwald-Hartwig datasets. Experimental results show that exploiting various kinds of local topological information helps improve the performance of the graph attention network.

To address the problems of low accuracy and poor generalization of current honeypot contract detection methods, a honeypot contract detection method for Ethereum based on source code structure and graph attention network was proposed.Firstly, in order to extract the structural information of the Solidity source code of the smart contract, the source code was parsed and converted into an XML parsing tree.Then, a set of feature words that could express the structural and content characteristics of the contract was selected, and the contract source code structure graph was constructed.Finally, in order to avoid the impact of dataset imbalance, the concepts of teacher model and student model were introduced based on the ensemble learning theory.Moreover, the graph attention network model was trained from the global and local perspectives, respectively, and the outputs of all models were fused to obtain the final contract detection result.The experiments demonstrate that CSGDetector has higher honeypot detection capability than the existing method KOLSTM, with increments of 1.27% and 7.21% on F1 measurement in two-class classification and multi-class classification experiments, respectively.When comparing with the existing method XGB, the average recall rate of CSGDetector in the masked honeypot detection experiments for different types of honeypot contracts is improved by 7.57%, which verifies the effectiveness of the method in improving the generalization performance of the algorithm.

IntroductionThe identification of microbe–drug associations can greatly facilitate drug research and development. Traditional methods for screening microbe-drug associations are time-consuming, manpower-intensive, and costly to conduct, so computational methods are a good alternative. However, most of them ignore the combination of abundant sequence, structural information, and microbe-drug network topology.MethodsIn this study, we developed a computational framework based on a modified graph attention variational autoencoder (MGAVAEMDA) to infer potential microbedrug associations by combining biological information with the variational autoencoder. In MGAVAEMDA, we first used multiple databases, which include microbial sequences, drug structures, and microbe-drug association databases, to establish two comprehensive feature matrices of microbes and drugs after multiple similarity computations, fusion, smoothing, and thresholding. Then, we employed a combination of variational autoencoder and graph attention to extract low-dimensional feature representations of microbes and drugs. Finally, the lowdimensional feature representation and graphical adjacency matrix were input into the random forest classifier to obtain the microbe–drug association score to identify the potential microbe-drug association. Moreover, in order to correct the model complexity and redundant calculation to improve efficiency, we introduced a modified graph convolutional neural network embedded into the variational autoencoder for computing low dimensional features.ResultsThe experiment results demonstrate that the prediction performance of MGAVAEMDA is better than the five state-of-the-art methods. For the major measurements (AUC =0.9357, AUPR =0.9378), the relative improvements of MGAVAEMDA compared to the suboptimal methods are 1.76 and 1.47%, respectively.DiscussionWe conducted case studies on two drugs and found that more than 85% of the predicted associations have been reported in PubMed. The comprehensive experimental results validated the reliability of our models in accurately inferring potential microbe-drug associations.

Recently, large-scale artificial intelligence models with billions of parameters have achieved good results in experiments, but their practical deployment on edge computing platforms is often subject to many constraints because of their resource requirements. These models require powerful computing platforms with a high memory capacity to store and process the numerous parameters and activations, which makes it challenging to deploy these large-scale models directly. Therefore, model compression techniques are crucial role in making these models more practical and accessible. In this article, a progressive channel pruning strategy combining graph attention network and transformer, namely GAT TransPruning, is proposed, which uses the graph attention networks (GAT) and the attention of transformer mechanism to determine the channel-to-channel relationship in large networks. This approach ensures that the network maintains its critical functional connections and optimizes the trade-off between model size and performance. In this study, VGG-16, VGG-19, ResNet-18, ResNet-34, and ResNet-50 are used as large-scale network models with the CIFAR-10 and CIFAR-100 datasets for verification and quantitative analysis of the proposed progressive channel pruning strategy. The experimental results reveal that the accuracy rate only drops by 6.58% when the channel pruning rate is 89% for VGG-19/CIFAR-100. In addition, the lightweight model inference speed is 9.10 times faster than that of the original large model. In comparison with the traditional channel pruning schemes, the proposed progressive channel pruning strategy based on the GAT and Transformer cannot only cut out the insignificant weight channels and effectively reduce the model size, but also ensure that the performance drop rate of its lightweight model is still the smallest even under high pruning ratio.

The incidence of brain diseases has increased yearly, threatening human life and health seriously. The Electroencephalogram (EEG) has been playing an important role in clinical practice for diagnosing brain diseases. However, due to the interference of noise and the limitations of manual observation, experts need to spend a lot of time and energy on EEG interpretation, and also is affected by subjective factors, which is prone to misjudgment. Therefore, the establishment of EEG-assisted diagnosis system is of great significance for the clinical diagnosis of brain diseases. With the application of artificial intelligence in EEG auxiliary system, researchers have proposed a series of EEG automatic analysis and anomaly detection algorithms based on deep learning. However, the existing algorithms still have some shortcomings such as inadequate extraction of potential spatio-temporal features in EEG signals. In this paper, the method based on GATs-LSTM is proposed. The comparative analysis of the final experiment shows that, It has demonstrated superior performance on the benchmark dataset with sensitive and specificity as 99.21%, 99.73% and 94.15%, 95.67%. The proposed work shows great results on the benchmark datasets and a big potential for clinics as a support system with medical experts monitoring the patients. [ABSTRACT FROM AUTHOR]

In this paper, we propose a new type of vision transformer (ViT) based on graph head attention (GHA). Because the multi-head attention (MHA) of a pure ViT requires multiple parameters and tends to lose the locality of an image, we replaced MHA with GHA by applying a graph to the attention head of the transformer. Consequently, the proposed GHA maintains both the locality and globality of the input patches and guarantees the diversity of the attention. The proposed GHA-ViT commonly outperforms pure ViT-based models using small-sized CIFAR-10/100, MNIST, and MNIST-F datasets and a medium-sized ImageNet-1K dataset in scratch training. A Top-1 accuracy of 81.7% was achieved for ImageNet-1K using GHA-B, which is a base model with approximately 29 M parameters. In addition, with CIFAR-10/100, the existing ViT and parameters are reduced 17-fold and the performance increased by 0.4/4.3%, respectively. The proposed GHA-ViT shows promising results in terms of the number of parameters and operations and the level of accuracy in comparison with other state-of-the-art ViT-lightweight models. [ABSTRACT FROM AUTHOR]

Artifacts are the main cause of degradation in CT image quality and diagnostic accuracy. Because of the complex texture of CT images, it is a challenging task to automatically detect artifacts from limited image samples. Recently, graph convolutional networks (GCNs) have achieved great success and shown promising results in medical imaging due to their powerful learning ability. However, GCNs do not take the attention mechanism into consideration. To overcome their limitations, we propose a novel Regional–Temporal Graph Attention Network for motion artifact detection from computed tomography images (RT-GAT). In this paper, head CT images are viewed as a heterogeneous graph by taking regional and temporal information into consideration, and the graph attention network is utilized to extract the features of the constructed graph. Then, the feature vector is input into the classifier to detect the motion artifacts. The experimental results demonstrate that our proposed RT-GAT method outperforms the state-of-the-art methods on a real-world CT dataset. [ABSTRACT FROM AUTHOR]

The fluctuation of wind power affects the operating safety and power consumption of the electric power grid and restricts the grid connection of wind power on a large scale. Therefore, wind power forecasting plays a key role in improving the safety and economic benefits of the power grid. This paper proposes a wind power predicting method based on a convolutional graph attention deep neural network with multi-wind farm data. Based on the graph attention network and attention mechanism, the method extracts spatial-temporal characteristics from the data of multiple wind farms. Then, combined with a deep neural network, a convolutional graph attention deep neural network model is constructed. Finally, the model is trained with the quantile regression loss function to achieve the wind power deterministic and probabilistic prediction based on multi-wind farm spatial-temporal data. A wind power dataset in the U.S. is taken as an example to demonstrate the efficacy of the proposed model. Compared with the selected baseline methods, the proposed model achieves the best prediction performance. The point prediction errors (i.e., root mean square error (RMSE) and normalized mean absolute percentage error (NMAPE)) are 0.304 MW and 1.177%, respectively. And the comprehensive performance of probabilistic prediction (i.e., continuously ranked probability score (CRPS)) is 0.580. Thus, the significance of multi-wind farm data and spatial-temporal feature extraction module is self-evident. [ABSTRACT FROM AUTHOR]

Social robot accounts controlled by artificial intelligence or humans are active in social networks, bringing negative impacts to network security and social life. Existing social robot detection methods based on graph neural networks suffer from the problem of many social network nodes and complex relationships, which makes it difficult to accurately describe the difference between the topological relations of nodes, resulting in low detection accuracy of social robots. This paper proposes a social robot detection method with the use of an improved neural network. First, social relationship subgraphs are constructed by leveraging the user's social network to disentangle intricate social relationships effectively. Then, a linear modulated graph attention residual network model is devised to extract the node and network topology features of the social relation subgraph, thereby generating comprehensive social relation subgraph features, and the feature-wise linear modulation module of the model can better learn the differences between the nodes. Next, user text content and behavioral gene sequences are extracted to construct social behavioral features combined with the social relationship subgraph features. Finally, social robots can bemore accurately identified by combining user behavioral and relationship features. By carrying out experimental studies based on the publicly available datasets TwiBot-20 and Cresci-15, the suggested method's detection accuracies can achieve 86.73% and 97.86%, respectively. Compared with the existing mainstream approaches, the accuracy of the proposed method is 2.2% and 1.35% higher on the two datasets. The results show that the method proposed in this paper can effectively detect social robots and maintain a healthy ecological environment of social networks. [ABSTRACT FROM AUTHOR]

In recent years, graph convolutional neural networks (GCNs) have become a popular research topic due to their outstanding performance in various complex network data mining tasks. However, current research on graph neural networks lacks understanding of the high-order structural features of networks, focusing mostly on node features and first-order neighbor features. This article proposes two new models, MGAT and MGATv2, by introducing high-order structure motifs that frequently appear in networks and combining them with graph attention mechanisms. By introducing a mixed information matrix based on motifs, the generation process of graph attention coefficients is improved, allowing the model to capture higher-order structural features. Compared with the latest research on various graph neural networks, both MGAT and MGATv2 achieve good results in node classification tasks. Furthermore, through various experimental studies on real datasets, we demonstrate that the introduction of network structural motifs can effectively enhance the expressive power of graph neural networks, indicating that both high-order structural features and attribute features are important components of network feature learning. [ABSTRACT FROM AUTHOR]