Search

Document-level Event Argument Extraction (DEAE) aims to extract structural event knowledge composed of arguments and roles beyond the sentence level. Existing methods mainly focus on designing prompts and using Abstract Meaning Representation (AMR) graph structure as additional features to enrich event argument representation. However, two challenges still remain: (1) the long-range dependency between event trigger and event arguments and (2) the distracting context in the document towards an event that can mislead the argument classification. To address these issues, we propose a novel document-level event argument extraction model named AMR Parser and Sparse Representation (APSR). Specifically, APSR sets inter- and intra-sentential encoders to capture the contextual information in different scopes. Especially, in the intra-sentential encoder, APSR designs three types of sparse event argument attention mechanisms to extract the long-range dependency. Then, APSR constructs AMR semantic graphs, which capture the interactions among concepts well. Finally, APSR fuses the inter- and intra-sentential representations and predicts what role a candidate span plays. Experimental results on the RAMS and WikiEvents datasets demonstrate that APSR achieves a superior performance compared with competitive baselines in terms of F1 by 1.27% and 3.12%, respectively.

Text classification is a machine learning technique employed to assign a given text to predefined categories, facilitating the automatic analysis and processing of textual data. However, an important problem is that the number of new text categories is growing faster than that of human annotation data, which makes many new categories of text data lack a lot of annotation data. As a result, the conventional deep neural network is forced to over-fit, which damages the application in the real world. As a solution to this problem, academics recommend addressing data scarcity through few-shot learning. One of the efficient methods is prompt-tuning, which transforms the input text into a mask prediction problem featuring [MASK]. By utilizing descriptors, the model maps output words to labels, enabling accurate prediction. Nevertheless, the previous prompt-based adaption approaches often relied on manually produced verbalizers or a single label to represent the entire label vocabulary, which makes the mapping granularity low, resulting in words not being accurately mapped to their label. To address these issues, we propose to enhance the verbalizer and construct the refined external knowledge into a prompt-tuning (REKP) model. We employ the external knowledge bases to increase the mapping space of tagged terms and design three refinement methods to remove noise data. We conduct comprehensive experiments on four benchmark datasets, namely AG’s News, Yahoo, IMDB, and Amazon. The results demonstrate that REKP can outperform the state-of-the-art baselines in terms of Micro-F1 on knowledge-enhanced text classification. In addition, we conduct an ablation study to ascertain the functionality of each module in our model, revealing that the refinement module significantly contributes to enhancing classification accuracy.

Graph Collaborative Filtering (GCF) methods have emerged as an effective recommendation approach, capturing users’ preferences over items by modeling user–item interaction graphs. However, these methods suffer from data sparsity in real scenarios, and their performance can be improved using contrastive learning. In this paper, we propose an optimized method, named LoRA-NCL, for GCF based on Neighborhood-enriched Contrastive Learning (NCL) and low-rank dimensionality reduction. We incorporate low-rank features obtained through matrix factorization into the NCL framework and employ LightGCN to extract high-dimensional representations. Extensive experiments on five public datasets demonstrate that the proposed method outperforms a competitive graph collaborative filtering base model, achieving 4.6% performance gains on the MovieLens dataset, respectively.

Relation extraction aims to predict the relation triple between the tail entity and head entity in a given text. A large body of works adopt meta-learning to address the few-shot issue faced by relation extraction, where each relation category only contains few labeled data for demonstration. Despite promising results achieved by existing meta-learning methods, these methods still struggle to distinguish the subtle differences between different relations with similar expressions. We argue this is largely owing to that these methods cannot capture unbiased and discriminative features in the very few-shot scenario. For alleviating the above problems, we propose a taxonomy-aware prototype network, which consists of a category-aware calibration module and a task-aware training strategy module. The former implicitly and explicitly calibrates the representation of prototype to become sufficiently unbiased and discriminative. The latter balances the weight between easy and hard instances, which enables our proposal to focus on data with more information during the training stage. Finally, comprehensive experiments are conducted on four typical meta tasks. Furthermore, our proposal presents superiority over the competitive baselines with an improvement of 3.30% in terms of average accuracy.

Intent recognition aims to identify users’ potential intents from their utterances, which is a key component in task-oriented dialog systems. A real challenge, however, is that the number of intent categories has grown faster than human-annotated data, resulting in only a small amount of data being available for many new intent categories. This lack of data leads to the overfitting of traditional deep neural networks on a small amount of training data, which seriously affects practical applications. Hence, some researchers have proposed few-shot learning should address the data-scarcity issue. One of the efficient methods is text augmentation, which always generates noisy or meaningless data. To address these issues, we propose leveraging the knowledge in pre-trained language models and constructed the cloze-style data augmentation (CDA) model. We employ unsupervised learning to force the augmented data to be semantically similar to the initial input sentences and contrastive learning to enhance the uniqueness of each category. Experimental results on CLINC-150 and BANKING-77 datasets show the effectiveness of our proposal by its beating of the competitive baselines. In addition, we conducted an ablation study to verify the function of each module in our models, and the results illustrate that the contrastive learning module plays the most important role in improving the recognition accuracy.

Chinese Medical Named Entity Recognition (Chinese-MNER) aims to identify potential entities and their categories from the unstructured Chinese medical text. Existing methods for this task mainly incorporate the dictionary knowledge on the basis of traditional BiLSTM-CRF or BERT architecture. However, the construction of high-quality dictionaries is typically time consuming and labor-intensive, which may also damage the robustness of NER models. What is more, the limited amount of annotated Chinese-MNER data can easily lead to the over-fitting problem while training. With the aim of dealing with the above problems, we put forward a BERT-BiLSTM-CRF model by integrating the part-of-speech (POS) tagging features and a Regularization method (BBCPR) for Chinese-MNER. In BBCPR, we first leverage a POS fusion layer to incorporate external syntax knowledge. Next, we design a novel REgularization mothod with Adversarial training and Dropout (READ) to improve the model robustness. Specifically, READ focuses on reducing the difference between the predictions of two sub-models through minimizing the bidirectional KL divergence between the adversarial output and original output distributions for the same sample. Comprehensive evaluations on two public data sets, namely, cMedQANER and cEHRNER from the Chinese Biomedical Language Understanding Evaluation benchmark (ChineseBLUE), demonstrate the superiority of our proposal in Chinese-MNER. In addition, ablation study shows that READ can effectively improve the model performance. Our proposal does well in exploring the technical terms and identifying the word boundary.

Knowledge-enhanced recommendation (KER) aims to integrate the knowledge graph (KG) into collaborative filtering (CF) for alleviating the sparsity and cold start problems. The state-of-the-art graph neural network (GNN)–based methods mainly focus on exploiting the connectivity between entities in the knowledge graph, while neglecting the interaction relation between items reflected in the user-item interactions. Moreover, the widely adopted BPR loss for model optimization fails to provide sufficient supervisions for learning discriminative representation of users and items. To address these issues, we propose the collaborative knowledge-enhanced recommendation (CKER) method. Specifically, CKER proposes a collaborative graph convolution network (CGCN) to learn the user and item representations from the connection between items in the constructed interaction graph and the connectivity between entities in the knowledge graph. Moreover, we introduce the self-supervised learning to maximize the mutual information between the interaction- and knowledge-aware user preferences by deriving additional supervision signals. We conduct comprehensive experiments on two benchmark datasets, namely Amazon-Book and Last-FM, and the experimental results show that CKER can outperform the state-of-the-art baselines in terms of recall and NDCG on knowledge-enhanced recommendation.

Fact verification aims to evaluate the authenticity of a given claim based on the evidence sentences retrieved from Wikipedia articles. Existing works mainly leverage the natural language inference methods to model the semantic interaction of claim and evidence, or further employ the graph structure to capture the relation features between multiple evidences. However, previous methods have limited representation ability in encoding complicated units of claim and evidences, and thus cannot support sophisticated reasoning. In addition, a limited amount of supervisory signals lead to the graph encoder could not distinguish the distinctions of different graph structures and weaken the encoding ability. To address the above issues, we propose a Knowledge-Enhanced Graph Attention network (KEGA) for fact verification, which introduces a knowledge integration module to enhance the representation of claims and evidences by incorporating external knowledge. Moreover, KEGA leverages an auxiliary loss based on contrastive learning to fine-tune the graph attention encoder and learn the discriminative features for the evidence graph. Comprehensive experiments conducted on FEVER, a large-scale benchmark dataset for fact verification, demonstrate the superiority of our proposal in both the multi-evidences and single-evidence scenarios. In addition, our findings show that the background knowledge for words can effectively improve the model performance.

The representation learning of the knowledge graph projects the entities and relationships in the triples into a low-dimensional continuous vector space. Early representation learning mostly focused on the information contained in the triplet itself but ignored other useful information. Since entities have different types of representations in different scenarios, the rich information in the types of entity levels is helpful for obtaining a more complete knowledge representation. In this paper, a new knowledge representation frame (TRKRL) combining rule path information and entity hierarchical type information is proposed to exploit interpretability of logical rules and the advantages of entity hierarchical types. Specifically, for entity hierarchical type information, we consider that entities have multiple representations of different types, as well as treat it as the projection matrix of entities, using the type encoder to model entity hierarchical types. For rule path information, we mine Horn rules from the knowledge graph to guide the synthesis of relations in paths. Experimental results show that TRKRL outperforms baselines on the knowledge graph completion task, which indicates that our model is capable of using entity hierarchical type information, relation paths information, and logic rules information for representation learning.

Session-based recommendation aims to model a user’s intent and predict an item that the user may interact with in the next step based on an ongoing session. Existing session-based recommender systems mainly aim to model the sequential signals based on Recurrent Neural Network (RNN) structures or the item transition relations between items with Graph Neural Network (GNN) based frameworks to identify a user’s intent for recommendation. However, in real scenarios, there may be strong sequential signals existing in users’ adjacent behaviors or multi-step transition relations among different items. Thus, either RNN- or GNN-based methods can only capture limited information for modeling complex user behavior patterns. RNNs pay attention to the sequential relations among consecutive items, while GNNs focus on structural information, i.e., how to enrich the item embedding with its adjacent items. In this paper, we propose a Collaborative Co-attention Network for Session-based Recommendation (CCN-SR) to incorporate both sequential and structural information, as well as capture the co-relations between them for obtaining an accurate session representation. To be specific, we first model the ongoing session with an RNN structure to capture the sequential information among items. Meanwhile, we also construct a session graph to learn the item representations with a GNN structure. Then, we design a co-attention network upon these two structures to capture the mutual information between them. The designed co-attention network can enrich the representation of each node in the session with both sequential and structural information, and thus generate a more comprehensive representation for each session. Extensive experiments are conducted on two public e-commerce datasets, and the results demonstrate that our proposed model outperforms state-of-the-art baseline model for session based recommendation in terms of both Recall and MRR. We also investigate different combination strategies and the experimental results verify the effectiveness of our proposed co-attention mechanism. Besides, our CCN-SR model achieves better performance than baseline models with different session lengths.

Session-based recommendation (SBRS) aims to make recommendations for users merely based on the ongoing session. Existing GNN-based methods achieve satisfactory performance by exploiting the pair-wise item transition pattern; however, they ignore the temporal evolution of the session graphs over different time-steps. Moreover, the widely applied cross-entropy loss with softmax in SBRS faces the serious overfitting problem. To deal with the above issues, we propose dynamic graph learning for session-based recommendation (DGL-SR). Specifically, we design a dynamic graph neural network (DGNN) to simultaneously take the graph structural information and the temporal dynamics into consideration for learning the dynamic item representations. Moreover, we propose a corrective margin softmax (CMS) to prevent overfitting in the model optimization by correcting the gradient of the negative samples. Comprehensive experiments are conducted on two benchmark datasets, that is, Diginetica and Gowalla, and the experimental results show the superiority of DGL-SR over the state-of-the-art baselines in terms of Recall@20 and MRR@20, especially on hitting the target item in the recommendation list.