Search

Distant supervised relation extraction is an efficient strategy of finding relational facts from unstructured text without labeled training data. A recent paradigm to develop relation extractors is using pre-trained Transformer language models to produce high-quality sentence representations. However, due to the original Transformer is weak at capturing local dependencies and phrasal structures, existing Transformer-based methods cannot identify various relational patterns in sentences. To address this issue, we propose a novel distant supervised relation extraction model, which employs a specific-designed pattern-aware self-attention network to automatically discover relational patterns for pre-trained Transformers in an end-to-end manner. Specifically, the proposed method assumes that the correlation between two adjacent tokens reflects the probability that they belong to the same pattern. Based on this assumption, a novel self-attention network is designed to generate the probability distribution of all patterns in a sentence. Then, the probability distribution is applied as a constraint in the first Transformer layer to encourage its attention heads to follow the relational pattern structures. As a result, fine-grained pattern information is enhanced in the pre-trained Transformer without losing global dependencies. Extensive experimental results on two popular benchmark datasets demonstrate that our model performs better than the state-of-the-art baselines. [ABSTRACT FROM AUTHOR]

Document-level relation extraction is a complex natural language processing task that predicts relations of entity pairs by capturing the critical semantic features on entity pairs from the document. However, current methods usually consider that the entity pairs contain the vast majority of information which can represent relational facts, and thus focus on modeling the entity pair, ignoring features on whole document and sentences. In the document-level relation extraction, the distance between entity pairs is relatively long. Judging the relation between entities usually requires reading many sentences or the whole document. Therefore, sentences and documents are particularly crucial for document-level relation extraction. In order to make full use of the multi-level information of sentences and documents, this paper proposes a document-level relation extraction framework with two advantages. First, we use the encoder to obtain the semantic features about the document and use the inter-sentence attention based on entity pairs to dynamically capture the features of multiple vital sentences. Second, we design a document gating that combines sentence-level features with document-level features to predict relations. Extensive experiments on a benchmark dataset have well-validated the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Food recommendation has attracted increasing attentions to various food-related applications and services. The food recommender models aim to match users' preferences with recipes, where the key lies in the representation learning of users and recipes. However, ranging from early content-based filtering and collaborative filtering methods to recent hybrid methods, the existing work overlooks the various food-related relations, especially the ingredient-ingredient relations, leading to incomprehensive representations. To bridge this gap, we propose a novel model Food recommendation with Graph Convolutional Network (FGCN), which exploits ingredient-ingredient, ingredient-recipe, and recipe-user relations deeply. FGCN employs the information propagation mechanism and adopts multiple embedding propagation layers to model high-order connectivity across different food-related relations and enhance the representations. Specifically, we develop three types of information propagation: (1) ingredient-ingredient information propagation, (2) ingredient-recipe information propagation, and (3) recipe-user information propagation. To validate the effectiveness and rationality of FGCN, we conduct extensive experiments on a real-world dataset. The results show that the proposed FGCN outperforms the state-of-the-art baselines. Further in-depth analyses reveal that FGCN could alleviate the sparsity issue in food recommendation. [ABSTRACT FROM AUTHOR]

Fundamental information extraction tasks, such as relation extraction and event detection, suffer from a data imbalance problem. To alleviate this problem, existing methods rely mostly on well-designed loss functions to reduce the negative influence of imbalanced data. However, this approach requires additional hyper-parameters and limits scalability. Furthermore, these methods can only benefit specific tasks and do not provide a unified framework across relation extraction and event detection. In this paper, a Classifier-Adaptation Knowledge Distillation (CAKD) framework is proposed to address these issues, thus improving relation extraction and event detection performance. The first step is to exploit sentence-level identification information across relation extraction and event detection, which can reduce identification errors caused by the data imbalance problem without relying on additional hyper-parameters. Moreover, this sentence-level identification information is used by a teacher network to guide the baseline model's training by sharing its classifier. Like an instructor, the classifier improves the baseline model's ability to extract this sentence-level identification information from raw texts, thus benefiting overall performance. Experiments were conducted on both relation extraction and event detection using the Text Analysis Conference Relation Extraction Dataset (TACRED) and Automatic Content Extraction (ACE) 2005 English datasets, respectively. The results demonstrate the effectiveness of the proposed framework. [ABSTRACT FROM AUTHOR]

Node classification in structural networks is a longstanding important problem in many real-world applications. Recent studies have shown that network embedding can greatly facilitate node classification by employing embedding algorithms to learn feature representations of nodes. Despite of promising performance, existing network embedding based methods are hard to capture the actual category features of a node because of the linearly inseparable problem in low-dimensional space; meanwhile they cannot incorporate both network structure information and node labels information into the representations simultaneously. To address the above problems, this paper presents a novel Deep Kernel Supervised Hashing (DKSH) method to learn hashing representations of nodes for node classification. Specifically, a deep multiple kernel learning is first employed to map nodes into suitable Hilbert space to deal with linearly inseparable problem. Then, instead of only considering structural similarity between two nodes, a novel similarity matrix is designed to merge both network structure information and node labels information. Supervised by the similarity matrix, the learned hashing representations can preserve the two kinds of information simultaneously from the learned Hilbert space. Extensive experiments show that the proposed method significantly outperforms the state-of-the-art baselines over three real-world benchmark datasets. [ABSTRACT FROM AUTHOR]

Convolutional neural networks (CNN) have been widely used in various tasks, largely due to their ability to efficiently extract n-gram features for text analysis and document representation. In this paper, we intend to insight the CNN model regarding its capability on text analysis. Vanilla CNNs do have weaknesses when it comes to the representation and feature extraction. Duplicate filters are inevitable with vanilla CNNs, which reduces the discriminative power of the representations. In addition, the current pooling operations either limit the CNN to the local optimum (i.e., max pooling) or they do not consider the importance of all features (i.e., mean pooling). In this paper, we propose two modules for vanilla CNNs to overcome these shortcomings. The first equips the CNN with discriminative filters (distinct filters with maximised divergence) and the second provides the ability to comprehensively extract all salient features. Specifically, our model increases the discriminative power of the model by maximizing the distance between different filters, and a novel global pooling mechanism for feature extraction. Validation tests against state-of-the-art baselines on five benchmark classification datasets achieve the competitive performance of our proposed model. Furthermore, visualization on upgrade filters and pooling features verify our hypothesis that the proposed model can receive discriminative filters and salient features. [ABSTRACT FROM AUTHOR]

Meta-embedding aims at assembling pre-trained embeddings from various sources and producing more expressively powerful word representations. Many natural language processing (NLP) tasks in a specific domain benefit from meta-embedding, especially when the task suffers from low resources. This paper proposes an unsupervised meta-embedding method that jointly models background knowledge from the source embeddings and domain-specific knowledge from the task domain. Specifically, embeddings from multiple sources for a word are dynamically aggregated to a single meta-embedding by a differentiable attention module. The embeddings derived from pre-training on a large-scale corpus provide complete background knowledge of word usage. Then, the meta-embedding is further enriched by exploring domain-specific knowledge from each task domain in two ways. First, contextual information in the raw corpus is considered to capture the semantics of words. Second, a graph representing domain-specific semantic structures is extracted from the raw corpus to highlight the relationships between salient words, then the graph is modeled by a powerful graph convolution network to effectively capture rich semantic structures among words in the task domain. Experiments conducted on two tasks, i.e., text classification and relation extraction, show that our model outputs more accurate word meta-embeddings for the task domain, compared to other state-of-the-art competitors.. [ABSTRACT FROM AUTHOR]