Search

Document-level relation extraction (DocRE) aims to reason about complex relational facts among entities by reading, inferring, and aggregating among entities over multiple sentences in a document. Existing studies construct document-level graphs to enrich interactions between entities. However, these methods pay more attention to the entity nodes and their connections, regardless of the rich knowledge entailed in the original corpus.In this paper, we propose a commonsense knowledge enhanced document-level graph representation, called CGDRE, which delves into the semantic knowledge of the original corpus and improves the ability of DocRE. Firstly, we use coreference contrastive learning to capture potential commonsense knowledge. Secondly, we construct a heterogeneous graph to enhance the graph structure information according to the original document and commonsense knowledge. Lastly, CGDRE infers relations on the aggregated graph and uses focal loss to train the model. Remarkably, it is amazing that CGDRE can effectively alleviate the long-tailed distribution problem in DocRE. Experiments on the public datasets DocRED, DialogRE, and MPDD show that CGDRE can significantly outperform other baselines, achieving a significant performance improvement. Extensive analyses demonstrate that the performance of our CGDRE is contributed by the capture of commonsense knowledge enhanced graph relation representation. [ABSTRACT FROM AUTHOR]

Grounding temporal video segments described in natural language queries effectively and efficiently is a crucial capability needed in vision-and-language fields. In this article, we deal with the fast video temporal grounding (FVTG) task, aiming at localizing the target segment with high speed and favorable accuracy. Most existing approaches adopt elaborately designed cross-modal interaction modules to improve the grounding performance, which suffer from the test-time bottleneck. Although several common space-based methods enjoy the high-speed merit during inference, they can hardly capture the comprehensive and explicit relations between visual and textual modalities. In this article, to tackle the dilemma of the speed–accuracy tradeoff, we propose a commonsense-aware cross-modal alignment network (C2AN) that incorporates commonsense-guided visual and text representations into a complementary common space for fast video temporal grounding. Specifically, the commonsense concepts are explored and exploited by extracting the structural semantic information from a language corpus. Then, a commonsense-aware interaction module is designed to obtain bridged visual and text features by utilizing the learned commonsense concepts. Finally, to maintain the original semantic information of textual queries, a cross-modal complementary common space is optimized to obtain matching scores for performing FVTG. Extensive results on two challenging benchmarks show that our C2AN method performs favorably against states of the art while running at high speed. Our code is available at https://github.com/ZiyueWu59/CCA [ABSTRACT FROM AUTHOR]

Copyright of Journal of Fuzhou University is the property of Journal of Fuzhou University, Editorial Department and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Humor is an important component of daily communication and usually causes laughter that promotes mental and physical health. Understanding humor is sometimes difficult for humans and may be more difficult for AIs since it usually requires deep commonsense. In this paper, we focus on automatic humor recognition by extrapolating conceptual commonsense-aware modules to Pre-trained Masked Language Models (PMLMs) to provide external knowledge. Specifically, keywords are extracted from an input text and conceptual commonsense embeddings associated with the keywords are obtained by using a COMET decoder. By using multi-head attention the representations of the input text and the commonsense are integrated. In this way we attempt to enable the proposed model to access commonsense knowledge and thus recognize humor that is not detectable only by PMLM. Through the experiments on two datasets we explore different sizes of PMLMs and different amounts of commonsense and find some sweet spots of PMLMs’ scales for integrating commonsense to perform humor recognition well. Our proposed models improve the F1 score by up to 1.7% and 4.1% on the haHackathon and humicroedit datasets respectively. The detailed analyses show our models also improve the sensitivity to humor while retaining the predictive tendency of the corresponding PMLMs.

Emotion Recognition in Conversation (ERC) is a task aimed at predicting the emotions conveyed by an utterance in a dialogue. It is common in ERC research to integrate intra-utterance, local contextual, and global contextual information to obtain the utterance vectors. However, there exist complex semantic dependencies among these factors, and failing to model these dependencies accurately can adversely affect the effectiveness of emotion recognition. Moreover, to enhance the semantic dependencies within the context, researchers commonly introduce external commonsense knowledge after modeling it. However, injecting commonsense knowledge into the model simply without considering its potential impact can introduce unexpected noise. To address these issues, we propose a dialogue emotion model based on local–global context encoder and commonsense knowledge fusion attention. The local–global context encoder effectively integrates the information of intra-utterance, local context, and global context to capture the semantic dependencies among them. To provide more accurate external commonsense information, we present a fusion module to filter the commonsense information through multi-head attention. Our proposed method has achieved competitive results on four datasets and exhibits advantages compared with mainstream models using commonsense knowledge. [ABSTRACT FROM AUTHOR]

Representation learning serves as a crucial link between knowledge graphs and neural models. Knowledge graphs are typical symbolic models and require representation learning to obtain vector representations of entities and relationships. Existing efforts focus more on the structural characteristics of knowledge itself, such as the connections between entities and relationships under the same type of knowledge, neglecting the potential value of commonsense knowledge in guiding and constraining. In fact, commonsense knowledge implies the belonging tendency of entities in triplets. In this paper, we propose a novel knowledge representation learning model, CoolGust, which is guided and constrained by commonsense knowledge and can effectively utilize commonsense knowledge to seamlessly guide the belonging relationships of entities and enhance the model performance. Commonsense knowledge is essential for guiding humans to make wise decisions in unknown scenarios. We find that by utilizing commonsense knowledge as guides and constraints for entities, hidden knowledge entanglement structures can be formed in complex network applications, thereby constructing decisions consistent with situational logic. To verify the effectiveness of our model, we conducted experimental verification of two tasks, link prediction, and triple classification, on three public datasets. The experimental results demonstrate the effectiveness and advancedness of our proposed method. [ABSTRACT FROM AUTHOR]

Karyotyping requires chromosome instances to be segmented and classified from the metaphase images. One of the difficulties in chromosome segmentation is that the chromosomes are randomly positioned in the image, and there is a great chance for chromosomes to be touched or overlap with others. It is always much easier for operators and automatic programs to tackle images without overlapping chromosomes than ones with largely overlapped chromosomes. In order to reduce the processing difficulty, adding a smart image selection procedure ahead of segmentation is practical and necessary. In this paper, we introduce the Smart Karyotyping Image Selection (SKIS) based on Commonsense Knowledge Reasoning. The initial experiment demonstrates that the proposed approach can select the expected images based on reasoning and benefit following karyotyping processes. [ABSTRACT FROM AUTHOR]

One of the big challenges in robotics is the generalization necessary for performing unknown tasks in unknown environments on unknown objects. For us humans, this challenge is simplified by the commonsense knowledge we can access. For cognitive robotics, representing and acquiring commonsense knowledge is a relevant problem, so we perform a systematic literature review to investigate the current state of commonsense knowledge exploitation in cognitive robotics. For this review, we combine a keyword search on six search engines with a snowballing search on six related reviews, resulting in 2,048 distinct publications. After applying pre-defined inclusion and exclusion criteria, we analyse the remaining 52 publications. Our focus lies on the use cases and domains for which commonsense knowledge is employed, the commonsense aspects that are considered, the datasets/resources used as sources for commonsense knowledge and the methods for evaluating these approaches. Additionally, we discovered a divide in terminology between research from the knowledge representation and reasoning and the cognitive robotics community. This divide is investigated by looking at the extensive review performed by Zech et al. (The International Journal of Robotics Research, 2019, 38, 518-562), with whom we have no overlapping publications despite the similar goals. [ABSTRACT FROM AUTHOR]

One of the big challenges in robotics is the generalization necessary for performing unknown tasks in unknown environments on unknown objects. For us humans, this challenge is simplified by the commonsense knowledge we can access. For cognitive robotics, representing and acquiring commonsense knowledge is a relevant problem, so we perform a systematic literature review to investigate the current state of commonsense knowledge exploitation in cognitive robotics. For this review, we combine a keyword search on six search engines with a snowballing search on six related reviews, resulting in 2,048 distinct publications. After applying pre-defined inclusion and exclusion criteria, we analyse the remaining 52 publications. Our focus lies on the use cases and domains for which commonsense knowledge is employed, the commonsense aspects that are considered, the datasets/resources used as sources for commonsense knowledge and the methods for evaluating these approaches. Additionally, we discovered a divide in terminology between research from the knowledge representation and reasoning and the cognitive robotics community. This divide is investigated by looking at the extensive review performed by Zech et al. (The International Journal of Robotics Research, 2019, 38, 518–562), with whom we have no overlapping publications despite the similar goals.

This article delves into the evolving frontier of ontology-driven natural language information processing. Through an in-depth examination, we put forth a novel linguistic processor architecture, uniquely integrating linguistic and ontological paradigms during semantic analysis. Distancing from conventional methodologies, our approach showcases a profound merger of knowledge extraction and representation techniques. A central highlight of our research is the development of an ontology-driven information system, architected with an innate emphasis on self-enhancement and adaptability. The system’s salient capability lies in its adept handling of elementary knowledge, combined with its dynamic aptitude to foster innovative concepts and relationships. A particular focus is accorded to the system’s application in scientific information processing, signifying its potential in revolutionizing knowledge-based applications within scientific domains. Through our endeavors, we aim to pave the way for more intuitive, precise, and expansive ontology-driven tools in the realm of knowledge extraction and representation. [ABSTRACT FROM AUTHOR]

This article delves into the evolving frontier of ontology-driven natural language information processing. Through an in-depth examination, we put forth a novel linguistic processor architecture, uniquely integrating linguistic and ontological paradigms during semantic analysis. Distancing from conventional methodologies, our approach showcases a profound merger of knowledge extraction and representation techniques. A central highlight of our research is the development of an ontology-driven information system, architected with an innate emphasis on self-enhancement and adaptability. The system’s salient capability lies in its adept handling of elementary knowledge, combined with its dynamic aptitude to foster innovative concepts and relationships. A particular focus is accorded to the system’s application in scientific information processing, signifying its potential in revolutionising knowledge-based applications within scientific domains. Through our endeavours, we aim to pave the way for more intuitive, precise, and expansive ontology-driven tools in the realm of knowledge extraction and representation. [ABSTRACT FROM AUTHOR]

National Key Laboratory of General Artificial Intelligence, Beijing Institute for General Artificial Intelligence (BIGAI), China Retrieving out-of-reach objects is a crucial task in virtual reality (VR). One of the most commonly used approaches for this task is the gesture-based approach, which allows for bare-hand, eyes-free, and direct retrieval. However, previous work has primarily focused on assigned gesture design, neglecting the context. This can make it challenging to accurately retrieve an object from a large number of objects due to the one-to-one mapping metaphor, limitations of finger poses, and memory burdens. There is a general consensus that objects and contexts are related, which suggests that the object expected to be retrieved is related to the context, including the scene and the objects with which users interact. As such, we propose a commonsense knowledge-driven joint reasoning approach for object retrieval, where human grasping gestures and context are modeled using an And-Or graph (AOG). This approach enables users to accurately retrieve objects from a large number of candidate objects by using natural grasping gestures based on their experience of grasping physical objects. Experimental results demonstrate that our proposed approach improves retrieval accuracy. We also propose an object retrieval system based on the proposed approach. Two user studies show that our system enables efficient object retrieval in virtual environments (VEs). [ABSTRACT FROM AUTHOR]

This article delves into the evolving frontier of ontology-driven natural language information processing. Through an in-depth examination, we put forth a novel linguistic processor architecture, uniquely integrating linguistic and ontological paradigms during semantic analysis. Distancing from conventional methodologies, our approach showcases a profound merger of knowledge extraction and representation techniques. A central highlight of our research is the development of an ontology-driven information system, architected with an innate emphasis on self-enhancement and adaptability. The system’s salient capability lies in its adept handling of elementary knowledge, combined with its dynamic aptitude to foster innovative concepts and relationships. A particular focus is accorded to the system’s application in scientific information processing, signifying its potential in revolutionising knowledge-based applications within scientific domains. Through our endeavours, we aim to pave the way for more intuitive, precise, and expansive ontology-driven tools in the realm of knowledge extraction and representation.

Harnessing commonsense knowledge poses a significant challenge for machine comprehension systems. This paper primarily focuses on incorporating a specific subset of commonsense knowledge, namely, script knowledge. Script knowledge is about sequences of actions that are typically performed by individuals in everyday life. Our experiments were centered around the MCScript dataset, which was the basis of the SemEval-2018 Task 11: Machine Comprehension using Commonsense Knowledge. As a baseline, we utilized our Three-Way Attentive Networks (TriANs) framework to model the interactions among passages, questions, and answers. Building upon the TriAN, we proposed to: (1) integrate a pre-trained language model to capture script knowledge; (2) introduce multi-layer attention to facilitate multi-hop reasoning; and (3) incorporate positional embeddings to enhance the model's capacity for event-ordering reasoning. In this paper, we present our proposed methods and prove their efficacy in improving script knowledge integration and reasoning. [ABSTRACT FROM AUTHOR]

Abstract Action recognition has been a popular area of computer vision research in recent years. The goal of this task is to recognise human actions in video frames. Most existing methods often depend on the visual features and their relationships inside the videos. The extracted features only represent the visual information of the current video itself and cannot represent the general knowledge of particular actions beyond the video. Thus, there are some deviations in these features, and the recognition performance still requires improvement. In this sudy, we present a novel spatio‐temporal knowledge module (STKM) to endow the current methods with commonsense knowledge. To this end, we first collect hybrid external knowledge from universal fields, which contains both visual and semantic information. Then graph convolution networks (GCN) are used to represent and aggregate this knowledge. The GCNs involve (i) a spatial graph to capture spatial relations and (ii) a temporal graph to capture serial occurrence relations among actions. By integrating knowledge and visual features, we can get better recognition results. Experiments on AVA, UCF101‐24 and JHMDB datasets show the robustness and generalisation ability of STKM. The results report a new state‐of‐the‐art 32.0 mAP on AVA v2.1. On UCF101‐24 and JHMDB datasets, our method also improves by 1.5 AP and 2.6 AP, respectively, over the baseline method.

Aspect-based sentiment analysis (ABSA) is a fine-grained sentiment analysis task, whose goal is to identify the sentiment polarity of the specific aspect term in a given sentence. Previous work has realized the importance of commonsense knowledge, semantic and syntax information for aspect-based sentiment analysis, while few approaches take them into account simultaneously. To tackle this problem, we propose a novel graph convolutional network to incorporate commonsense knowledge, syntax and semantics information for this task. Specifically, we first construct an aspect-specific dependency tree rooted at aspect by reshaping an ordinary dependency parse tree and then integrate commonsense knowledge into the refined tree. Based on it, a semantic graph convolutional network is utilized to capture semantics information and a syntax-knowledge graph convolutional network with range-aware weight mechanism is adopted to encode important aspect-relevant commonsense knowledge and syntax information. Finally, an information exchange module is applied to interact commonsense knowledge, syntax and semantics information for classification. Experimental results demonstrate that our proposed model outperforms state-of-the-art models. [ABSTRACT FROM AUTHOR]

Action recognition has been a popular area of computer vision research in recent years. The goal of this task is to recognise human actions in video frames. Most existing methods often depend on the visual features and their relationships inside the videos. The extracted features only represent the visual information of the current video itself and cannot represent the general knowledge of particular actions beyond the video. Thus, there are some deviations in these features, and the recognition performance still requires improvement. In this sudy, we present a novel spatio‐temporal knowledge module (STKM) to endow the current methods with commonsense knowledge. To this end, we first collect hybrid external knowledge from universal fields, which contains both visual and semantic information. Then graph convolution networks (GCN) are used to represent and aggregate this knowledge. The GCNs involve (i) a spatial graph to capture spatial relations and (ii) a temporal graph to capture serial occurrence relations among actions. By integrating knowledge and visual features, we can get better recognition results. Experiments on AVA, UCF101‐24 and JHMDB datasets show the robustness and generalisation ability of STKM. The results report a new state‐of‐the‐art 32.0 mAP on AVA v2.1. On UCF101‐24 and JHMDB datasets, our method also improves by 1.5 AP and 2.6 AP, respectively, over the baseline method. [ABSTRACT FROM AUTHOR]

Sarcasm is a linguistic phenomenon indicating a difference between literal meanings and implied intentions. It is commonly used on blogs, e-commerce platforms, and social media. Numerous NLP tasks, such as opinion mining and sentiment analysis systems, are hampered by its linguistic nature in detection. Traditional techniques concentrated mostly on textual incongruity. Recent research demonstrated that the addition of commonsense knowledge into sarcasm detection is an effective new method. However, existing techniques cannot effectively capture sentence "incongruity" information or take good advantage of external knowledge, resulting in imperfect detection performance. In this work, new modules are proposed for maximizing the utilization of the text, the commonsense knowledge, and their interplay. At first, we propose an adaptive incongruity extraction module to compute the distance between each word in the text and commonsense knowledge. Two adaptive incongruity extraction modules are applied to text and commonsense knowledge, respectively, which can obtain two adaptive incongruity attention matrixes. Therefore, each of the words in the sequence receives a new representation with enhanced incongruity semantics. Secondly, we propose the incongruity cross-attention module to extract the incongruity between the text and the corresponding commonsense knowledge, thereby allowing us to pick useful commonsense knowledge in sarcasm detection. In addition, we propose an improved gate module as a feature fusion module of text and commonsense knowledge, which determines how much information should be considered. Experimental results on publicly available datasets demonstrate the superiority of our method in achieving state-of-the-art performance on three datasets as well as enjoying improved interpretability. [ABSTRACT FROM AUTHOR]

Empathy plays an important role in human conversations as an ability that enables individuals to understand the emotions and situations of others. Integrating empathy into dialogue systems is a crucial step in making them humanized. Relevant psychological studies have shown that a complete, high-quality empathetic dialogue should consist of the following two stages: (1) Empathetic Perception: the listener needs to perceive the emotional state of the speaker from both cognitive and affective aspects; (2) Empathetic Expression: the appropriate expression is chosen to respond to the perceived information. However, many existing studies on empathetic response generation only focus on one of these stages, resulting in incomplete and insufficiently empathetic responses. To this end, we propose the EmpCI, a two-stage empathetic response generation model that utilizes commonsense knowledge and mixed empathetic intent, respectively. Specifically, we use commonsense knowledge in the first stage to enhance the model's perception of the user's emotion and introduce mixed empathetic intent in the second stage to generate responses with appropriate expressions for the perceived information. Finally, we evaluated the EmpCI on the EmpatheticDialogues dataset, and extensive experiment results show that the proposed model outperforms the baselines in both perceiving users' emotions and generating empathetic responses. [ABSTRACT FROM AUTHOR]

Grounding on a commonsense knowledge subgraph can help the model generate more informative and diverse dialogue responses. Prior Traverse-based works explicitly retrieve a subgraph from the external knowledge base (eKB). Notably, the available knowledge is strictly restricted by the eKB. To break this restriction, Generative Retrieval methods externalize knowledge from the language model. However, they always generate boring knowledge due to their one-pass externalization procedure. This work proposes a novel TiLM Traverse in Language Model (TiLM) , which uses three 'Chain-of-Thought' sub-tasks, i.e., Query Entity Production , Topic Entity Prediction , and Knowledge Subgraph Completion , to build a high-quality knowledge subgraph to ground the next Response Generation without explicitly accessing the eKB in inference. Experimental results on both Chinese and English datasets demonstrate TiLM 's outstanding performance even only with a small scale of parameters. • We propose a novel TiLM for commonsense knowledge-ground dialogue response generations. • TiLM can generate high-quality knowledge subgraphs preserving structural information. • TiLM improves the performance without introducing more parameters. • Extensive experiments on two datasets have verified the effectiveness of TiLM. [ABSTRACT FROM AUTHOR]

In recent years, neural networks have been widely used in natural language processing, especially in sentence similarity modeling. Most of the previous studies focused on the current sentence, ignoring the commonsense knowledge related to the current sentence in the task of sentence similarity modeling. Commonsense knowledge can be remarkably useful for understanding the semantics of sentences. CK-Encoder, which can effectively acquire commonsense knowledge to improve the performance of sentence similarity modeling, is proposed in this paper. Specifically, the model first generates a commonsense knowledge graph of the input sentence and calculates this graph by using the graph convolution network. In addition, CKER, a framework combining CK-Encoder and sentence encoder, is introduced. Experiments on two sentence similarity tasks have demonstrated that CK-Encoder can effectively acquire commonsense knowledge to improve the capability of a model to understand sentences.

A popular chatbot can generate natural and human-like responses, and the crucial technology is the ability to understand and appreciate the emotions and demands expressed from the perspective of the user. However, some empathetic dialogue generation models only specialise in commonsense and neglect emotion, which can only get a one-sided understanding of the user's situation and makes the model unable to express emotion better. In this paper, we propose a novel affective feature knowledge interactive model named AFKI, to enhance response generation performance, which enriches conversation history to obtain emotional interactive context by leveraging fine-grained emotional features and commonsense knowledge. Furthermore, we utilise an emotional interactive context encoder to learn higher-level affective interaction information and distill the emotional state feature to guide the empathetic response generation. The emotional features are to well capture the subtle differences of the user's emotional expression, and the commonsense knowledge improves the representation of affective information on generated responses. Extensive experiments on the empathetic conversation task demonstrate that our model generates multiple responses with higher emotion accuracy and stronger empathetic ability compared with baseline model approaches for empathetic response generation. [ABSTRACT FROM AUTHOR]

Zero-shot learning — the problem of training and testing on a completely disjoint set of classes — relies greatly on its ability to transfer knowledge from train classes to test classes. Traditionally semantic embeddings consisting of human-defined attributes or distributed word embeddings are used to facilitate this transfer by improving the association between visual and semantic embeddings. In this paper, we take advantage of explicit relations between nodes defined in ConceptNet, a commonsense knowledge graph, to generate commonsense embeddings of the class labels by using a graph convolution network-based autoencoder. Our experiments performed on three standard benchmark datasets surpass the strong baselines when we fuse our commonsense embeddings with existing semantic embeddings, i.e., human-defined attributes and distributed word embeddings. This work paves the path to more brain-inspired approaches to zero-short learning. [ABSTRACT FROM AUTHOR]

Harnessing commonsense knowledge poses a significant challenge for machine comprehension systems. This paper primarily focuses on incorporating a specific subset of commonsense knowledge, namely, script knowledge. Script knowledge is about sequences of actions that are typically performed by individuals in everyday life. Our experiments were centered around the MCScript dataset, which was the basis of the SemEval-2018 Task 11: Machine Comprehension using Commonsense Knowledge. As a baseline, we utilized our Three-Way Attentive Networks (TriANs) framework to model the interactions among passages, questions, and answers. Building upon the TriAN, we proposed to: (1) integrate a pre-trained language model to capture script knowledge; (2) introduce multi-layer attention to facilitate multi-hop reasoning; and (3) incorporate positional embeddings to enhance the model’s capacity for event-ordering reasoning. In this paper, we present our proposed methods and prove their efficacy in improving script knowledge integration and reasoning.

Compared to sentence-level relation extraction, practical document-level relation extraction (DocRE) is a more challenging task for which multi-entity problems need to be resolved. It aims at extracting relationships between two entities over multiple sentences at once while taking into account significant cross-sentence features. Learning long-distance semantic relation representation across sentences in a document, however, is a widespread and difficult task. To address the above issues, we propose a novel S elf-supervised C ommonsense-enhanced D oc RE approach, named as SCDRE , bypassing the need for external knowledge. The methodology begins by harnessing self-supervised learning to capture the commonsense knowledge pertaining to each entity within an entity pair, drawing insights from the commonsense entailed text. This acquired knowledge subsequently serves as the foundation for transforming cross-sentence entity pairs into alias counterparts achieved by the coreference commonsense replacement. The focus then shifts to semantic relation representation learning, applied to these alias entity pairs. Through a process of entity pair rich attention fusion, these alias pairs are seamlessly and automatically translated back into the target entity pairs. This innovation harnesses self-supervised learning and contextual commonsense to distinguish SCDRE as a unique and self-contained approach, promising an enhanced ability to extract relationships from documents. We examine our model on three publicly accessible datasets, DocRED, DialogRE and MPDD, and the results show that it performs significantly better than strong baselines by 2.03% F1, and commonsense knowledge has an important contribution to the DocRE by the ablation experimental analysis. [ABSTRACT FROM AUTHOR]