Search

Chemical-induced disease (CID) relation extraction from biomedical articles plays an important role in disease treatment and drug development. Existing methods are insufficient for capturing complete document level semantic information due to ignoring semantic information of entities in different sentences. In this work, we proposed an effective document-level relation extraction model to automatically extract intra-/inter-sentential CID relations from articles. Firstly, our model employed BERT to generate contextual semantic representations of the title, abstract and shortest dependency paths (SDPs). Secondly, to enhance the semantic representation of the whole document, cross attention with self-attention (named cross2self-attention) between abstract, title and SDPs was proposed to learn the mutual semantic information. Thirdly, to distinguish the importance of the target entity in different sentences, the Gaussian probability distribution was utilized to compute the weights of the co-occurrence sentence and its adjacent entity sentences. More complete semantic information of the target entity is collected from all entities occurring in the document via our presented document-level R-BERT (DocR-BERT). Finally, the related representations were concatenated and fed into the softmax function to extract CIDs. We evaluated the model on the CDR corpus provided by BioCreative V. The proposed model without external resources is superior in performance as compared with other state-of-the-art models (our model achieves 53.5%, 70%, and 63.7% of the F1-score on inter-/intra-sentential and overall CDR dataset). The experimental results indicate that cross2self-attention, the Gaussian probability distribution and DocR-BERT can effectively improve the CID extraction performance. Furthermore, the mutual semantic information learned by the cross self-attention from abstract towards title can significantly influence the extraction performance of document-level biomedical relation extraction tasks.

Word segmentation is an essential and challenging task in natural language processing, especially for the Chinese language due to its high linguistic complexity. Existing methods for Chinese word segmentation, including statistical machine learning methods and neural network methods, usually have good performance in specific knowledge domains. Given the increasing importance of interdisciplinary and cross-domain studies, one of the challenges in cross-domain word segmentation is to handle the out-of-vocabulary (OOV) words. Existing methods show unsatisfactory performance to meet the practical standard. To this end, we propose a document-level context-aware model that can automatically perceive and identify OOV words from different domains. Our method jointly implements a word-based and a character-based model and then processes the results with a newly proposed reconstruction model. We evaluate the new method by designing and conducting comprehensive experiments on two real-world datasets (e.g., news from different domains). The results demonstrate the superiority of our method over the state-of-the-art models in handling texts from different domains. Importantly, when doing the word segmentation under the cross-domain scenario, our proposed method can improve the performance of OOV words recognition.

Aspect-based sentiment classification (ABSC) is a fine-grained analysis task that obtains different sentiment polarities contained in a single text from the views of different aspects. Its practicability draws so much attention from researchers that the number of related works grows explosively. However, existing works mainly aim to obtain polarities from short texts (shorter than 100 words), only a few works analyze documents (shorter than 500 words), but almost no work analyzes long documents (LD, longer than 500 words). This situation makes ABSC powerless when dealing with some texts like in-depth analysis articles. In this paper, we make ABSC step into the LD level by proposing the Hierarchical Aspect-Oriented Framework for Long Document (HAOFL). HAOFL solves two challenges that rarely appear in short texts and normal documents. The first is the too-long input sequence that can cause the model to forget previously learned information or ignore the tailed unlearned information. The second is the unstable sentiment information of the target aspect contained in LD, which increases the difficulty for a model to draw a proper result. HAOFL constructs the data transformation module, dependency processing module, and sentiment aggregation module to solve these two challenges. Numerical experiments prove HAOFL can solve the aforementioned challenges and achieve superior performance in an effective and resource-saving way. With HAOFL, the performances of popular ABSC models on LD are improved at most 8.69% of accuracy and 11.37% of F1-score. In terms of resource-consuming, up to 82.10% of training time and 71.03% of GPU memory are saved.

BACKGROUND Anaesthesiology is one of the safest fields in medicine today in relation to mortality. Deaths directly because of anaesthesia have fortunately now become rare exceptions. Nevertheless, important findings can still be drawn from the rare deaths that still occur. OBJECTIVE The aim of this study was to identify and analyse the causes of deaths related to anaesthesia alone over a 10-year period. DESIGN Retrospective structured analysis of a database of medical liability claims. SETTING Hospitals at all levels of care in Germany. PATIENTS The database of a large insurance broker included data for 81 413 completed liability claims over the 10-year period from 2006 to 2015. Among 1914 cases associated with anaesthetic procedures, 56 deaths were identified. Of these, 30 clearly involved anaesthesia (Edwards category 1) and were included in the evaluation. INTERVENTIONS None (retrospective database analysis). MAIN OUTCOME MEASURES Causes of anaesthesia-related death identified from medical records, court records, expert opinions and autopsy reports. RESULTS The 30 deaths were analysed in detail at the case and document level. They included high proportions of 'potentially avoidable' deaths, at 86.6%, and what are termed 'never events', at 66.7%. Problems with the airway were the cause in 40% and problems with correct monitoring in 20%. In addition, communication problems were identified as a 'human factor' in 50% of the cases. CONCLUSION The majority of the anaesthesia-related deaths investigated could very probably have been avoided with simple anaesthesiological measures if routine guidelines had been followed and current standards observed. Actions to be taken are inferred from these results, and recommendations are made. In future, greater care must be taken to ensure that the level of safety already achieved in anaesthesiology can be maintained despite demographic developments and increasing economic pressures.

Document-level sentiment classification is dedicated to predicting the sentiment polarity of document-level reviews posted by users about products and services. Many methods use neural networks have achieved very successful results on sentiment classification tasks. These methods usually focus on mining useful information from the text of the review documents. However, they ignore the importance of users’ review habits. The reviews posted by the same user when commenting on different products contain similar review habits, and reviews that contain highly similar review habits often have similar sentiment ratings. In this paper, we propose a novel sentiment classification algorithm that utilizes user’s review habits to enhance hierarchical neural networks, namely as HUSN. Firstly, we divide the reviews in the training set according to the users. All the reviews of each user are aggregated together and called the historical reviews of this user. Secondly, the target review in the test set and its multiple historical reviews in the training set are sent to the Long Short-Term Memory based hierarchical neural network to obtain the corresponding review document representations containing the user’s review habits. Finally, we calculate the similarities between the target review document representation and multiple historical review document representations. The higher the similarity, the closer the review habits of different reviews from the same user, and the closer the corresponding sentiment ratings. Experimental results show that the similarities between the review habits of different reviews from the same user can further improve the performance of document-level sentiment classification. The HUSN algorithm performs better than all baseline methods on three publicly available document-level review datasets.

In this study, we aimed to improve the performance results of Arabic sentiment analysis. This can be achieved by investigating the most successful machine learning method and the most useful feature vector to classify sentiments in both term and document levels into two (positive or negative) categories. Moreover, specification of one polarity degree for the term that has more than one is investigated. Also to handle the negations and intensifications, some rules are developed. According to the obtained results, Artificial Neural Network classifier is nominated as the best classifier in both term and document level sentiment analysis (SA) for Arabic Language. Furthermore, the average F-score achieved in the term level SA for both positive and negative testing classes is 0.92. In the document level SA, the average F-score for positive testing classes is 0.94, while for negative classes is 0.93., 10 pages, 3 figures, Published in: 2019 IEEE 7th Palestinian International Conference on Electrical and Computer Engineering (PICECE), Date of Conference: 26-27 March 2019

We study the properties of \begin{document}$ \Phi $\end{document} -irregular sets (sets of points for which the Birkhoff average diverges) in dynamical systems with the shadowing property. We estimate the topological entropy of \begin{document}$ \Phi $\end{document} -irregular set in terms of entropy on chain recurrent classes and prove that \begin{document}$ \Phi $\end{document} -irregular sets of full entropy are typical. We also consider \begin{document}$ \Phi $\end{document} -level sets (sets of points whose Birkhoff average is in a specified interval), relating entropy they carry with the entropy of some ergodic measures. Finally, we study the problem of large deviations considering the level sets with respect to reference measures.

Document-level financial event extraction (DFEE) is the task of detecting events and extracting the corresponding event arguments in financial documents, which plays an important role in information extraction in the financial domain. This task is challenging as the financial documents are generally long text and event arguments of one event may be scattered in different sentences. To address this issue, we proposed a novel Prior Information Enhanced Extraction framework (PIEE) for DFEE, leveraging prior information from both event types and pre-trained language models. Specifically, PIEE consists of three components: event detection, event argument extraction, and event table filling. In event detection, we identify the event type. Then, the event type is explicitly used for event argument extraction. Meanwhile, the implicit information within language models also provides considerable cues for event arguments localization. Finally, all the event arguments are filled in an event table by a set of predefined heuristic rules. To demonstrate the effectiveness of our proposed framework, we participated in the share task of CCKS2020 Task 4-2: Document-level Event Arguments Extraction. On both Leaderboard A and Leaderboard B, PIEE took the first place and significantly outperformed the other systems.

Background Both intra- and inter-sentential semantic relations in biomedical texts provide valuable information for biomedical research. However, most existing methods either focus on extracting intra-sentential relations and ignore inter-sentential ones or fail to extract inter-sentential relations accurately and regard the instances containing entity relations as being independent, which neglects the interactions between relations. We propose a novel sequence labeling-based biomedical relation extraction method named Bio-Seq. In the method, sequence labeling framework is extended by multiple specified feature extractors so as to facilitate the feature extractions at different levels, especially at the inter-sentential level. Besides, the sequence labeling framework enables Bio-Seq to take advantage of the interactions between relations, and thus, further improves the precision of document-level relation extraction. Results Our proposed method obtained an F1-score of 63.5% on BioCreative V chemical disease relation corpus, and an F1-score of 54.4% on inter-sentential relations, which was 10.5% better than the document-level classification baseline. Also, our method achieved an F1-score of 85.1% on n2c2-ADE sub-dataset. Conclusion Sequence labeling method can be successfully used to extract document-level relations, especially for boosting the performance on inter-sentential relation extraction. Our work can facilitate the research on document-level biomedical text mining.

This paper shows a significant comparison of two primary bibliographic data sources at the document level of Scopus and Dimensions. The emphasis is on the differences in their document coverage by institution level of aggregation. The main objective is to assess whether Dimensions offers at the institutional level good new possibilities for bibliometric analysis as at the global level. The results of a comparative study of the citation count profiles of articles published by faculty members of Prince of Songkla University (PSU) in Dimensions and Scopus from the year the databases first included PSU-authored papers (1970 and 1978, respectively) through the end of June 2020. Descriptive statistics and correlation analysis of 19,846 articles indexed in Dimensions and 13,577 indexed in Scopus. The main finding was that the number of citations received by Dimensions was highly correlated with citation counts in Scopus. Spearman’s correlation between citation counts in Dimensions and Scopus was a high and mighty relationship. The findings mainly affect Dimensions’ possibilities as instruments for carrying out bibliometric analysis of university members’ research productivity. University researchers can use Dimensions to retrieve information, and the design policies can be used to evaluate research using scientific databases.

Understanding patient opinions expressed towards healthcare services in online platforms could allow healthcare professionals to respond to address patients' concerns in a timely manner. Extracting patient opinion towards various aspects of health services is closely related to aspect-based sentiment analysis (ABSA) in which we need to identify both opinion targets and target-specific opinion expressions. The lack of aspect-level annotations however makes it difficult to build such an ABSA system. This paper proposes a joint learning framework for simultaneous unsupervised aspect extraction at the sentence level and supervised sentiment classification at the document level. It achieves 98.2% sentiment classification accuracy when tested on the reviews about healthcare services collected from Yelp, outperforming several strong baselines. Moreover, our model can extract coherent aspects and can automatically infer the distribution of aspects under different polarities without requiring aspect-level annotations for model learning.

Event is a common but non-negligible knowledge type. How to identify events from texts, extract their arguments, even analyze the relations between different events are important for many applications. This paper summaries some constructed event-centric knowledge graphs and the recent typical approaches for event and event relation extraction, besides task description, widely used evaluation datasets, and challenges. Specifically, in the event extraction task, we mainly focus on three recent important research problems: 1) how to learn the textual semantic representations for events in sentence-level event extraction; 2) how to extract relations across sentences or in a document level; 3) how to acquire or augment labeled instances for model training. In event relation extraction, we focus on the extraction approaches for three typical event relation types, including coreference, causal and temporal relations, respectively. Finally, we give out our conclusion and potential research issues in the future.

Document-level sentiment analysis is a challenging task given the large size of the text, which leads to an abundance of words and opinions, at times contradictory, in the same document. This analysis is particularly useful in analyzing press articles and blog posts about a particular product or company, and it requires a high concentration, especially when the topic being discussed is sensitive. Nevertheless, most existing models and techniques are designed to process short text from social networks and collaborative platforms. In this paper, we propose a combination of Convolutional Neural Networks (CNN) and Bidirectional Long Short-Term Memory (BiLSTM) models, with Doc2vec embedding, suitable for opinion analysis in long texts. The CNN-BiLSTM model is compared with CNN, LSTM, BiLSTM and CNN-LSTM models with Word2vec/Doc2vec embeddings. The Doc2vec with CNN-BiLSTM model was applied on French newspapers articles and outperformed the other models with 90.66% accuracy.

Latent Dirichlet Allocation (LDA) is a topic model that represents a document as a distribution of multiple topics. It expresses each topic as a distribution of multiple words by mining semantic relationships hidden in text. However, traditional LDA ignores some of the semantic features hidden inside the document semantic structure of medium and long texts. Instead of using the original LDA to model the topic at the document level, it is better to refine the document into different semantic topic units. In this paper, we propose an improved LDA topic model based on partition (LDAP) for medium and long texts. LDAP not only preserves the benefits of the original LDA but also refines the modeled granularity from the document level to the semantic topic level, which is particularly suitable for the topic modeling of the medium and long text. The extensive experimental classification results on Fudan University corpus and Sougou Lab corpus demonstrate that LDAP achieves better performance compared with other topic models, such as LDA, HDP, LSA and doc2vec.

The world has recently seen unprecedented growth of social networks that have enabled internet users to share and communicate their views in various areas of life. Analysis of opinion is one type of text mining that aims to discover human views and emotions about any particular event, or topic. In the experiment of this research, we perform document level sentiment analysis in the form of positive and negative sentiments using both the traditional models and recent deep learning models. Three traditional methods (i.e. Naive Bayes (NB), Logistic Regression (LR), and Voting Ensemble) and three deep learning layout (i.e. Single Convolutional Neural Network (CNN), Single CNN + FastText and Ensemble(2-Layer CNN + Gated Recurrent Unit (GRU))) have been employed for two different datasets to evaluate the opinion classification. In our experiment, accuracy, f1-score, and RMSE have been considered as evaluation parameters and the optimistic results show that for airline review dataset, Single CNN + FastText comes out with the highest performance whereas for hotel review dataset, 2-L CNN + GRU comes out with the highest performance.

A huge amount of data is available on the internet and social media platforms. It is necessary to use this in data in an efficient way. The growth of the companies depends on the satisfaction of their customers. This makes the sentiment analysis an important and challenging activity. In this literature review, we identified various machine learning algorithms that can be used for this purpose.

The advent of large pre-trained language models has made it possible to make high-quality predictions on how to add or change a sentence in a document. However, the high branching factor inherent to text generation impedes the ability of even the strongest language models to offer useful editing suggestions at a more global or document level. We introduce a new task, document sketching, which involves generating entire draft documents for the writer to review and revise. These drafts are built from sets of documents that overlap in form - sharing large segments of potentially reusable text - while diverging in content. To support this task, we introduce a Wikipedia-based dataset of analogous documents and investigate the application of weakly supervised methods, including use of a transformer-based mixture of experts, together with reinforcement learning. We report experiments using automated and human evaluation methods and discuss relative merits of these models., Findings of ACL 2021

Document-level event extraction aims to recognize event information from a whole piece of article. Existing methods are not effective due to two challenges of this task: a) the target event arguments are scattered across sentences; b) the correlation among events in a document is non-trivial to model. In this paper, we propose Heterogeneous Graph-based Interaction Model with a Tracker (GIT) to solve the aforementioned two challenges. For the first challenge, GIT constructs a heterogeneous graph interaction network to capture global interactions among different sentences and entity mentions. For the second, GIT introduces a Tracker module to track the extracted events and hence capture the interdependency among the events. Experiments on a large-scale dataset (Zheng et al., 2019) show GIT outperforms the previous methods by 2.8 F1. Further analysis reveals GIT is effective in extracting multiple correlated events and event arguments that scatter across the document. Our code is available at https://github.com/RunxinXu/GIT., Comment: Accepted by ACL-IJCNLP 2021 main conference (Long Paper)

Open Information Extraction (OpenIE) aims to extract structured relational tuples (subject, relation, object) from sentences and plays critical roles for many downstream NLP applications. Existing solutions perform extraction at sentence level, without referring to any additional contextual information. In reality, however, a sentence typically exists as part of a document rather than standalone; we often need to access relevant contextual information around the sentence before we can accurately interpret it. As there is no document-level context-aware OpenIE dataset available, we manually annotate 800 sentences from 80 documents in two domains (Healthcare and Transportation) to form a DocOIE dataset for evaluation. In addition, we propose DocIE, a novel document-level context-aware OpenIE model. Our experimental results based on DocIE demonstrate that incorporating document-level context is helpful in improving OpenIE performance. Both DocOIE dataset and DocIE model are released for public., To appear in Findings of ACL 2021

Sentiment Analysis is the computer science field that comprises techniques that aim to automatically extract opinions from texts. Usually, these techniques assign a Sentiment Orientation to the whole document (Document Level Sentiment Analysis). But a document can express sentiment about several aspects of an entity. Methods that extract those aspects, paired with the sentiment about them, operate in the Aspect Level. Aspect-Based Sentiment Analysis approaches can be split into two stages: Aspect Extraction and Aspect Sentiment Classification. The literature presents works mainly focused on reviews about hotels, smartphones, or restaurants. In this work, we present an approach for Aspect Extraction based on Multilingual (Google's) and Portuguese (BERTimbau) BERT pre-trained models. Our experiments show that Aspect Extraction based on BERT pre-trained for Portuguese achieved Balanced Accuracy of up to 93% on a corpus of reviews about the accommodation sector.

Document-level sentiment classification is to assign an overall sentiment polarity to an opinion document. Some researchers have already realized that, in addition to document texts, extensional-information such as product features and user preferences can be quite useful. Many previous studies represent them as ID-type extensional-information and incorporate them into deep learning models. However, they ignore the descriptive extensional information that is also useful for document representations. This paper covers the following aspects: (1) introduces the Description of Opinion Target (DOT), a new extensional-information for document-level sentiment classification, (2) builds the Document-level Sentiment ClassificatioN with EXTensional-information (DSC_NEXT) dataset which consists of three datasets: IMDB_NEXT, Yelp_NEXT and CMRDB_NEXT and (3) validates the effectiveness of DOT by performing experiments based on current state-of-the-art (SOTA) document-level sentiment analysis methods. Implications for using extensional-information in neural network models are also considered.

Document-level sentiment classification is a fundamental task in Natural Language Processing (NLP). Previous studies have demonstrated the importance of personalized sentiment classification by taking user preference and product characteristics on the sentiment ratings into consideration. The state-of-the-art approaches incorporate such information via attention mechanism, where the attention weights are calculated after the texts are encoded into the low-dimensional vectors with LSTM-based models. However, user and product information may be discarded in the process of generating the semantic representations. In this paper, we propose a novel User-Product gated LSTM network (UP-LSTM), which incorporates user and product information into LSTM cells at the same time of generating text representations. Therefore, UP-LSTM can dynamically produce user- and product-aware contextual representations of texts. Moreover, we devise another version of it to improve the training efficiency. We conduct a comprehensive evaluation with three real world datasets. Experimental results show that our model outperforms previous approaches by an obvious margin.

Document-level relation extraction (RE) aims to extract relations between entities within a document. Unlike sentence-level RE, it requires integrating evidences across multiple sentences. However, current models still lack the ability to effectively obtain relevant evidences for relation inference from multi-granularity information in the document. In this paper, we propose Hierarchical Aggregation and Inference Network (HAIN), performing the model to effectively predict relations by using global and local information from the document. Specifically, HAIN first constructs a meta dependency graph (mDG) to capture rich long distance global dependency information across the document. It also constructs a mention interaction graph (MG) to model complex local interactions among different mentions. Finally, it creates an entity inference graph (EG), based on which we design a novel hybrid attention mechanism to integrate relevant global and local information for entities. Experimental results demonstrate that our model achieves superior performance on a large-scale document-level dataset (DocRED). Extensive analyses also show that the model is particularly effective in extracting relations between entities across multiple sentences and mentions.