Search

In Electronic Health Record (EHR) systems, key patient information is often captured in the form of unstructured clinical notes. The information from these notes can be extracted using Clinical Natural Language Processing (NLP). Training corpus is a key factor in development of efficient clinical NLP models. Clinical NLP corpus construction is complex and multifaceted. There are several challenges in corpus construction, but one challenge often not researched well is cohort selection aspect. In this study, we present methods employed and challenges encountered in cohort selection for construction of a clinical NLP corpus. In specific we present our methods in selection of cancer pathology reports to construct a corpus for automatic deidentification. 2100 pathology reports were extracted from 1833 (518 male and 1313 female) cancer patients using Health Level-7 (HL7) message standard. In terms of the age group distribution, the age group 60–70 years was highest with 872 patients. Our findings suggest deciphering the segment information from HL7 messages that are collected from different hospitals is a challenging task. The quality of HL7 messages also varied significantly with inconsistent tags making it difficult to identify reports that meet criteria set a priori. One of key lessons learned is linking the HL7 reports data with additional EMR data such as admissions, would help in identifying high quality reports and resolve duplicates. Also, our findings suggest that, in general the EHR data quality is poor with varying clinical coding and metadata standards between different hospitals. It is vital to identify and address these challenges for development of a high-quality corpus.

Currently, there are only a limited number of Japanese-Chinese bilingual corpora of a sufficient amount that can be used as training data for neural machine translation (NMT). In particular, there are few corpora that include spoken language such as daily conversation. In this research, we attempt to construct a Japanese-Chinese bilingual corpus of a certain scale by crawling the subtitle data of movies and TV series from the websites. We calculated the BLEU scores of the constructed WCC-JC (Web Crawled Corpus—Japanese and Chinese) and the other compared corpora. We also manually evaluated the translation results using the translation model trained on the WCC-JC to confirm the quality and effectiveness.

With the rapidly increasing use of social media platforms, much of our lives is spent online. Despite the great advantages of using social media, unfortunately, the spread of hate, cyberbullying, harassment, and trolling can be very common online. Many extremists use social media platforms to communicate their messages of hatred and spread violence, which may result in serious psychological consequences and even contribute to real-world violence. Thus, the aim of this research was to build the HateMotiv corpus, a freely available dataset that is annotated for types of hate crimes and the motivation behind committing them. The dataset was developed using Twitter as an example of social media platforms and could provide the research community with a very unique, novel, and reliable dataset. The dataset is unique as a consequence of its topic-specific nature and its detailed annotation. The corpus was annotated by two annotators who are experts in annotation based on unified guidelines, so they were able to produce an annotation of a high standard with F-scores for the agreement rate as high as 0.66 and 0.71 for type and motivation labels of hate crimes, respectively.

Nowadays, an increasing portion of our lives is spent interacting online through social media platforms, thanks to the widespread adoption of the latest technology and the proliferation of smartphones. Obtaining news from social media platforms is fast, easy, and less expensive compared with other traditional media platforms, e.g., television and newspapers. Therefore, social media is now being exploited to disseminate fake news and false information. This research aims to build the FakeAds corpus, which consists of tweets for product advertisements. The aim of the FakeAds corpus is to study the impact of fake news and false information in advertising and marketing materials for specific products and which types of products (i.e., cosmetics, health, fashion, or electronics) are targeted most on Twitter to draw the attention of consumers. The corpus is unique and novel, in terms of the very specific topic (i.e., the role of Twitter in disseminating fake news related to production promotion and advertisement) and also in terms of its fine-grained annotations. The annotation guidelines were designed with guidance by a domain expert, and the annotation is performed by two domain experts, resulting in a high-quality annotation, with agreement rate F-scores as high as 0.815.

Live blogs are an increasingly popular news format to cover breaking news and live events in online journalism. Online news websites around the world are using this medium to give their readers a minute by minute update on an event. Good summaries enhance the value of the live blogs for a reader, but are often not available. In this article, (a) we first define the task of summarizing a live blog, (b) study ways of automatically collecting corpora for live blog summarization, and (c) understand the complexity of the task by empirically evaluating well-known state-of-the-art unsupervised and supervised summarization systems on our new corpus. We show that live blog summarization poses new challenges in the field of news summarization, since frequency and positional signals cannot be used. We make our tools publicly available to reconstruct the corpus and to conduct our empirical experiments. This encourages the research community to build upon and replicate our results. [ABSTRACT FROM AUTHOR]

Motivation Annotation tools are applied to build training and test corpora, which are essential for the development and evaluation of new natural language processing algorithms. Further, annotation tools are also used to extract new information for a particular use case. However, owing to the high number of existing annotation tools, finding the one that best fits particular needs is a demanding task that requires searching the scientific literature followed by installing and trying various tools. Methods We searched for annotation tools and selected a subset of them according to five requirements with which they should comply, such as being Web-based or supporting the definition of a schema. We installed the selected tools (when necessary), carried out hands-on experiments and evaluated them using 26 criteria that covered functional and technical aspects. We defined each criterion on three levels of matches and a score for the final evaluation of the tools. Results We evaluated 78 tools and selected the following 15 for a detailed evaluation: BioQRator, brat, Catma, Djangology, ezTag, FLAT, LightTag, MAT, MyMiner, PDFAnno, prodigy, tagtog, TextAE, WAT-SL and WebAnno. Full compliance with our 26 criteria ranged from only 9 up to 20 criteria, which demonstrated that some tools are comprehensive and mature enough to be used on most annotation projects. The highest score of 0.81 was obtained by WebAnno (of a maximum value of 1.0). [ABSTRACT FROM AUTHOR]

Review of A Guide to Using Corpora for English Language Learners, by Robert Poole.

This paper presents the construction of a corpus of writings by Slovene learners of Japanese as a foreign language at the beginner and intermediate levels and an analysis of the grammar errors contained within it, with the purpose of providing a simple and effective means of acquiring data on errors made by students of Japanese as a second language. Additionally, an error analysis of the grammar errors in the corpus and a comparison of the most common errors found on both levels, reveals the types of errors that carry over from the beginner to the intermediate level, negatively affecting the learning process. By compiling and analyzing a collection of 182 written texts written by Japanese learners, 492 cases of grammar misuse were observed on the beginner and 564 on the intermediate level. A comparative analysis of the most common types of grammar misuse on each level highlights the types of errors that seem to carry over from the beginner to the intermediate level. The findings can be useful to Japanese language learners as well as teachers. Furthermore, the learner’s corpus created in the process marks the first step towards the creation of a larger, annotated and publicly accessible learner corpus of writings by Slovenian learners of Japanese to be used for further research in the field of second language acquisition.

Argumentation Machines search for arguments in natural language from information sources on the Web and reason with them on the knowledge level to actively support the deliberation and synthesis of arguments for a particular user query. The recap project is part of the Priority Program ratio and aims at novel contributions to and confluence of methods from information retrieval, knowledge representation, as well as case-based reasoning for the development of future argumentation machines. In this paper we summarise recent research results from the project. In particular, a new German corpus of 100 semantically annotated argument graphs from the domain of education politics has been created and is made available to the argumentation research community. Further, we discuss a comprehensive investigation in finding arguments and argument graphs. We introduce a probabilistic ranking framework for argument retrieval, i.e. for finding good premises for a designated claim. For finding argument graphs, we developed methods for case-based argument retrieval considering the graph structure of an argument together with textual and ontology-based similarity measures applied to claims, premises, and argument schemes. [ABSTRACT FROM AUTHOR]

Background: In this study, we focus on building a fine-grained entity annotation corpus with the corresponding annotation guideline of traditional Chinese medicine (TCM) clinical records. Our aim is to provide a basis for the fine-grained corpus construction of TCM clinical records in future.Methods: We developed a four-step approach that is suitable for the construction of TCM medical records in our corpus. First, we determined the entity types included in this study through sample annotation. Then, we drafted a fine-grained annotation guideline by summarizing the characteristics of the dataset and referring to some existing guidelines. We iteratively updated the guidelines until the inter-annotator agreement (IAA) exceeded a Cohen's kappa value of 0.9. Comprehensive annotations were performed while keeping the IAA value above 0.9.Results: We annotated the 10,197 clinical records in five rounds. Four entity categories involving 13 entity types were employed. The final fine-grained annotated entity corpus consists of 1104 entities and 67,799 tokens. The final IAAs are 0.936 on average (for three annotators), indicating that the fine-grained entity recognition corpus is of high quality.Conclusions: These results will provide a foundation for future research on corpus construction and named entity recognition tasks in the TCM clinical domain. [ABSTRACT FROM AUTHOR]

Copyright of Acta Linguistica Asiatica is the property of Acta Linguistica Asiatica and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

In this paper, we propose an Ontology-Style Relation (OSR) annotation approach. In conventional Relation Extraction (RE) datasets, relations are annotated as a link between two entity mentions. In contrast, in our OSR annotation, a relation is annotated as a relation mention (i.e. , not a link but a node) and rdfs:domain and rdfs:range links are annotated from the relation mention to its argument entity mentions. This approach has the following benefits: (1) the relation annotations can be easily converted to Resource Description Framework (RDF) triples to populate an Ontology, (2) some part of conventional RE tasks can be tackled as Named Entity Recognition (NER) tasks, and the relation classes are limited to several RDF properties, and (3) OSR annotations can be used for clear documentations of Ontology contents. We conducted two kinds of evaluation to investigate effects of OSR annotation. We converted (1) an in-house corpus of Japanese Rules of the Road (RoR) in conventional annotations into the OSR annotations and built a novel OSR-RoR corpus and (2) SemEval-2010 Task 8 dataset into the OSR annotations (called OSR-SemEval corpus). We compared the NER and RE performance using neural NER/RE tool DyGIE++ on the conventional and OSR annotations. The experimental results show that the OSR annotations make the RE task easier while introducing slight complexity into the NER task. • The relation annotations can be easily converted to Resource Description Framework (RDF) triples to populate an Ontology. • Some part of conventional RE tasks can be tackled as Named Entity Recognition (NER) tasks. The relation classes are limited to several RDF properties. • OSR annotations can be used for clear documentations of Ontology contents. [ABSTRACT FROM AUTHOR]

Amid the ongoing mental health crisis, there is an increasing need to discern possible signs of mental disturbance manifested in social media text. During in-person therapy sessions, mental health professionals employ manual methods to identify the root causes and outcomes of latent factors contributing to mental disturbances, which is a painstaking and time-intensive process. Neglecting multi-dimensional aspects of well-being (i.e., wellness dimensions) over time can adversely affect an individual's mind. To enable such fine-grained mental health screening, we define the task of identifying wellness concepts and classifying in pre-defined dimensions in Reddit posts. We construct a novel dataset called WellXplain , which consists of 3,092 instances and a total of 72,813 words. Our experts developed an annotation scheme based on a well-adapted Halbert L. Dunn's theory of wellness dimensions. Further, the data encompasses human-annotated text spans as pertinent explanations for decision-making during wellness concept classification. We anticipate that releasing the dataset and evaluating the baselines will facilitate the development of new language models for concept extraction and classification in healthcare domain. • Introducing the dataset importance in mental healthcare simulations. • Creating a corpus for well-being concept analysis. • Exploring domain-specific transformers and language models. • Assessing traditional multi-class classifier and its reliability. [ABSTRACT FROM AUTHOR]

Abstract Background Cardiovascular disease (CVD) has become the leading cause of death in China, and most of the cases can be prevented by controlling risk factors. The goal of this study was to build a corpus of CVD risk factor annotations based on Chinese electronic medical records (CEMRs). This corpus is intended to be used to develop a risk factor information extraction system that, in turn, can be applied as a foundation for the further study of the progress of risk factors and CVD. Results We designed a light annotation task to capture CVD risk factors with indicators, temporal attributes and assertions that were explicitly or implicitly displayed in the records. The task included: 1) preparing data; 2) creating guidelines for capturing annotations (these were created with the help of clinicians); 3) proposing an annotation method including building the guidelines draft, training the annotators and updating the guidelines, and corpus construction. Meanwhile, we proposed some creative annotation guidelines: (1) the under-threshold medical examination values were annotated for our purpose of studying the progress of risk factors and CVD; (2) possible and negative risk factors were concerned for the same reason, and we created assertions for annotations; (3) we added four temporal attributes to CVD risk factors in CEMRs for constructing long term variations. Then, a risk factor annotated corpus based on de-identified discharge summaries and progress notes from 600 patients was developed. Built with the help of clinicians, this corpus has an inter-annotator agreement (IAA) F1-measure of 0.968, indicating a high reliability. Conclusion To the best of our knowledge, this is the first annotated corpus concerning CVD risk factors in CEMRs and the guidelines for capturing CVD risk factor annotations from CEMRs were proposed. The obtained document-level annotations can be applied in future studies to monitor risk factors and CVD over the long term.

In this commentary, I highlight some of the novel contributions of Nathaniel Condit-Schultz's "MCFlow: A Digital Corpus of Rap Transcriptions" and discuss issues of rhyme definition, sampling and corpus construction, feature representation, and historical narratives.

The main purposes of this article are to provide an overview of a research project on a longitudinal learner spoken corpus and to share procedures related to the transcription of learners’ utterances from audio files using automated speech recognition (ASR) technology (IBM Watson Speech-to-text). The data of the corpus were collected twice or thrice a year for three consecutive years from 2016, creating eight data collection points altogether. They were gathered from 120 secondary school students who had been learning English in an English as a Foreign Language context for three years. The students were asked to take a monologue speaking test, the Telephone Standard Speaking Test, consisting of various tasks. The overall discussion of the article focuses on the details of this project and highlights how a methodological approach of combining electronic learner language data and ASR technology is useful in constructing learner spoken corpora.

Copyright of Contributions to Contemporary History / Prispevki za Novejšo Zgodovino is the property of Prispevki za Novejso Zgodovino and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

This paper describes the process by which we have constructed a corpus of heterogeneous texts about non-human animals. It aims to contribute both methodologically – with respect to the challenges of compiling a thematic corpus – and substantively – in relation to the identification of some features of discourse about animals. Having introduced the research project and its guiding questions, the paper describes the principles of data selection and the procedures used in the analysis. We highlight the methods we devised both to avoid the potential circularity associated with pre-determined search terms, and to overcome the limitations of a relatively small corpus containing a wide range of relevant vocabulary. We go on to report some initial findings on the most frequent animal naming terms and adjectives describing them, including a small case study of the adjectives live and dead. The paper concludes by indicating the ways in which the iterative methods we have employed are open to further extension, and points to some methodological and substantive implications of this enterprise. [ABSTRACT FROM AUTHOR]

This paper describes the construction process involved in creating a robust local learner corpus of texts produced by international students in a first-year writing course at a large public, mid-western university in the U.S. We show how involving faculty members and graduate students of our local writing program in the process of learner corpus analysis provides them with opportunities to develop their skills and knowledge as writing instructors, course designers, and, ultimately, knowledge producers. An additional benefit of such an undertaking is that the corpus can become part of the infrastructure of a research community that allows continued contributions by others individually and collaboratively. We also illustrate the usefulness of our local learner corpus for research, teaching, mentoring, and collaboration within our writing program with examples of the research projects and teaching interventions we have developed. [ABSTRACT FROM AUTHOR]

Removal of boilerplate is one of the essential tasks in web corpus construction and web indexing. Boilerplate (redundant and automatically inserted material like menus, copyright notices, navigational elements, etc.) is usually considered to be linguistically unattractive for inclusion in a web corpus. Also, search engines should not index such material because it can lead to spurious results for search terms if these terms appear in boilerplate regions of the web page. In this paper, I present and evaluate a supervised machine-learning approach to general-purpose boilerplate detection for languages based on Latin alphabets using Multi-Layer Perceptrons (MLPs). It is both very efficient and very accurate (between 95 % and $$99\,\%$$ correct classifications, depending on the input language). I show that language-specific classifiers greatly improve the accuracy of boilerplate detectors. The single features used for the classification are evaluated with regard to the merit they contribute to the classification. Furthermore, I show that the accuracy of the MLP is on a par with that of a wide range of other classifiers. My approach has been implemented in the open-source texrex web page cleaning software, and large corpora constructed using it are available from the COW initiative, including the CommonCOW corpora created from CommonCrawl datasets. [ABSTRACT FROM AUTHOR]

This paper introduces the Spoken British National Corpus 2014, an 11.5-million-word corpus of orthographically transcribed conversations among L1 speakers of British English from across the UK, recorded in the years 2012-2016. After showing that a survey of the recent history of corpora of spoken British English justifies the compilation of this new corpus, we describe the main stages of the Spoken BNC2014's creation: design, data and metadata collection, transcription, XML encoding, and annotation. In doing so we aim to (i) encourage users of the corpus to approach the data with sensitivity to the many methodological issues we identified and attempted to overcome while compiling the Spoken BNC2014, and (ii) inform (future) compilers of spoken corpora of the innovations we implemented to attempt to make the construction of corpora representing spontaneous speech in informal contexts more tractable, both logistically and practically, than in the past. [ABSTRACT FROM AUTHOR]

Objective: To build a comprehensive corpus covering syntactic and semantic annotations of Chinese clinical texts with corresponding annotation guidelines and methods as well as to develop tools trained on the annotated corpus, which supplies baselines for research on Chinese texts in the clinical domain.Materials and Methods: An iterative annotation method was proposed to train annotators and to develop annotation guidelines. Then, by using annotation quality assurance measures, a comprehensive corpus was built, containing annotations of part-of-speech (POS) tags, syntactic tags, entities, assertions, and relations. Inter-annotator agreement (IAA) was calculated to evaluate the annotation quality and a Chinese clinical text processing and information extraction system (CCTPIES) was developed based on our annotated corpus.Results: The syntactic corpus consists of 138 Chinese clinical documents with 47,426 tokens and 2612 full parsing trees, while the semantic corpus includes 992 documents that annotated 39,511 entities with their assertions and 7693 relations. IAA evaluation shows that this comprehensive corpus is of good quality, and the system modules are effective.Discussion: The annotated corpus makes a considerable contribution to natural language processing (NLP) research into Chinese texts in the clinical domain. However, this corpus has a number of limitations. Some additional types of clinical text should be introduced to improve corpus coverage and active learning methods should be utilized to promote annotation efficiency.Conclusions: In this study, several annotation guidelines and an annotation method for Chinese clinical texts were proposed, and a comprehensive corpus with its NLP modules were constructed, providing a foundation for further study of applying NLP techniques to Chinese texts in the clinical domain. [ABSTRACT FROM AUTHOR]

Web news provides a quick and convenient means to create collections of large documents. The creation of a web news corpus has typically required the construction of a set of HTML parsing rules to identify content text. In general, these parsing rules are written manually and treat different web pages differently. We address this issue and propose a news content recognition algorithm that is language and layout independent. Our method first scans a given HTML document and roughly localizes a set of candidate news areas. Next, we apply a designed scoring function to rank the best content. To validate this approach, we evaluate the systems performance using 1092 items of multilingual web news data covering 17 global regions and 11 distinct languages. We compare these data with nine published content extraction systems using standard settings. The results of this empirical study show that our method outperforms the second-best approach (Boilerpipe) by 6.04 and 10.79 % with regard to the relative micro and macro F-measures, respectively. We also apply our system to monitor online RSS news distribution. It collected 0.4 million news articles from 200 RSS channels in 20 days. This sample quality test shows that our method achieved 93 % extraction accuracy for large news streams. [ABSTRACT FROM AUTHOR]

An overview of the proposed methodology is demonstrated in Figure 1. The presented approach consists of two important modules: the information retrieval (IR) module and the pretrained BERT model. The IR module aims to choose segments of a set of texts that includes a list of answers. Then, the BERT model recognizes an answer span from the sentence clause based on the combination of segments. The core stages of the presented approach are as follows: (1) domain-geoscience ontology construction, (2) data preprocessing, (3) dataset development, (4) BERT module training and testing, and (5) approach evaluation and prototype development. The first step constructs the geological domain ontology for building the original corpus of the QA system. The second step is the data preprocessing used to remove meaningless words from the ontology description for further construction of the dataset. The third step is to convert the questions and paragraphs into suitable data structures to form the training dataset. The fourth step is to train the pretrained BERT model and test the answers generated from the user input questions. The last step is to build a prototype QA system and evaluate it. [Display omitted] • Proposed a method and process for constructing automatic question-and-answer datasets based on the names and definitions in the ontology. • A robust end-to-end approach is developed to improve the efficiency and effectiveness of retrieving terminology queries in the geological domain. • Developed a prototype chatbot system based on the WeChat platform and construct experiments for evaluation. Mineral exploration reports and documents are a rich data source that contains a large amount of geological environments in which mineral deposits form. Among them, it is difficult to extract the required answers from the large amount of geological data. Despite the availability of search engines and digital databases that can be used to store geological data, users are unable to retrieve the information needed for a specific field in a timely manner. As a result, users usually have to contend with the burden of browsing and filtering information, which can be a time-consuming process. To address this issue, we propose a robust end-to-end approach that can improve the efficiency and effectiveness of retrieving queries related to mineral exploration terms. First, we present an automated workflow for constructing automatic question-and-answer datasets based on the names and definitions in the mineral exploration ontology. The Bidirectional Encoder Representation from Transformers (BERT) model is trained to test the answers generated from the user input question. Finally, a prototype chatbot system based on the WeChat platform and constructed experiments for evaluation is presented. Our proposed method has powerful feature representation and learning capabilities and thus has the potential to be adopted by other specialized fields (especially where a large number of mineral exploration ontologies already exist). [ABSTRACT FROM AUTHOR]

Cardiovascular disease has a significant impact on both society and patients, making it necessary to conduct knowledge-based research such as research that utilizes knowledge graphs and automated question answering. However, the existing research on corpus construction for cardiovascular disease is relatively limited, which has hindered further knowledge-based research on this disease. Electronic medical records contain patient data that span the entire diagnosis and treatment process and include a large amount of reliable medical information. Therefore, we collected electronic medical record data related to cardiovascular disease, combined the data with relevant work experience and developed a standard for labeling cardiovascular electronic medical record entities and entity relations. By building a sentence-level labeling result dictionary through the use of a rule-based semi-automatic method, a cardiovascular electronic medical record entity and entity relationship labeling corpus (CVDEMRC) was constructed. The CVDEMRC contains 7691 entities and 11,185 entity relation triples, and the results of consistency examination were 93.51% and 84.02% for entities and entity-relationship annotations, respectively, demonstrating good consistency results. The CVDEMRC constructed in this study is expected to provide a database for information extraction research related to cardiovascular diseases.

In this commentary, I highlight some of the novel contributions of Nathaniel Condit-Schultz's "MCFlow: A Digital Corpus of Rap Transcriptions" and discuss issues of rhyme definition, sampling and corpus construction, feature representation, and historical narratives. [ABSTRACT FROM AUTHOR]