Search

Combining different input modalities beyond text is a key challenge for natural language processing. Previous work has been inconclusive as to the true utility of images as a supplementary information source for text classification tasks, motivating this large-scale human study of labelling performance given text-only, images-only, or both text and images. To this end, we create a new dataset accompanied with a novel annotation method—Japanese Entity Labeling with Dynamic Annotation—to deepen our understanding of the effectiveness of images for multi-modal text classification. By performing careful comparative analysis of human performance and the performance of state-of-the-art multi-modal text classification models, we gain valuable insights into differences between human and model performance, and the conditions under which images are beneficial for text classification.

Introduction Patients with ambulatory advanced heart failure (HF) are increasingly considered for durable mechanical circulatory support (MCS) and heart transplantation and their effective triage requires careful assessment of the clinical trajectory. Methods REVIVAL, a prospective, observational study, enrolled 400 ambulatory advanced HF patients from 21 MCS/transplant centers in 2015-2016. Study design included a clinical re-assessment of Interagency Registry for Mechanically Assisted Circulatory Support (INTERMACS) profile within 120 days after enrollment. The prognostic impact of a worsening INTERMACS Profile assigned by the treating physician was assessed at 1 year after the Early Relook. Results Early Relook was done in 325 of 400 patients (81%), of whom 24% had a worsened INTERMACS Profile, associated with longer HF history and worse baseline INTERMACS profile, but no difference in baseline LVEF (median 0.20), 6-minute walk, quality of life, or other baseline parameters. Early worsening predicted higher rate of the combined primary endpoint of death, urgent MCS, or urgent transplant by 1 year after Early Relook, (28% vs 15%), with hazard ratio 2.2 (95% CI 1.2- 3.8; p = .006) even after adjusting for baseline INTERMACS Profile and Seattle HF Model score. Deterioration to urgent MCS occurred in 14% vs 5% (p = .006) during the year after Early Relook. Conclusions Early Relook identifies worsening of INTERMACS Profile in a significant population of ambulatory advanced HF, who had worse outcomes over the subsequent year. Early reassessment of ambulatory advanced HF patients should be performed to better define the trajectory of illness and inform triage to advanced therapies.

State-of-the-art classification and regression models are often not well calibrated, and cannot reliably provide uncertainty estimates, limiting their utility in safety-critical applications such as clinical decision-making. While recent work has focused on calibration of classifiers, there is almost no work in NLP on calibration in a regression setting. In this paper, we quantify the calibration of pre- trained language models for text regression, both intrinsically and extrinsically. We further apply uncertainty estimates to augment training data in low-resource domains. Our experiments on three regression tasks in both self-training and active-learning settings show that uncertainty estimation can be used to increase overall performance and enhance model generalization.

Existing question answering systems struggle to answer factoid questions when geospatial information is involved. This is because most systems cannot accurately detect the geospatial semantic elements from the natural language questions, or capture the semantic relationships between those elements. In this paper, we propose a geospatial semantic encoding schema and a semantic graph representation which captures the semantic relations and dependencies in geospatial questions. We demonstrate that our proposed graph representation approach aids in the translation from natural language to a formal, executable expression in a query language. To decrease the need for people to provide explanatory information as part of their question and make the translation fully automatic, we treat the semantic encoding of the question as a sequential tagging task, and the graph generation of the query as a semantic dependency parsing task. We apply neural network approaches to automatically encode the geospatial questions into spatial semantic graph representations. Compared with current template-based approaches, our method generalises to a broader range of questions, including those with complex syntax and semantics. Our proposed approach achieves better results on GeoData201 than existing methods.

Conversing about places with a computer poses a range of challenges to current AI.

Objectives This study sought to determine cardiopulmonary exercise (CPX) predictors of the combined outcome of durable mechanical circulatory support (MCS), transplantation, or death at 1 year among patients with ambulatory advanced heart failure (HF). Background Optimal CPX predictors of outcomes in contemporary ambulatory advanced HF patients are unclear. Methods REVIVAL (Registry Evaluation of Vital Information for ventricular assist devices [VADs] in Ambulatory Life) enrolled 400 systolic HF patients, INTERMACS (Interagency Registry for Mechanically Assisted Circulatory Support) profiles 4-7. CPX was performed by 273 subjects 2 ± 1 months after study enrollment. Discriminative power of maximal (peak oxygen consumption [peak V O 2]; V O 2 pulse, circulatory power [CP]; peak systolic blood pressure • peak V O 2], peak end-tidal pressure C O 2 [PEtC O 2], and peak Borg scale score) and submaximal CPX parameters (ventilatory efficiency [VE/VC O 2 slope]; V O 2 at anaerobic threshold [V O 2AT]; and oxygen uptake efficiency slope [OUES]) to predict the composite outcome were assessed by univariate and multivariate Cox regression and Harrell’s concordance statistic. Results At 1 year, there were 39 events (6 transplants, 15 deaths, 18 MCS implantations). Peak V O 2, V O 2AT, OUES, peak PEtCO2, and CP were higher in the no-event group (all p Conclusions Among patients with ambulatory advanced HF, the strongest maximal and submaximal CPX predictor of MCS implantation, transplantation, or death at 1 year were CP and VE/VCO2, respectively. The patient-reported measure of exercise effort (Borg scale score) contributed substantially to the prediction of outcomes, a surprising and novel finding that warrants further investigation. (Registry Evaluation of Vital Information for VADs in Ambulatory Life [REVIVAL]; NCT01369407 )

Background Assessment of the quality of medical evidence available on the web is a critical step in the preparation of systematic reviews. Existing tools that automate parts of this task validate the quality of individual studies but not of entire bodies of evidence and focus on a restricted set of quality criteria. Objective We proposed a quality assessment task that provides an overall quality rating for each body of evidence (BoE), as well as finer-grained justification for different quality criteria according to the Grading of Recommendation, Assessment, Development, and Evaluation formalization framework. For this purpose, we constructed a new data set and developed a machine learning baseline system (EvidenceGRADEr). Methods We algorithmically extracted quality-related data from all summaries of findings found in the Cochrane Database of Systematic Reviews. Each BoE was defined by a set of population, intervention, comparison, and outcome criteria and assigned a quality grade (high, moderate, low, or very low) together with quality criteria (justification) that influenced that decision. Different statistical data, metadata about the review, and parts of the review text were extracted as support for grading each BoE. After pruning the resulting data set with various quality checks, we used it to train several neural-model variants. The predictions were compared against the labels originally assigned by the authors of the systematic reviews. Results Our quality assessment data set, Cochrane Database of Systematic Reviews Quality of Evidence, contains 13,440 instances, or BoEs labeled for quality, originating from 2252 systematic reviews published on the internet from 2002 to 2020. On the basis of a 10-fold cross-validation, the best neural binary classifiers for quality criteria detected risk of bias at 0.78 F1 (P=.68; R=0.92) and imprecision at 0.75 F1 (P=.66; R=0.86), while the performance on inconsistency, indirectness, and publication bias criteria was lower (F1 in the range of 0.3-0.4). The prediction of the overall quality grade into 1 of the 4 levels resulted in 0.5 F1. When casting the task as a binary problem by merging the Grading of Recommendation, Assessment, Development, and Evaluation classes (high+moderate vs low+very low-quality evidence), we attained 0.74 F1. We also found that the results varied depending on the supporting information that is provided as an input to the models. Conclusions Different factors affect the quality of evidence in the context of systematic reviews of medical evidence. Some of these (risk of bias and imprecision) can be automated with reasonable accuracy. Other quality dimensions such as indirectness, inconsistency, and publication bias prove more challenging for machine learning, largely because they are much rarer. This technology could substantially reduce reviewer workload in the future and expedite quality assessment as part of evidence synthesis.

While pretrained language models (LMs) have driven impressive gains over morpho-syntactic and semantic tasks, their ability to model discourse and pragmatic phenomena is less clear. As a step towards a better understanding of their discourse modeling capabilities, we propose a sentence intrusion detection task. We examine the performance of a broad range of pretrained LMs on this detection task for English. Lacking a dataset for the task, we introduce INSteD, a novel intruder sentence detection dataset, containing 170,000+ documents constructed from English Wikipedia and CNN news articles. Our experiments show that pretrained LMs perform impressively in in-domain evaluation, but experience a substantial drop in the cross-domain setting, indicating limited generalization capacity. Further results over a novel linguistic probe dataset show that there is substantial room for improvement, especially in the cross- domain setting.

Background Publication of registered clinical trials is a critical step in the timely dissemination of trial findings. However, a significant proportion of completed clinical trials are never published, motivating the need to analyze the factors behind success or failure to publish. This could inform study design, help regulatory decision-making, and improve resource allocation. It could also enhance our understanding of bias in the publication of trials and publication trends based on the research direction or strength of the findings. Although the publication of clinical trials has been addressed in several descriptive studies at an aggregate level, there is a lack of research on the predictive analysis of a trial’s publishability given an individual (planned) clinical trial description. Objective We aimed to conduct a study that combined structured and unstructured features relevant to publication status in a single predictive approach. Established natural language processing techniques as well as recent pretrained language models enabled us to incorporate information from the textual descriptions of clinical trials into a machine learning approach. We were particularly interested in whether and which textual features could improve the classification accuracy for publication outcomes. Methods In this study, we used metadata from ClinicalTrials.gov (a registry of clinical trials) and MEDLINE (a database of academic journal articles) to build a data set of clinical trials (N=76,950) that contained the description of a registered trial and its publication outcome (27,702/76,950, 36% published and 49,248/76,950, 64% unpublished). This is the largest data set of its kind, which we released as part of this work. The publication outcome in the data set was identified from MEDLINE based on clinical trial identifiers. We carried out a descriptive analysis and predicted the publication outcome using 2 approaches: a neural network with a large domain-specific language model and a random forest classifier using a weighted bag-of-words representation of text. Results First, our analysis of the newly created data set corroborates several findings from the existing literature regarding attributes associated with a higher publication rate. Second, a crucial observation from our predictive modeling was that the addition of textual features (eg, eligibility criteria) offers consistent improvements over using only structured data (F1-score=0.62-0.64 vs F1-score=0.61 without textual features). Both pretrained language models and more basic word-based representations provide high-utility text representations, with no significant empirical difference between the two. Conclusions Different factors affect the publication of a registered clinical trial. Our approach to predictive modeling combines heterogeneous features, both structured and unstructured. We show that methods from natural language processing can provide effective textual features to enable more accurate prediction of publication success, which has not been explored for this task previously.

The perceived quality of a document is affected by various factors, including grammat- icality, readability, stylistics, and expertise depth, making the task of document quality assessment a complex one. In this paper, we explore this task in the context of assessing the quality of Wikipedia articles and academic papers. Observing that the visual rendering of a document can capture implicit quality indicators that are not present in the document text — such as images, font choices, and visual layout — we propose a joint model that combines the text content with a visual rendering of the document for document qual- ity assessment. Our joint model achieves state-of-the-art results over five datasets in two domains (Wikipedia and academic papers), which demonstrates the complementarity of textual and visual features, and the general applicability of our model. To examine what kinds of features our model has learned, we further train our model in a multi-task learning setting, where document quality assessment is the primary task and feature learning is an auxiliary task. Experimental results show that visual embeddings are better at learning structural features while textual embeddings are better at learning readability scores, which further verifies the complementarity of visual and textual features.

Background Worsening heart failure (HF) and health-related quality of life (HRQOL) have been shown to impact the decision to proceed with left ventricular assist device (LVAD) implantation, but little is known about how socioeconomic factors influence expressed patient preference for LVAD. Methods and Results Ambulatory patients with advanced systolic HF (n=353) reviewed written information about LVAD therapy and completed a brief survey to indicate whether they would want an LVAD to treat their current level of HF. Ordinal logistic regression analyses identified clinical and demographic predictors of LVAD preference. Higher New York Heart Association (NYHA) class, worse HRQOL measured by Kansas City Cardiomyopathy Questionnaire, lower education level, and lower income were significant univariable predictors of patients wanting an LVAD. In the multivariable model, higher NYHA class (OR [odds ratio]: 1.43, CI [confidence interval]: 1.08–1.90, P = .013) and lower income level (OR: 2.10, CI: 1.18 – 3.76, P = .012 for $80,000) remained significantly associated with wanting an LVAD. Conclusion Among ambulatory patients with advanced systolic HF, treatment preference for LVAD was influenced by level of income independent of HF severity. Understanding the impact of socioeconomic factors on willingness to consider LVAD therapy may help tailor counseling towards individual needs.

Due to the Covid-19 pandemic, universities were required to shift teaching online in 2020, including exams. This rapid move required many changes and compromises to be made in order for students to take their exams remotely. We report on the experiences of running online exams for a large CS1 course at an Australian university, describing the changes made to the online learning platform to support this task, as well as the authoring of the exam and the support provided to students. Several clear instances of academic misconduct are presented to illustrate some of the issues endemic to online exams. Broadly, this approach to examination was a success that we plan to continue in future years, and we reflect on how the approach can be improved.

BACKGROUND Assessment of the quality of medical evidence available on the web is a critical step in the preparation of systematic reviews. Existing tools that automate parts of this task validate the quality of individual studies but not of entire bodies of evidence and focus on a restricted set of quality criteria. OBJECTIVE We proposed a quality assessment task that provides an overall quality rating for each body of evidence (BoE), as well as finer-grained justification for different quality criteria according to the Grading of Recommendation, Assessment, Development, and Evaluation formalization framework. For this purpose, we constructed a new data set and developed a machine learning baseline system (EvidenceGRADEr). METHODS We algorithmically extracted quality-related data from all summaries of findings found in the Cochrane Database of Systematic Reviews. Each BoE was defined by a set of population, intervention, comparison, and outcome criteria and assigned a quality grade (high, moderate, low, or very low) together with quality criteria (justification) that influenced that decision. Different statistical data, metadata about the review, and parts of the review text were extracted as support for grading each BoE. After pruning the resulting data set with various quality checks, we used it to train several neural-model variants. The predictions were compared against the labels originally assigned by the authors of the systematic reviews. RESULTS Our quality assessment data set, Cochrane Database of Systematic Reviews Quality of Evidence, contains 13,440 instances, or BoEs labeled for quality, originating from 2252 systematic reviews published on the internet from 2002 to 2020. On the basis of a 10-fold cross-validation, the best neural binary classifiers for quality criteria detected risk of bias at 0.78 F1 (P=.68; R=0.92) and imprecision at 0.75 F1 (P=.66; R=0.86), while the performance on inconsistency, indirectness, and publication bias criteria was lower (F1 in the range of 0.3-0.4). The prediction of the overall quality grade into 1 of the 4 levels resulted in 0.5 F1. When casting the task as a binary problem by merging the Grading of Recommendation, Assessment, Development, and Evaluation classes (high+moderate vs low+very low-quality evidence), we attained 0.74 F1. We also found that the results varied depending on the supporting information that is provided as an input to the models. CONCLUSIONS Different factors affect the quality of evidence in the context of systematic reviews of medical evidence. Some of these (risk of bias and imprecision) can be automated with reasonable accuracy. Other quality dimensions such as indirectness, inconsistency, and publication bias prove more challenging for machine learning, largely because they are much rarer. This technology could substantially reduce reviewer workload in the future and expedite quality assessment as part of evidence synthesis.

Background As antimicrobial prescribers, veterinarians contribute to the emergence of MDR pathogens. Antimicrobial stewardship programmes are an effective means of reducing the rate of development of antimicrobial resistance. A key component of antimicrobial stewardship programmes is selecting an appropriate antimicrobial agent for the presenting complaint and using an appropriate dose rate for an appropriate duration. Objectives To describe antimicrobial usage, including dose, for common indications for antimicrobial use in companion animal practice. Methods Natural language processing (NLP) techniques were applied to extract and analyse clinical records. Results A total of 343 668 records for dogs and 109 719 records for cats administered systemic antimicrobials from 1 January 2013 to 31 December 2017 were extracted from the database. The NLP algorithms extracted dose, duration of therapy and diagnosis completely for 133 046 (39%) of the records for dogs and 40 841 records for cats (37%). The remaining records were missing one or more of these elements in the clinical data. The most common reason for antimicrobial administration was skin disorders (n = 66 198, 25%) and traumatic injuries (n = 15 932, 19%) in dogs and cats, respectively. Dose was consistent with guideline recommendations in 73% of cases where complete clinical data were available. Conclusions Automated extraction using NLP methods is a powerful tool to evaluate large datasets and to enable veterinarians to describe the reasons that antimicrobials are administered. However, this can only be determined when the data presented in the clinical record are complete, which was not the case in most instances in this dataset. Most importantly, the dose administered varied and was often not consistent with guideline recommendations.

While automatic summarization evaluation methods developed for English are routinely applied to other languages, this is the first attempt to systematically quantify their panlinguistic efficacy. We take a summarization corpus for eight different languages, and manually annotate generated summaries for focus (precision) and coverage (recall). Based on this, we evaluate 19 summarization evaluation metrics, and find that using multilingual BERT within BERTScore performs well across all languages, at a level above that for English., Findings of ACL 2021

Objectives This study sought to explore clinical characteristics and outcomes in women and men with ambulatory advanced heart failure (HF). Background Women have been underrepresented in studies of advanced HF and have an increased mortality on the transplant waiting list and early after mechanical circulatory support (MCS). An increased understanding of the differential burden of HF between women and men is required to inform the use of mechanical circulatory support in ambulatory advanced HF patients. Methods REVIVAL (Registry Evaluation for Vital Information on Ventricular Assist Devices in Ambulatory Life) is a prospective, observational study of 400 outpatients with chronic systolic HF, New York Heart Association functional class II to IV, and 1 additional high-risk feature. Clinical characteristics, quality of life, and functional capacity were compared between women and men, as was a primary composite endpoint of death, durable MCS, or urgent heart transplantation at 1 year. Results REVIVAL enrolled 99 women (25% of the cohort) who had similar age, ejection fraction, INTERMACS (Interagency Registry for Mechanically Assisted Circulatory Support) profiles, medication use, and willingness to consider MCS as the men enrolled; however, women reported significantly greater limitations in quality of life with respect to physical limitation, reduced 6-min walk distance, and more frequent symptoms of depression. Nevertheless, 1-year combined risk of death, durable MCS, or urgent transplantation did not differ between women and men (24% vs. 22%; p = 0.94). Conclusions This study represents the largest report to date of women with ambulatory advanced HF receiving contemporary therapies. Systematic elicitation of patient-reported outcome measures uncovered an added burden of HF in women and may be an appropriate target of augmented support and intervention.

The CRISPR/Cas system could provide an efficient and reliable means of editing the human genome and has the potential to revolutionize modern medicine; however, rapid developments are raising complex ethical issues. There has been significant scientific debate regarding the acceptability of some applications of CRISPR/Cas, with leaders in the field highlighting the need for the lay public's views to shape expert discussion. As such, we sought to determine the factors that influence public opinion on gene editing. We created a 17-item online survey translated into 11 languages and advertised worldwide. Topic modeling was used to analyze textual responses to determine what factors influenced respondents' opinions toward human somatic or embryonic gene editing, and how this varied among respondents with differing attitudes and demographic backgrounds. A total of 3,988 free-text responses were analyzed. Respondents had a mean age of 32 (range, 11-90) years, and 37% were female. The most prevalent topics cited were Future Generations, Research, Human Editing, Children, and Health. Respondents who disagreed with gene editing for health-related purposes were more likely to cite the topic Better Understanding than those who agreed to both somatic and embryonic gene editing. Respondents from Western backgrounds more frequently discussed Future Generations, compared with participants from Eastern countries. Religious respondents did not cite the topic Religious Beliefs more frequently than did nonreligious respondents, whereas Christian respondents were more likely to cite the topic Future Generations. Our results suggest that public resistance to human somatic or embryonic gene editing does not stem from an inherent mistrust of genome modification, but rather a desire for greater understanding. Furthermore, we demonstrate that factors influencing public opinion vary greatly amongst demographic groups. It is crucial that the determinants of public attitudes toward CRISPR/Cas be well understood so that the technology does not suffer the negative public sentiment seen with previous genetic biotechnologies.

The COVID-19 pandemic has driven ever-greater demand for tools which enable efficient exploration of biomedical literature. Although semi-structured information resulting from concept recognition and detection of the defining elements of clinical trials (e.g. PICO criteria) has been commonly used to support literature search, the contributions of this abstraction remain poorly understood, especially in relation to text-based retrieval. In this study, we compare the results retrieved by a standard search engine with those filtered using clinically-relevant concepts and their relations. With analysis based on the annotations from the TREC-COVID shared task, we obtain quantitative as well as qualitative insights into characteristics of relational and concept-based literature exploration. Most importantly, we find that the relational concept selection filters the original retrieved collection in a way that decreases the proportion of unjudged documents and increases the precision, which means that the user is likely to be exposed to a larger number of relevant documents., Accepted at HealthNLP'21

Existing work on probing of pretrained language models (LMs) has predominantly focused on sentence-level syntactic tasks. In this paper, we introduce document-level discourse probing to evaluate the ability of pretrained LMs to capture document-level relations. We experiment with 7 pretrained LMs, 4 languages, and 7 discourse probing tasks, and find BART to be overall the best model at capturing discourse -- but only in its encoder, with BERT performing surprisingly well as the baseline model. Across the different models, there are substantial differences in which layers best capture discourse information, and large disparities between models., Accepted at NAACL 2021

Chemical patents represent a valuable source of information about new chemical compounds, which is critical to the drug discovery process. Automated information extraction over chemical patents is, however, a challenging task due to the large volume of existing patents and the complex linguistic properties of chemical patents. The Cheminformatics Elsevier Melbourne University (ChEMU) evaluation lab 2020, part of the Conference and Labs of the Evaluation Forum 2020 (CLEF2020), was introduced to support the development of advanced text mining techniques for chemical patents. The ChEMU 2020 lab proposed two fundamental information extraction tasks focusing on chemical reaction processes described in chemical patents: (1)chemical named entity recognition, requiring identification of essential chemical entities and their roles in chemical reactions, as well as reaction conditions; and (2)event extraction, which aims at identification of event steps relating the entities involved in chemical reactions. The ChEMU 2020 lab received 37 team registrations and 46 runs. Overall, the performance of submissions for these tasks exceeded our expectations, with the top systems outperforming strong baselines. We further show the methods to be robust to variations in sampling of the test data. We provide a detailed overview of the ChEMU 2020 corpus and its annotation, showing that inter-annotator agreement is very strong. We also present the methods adopted by participants, provide a detailed analysis of their performance, and carefully consider the potential impact of data leakage on interpretation of the results. The ChEMU 2020 Lab has shown the viability of automated methods to support information extraction of key information in chemical patents.

Chemical patents serve as an indispensable source of information about new discoveries of chemical compounds. The ChEMU (Cheminformatics Elsevier Melbourne University) lab addresses information extraction over chemical patents, and aims to advance the state of the art on this topic. ChEMU lab 2021, as part of the 12th Conference and Labs of the Evaluation Forum (CLEF-2021), will be the second ChEMU lab. ChEMU 2021 will provide two distinct tasks related to reference resolution in chemical patents. Task 1—Chemical Reaction Reference Resolution—focuses on paragraph-level references and aims to identify the chemical reactions or general conditions specified in one reaction description referred to by another. Task 2—Anaphora Resolution—focuses on expression-level references and aims to identify the reference relationships between expressions in chemical reaction descriptions. In this paper, we introduce ChEMU 2021, including its motivation, goals, tasks, resources, and evaluation framework.

In this paper, we provide an overview of the Cheminformatics Elsevier Melbourne University (ChEMU) evaluation lab 2021, part of the Conference and Labs of the Evaluation Forum 2021 (CLEF 2021). The ChEMU evaluation lab focuses on information extraction over chemical reactions from patent texts. As the second instance of our ChEMU lab series, we build upon the ChEMU corpus developed for ChEMU 2020, extending it for two distinct tasks related to reference resolution in chemical patents. Task 1—Chemical Reaction Reference Resolution—focuses on paragraph-level references and aims to identify the chemical reactions or general conditions specified in one reaction description referred to by another. Task 2—Anaphora Resolution—focuses on expression-level references and aims to identify the reference relationships between expressions in chemical reaction descriptions. Herein, we describe the resources created for these tasks and the evaluation methodology adopted. We also provide a brief summary of the results obtained in this lab, finding that one submission achieves substantially better results than our baseline models.

We present COVID-SEE, a system for medical literature discovery based on the concept of information exploration, which builds on several distinct text analysis and natural language processing methods to structure and organise information in publications, and augments search through a visual overview of a collection enabling exploration to identify key articles of interest. We developed this system over COVID-19 literature to help medical professionals and researchers explore the literature evidence, and improve findability of relevant information. COVID-SEE is available at http://covid-see.com.

Chemical patents contain rich coreference and bridging links, which are the target of this research. Specially, we introduce a novel annotation scheme, based on which we create the ChEMU-Ref dataset from reaction description snippets in English-language chemical patents. We propose a neural approach to anaphora resolution, which we show to achieve strong results, especially when jointly trained over coreference and bridging links.

We introduce a top-down approach to discourse parsing that is conceptually simpler than its predecessors (Kobayashi et al., 2020; Zhang et al., 2020). By framing the task as a sequence labelling problem where the goal is to iteratively segment a document into individual discourse units, we are able to eliminate the decoder and reduce the search space for splitting points. We explore both traditional recurrent models and modern pre-trained transformer models for the task, and additionally introduce a novel dynamic oracle for top-down parsing. Based on the Full metric, our proposed LSTM model sets a new state-of-the-art for RST parsing., Accepted at EACL 2021

A key part of the NLP ethics movement is responsible use of data, but exactly what that means or how it can be best achieved remain unclear. This position paper discusses the core legal and ethical principles for collection and sharing of textual data, and the tensions between them. We propose a potential checklist for responsible data (re-)use that could both standardise the peer review of conference submissions, as well as enable a more in-depth view of published research across the community. Our proposal aims to contribute to the development of a consistent standard for data (re-)use, embraced across NLP conferences., Comment: Findings of EMNLP 2021

Images are core components of multi-modal learning in natural language processing (NLP), and results have varied substantially as to whether images improve NLP tasks or not. One confounding effect has been that previous NLP research has generally focused on sophisticated tasks (in varying settings), generally applied to English only. We focus on text classification, in the context of assigning named entity classes to a given Wikipedia page, where images generally complement the text and the Wikipedia page can be in one of a number of different languages. Our experiments across a range of languages show that images complement NLP models (including BERT) trained without external pre-training, but when combined with BERT models pre-trained on large-scale external data, images contribute nothing.

Adversarial learning can learn fairer and less biased models of language than standard methods. However, current adversarial techniques only partially mitigate model bias, added to which their training procedures are often unstable. In this paper, we propose a novel approach to adversarial learning based on the use of multiple diverse discriminators, whereby discriminators are encouraged to learn orthogonal hidden representations from one another. Experimental results show that our method substantially improves over standard adversarial removal methods, in terms of reducing bias and the stability of training., Comment: EACL 2021 (5 pages + 1 references)

Neutralisation techniques, e.g. denial of responsibility and denial of victim, are used in the narrative of climate change scepticism to justify lack of action or to promote an alternative view. We first draw on social science to introduce the problem to the community of nlp, present the granularity of the coding schema and then collect manual annotations of neutralised techniques in text relating to climate change, and experiment with supervised and semi- supervised BERT-based models.

Although the Indonesian language is spoken by almost 200 million people and the 10th most spoken language in the world, it is under-represented in NLP research. Previous work on Indonesian has been hampered by a lack of annotated datasets, a sparsity of language resources, and a lack of resource standardization. In this work, we release the IndoLEM dataset comprising seven tasks for the Indonesian language, spanning morpho-syntax, semantics, and discourse. We additionally release IndoBERT, a new pre-trained language model for Indonesian, and evaluate it over IndoLEM, in addition to benchmarking it against existing resources. Our experiments show that IndoBERT achieves state-of-the-art performance over most of the tasks in IndoLEM., Accepted at COLING 2020 - The 28th International Conference on Computational Linguistics

Existing metonymy resolution approaches rely on features extracted from external resources like dictionaries and hand-crafted lexical resources. In this paper, we propose an end-to-end word-level classification approach based only on BERT, without dependencies on taggers, parsers, curated dictionaries of place names, or other external resources. We show that our approach achieves the state-of-the-art on 5 datasets, surpassing conventional BERT models and benchmarks by a large margin. We also show that our approach generalises well to unseen data., 12 pages

As part of growing NLP capabilities, coupled with an awareness of the ethical dimensions of research, questions have been raised about whether particular datasets and tasks should be deemed off-limits for NLP research. We examine this question with respect to a paper on automatic legal sentencing from EMNLP 2019 which was a source of some debate, in asking whether the paper should have been allowed to be published, who should have been charged with making such a decision, and on what basis. We focus in particular on the role of data statements in ethically assessing research, but also discuss the topic of dual use, and examine the outcomes of similar debates in other scientific disciplines., 6 pages; accepted for ACL2020

Background Cefovecin is a long-acting third-generation cephalosporin commonly used in veterinary medicine. Third-generation cephalosporins are critically important antimicrobials that should only be used after culture and susceptibility testing. The authors describe the common indications for cefovecin use in dogs and cats, and the frequency of culture and susceptibility testing. Materials and methods A cross-sectional study was performed using clinical records extracted from VetCompass Australia. A previously described method was used to identify records containing cefovecin. The reason for cefovecin use was annotated in situ in each consultation text. Results Over a six-month period (February and September 2018), 5180 (0.4 per cent) consultations involved cefovecin administration, of which 151 were excluded. Cats were administered cefovecin more frequently than dogs (1.9 per cent of cat consultations and 0.1 per cent of dog consultations). The most common reasons for cefovecin administration to cats were cat fight injuries and abscesses (28 per cent) and dermatitis (13 per cent). For dogs, the most common reasons for cefovecin administration were surgical prophylaxis (24 per cent) and dermatitis (19 per cent). Culture and susceptibility testing were reported in 16 cases (0.3 per cent). Conclusion Cefovecin is used in many scenarios in dogs and cats where antimicrobials may be either not indicated or where an antimicrobial of lower importance to human health is recommended.

Programming exercises lend themselves to learning environments that can provide automated testing and formative feedback to allow students to learn independently and at their own pace. Even so, there are still situations where there is no substitute for interaction with a human expert to provide guidance and encouragement to students. We describe how online tutoring is used within two very different contexts on the same platform. While both contexts use a specific learning environment, many of the experiences encountered in providing online tutoring support provide important lessons for educators that generalise to using any comparable environment.