Search

A basic step in any annotation effort is the measurement of the Inter Annotator Agreement (IAA). An important factor that can affect the IAA is the presence of annotator bias. In this paper we introduce a new interpretation and application of the Item Response Theory (IRT) to detect annotators’ bias. Our interpretation of IRT offers an original bias identification method that can be used to compare annotators’ bias and characterise annotation disagreement. Our method can be used to spot outlier annotators, improve annotation guidelines and provide a better picture of the annotation reliability. Additionally, because scales for IAA interpretation are not generally agreed upon, our bias identification method is valuable as a complement to the IAA value which can help with understanding the annotation disagreement.

In the world of public misinformation, there are many cases where the information is not false or fabricated, but rather has been manipulated using more subtle techniques such as word replacements, selection of details, omissions and argument distortion. These techniques can have the effect of influencing the reader’s frame of mind towards the events reported. We currently lack the necessary tools to uncover such manipulations automatically. In this position paper, we propose an integrated analysis framework and pipeline to identify various narrative signals in news articles; such as structural roles, framing, and subjectivity. By comparing these at the document level and sentence level, it will be possible to highlight differences of narrative techniques used to report the same news events.

Since the early years of the past century, many scholars have focused their efforts towards designing models to better understand the way listeners perceive musical tension. From the existing models, Lerdahl's has shown strong correlations against tension judgements provided by human listeners and has been used to make accurate predictions of musical tension. However, a full automation of Lerdahl's model of tension has not yet been made available. This paper presents a computational approach to automatically calculate musical tension according to Lerdahl's model, with a publicly available implementation.

The increasing use of social media around global news events, such as the London Olympics in 2012, raises questions for international broadcasters about how to engage with users via social media in order to best achieve their individual missions. Twitter is a highly diverse social network whose conversations are multi-directional involving individual users, political and cultural actors, athletes and a range of media professionals. In so doing, users form networks of influence via their interactions affecting the ways that information is shared about specific global events.\ud \ud This article attempts to understand how networks of influence are formed among Twitter users, and the relative influence of global news media organisations and information providers in the Twittersphere during such global news events. We build an analysis around a set of tweets collected during the 2012 London Olympics. To understand how different users influence the conversations across Twitter, we compare three types of accounts: those belonging to a number of well-known athletes, those belonging to some well-known commentators employed by the BBC, and a number of corporate accounts belonging to the BBC World Service and the official London Twitter account. We look at the data from two perspectives. First, to understand the structure of the social groupings formed among Twitter users, we use a network analysis to model social groupings in the Twittersphere across time and space. Second, to assess the influence of individual tweets, we investigate the ageing factor of tweets, which measures how long users continue to interact with a particular tweet after it is originally posted.\ud \ud We consider what the profile of particular tweets from corporate and athletes’ accounts can tell us about how networks of influence are forged and maintained. We use these analyses to answer the questions: How do different types of accounts help shape the social networks? and, What determines the level and type of influence of a particular account?

Current game music systems typically involve the playback of prerecorded audio tracks which are crossfaded in response to game events such as level changes. However, crossfading can limit the expressive power of musical transitions, and can make fine grained structural variations difficult to achieve. We therefore describe an alternative approach in which music is algorithmically generated based on a set of high-level musical features that can be controlled in real-time according to a player’s progression through a game narrative. We outline an implementation of the approach in an actual game, focusing primarily on how the music system traces the game’s emotional narrative by periodically querying certain narrative parameters and adjusting the musical features of its output accordingly.

This poster presents the work of the ComTax project to develop a community-driven curation process among practicing scientists and citizen scientists. The project provides tools to help scientists identify and validate appropriate taxonomic names from the scanned historical literature. The system operates on scanned documents, typically taken from the Biodiversity Heritage Library, although documents sourced from other repositories could be used.\ud \ud The system is intended to be used on uncorrected text after optical character recognition (OCR) on the scanned images. The key stages are:\ud \ud 1. Identify possible taxonomic names in the scanned text using machine learning techniques.\ud \ud 2. Verify the extracted names against existing databases. If present, the source scanned text can be automatically marked-up with the name.\ud \ud 3. Unverified names might mean they are not currently recorded in the verification databases, typically because the old name in the literature has been reclassified, or because erroneous OCR means that the name is incorrectly transcribed in the scanned text. In either case:\ud \ud 3.1. Present the proposed name to domain experts or citizen scientists for validation or correction, potentially through a voting mechanism to collect expert judgments on the putative taxonomic name.\ud \ud 3.2. Mark-up the scanned text with the corrected spelling of the name and offer validated taxonomic names for further use by the community.\ud \ud This poster will describe the technical challenges facing the ComTax project, and highlight potential extensions of the work to the curation of other entities of interest in the legacy literature or of different disciplines.

Digitized biodiversity literature provides a wealth of content for using biodiversity knowledge by machines. However, identifying taxonomic names and the associated semantic metadata is a difficult and labour intensive process. We present a system to support human assisted creation of semantic metadata. Information extraction techniques auto-matically identify taxonomic names from scanned documents. They are then presented to users for manual correction or verification. The tools that support the curation process include taxonomic name identification and mapping, and community-driven taxonomic name verification. Our research shows the potential for these information extrac-tion techniques to support research and curation in disciplines dependent upon scanned documents

Biological taxonomy is the classification of living and fossil organisms. Taxonomists have identified and named some 1.8 million species of animals, plants, and microorganisms, a fraction of Earth's estimated 5‐30 million species. Part of the effort of taxonomy lies in developing and curating taxonomic databases, which support access to the taxonomy literature, and provide basic knowledge needed for management and conservation of biodiversity.\ud \ud A major difficulty facing this task is incorporating knowledge that is currently contained only in the historical literature. Extracting this knowledge is a difficult and labour‐intensive process, as scanning errors and other variations in nomenclature mean that particular names must be manually verified as part of the process. For example, Actinobacillus actionomy, Actinobacillus actionomyce, and Actinobacillus actionomycetam could all be variants of the same name. ComTax is an ongoing project to develop a community‐driven curation process among taxonomists, by providing tools to help them identify and validate taxonomic names from the scanned historical literature. The system operates on scanned documents following optical character recognition (OCR). The key stages are:\ud \ud 1. Identify possible taxonomic names from scanned text. Names might be new either because they do not appear in existing databases, or because they have been incorrectly identified by OCR.\ud \ud 2. Present the proposed name to a domain expert for validation or correction.\ud \ud 3. Present validated taxonomic names for curation. Organizations like Global Biodiversity Information Facility (GBIF) manage the curation of taxonomic databases. This poster describes the technical challenges facing the ComTax project, and will discuss our work with the Natural History Museum to integrate the curation process into taxonomists' workflow. We also demonstrate the relevance of this work within the wider context of biological taxonomy curation.

peer-reviewed Stakeholders frequently use speculative language when they need to convey their requirements with some degree of uncertainty. Due to the intrinsic vagueness of speculative language, speculative requirements risk being misunderstood, and related uncertainty overlooked, and may benefit from careful treatment in the requirements engineering process. In this paper, we present a linguistically-oriented approach to automatic detection of uncertainty in natural language (NL) requirements. Our approach comprises two stages. First we identify speculative sentences by applying a machine learning algorithm called Conditional Random Fields (CRFs) to identify uncertainty cues. The algorithm exploits a rich set of lexical and syntactic features extracted from requirements sentences. Second, we try to determine the scope of uncertainty. We use a rule-based approach that draws on a set of hand-crafted linguistic heuristics to determine the uncertainty scope with the help of dependency structures present in the sentence parse tree. We report on a series of experiments we conducted to evaluate the performance and usefulness of our system.

We present the ABLE document collection, which consists of a set of annotated volumes of the Bulletin of the British Museum (Natural History). These follow our work on automating the markup of scanned copies of the biodiversity literature, for the purpose of supporting working taxonomists. We consider an enhanced TEI XML markup language, which is used as an intermediate stage in translating from the initial XML obtained from Optical Character Recognition to the target taXMLit. The intermediate representation allows additional information from external sources such as a taxonomic thesaurus to be incorporated before the final translation into taXMLit.

Accurately capturing the quantifier scope behaviour of coordinated NPs can be problematic for underspecification systems that define constraints over semantic constructors. We present an extension to a hole-semantics like language that allows a natural representation of coordinated NPs, and a translation from partial scope requirements into constraints on the constructors. We conclude that the efficient decision procedures developed for constraints on semantic constructors enable the possible meanings of sentences containing coordinated NPs to be fully underspecified.

In this paper we present heuristics for resolving coordination ambiguities. We test the hypothesis that the most likely reading of a coordination can be predicted using word distribution information from a generic corpus. Our heuristics are based upon the relative frequency of the coordination in the corpus, the distributional similarity of the coordinated words, and the collocation frequency between the coordinated words and their modifiers. These heuristics have varying but useful predictive power. They also take into account our view that many ambiguities cannot be effectively disambiguated, since human perceptions vary widely.