Search

A long query provides more useful hints for searching relevant documents, but it is likely to introduce noise which affects retrieval performance. In order to smooth such adverse effect, it is important to reduce noisy terms, introduce and boost additional relevant terms. This paper presents a comprehensive framework, called Aspect Hidden Markov Model (AHMM), which integrates query reduction and expansion, for retrieval with long queries. It optimizes the probability distribution of query terms by utilizing intra-query term dependencies as well as the relationships between query terms and words observed in relevance feedback documents. Empirical evaluation on three large-scale TREC collections demonstrates that our approach, which is automatic, achieves salient improvements over various strong baselines, and also reaches a comparable performance to a state of the art method based on user’s interactive query term reduction and expansion.

Query expansion is generally a useful technique in improving search performance. However, some expanded query terms obtained by traditional statistical methods (e.g., pseudo-relevance feedback) may not be relevant to the user's information need, while some relevant terms may not be contained in the feedback documents at all. Recent studies utilize external resources to detect terms that are related to the query, and then adopt these terms in query expansion. In this paper, we present a study in the use of Freebase, which is an open source general-purpose ontology, as a source for deriving expansion terms. FreeBase provides a graph-based model of human knowledge, from which a rich and multi-step structure of instances related to the query concept can be extracted, as a complement to the traditional statistical approaches to query expansion. We propose a novel method, based on the well-principled Dempster-Shafer's (D-S) evidence theory, to measure the certainty of expansion terms from the Freebase structure. The expanded query model is then combined with a state of the art statistical query expansion model - the Relevance Model (RM3). Experiments show that the proposed method achieves significant improvements over RM3.

This poster presents the work of the ComTax project to develop a community-driven curation process among practicing scientists and citizen scientists. The project provides tools to help scientists identify and validate appropriate taxonomic names from the scanned historical literature. The system operates on scanned documents, typically taken from the Biodiversity Heritage Library, although documents sourced from other repositories could be used.\ud \ud The system is intended to be used on uncorrected text after optical character recognition (OCR) on the scanned images. The key stages are:\ud \ud 1. Identify possible taxonomic names in the scanned text using machine learning techniques.\ud \ud 2. Verify the extracted names against existing databases. If present, the source scanned text can be automatically marked-up with the name.\ud \ud 3. Unverified names might mean they are not currently recorded in the verification databases, typically because the old name in the literature has been reclassified, or because erroneous OCR means that the name is incorrectly transcribed in the scanned text. In either case:\ud \ud 3.1. Present the proposed name to domain experts or citizen scientists for validation or correction, potentially through a voting mechanism to collect expert judgments on the putative taxonomic name.\ud \ud 3.2. Mark-up the scanned text with the corrected spelling of the name and offer validated taxonomic names for further use by the community.\ud \ud This poster will describe the technical challenges facing the ComTax project, and highlight potential extensions of the work to the curation of other entities of interest in the legacy literature or of different disciplines.

Digitized biodiversity literature provides a wealth of content for using biodiversity knowledge by machines. However, identifying taxonomic names and the associated semantic metadata is a difficult and labour intensive process. We present a system to support human assisted creation of semantic metadata. Information extraction techniques auto-matically identify taxonomic names from scanned documents. They are then presented to users for manual correction or verification. The tools that support the curation process include taxonomic name identification and mapping, and community-driven taxonomic name verification. Our research shows the potential for these information extrac-tion techniques to support research and curation in disciplines dependent upon scanned documents

Biological taxonomy is the classification of living and fossil organisms. Taxonomists have identified and named some 1.8 million species of animals, plants, and microorganisms, a fraction of Earth's estimated 5‐30 million species. Part of the effort of taxonomy lies in developing and curating taxonomic databases, which support access to the taxonomy literature, and provide basic knowledge needed for management and conservation of biodiversity.\ud \ud A major difficulty facing this task is incorporating knowledge that is currently contained only in the historical literature. Extracting this knowledge is a difficult and labour‐intensive process, as scanning errors and other variations in nomenclature mean that particular names must be manually verified as part of the process. For example, Actinobacillus actionomy, Actinobacillus actionomyce, and Actinobacillus actionomycetam could all be variants of the same name. ComTax is an ongoing project to develop a community‐driven curation process among taxonomists, by providing tools to help them identify and validate taxonomic names from the scanned historical literature. The system operates on scanned documents following optical character recognition (OCR). The key stages are:\ud \ud 1. Identify possible taxonomic names from scanned text. Names might be new either because they do not appear in existing databases, or because they have been incorrectly identified by OCR.\ud \ud 2. Present the proposed name to a domain expert for validation or correction.\ud \ud 3. Present validated taxonomic names for curation. Organizations like Global Biodiversity Information Facility (GBIF) manage the curation of taxonomic databases. This poster describes the technical challenges facing the ComTax project, and will discuss our work with the Natural History Museum to integrate the curation process into taxonomists' workflow. We also demonstrate the relevance of this work within the wider context of biological taxonomy curation.

peer-reviewed Stakeholders frequently use speculative language when they need to convey their requirements with some degree of uncertainty. Due to the intrinsic vagueness of speculative language, speculative requirements risk being misunderstood, and related uncertainty overlooked, and may benefit from careful treatment in the requirements engineering process. In this paper, we present a linguistically-oriented approach to automatic detection of uncertainty in natural language (NL) requirements. Our approach comprises two stages. First we identify speculative sentences by applying a machine learning algorithm called Conditional Random Fields (CRFs) to identify uncertainty cues. The algorithm exploits a rich set of lexical and syntactic features extracted from requirements sentences. Second, we try to determine the scope of uncertainty. We use a rule-based approach that draws on a set of hand-crafted linguistic heuristics to determine the uncertainty scope with the help of dependency structures present in the sentence parse tree. We report on a series of experiments we conducted to evaluate the performance and usefulness of our system.

peer-reviewed Many requirements documents are written in natural language (NL). However, with the flexibility of NL comes the risk of introducing unwanted ambiguities in the requirements and misunderstandings between stakeholders. In this paper, we describe an automated approach to identify potentially nocuous ambiguity, which occurs when text is interpreted differently by different readers. We concentrate on anaphoric ambiguity, which occurs when readers may disagree on how pronouns should be interpreted. We describe a number of heuristics, each of which captures information that may lead a reader to favour a particular interpretation of the text. We use these heuristics to build a classifier, which in turn predicts the degree to which particular interpretations are preferred. We collected multiple human judgements on the interpretation of requirements exhibiting anaphoric ambiguity, and show how the distribution of these judgements can be used to assess whether a particular instance of ambiguity is nocuous. Given a requirements document written in natural language, our approach can identify sentences which contain anaphoric ambiguity, and use the classifier to alert the requirements writer of text that runs the risk of misinterpretation. We report on a series of experiments that we conducted to evaluate the performance of the automated system we developed to support our approach. The results show that the system achieves high recall with a consistent improvement on baseline precision subject to some ambiguity tolerance levels, allowing us to explore and highlight realistic and potentially problematic ambiguities in actual requirements documents.

Information retrieval has become very popular over the last\ud decade with the advent of the Web. Nevertheless, searching on the Web is very different to searching on smaller, often more structured collections such as intranets and digital libraries. Such collections are the focus of the AutoAdapt project. The project seeks to aid user search by providing well structured domain knowledge to assist query modification and navigation.\ud \ud There are two challenges: acquiring the domain knowledge and adapting it automatically to the specific interest of the user community. The paper introduces an implemented prototype that serves as a starting point on the\ud way to truly adaptive search.

This paper describes the prototype interactive search system being developed within the AutoAdapt project1. The AutoAdapt project seeks to enhance the user experience in searching for information and navigating within selected domain collections by providing structured representations of domain knowledge to be directly explored, logged, adapted and updated to reflect user needs. We propose that this structure is a valuable stepping-stone in context-rich logging of user activities within the information seeking environment. Here we describe the primary components that have been implemented and the user interactions that it will support.

In this paper we present heuristics for resolving coordination ambiguities. We test the hypothesis that the most likely reading of a coordination can be predicted using word distribution information from a generic corpus. Our heuristics are based upon the relative frequency of the coordination in the corpus, the distributional similarity of the coordinated words, and the collocation frequency between the coordinated words and their modifiers. These heuristics have varying but useful predictive power. They also take into account our view that many ambiguities cannot be effectively disambiguated, since human perceptions vary widely.

This paper proposes two unconventional metrics as animportant tool for assessment research: the Manhattan(L1) and the Euclidean (L2) distance measures. We used them to evaluate the results of a Latent Semantic Analysis(LSA) system to assess short answers to two questionsabout HTML in an introductory computer science class.This is the only study, as far as we know, that addresses the question of how well an LSA-based system can evaluate answers in the very specific and technical language of HTML. We found that, although there are several ways to measure automatic assessment results in the literature, they were not useful for our purposes. We want to compare the marks given by LSA to marks awarded by a human tutor. We demonstrate how L1 and L2 quantify the results of varying the amount of training data necessary to enable LSA to mark the answers to two HTML questions. Although this paper describes the use of the metrics in one particular case, it has more general applicability. Much fine-tuning of an LSA marking system is required for good results. A researcher needs an easy way to evaluate the results of various modifications to the system. The Manhattan and the Euclidean distance measures provide this functionality.

With the growing amount of semantic data being published on the Web the problem of finding individuals in different datasets which correspond to the same entity is gaining importance. Given that datasets are often structured using different ontologies, automatic schema-matching techniques have to be utilized before proceeding with data-level alignment. In this paper we discuss how ontology schema mis- matches influence data-level alignment based on our first experience with implementing a data fusion tool for a multi-ontology environment.

peer-reviewed Nocuous ambiguity occurs when a linguistic expression is interpreted differently by different readers in a given context. We present an approach to automatically identify nocuous ambiguity that is likely to lead to misunderstandings among readers. Our model is built on a machine learning architecture. It learns from a set of heuristics each of which predicts a factor that may lead a reader to favor a particular interpretation. An ambiguity threshold indicates the extent to which ambiguity can be tolerated in the application domain. Collections of human judgments are used to train heuristics and set ambiguity thresholds, and for evaluation. We report results from applying the methodology to coordination and anaphora ambiguity. Results show that the method can identify nocuous ambiguity in text, and may be widened to cover further types of ambiguity. We discuss approaches to evaluation.

Following the withdrawal of the filtering track from the latest TREC conferences, there is a niche for new evaluation standards. Towards this end, we suggest, based on variations of TREC's routing subtask, two new evaluation methodologies. The first can be used for evaluating single, multi-topic profiles and the second for testing the ability of a multi-topic profile to adapt to both modest variations and radical drifts in user interests.