Search

The usefulness evaluation model proposed by Cole et al. in 2009 [2] focuses on the evaluation of interactive IR systems by their support towards the user's overall goal, sub goals and tasks. This is a more human focus of the IR evaluation process than with classical TREC-oriented studies and gives a more holistic view on the IR evaluation process. However, yet there is no formal framework how the usefulness model can be operationalized. Additionally, a lot of information needed for the operationalization is only available in explicit user studies where for example the overall goal and the tasks are prompted from the users or are predefined. Measuring the usefulness of IR systems outside the laboratory is a challenging task as most often only log data of user interaction is available. But, an operationalization of the usefulness model based on interaction data could be applied to diverse systems and evaluation results would be comparable. In this article we discuss the challenges for measuring the usefulness of IIR systems with log-based approaches., Comment: Proceedings of 1st International Workshop on System And User Centered Evaluation Approaches in Interactive Information Retrieval at CHIIR 2016

In Interactive Information Retrieval (IIR) different services such as search term suggestion can support users in their search process. The applicability and performance of such services is either measured with different user-centered studies (like usability tests or laboratory experiments) or, in the context of IR, with their contribution to measures like precision and recall. However, each evaluation methodology has its certain disadvantages. For example, user-centered experiments are often costly and small-scaled; IR experiments rely on relevance assessments and measure only relevance of documents. In this work we operationalize the usefulness model of Cole et al. (2009) on the level of system support to measure not only the local effect of an IR service, but the impact it has on the whole search process. We therefore use a log-based evaluation approach which models user interactions within sessions with positive signals and apply it for the case of a search term suggestion service. We found that the usage of the service significantly often implicates the occurrence of positive signals during the following session steps., Comment: In CHIIR '16 Proceedings of the 2016 ACM on Conference on Human Information Interaction and Retrieval , 153-162

The ability to browse an information space in a structured way by exploiting similarities and dissimilarities between information objects is crucial for knowledge discovery. Knowledge maps use visualizations to gain insights into the structure of large-scale information spaces, but are still far away from being applicable for searching. The paper proposes a use case for enhancing search term recommendations by heat map visualizations of co-word relation-ships taken from indexing vocabulary. By contrasting areas of different "heat" the user is enabled to indicate mainstream areas of the field in question more easily., Comment: URL workshop proceedings: http://ceur-ws.org/Vol-1311/

Knowledge maps are promising tools for visualizing the structure of large-scale information spaces, but still far away from being applicable for searching. The first international workshop on "Knowledge Maps and Information Retrieval (KMIR)", held as part of the International Conference on Digital Libraries 2014 in London, aimed at bringing together experts in Information Retrieval (IR) and knowledge mapping in order to discuss the potential of interactive knowledge maps for information seeking purposes., Comment: URL workshop proceedings: http://ceur-ws.org/Vol-1311/

This workshop brings together experts of communities which often have been perceived as different once: bibliometrics / scientometrics / informetrics on the one side and information retrieval on the other. Our motivation as organizers of the workshop started from the observation that main discourses in both fields are different, that communities are only partly overlapping and from the belief that a knowledge transfer would be profitable for both sides. Bibliometric techniques are not yet widely used to enhance retrieval processes in digital libraries, although they offer value-added effects for users. On the other side, more and more information professionals, working in libraries and archives are confronted with applying bibliometric techniques in their services. This way knowledge exchange becomes more urgent. The first workshop set the research agenda, by introducing in each other methods, reporting about current research problems and brainstorming about common interests. This follow-up workshop continues the overall communication, but also puts one problem into the focus. In particular, we will explore how statistical modelling of scholarship can improve retrieval services for specific communities, as well as for large, cross-domain collections like Mendeley or ResearchGate. This second BIR workshop continues to raise awareness of the missing link between Information Retrieval (IR) and bibliometrics and contributes to create a common ground for the incorporation of bibliometric-enhanced services into retrieval at the scholarly search engine interface., Comment: 4 pages, 37th European Conference on Information Retrieval, BIR workshop

We propose an online access panel to support the evaluation process of Interactive Information Retrieval (IIR) systems - called IIRpanel. By maintaining an online access panel with users of IIR systems we assume that the recurring effort to recruit participants for web-based as well as for lab studies can be minimized. We target on using the online access panel not only for our own development processes but to open it for other interested researchers in the field of IIR. In this paper we present the concept of IIRpanel as well as first implementation details., Comment: 2 pages, 1 figure, 2014 IEEE/ACM Joint Conference on Digital Libraries (JCDL), London, 8th-12th September 2014

Information systems usually show as a particular point of failure the vagueness between user search terms and the knowledge orders of the information space in question. Some kind of guided searching therefore becomes more and more important in order to precisely discover information without knowing the right search terms. Knowledge maps of digital library collections are promising navigation tools through knowledge spaces but still far away from being applicable for searching digital libraries. However, there is no continuous knowledge exchange between the "map makers" on the one hand and the Information Retrieval (IR) specialists on the other hand. Thus, there is also a lack of models that properly combine insights of the two strands. The proposed workshop aims at bringing together these two communities: experts in IR reflecting on visual enhanced search interfaces and experts in knowledge mapping reflecting on visualizations of the content of a collection that might also present a context for a search term in a visual manner. The intention of the workshop is to raise awareness of the potential of interactive knowledge maps for information seeking purposes and to create a common ground for experiments aiming at the incorporation of knowledge maps into IR models at the level of the user interface., Comment: 6 pages, accepted workshop proposal for Digital Libraries 2014

This first "Bibliometric-enhanced Information Retrieval" (BIR 2014) workshop aims to engage with the IR community about possible links to bibliometrics and scholarly communication. Bibliometric techniques are not yet widely used to enhance retrieval processes in digital libraries, although they offer value-added effects for users. In this workshop we will explore how statistical modelling of scholarship, such as Bradfordizing or network analysis of co-authorship network, can improve retrieval services for specific communities, as well as for large, cross-domain collections. This workshop aims to raise awareness of the missing link between information retrieval (IR) and bibliometrics / scientometrics and to create a common ground for the incorporation of bibliometric-enhanced services into retrieval at the digital library interface. Our interests include information retrieval, information seeking, science modelling, network analysis, and digital libraries. The goal is to apply insights from bibliometrics, scientometrics, and informetrics to concrete practical problems of information retrieval and browsing., Comment: 4 pages, Bibliometric-enhanced Information Retrieval Workshop at ECIR 2014, Amsterdam, NL

Bibliometric techniques are not yet widely used to enhance retrieval processes in digital libraries, although they offer value-added effects for users. In this workshop we will explore how statistical modelling of scholarship, such as Bradfordizing or network analysis of coauthorship network, can improve retrieval services for specific communities, as well as for large, cross-domain collections. This workshop aims to raise awareness of the missing link between information retrieval (IR) and bibliometrics/scientometrics and to create a common ground for the incorporation of bibliometric-enhanced services into retrieval at the digital library interface., Comment: 6 pages, accepted workshop proposal for ECIR 2014

Online Social Networks usually provide no or limited way to access scholarly information provided by Digital Libraries (DLs) in order to share and discuss scholarly content with other online community members. The paper addresses the potentials of Social Networking sites (SNSs) for science and proposes initial use cases as well as a basic bi-directional model called ScholarLib for linking SNSs to scholarly DLs. The major aim of ScholarLib is to make scholarly information provided by DLs accessible at SNSs, and vice versa, to enhance retrieval quality at DL side by social information provided by SNSs., Comment: 5 pages, ACM Web Science 2012

The paper introduces scholarly Information Retrieval (IR) as a further dimension that should be considered in the science modeling debate. The IR use case is seen as a validation model of the adequacy of science models in representing and predicting structure and dynamics in science. Particular conceptualizations of scholarly activity and structures in science are used as value-added search services to improve retrieval quality: a co-word model depicting the cognitive structure of a field (used for query expansion), the Bradford law of information concentration, and a model of co-authorship networks (both used for re-ranking search results). An evaluation of the retrieval quality when science model driven services are used turned out that the models proposed actually provide beneficial effects to retrieval quality. From an IR perspective, the models studied are therefore verified as expressive conceptualizations of central phenomena in science. Thus, it could be shown that the IR perspective can significantly contribute to a better understanding of scholarly structures and activities., Comment: 26 pages, to appear in Scientometrics

The first step to handle semantic heterogeneity should be the attempt to enrich the semantic information about documents, i.e. to fill up the gaps in the documents meta-data automatically. Section 2 describes a set of cascading deductive and heuristic extraction rules, which were developed in the project CARMEN for the domain of Social Sciences. The mapping between different terminologies can be done by using intellectual, statistical and/or neural network transfer modules. Intellectual transfers use cross-concordances between different classification schemes or thesauri. Section 3 describes the creation, storage and handling of such transfers., Comment: Technical Report (Arbeitsbericht) GESIS - Leibniz Institute for the Social Sciences

The paper proposes three different kinds of science models as value-added services that are integrated in the retrieval process to enhance retrieval quality. The paper discusses the approaches Search Term Recommendation, Bradfordizing and Author Centrality on a general level and addresses implementation issues of the models within a real-life retrieval environment., Comment: 14 pages, 3 figures, ISI 2011

This paper is about a better understanding on the structure and dynamics of science and the usage of these insights for compensating the typical problems that arises in metadata-driven Digital Libraries. Three science model driven retrieval services are presented: co-word analysis based query expansion, re-ranking via Bradfordizing and author centrality. The services are evaluated with relevance assessments from which two important implications emerge: (1) precision values of the retrieval service are the same or better than the tf-idf retrieval baseline and (2) each service retrieved a disjoint set of documents. The different services each favor quite other - but still relevant - documents than pure term-frequency based rankings. The proposed models and derived retrieval services therefore open up new viewpoints on the scientific knowledge space and provide an alternative framework to structure scholarly information systems., Comment: 8 pages, 4 figures, Cologne Conference on Interoperability and Semantics in Knowledge Organization

This paper is about an information retrieval evaluation on three different retrieval-supporting services. All three services were designed to compensate typical problems that arise in metadata-driven Digital Libraries, which are not adequately handled by a simple tf-idf based retrieval. The services are: (1) a co-word analysis based query expansion mechanism and re-ranking via (2) Bradfordizing and (3) author centrality. The services are evaluated with relevance assessments conducted by 73 information science students. Since the students are neither information professionals nor domain experts the question of inter-rater agreement is taken into consideration. Two important implications emerge: (1) the inter-rater agreement rates were mainly fair to moderate and (2) after a data-cleaning step which erased the assessments with poor agreement rates the evaluation data shows that the three retrieval services returned disjoint but still relevant result sets., Comment: 7 pages, 3 figures, LWA 2010, Workshop IR

This paper is a short description of an information retrieval system enhanced by three model driven retrieval services: (1) co-word analysis based query expansion, re-ranking via (2) Bradfordizing and (3) author centrality. The different services each favor quite other - but still relevant - documents than pure term-frequency based rankings. Each service can be interactively combined with each other to allow an iterative retrieval refinement., Comment: 2 pages, 1 figure, ASIST 2010 conference, Pittsburgh, PA, USA