Search

The empirical nature of Information Retrieval (IR) mandates strong experimental practices. A keystone of such experimental practices is the Cranfield evaluation paradigm. Within this paradigm, the collection of relevance judgments has been the subject of intense scientific investigation. This is because, on one hand, consistent, precise, and numerous judgements are keys to reducing evaluation uncertainty and test collection bias; on the other hand, however, relevance judgements are costly to collect. The selection of which documents to judge for relevance, known as pooling method, has therefore a great impact on IR evaluation. In this paper we focus on the bias introduced by the pooling method, known as pool bias, which affects the reusability of test collections, in particular when building test collections with a limited budget. In this paper we formalize and evaluate a set of 22 pooling strategies based on: traditional strategies, voting systems, retrieval fusion methods, evaluation measures, and multi-armed bandit models. To do this we run a large-scale evaluation by considering a set of 9 standard TREC test collections, in which we show that the choice of the pooling strategy has significant effects on the cost needed to obtain an unbiased test collection. We also identify the least biased pooling strategy in terms of pool bias according to three IR evaluation measures: AP, NDCG, and P@10. [ABSTRACT FROM AUTHOR]

Abstract To support the objectives of the journal, to publish new research and insights covering a broad spectrum of Intellectual Property information retrieval and patent analytics related practices and methods, the editors, together with the team at IFI CLAIMS® Patent Services, have put together a patent research collection, publicly available and for free, to foster scientific good practice: comparability, reproducibility, transparency and repeatability of experiments and results. The WPI collection is for this reason static. It will not be updated with new data. Otherwise, changes in experimental results would be impossible to trace with confidence to the new methods being proposed and not seen as a result of changes in the dataset. The WPI collection complements existing test collections, which are vertical (one domain or one authority over many years). Instead, the WPI collection is horizontal: it includes all technical domains from the major patenting authorities over the relatively short time span of two years. [ABSTRACT FROM AUTHOR]

Every information retrieval (IR) model embeds in its scoring function a form of term frequency (TF) quantification. The contribution of the term frequency is determined by the properties of the function of the chosen TF quantification, and by its TF normalization. The first defines how independent the occurrences of multiple terms are, while the second acts on mitigating the a priori probability of having a high term frequency in a document (estimation usually based on the document length). New test collections, coming from different domains (e.g. medical, legal), give evidence that not only document length, but in addition, verboseness of documents should be explicitly considered. Therefore we propose and investigate a systematic combination of document verboseness and length. To theoretically justify the combination, we show the duality between document verboseness and length. In addition, we investigate the duality between verboseness and other components of IR models. We test these new TF normalizations on four suitable test collections. We do this on a well defined spectrum of TF quantifications. Finally, based on the theoretical and experimental observations, we show how the two components of this new normalization, document verboseness and length, interact with each other. Our experiments demonstrate that the new models never underperform existing models, while sometimes introducing statistically significantly better results, at no additional computational cost. [ABSTRACT FROM AUTHOR]

Since its inception in 2013, one of the key contributions of the CLEF eHealth evaluation campaign has been the organization of an ad-hoc information retrieval (IR) benchmarking task. This IR task evaluates systems intended to support laypeople searching for and understanding health information. Each year the task provides registered participants with standard IR test collections consisting of a document collection and topic set. Participants then return retrieval results obtained by their IR systems for each query, which are assessed using a pooling procedure. In this article we focus on CLEF eHealth 2013 and 2014s retrieval task, which saw topics created based on patients’ information needs associated with their medical discharge summaries. We overview the task and datasets created, and the results obtained by participating teams over these two years. We then provide a detailed comparative analysis of the results, and conduct an evaluation of the datasets in the light of these results. This twofold study of the evaluation campaign teaches us about technical aspects of medical IR, such as the effectiveness of query expansion; the quality and characteristics of CLEF eHealth IR datasets, such as their reliability; and how to run an IR evaluation campaign in the medical domain. [ABSTRACT FROM AUTHOR]

We take a look back at the conferences we attended in 2018. As every year, we try to cover a mix of small and large, academic and industry conferences, to get the pulse of our patent information industry, but also to make sure that developments in computer-based information processing are on our radar. The buzz around anything "AI" that we observed last year continues strongly. We report here on three conferences: the European Conference on Information Retrieval, Patent Information User Group conference, and the European Patent Office Patent Information Conference. [ABSTRACT FROM AUTHOR]

The article focuses on the credibility of information retrieval (IR). Topics discussed include main research directions that tackle different aspects of credibility in information retrieval; new evaluation benchmarks and user studies on IR; and diversity of search results which increase the probability of a positive user satisfaction.

The article focuses on information retrieval (IR) research areas and how they can be viewed in terms of credibility. Topics discussed include probabilistic retrieval together with methods to re-generate probabilities of relevance from retrieval status values; diversity of search results as a method to increase the probability of a positive user satisfaction; and evaluation campaigns as a direct measure of reliability of an IR method at a specific task.

The article focuses on credibility in social media information retrieval. Topics discussed include works on community question answering (CQA), credibility focused on Twitter Web service, an overview of general approaches used for assessing credibility in social media; and how microblogging services Twitter being considered more similar to blogs than social networks.

The article focuses on credibility in multimedia information retrieval with specific reference to video content credibility analysis. Topics discussed include an overview of general approaches used for assessing credibility in multimedia; work concerning the combination of textual and visual or audio features for automatic credibility estimation; and digital traces left in multimedia content from potential alterations.

The article focuses on credibility in social media information retrieval. Topics discussed include works on credibility focused on Twitter Web service, an overview of general approaches used for assessing credibility in social media, focusing on link based methods; and how microblogging services, such as Twitter, being considered more similar to blogs than social networks.

The article focuses on credibility components in healthcare information retrieval. Topics discussed include the different requirements and expectations users have, function of the domain of their information need, and the importance of healthcare information for both health professionals and the general public.

The article focuses on the domain specific credibility of healthcare information retrieval. Topics discussed include credibility of volunteered geographic information (VGI), the requirements and expectations users have, function of the domain of their information need, and the importance of healthcare information for both health professionals and the general public.

The article discusses various issues related to blog credibility in information retreival. Topics discussed include the different requirements and expectations computer users have, blogger expertise and offline identity disclosure, blogger's trustworthiness and value system, and information quality such as completeness, accuracy, appropriateness and timeliness.

The article focuses on how credibility factors affect a user's information seeking behavior from the Web. It highlights the 3S-model for evaluating credibility which was introduced by researcher Lucassen and Schraagen that emphasized on the importance of semantic features, writing style, and source experience.

The article focuses on the importance of analyzing credibility of information as credibility affects Web-based information seeking behavior. It includes general guidelines for improving the credibility of Web sites, analyzing credibility, and understanding the readers' attitude towards information authenticity.

The article focuses on the Quality component of the Credibility Components in the Web-based information access systems. It includes topics such as Quality as the super-ordinate concept linked to the interest that certain content can raise, quality analysis of text and textual features used to reveal quality content of the information.

The article explains the components of the Internet based information retrieval system, and the credibility requirements related to it. It includes components particularly for the consideration of credibility, such as trustworthiness, expertise, and reliability. It also highlights quality, which is often seen as an intrinsic characteristic of content shared on the Web.

An introduction is presented in which the editor discusses various articles within the issue on topics including Information retrieval (IR) in the intellectual property, cross-language retrieval, categorisation and image search in patents .

The article offers a detailed study related to image and chemical structure retrieval in the patent domain. It also discusses the importance of the information contained in the non-textual parts of the patent and algorithms formed to use this information in search. It further mentions the characteristics of drawings in patents, Computer-aided Patent Drawing Retrieval, and Image Classification.

Abstract: There is little need to emphasize the importance of chemoinformatics and chemical information retrieval. However, what seems to require a lot more effort in convincing members of the community is the need for standardized evaluation procedures and measures. How confident are we, as users, that the search tools we used have given us all the information that we were looking for? It is unrealistic to believe a marketing campaign which claims that a specific patent IR (information retrieval) system can find all relevant results for a search topic. And if we don’t trust marketing campaigns, how can we get an idea of what to expect from a patent search engine? One of the most prominent initiatives to work in this direction, of evaluating chemical IR tools, has started in 2009 with the support of NIST (National Institute for Standards and Technology): the TREC Chemical IR Track focuses on evaluation of search technologies for retrieval and knowledge discovery of digitally stored information on chemical patents and academic journal articles on chemistry. This paper describes our 2009 experience, presents the official results of the participating groups, and lays down the targets for 2010. [Copyright &y& Elsevier]

An appendix is presented on manually built datasets related to credibility of information retrieval (IR) systems.

The article focuses on credibility in multimedia information retrieval, with specific reference to credibility of online audio content. Topics discussed include an overview of general approaches used for assessing credibility in multimedia; work concerning the online audio content for automatic credibility estimation; and digital traces left in multimedia content from potential alterations.

The article focuses on credibility in multimedia information retrieval, with specific reference to credibility of images. Topics discussed include an overview of general approaches used for assessing credibility in multimedia; work concerning visual features for automatic credibility estimation; and digital traces left in multimedia content from potential alterations.

The article focuses on factors that would help predict credibility of Web accessed information. It presents information on topics such as, the machine learning algorithms to predict the credibility of the source or content on the Web, features that are relevant for Web credibility assessment, and social features that are limited with respect to the transitory nature of credibility.

The article focuses on the Credibility Components in the Web-based information access systems which includes expertise, trustworthiness, quality, and reliability. The topics highlighted in the article are importance of the content and of the observable behavior of the source, additional characteristics that influence credibility, and relation between credible source and credible content.

An appendix is presented on automatically built datasets related to credibility of information retrieval (IR) systems.

An appendix is presented on the research work on the credibility of information retrieval (IR) systems.

The article focuses on credibility in multimedia information retrieval. Topics discussed include an overview of general approaches used for assessing credibility in multimedia; work concerning the online video and audio content for automatic credibility estimation; and digital traces left in multimedia content from potential alterations.

The article focuses on credibility in social media information retrieval. Topics discussed include an overview of general approaches used for assessing credibility in social media; works on community question answering (CQA), credibility focused on Twitter Web service, and how microblogging services Twitter being considered more similar to blogs than social networks.

The article focuses on credibility in multimedia information retrieval. Topics discussed include an overview of general approaches used for assessing credibility in multimedia; work concerning the combination of textual and visual or audio features for automatic credibility estimation; and digital traces left in multimedia content from potential alterations.

The article focuses on the Reliability component of the Credibility Components in the Web-based information access systems, which is generally perceived as dependability and consistency in quality. It states that reliability is also viewed as a subarea of credibility, and is one of the most influential aspects that should be considered for assessing the relevance of a Web publication.

The article focuses on the Trustworthiness component of the Credibility Components in the Web-based information access systems. In the context of social media, trustworthy sources are used as an indicator for the credibility of a given piece of information. It explains the relation between global framework for trust/distrust relationships to understand the Trustworthiness component of the Credibility Components.

The article focuses on the Expertise component of the Credibility Components in the Web-based information access systems. It states that expertise information is likely to change over time, and mentions about various approaches proposed to find automatically experts on Community Question Answering stands for collection of information.

Readers of this journal are well aware that automation technology has played a significant role in searching for patent information and, as artificial intelligence is once again (after the first, 1960s, and second, 1980s, golden eras of AI) a trending topic at both academic and industry conferences, the editorial team of this journal would like to encourage contributions that cover any aspect of automation as applied to intellectual property information. As a new Associate Editor of World Patent Information, I take the opportunity to advertise the availability of the editorial team to submissions from computer science teams, in addition to the existing contributions from the IP community. By way of introduction, I have taken the liberty to provide an extremely brief overview of efforts and contributions that the computer science community has made to the field of intellectual property. This summary focuses on patents, but that is not to say that trademarks or other forms of intellectual property have not triggered the interest of computer scientists. In fact, to call this summary short is already giving it too much credit. It is but a seed, upon which I hope that a forest of contributions and publications from my fellow computer scientists will grow. [ABSTRACT FROM AUTHOR]

The article focuses on credibility components in information retrieval. Topics discussed include the different requirements and expectations users have in the healthcare, function of the blog domain of their information need, and the importance of healthcare information for both health professionals and the general public.

The article focuses on the Credibility Components in the Web-based information access systems which includes expertise, trustworthiness, quality, and reliability.