Search

In information retrieval, relevance is perceived as a multidimensional and dynamic concept influenced by user, task, and domain factors. Relying on this perspective, researchers have introduced multidimensional relevance models addressing diverse search tasks across numerous knowledge domains. Through our systematic review of 72 studies, we categorize research based on domain specificity and the distinct relevance aspects employed for estimating multidimensional relevance. Moreover, we highlight the approaches used to aggregate scores related to these factors and rank information items. Our insights underline the importance of concise definitions and unified methods for estimating relevance factors within and across domains. Finally, we identify benchmark collections for evaluations based on multiple relevance aspects while underscoring the necessity for new ones. Our findings suggest that large language models hold considerable promise for shaping future research in this field, mainly due to their relevance labeling abilities. This article is categorized under: Application Areas > Science and Technology Technologies > Computational Intelligence.

The fine-tuning paradigm has been widely adopted to train neural models tailored for specific tasks. However, the recent upsurge of Large Language Models (LLMs), characterized by billions of parameters, has introduced profound computational challenges to the fine-tuning process. This has fueled intensive research on Parameter-Efficient Fine-Tuning (PEFT) techniques, usually involving the training of a selective subset of the original model parameters. One of the most used approaches is Adapters, which add trainable lightweight layers to the existing pretrained weights. Within this context, we propose AdaKron, an Adapter-based fine-tuning with the Kronecker product. In particular, we leverage the Kronecker product to combine the output of two small networks, resulting in a final vector whose dimension is the product of the dimensions of the individual outputs, allowing us to train only 0.55% of the model's original parameters. We evaluate AdaKron performing a series of experiments on the General Language Understanding Evaluation (GLUE) benchmark, achieving results in the same ballpark as recent state-of-the-art PEFT methods, despite training fewer parameters.

Order-Sorted Feature (OSF) logic is a knowledge representation and reasoning language based on function-denoting feature symbols and set-denoting sort symbols ordered in a subsumption lattice. OSF logic allows the construction of record-like terms that represent classes of entities and that are themselves ordered in a subsumption relation. The unification algorithm for such structures provides an efficient calculus of type subsumption, which has been applied in computational linguistics and implemented in constraint logic programming languages such as LOGIN and LIFE and automated reasoners such as CEDAR. This work generalizes OSF logic to a fuzzy setting. We give a flexible definition of a fuzzy subsumption relation which generalizes Zadeh's inclusion between fuzzy sets. Based on this definition we define a fuzzy semantics of OSF logic where sort symbols and OSF terms denote fuzzy sets. We extend the subsumption relation to OSF terms and prove that it constitutes a fuzzy partial order with the property that two OSF terms are subsumed by one another in the crisp sense if and only if their subsumption degree is greater than 0. We show how to find the greatest lower bound of two OSF terms by unifying them and how to compute the subsumption degree between two OSF terms, and we provide the complexity of these operations.

Retrieval with extremely long queries and documents is a well-known and challenging task in information retrieval and is commonly known as Query-by-Document (QBD) retrieval. Specifically designed Transformer models that can handle long input sequences have not shown high effectiveness in QBD tasks in previous work. We propose a Re-Ranker based on the novel Proportional Relevance Score (RPRS) to compute the relevance score between a query and the top-k candidate documents. Our extensive evaluation shows RPRS obtains significantly better results than the state-of-the-art models on five different datasets. Furthermore, RPRS is highly efficient, since all documents can be pre-processed, embedded, and indexed before query time that gives our re-ranker the advantage of having a complexity of O(N), where N is the total number of sentences in the query and candidate documents. Furthermore, our method solves the problem of the low-resource training in QBD retrieval tasks as it does not need large amounts of training data and has only three parameters with a limited range that can be optimized with a grid search even if a small amount of labeled data is available. Our detailed analysis shows that RPRS benefits from covering the full length of candidate documents and queries.

In this paper, we propose a novel approach to consider multiple dimensions of relevance in cross-encoder re-ranking. On the one hand, cross-encoders constitute an effective solution for re-ranking when considering a single relevance dimension such as topicality, but are not designed to straightforwardly account for additional relevance dimensions. On the other hand, the majority of re-ranking models accounting for multdimensional relevance are often based on the aggregation of multiple relevance scores at the re-ranking stage, leading to potential compensatory effects. To address these issues, in the proposed solution we enhance the candidate documents retrieved by a first-stage lexical retrieval model with suitable relevance statements related to distinct relevance dimensions, and then perform a re-ranking on them with cross-encoders. In this work we focus, in particular, on an extra dimension of relevance beyond topicality, namely, credibility, to address health misinformation in the Consumer Health Search task. Experimental evaluations are performed by considering publicly available datasets; our results show that the proposed approach statistically outperforms state-of-the-art aggregation-based and cross-encoder re-rankers.

Introduction: Clinical trials (CTs) often fail due to inadequate patient recruitment. Finding eligible patients involves comparing the patient's information with the CT eligibility criteria. Automated patient matching offers the promise of improving the process, yet the main difficulties of CT retrieval lie in the semantic complexity of matching unstructured patient descriptions with semi-structured, multi-field CT documents and in capturing the meaning of negation coming from the eligibility criteria. Objectives: This paper tackles the challenges of CT retrieval by presenting an approach that addresses the patient-to-trials paradigm. Our approach involves two key components in a pipeline-based model: (i) a data enrichment technique for enhancing both queries and documents during the first retrieval stage, and (ii) a novel re-ranking schema that uses a Transformer network in a setup adapted to this task by leveraging the structure of the CT documents. Methods: We use named entity recognition and negation detection in both patient description and the eligibility section of CTs. We further classify patient descriptions and CT eligibility criteria into current, past, and family medical conditions. This extracted information is used to boost the importance of disease and drug mentions in both query and index for lexical retrieval. Furthermore, we propose a two-step training schema for the Transformer network used to re-rank the results from the lexical retrieval. The first step focuses on matching patient information with the descriptive sections of trials, while the second step aims to determine eligibility by matching patient information with the criteria section. Results: Our findings indicate that the inclusion criteria section of the CT has a great influence on the relevance score in lexical models, and that the enrichment techniques for queries and documents improve the retrieval of relevant trials. The re-ranking strategy, based on our training schema, consistent

Big data have become a core technology for providing innovative solutions in numerical applications and services in many fields. Embedded in these big data is valuable information and knowledge. This calls for data science and analytics, which has emerged as an important paradigm for driving the new economy and domains (e.g., Internet of Things, social and mobile networks, cloud computing), reforming classic disciplines (e.g., telecommunications, biology, health and social science), as well as upgrading core business and economic activity. In this article, we focus on both theoretical and practical data science and analytics. We summarize and highlight some of its challenges and solutions, which are covered in the eight articles in the current Special Issue on "theoretical and practical data science and analytics."

The DoSSIER team from the University of Milano Bicocca participated in the Simple Text lab from Clef 2023. We focus on analysing the distribution of topics (or themes) for the content selection task and its potential effects in retrieval. We introduce Domain-knowledge through an ontology with different classification algorithms for annotating documents. This allows us to study the diversity in retrieval results and show how constraining them positively impacts the information-gathering task. We constrain results to concentrate them to specific sets of themes built using pseudo-relevance feedback.

This study investigates the effect that various patient-related information extracted from unstructured clinical notes has on two different tasks, i.e., patient allocation in clinical trials and medical literature retrieval. Specifically, we combine standard and transformer-based methods to extract entities (e.g., drugs, medical problems), disambiguate their meaning (e.g., family history, negations), or expand them with related medical concepts to synthesize diverse query representations. The empirical evaluation showed that certain query representations positively affect retrieval effectiveness for patient allocation in clinical trials, but no statistically significant improvements have been identified in medical literature retrieval. Across the queries, it has been found that removing negated entities using a domain-specific pre-trained transformer model has been more effective than a standard rule-based approach. In addition, our experiments have shown that removing information related to family history can further improve patient allocation in clinical trials.

In this paper we propose a novel approach for combining first-stage lexical retrieval models and Transformer-based re-rankers: we inject the relevance score of the lexical model as a token in the middle of the input of the cross-encoder re-ranker. It was shown in prior work that interpolation between the relevance score of lexical and BERT-based re-rankers may not consistently result in higher effectiveness. Our idea is motivated by the finding that BERT models can capture numeric information. We compare several representations of the BM25 score and inject them as text in the input of four different cross-encoders. We additionally analyze the effect for different query types, and investigate the effectiveness of our method for capturing exact matching relevance. Evaluation on the MSMARCO Passage collection and the TREC DL collections shows that the proposed method significantly improves over all cross-encoder re-rankers as well as the common interpolation methods. We show that the improvement is consistent for all query types. We also find an improvement in exact matching capabilities over both BM25 and the cross-encoders. Our findings indicate that cross-encoder re-rankers can efficiently be improved without additional computational burden and extra steps in the pipeline by explicitly adding the output of the first-stage ranker to the model input, and this effect is robust for different models and query types.

The problem of personalization in Information Retrieval has been under study for a long time. A well-known issue related to this task is the lack of publicly available datasets to support a comparative evaluation of personalized search systems. To contribute in this respect, this paper introduces SE-PEF (StackExchange - Personalized Expert Finding), a resource useful for designing and evaluating personalized models related to the Expert Finding (EF) task. The contributed dataset includes more than 250k queries and 565k answers from 3 306 experts, which are annotated with a rich set of features modeling the social interactions among the users of a popular cQA platform. The results of the preliminary experiments conducted show the appropriateness of SE-PEF to evaluate and to train effective EF models.

As a result of the widespread use of intelligent assistants, personalization in dialogue systems has become a hot topic in both research and industry. Typically, training such systems is computationally expensive, especially when using recent large language models. To address this challenge, we develop an approach to personalize dialogue systems using adapter layers and topic modelling. Our implementation enables the model to incorporate user-specific information, achieving promising results by training only a small fraction of parameters.

Introduction: People are today increasingly relying on health information they find online to make decisions that may impact both their physical and mental wellbeing. Therefore, there is a growing need for systems that can assess the truthfulness of such health information. Most of the current literature solutions use machine learning or knowledge-based approaches treating the problem as a binary classification task, discriminating between correct information and misinformation. Such solutions present several problems with regard to user decision making, among which: (i) the binary classification task provides users with just two predetermined possibilities with respect to the truthfulness of the information, which users should take for granted; indeed, (ii) the processes by which the results were obtained are often opaque and the results themselves have little or no interpretation. Methods: To address these issues, we approach the problem as an ad hoc retrieval task rather than a classification task, with reference, in particular, to the Consumer Health Search task. To do this, a previously proposed Information Retrieval model, which considers information truthfulness as a dimension of relevance, is used to obtain a ranked list of both topically-relevant and truthful documents. The novelty of this work concerns the extension of such a model with a solution for the explainability of the results obtained, by relying on a knowledge base consisting of scientific evidence in the form of medical journal articles. Results and discussion: We evaluate the proposed solution both quantitatively, as a standard classification task, and qualitatively, through a user study to examine the “explained” ranked list of documents. The results obtained illustrate the solution's effectiveness and usefulness in making the retrieved results more interpretable by Consumer Health Searchers, both with respect to topical relevance and truthfulness.

Research aimed at finding solutions to the problem of the diffusion of distinct forms of non-genuine information online across multiple domains has attracted growing interest in recent years, from opinion spam to fake news detection. Currently, partly due to the COVID-19 virus outbreak and the subsequent proliferation of unfounded claims and highly biased content, attention has focused on developing solutions that can automatically assess the genuineness of health information. Most of these approaches, applied both to Web pages and social media content, rely primarily on the use of handcrafted features in conjunction with Machine Learning. In this article, instead, we propose a health misinformation detection model that exploits as features the embedded representations of some structural and content characteristics of Web pages, which are obtained using an embedding model pre-trained on medical data. Such features are employed within a deep learning classification model, which categorizes genuine health information versus health misinformation. The purpose of this article is therefore to evaluate the effectiveness of the proposed model, namely Vec4Cred, with respect to the problem considered. This model represents an evolution of a previous one, with respect to which new features and architectural choices have been considered and illustrated in this work.

A prominent problem in knowledge representation is how to answer queries taking into account also the implicit consequences of an ontology representing domain knowledge. While this problem has been widely studied within the realm of description logic ontologies, it has been surprisingly neglected within the context of vague or imprecise knowledge, particularly from the point of view of mathematical fuzzy logic. In this paper, we study the problem of answering conjunctive queries and threshold queries w.r.t. ontologies in fuzzy DL-Lite. Specifically, we show through a rewriting approach that threshold query answering w.r.t. consistent ontologies remains in ${AC}^{0}$ in data complexity, but that conjunctive query answering is highly dependent on the selected triangular norm, which has an impact on the underlying semantics. For the idempotent Gödel t-norm, we provide an effective method based on a reduction to the classical case.

Semantic Query Labeling is the task of locating the constituent parts of a query and assigning domain-specific semantic labels to each of them. It allows unfolding the relations between the query terms and the documents’ structure while leaving unaltered the keyword-based query formulation. In this paper, we investigate the pre-training of a semantic query-tagger with synthetic data generated by leveraging the documents’ structure. By simulating a dynamic environment, we also evaluate the consistency of performance improvements brought by pre-training as real-world training data becomes available. The results of our experiments suggest both the utility of pre-training with synthetic data and its improvements’ consistency over time.

In recent years, a multitude of e-commerce websites arose. Product Search is a fundamental part of these websites, which is often managed as a traditional retrieval task. However, Product Search has the ultimate goal of satisfying specific and personal user needs, leading users to find and purchase what they are looking for, based on their preferences. To maximize users’ satisfaction, Product Search should be treated as a personalized task. In this paper, we propose and evaluate a simple yet effective personalized results re-ranking approach based on the fusion of the relevance score computed by a well-known ranking model, namely BM25, with the scores deriving from multiple user/item representations. Our main contributions are: (1) we propose a score fusion-based approach for personalized re-ranking that leverages multiple user/item representations, (2) our approach accounts for both content-based features and collaborative information (i.e. features extracted from the user–item interactions graph), (3) the proposed approach is fast and scalable, can be easily added on top of any search engine and it can be extended to include additional features. The performed comparative evaluations show that our model can significantly increase the retrieval effectiveness of the underlying retrieval model and, in the great majority of cases, outperforms modern Neural Network-based personalized retrieval models for Product Search.

ECIR 2023, Item does not contain fulltext

Personalized Query Expansion, the task of expanding queries with additional terms extracted from the user-related vocabulary, is a well-known solution to improve the retrieval performance of a system w.r.t. short queries. Recent approaches rely on word embeddings to select expansion terms from user-related texts. Although promising results have been delivered with former word embedding techniques, we argue that these methods are not suited for contextual word embeddings, which produce a unique vector representation for each term occurrence. In this article, we propose a Personalized Query Expansion method designed to solve the issues arising from the use of contextual word embeddings with the current Personalized Query Expansion approaches based on word embeddings. Specifically, we employ a clustering-based procedure to identify the terms that better represent the user interests and to improve the diversity of those selected for expansion, achieving improvements of up to 4% w.r.t. the best-performing baseline in terms of MAP@100. Moreover, our approach outperforms previous ones in terms of efficiency, allowing us to achieve sub-millisecond expansion times even in data-rich scenarios. Finally, we introduce a novel metric to evaluate the expansion terms’ diversity and empirically show the unsuitability of previous approaches based on word embeddings when employed along with contextual word embeddings, which cause the selection of semantically overlapping expansion terms.

This is a report on the thirteenth edition of the Conference and Labs of the Evaluation Forum (CLEF 2022), held on September 5--8, 2022, in Bologna, Italy. CLEF was a four-day hybrid event combining a conference and an evaluation forum. The conference featured keynotes by Benno Stein and Rita Cucchiara, and presentation of peer-reviewed research papers covering a wide range of topics, in addition to many posters. The evaluation forum consisted of fourteen labs: ARQMath, BioASQ, CheckThat!, ChEMU, eRisk, HIPE, iDPP, ImageCLEF, JokeR, LeQua, LifeCLEF, PAN, SimpleText, and Touché, addressing a wide range of tasks, media, languages, and ways to go beyond standard test collections.

In domain-specific search, effectiveness depends on considering multiple dimensions of relevance, beyond topicality, related to the domain involved. Estimating additional relevance dimensions can affect efficiency because the computation of relevance scores is time-consuming if performed on the full document, especially for query-dependent dimensions. Hence, this article introduces an approach for improving effectiveness in domain-specific search by considering multiple dimensions of relevance, namely topicality, correctness, and credibility. To address efficiency, we propose a re-ranking approach that estimates domain-specific relevance scores on document summaries, rather than full documents. We validate the proposed solution by performing the Ad-Hoc Retrieval task from the TREC 2020 Health Misinformation Track, a domain that crucially relies on the considered relevance dimensions. Our findings underscore the potential of our approach with respect to both effectiveness and efficiency.

The fast development of novel approaches derived from the Transformers architecture has led to outstanding performance in different scenarios, from Natural Language Processing to Computer Vision. Recently, they achieved impressive results even in the challenging task of non-rigid shape matching. However, little is known about the capability of the Transformer-encoder architecture for the shape matching task, and its performances still remained largely unexplored. In this paper, we step back and investigate the contribution made by the Transformer-encoder architecture compared to its more recent alternatives, focusing on why and how it works on this specific task. Thanks to the versatility of our implementation, we can harness the bi-directional structure of the correspondence problem, making it more interpretable. Furthermore, we prove that positional encodings are essential for processing unordered point clouds. Through a comprehensive set of experiments, we find that attention and positional encoding are (almost) all you need for shape matching. The simple Transformer-encoder architecture, coupled with relative position encoding in the attention mechanism, is able to obtain strong improvements, reaching the current state-of-the-art.

Fake news is a major challenge in social media, particularly in the health domain where it can lead to severe consequences for both individuals and society as a whole. To contribute to combating this problem, we present a novel solution for improving the accuracy of detecting fake health news, utilizing a fine-Tuned BERT model that integrates both user-and content-related socio-contextual information. Specifically, this information is combined with the textual content itself to form a socio-contextual input sequence for the BERT model. By fine-Tuning such a model with respect to the health misinformation detection task, the resulting classifier can accurately predict the category to which each piece of content belongs, i.e., either "real health news"or "fake health news". We validate our solution through a series of experiments conducted on distinct publicly available datasets constituted by health-related tweets. These results illustrate the superiority of the proposed solution compared to the standard BERT baseline model and other advanced models. Indeed, they show that the integration of socio-contextual information in the detection process positively contributes to increasing the overall accuracy of the fake health news detection task. The study also suggests, in a preliminary way, how such information could be used for the explainability of the model itself.

Searching for online information is nowadays a critical task in a scenario characterized by information overload and misinformation. To address these issues, it is necessary to provide users with both topically relevant and truthful information. Re-ranking is a strategy often used in Information Retrieval (IR) to consider multiple dimensions of relevance. However, re-rankers often analyze the full text of documents to obtain an overall relevance score at the re-ranking stage, which can lead to sub-optimal results. Some recent Transformer-based re-rankers actually consider text passages rather than the entire document, but focus only on topical relevance. Transformers are also being used in non-IR solutions to identify information truthfulness, but just to perform a binary classification task. Therefore, in this article, we propose an IR model based on re-ranking that focuses on suitably identified text passages from documents for retrieving both topically relevant and truthful information. This approach significantly reduces the noise introduced by query-unrelated content in long documents and allows us to evaluate the document’s truthfulness against it, enabling more effective retrieval. We tested the effectiveness of the proposed solution in the context of the Consumer Health Search task, considering publicly available datasets. Our results show that the proposed approach statistically outperforms full-text retrieval models in the context of multidimensional relevance, such as those based on aggregation, and monodimensional relevance Transformer-based re-rankers, such as BERT-based re-rankers.

In this paper, we provide an overview of the ninth annual edition of the CLEF eHealth evaluation lab. CLEF eHealth 2021 continues our evaluation resource building efforts around the easing and support of patients, their next-of-kins, health care professionals, and health scientists in understanding, accessing, and authoring electronic health information in a multilingual setting. The 2021 lab offered two tasks: Task 1 on multilingual Information Extraction (IE), this year extending to a corpus of Spanish radiology reports; and Task 2 on Consumer Health Search (CHS) that builds on the previous year’s Information Retrieval (IR) tasks. In total, 11 teams took part in these tasks (7 in Task 1 on IE and 4 in Task 2 on IR). Herein, we describe the resources created for these tasks and the evaluation methodology adopted, and we provide a brief summary of the participants of this year’s challenges as well as the results obtained. As in previous years, the organizers have made data, tools, and more specific overview papers associated with the lab tasks available for future research and development.

Language embeddings are a promising approach for handling natural language expressions. Current embeddings encompass a large language corpus, and need to be retrained to deal with specific sub-domains. On the other hand, these embeddings often disregard even basic domain knowledge, making them specially fragile when handling technical, specific, knowledge domains, and requiring costly retraining. To alleviate this issue, we propose a combined approach where the embedding is seen as a model of a logical knowledge base. Through a continuous learning approach, the embedding improves its satisfaction of the knowledge base, and in turn produces better training examples by labelling previously unseen text. In this position paper we describe the general framework for this continuous learning, along with its main features.

Taxonomies are frequently used to represent knowledge; a taxonomy is a set of concepts partially ordered by a subsumption (is-a) relation. Recently, the CEDAR project has proposed a CEDAR Semantic Web reasoner based on OSF logic, which has shown very promising results with respect to both efficiency and scalability. Indeed, the basic operations behind the reasoning mechanism of CEDAR amount to answering Boolean queries on a taxonomy. For instance, CEDAR answers a conjunctive query on a taxonomy by computing the greatest lower bound of a subset of its elements. This paper proposes a generalization of this operation to the setting where the taxonomy is regarded as a fuzzy partially ordered set and an answer to a conjunctive query is associated with a satisfaction (approximation) degree. A conjunctive query on the fuzzy taxonomy can be answered by applying the same encoding technique of the CEDAR reasoner, while the approximation degree of an answer can be computed by adapting shortest path algorithms for directed acyclic graphs that can be further optimized thanks to the information provided by the encoding.

Fake news are spread by exploiting specific linguistic patterns aimed at triggering negative emotions and persuading the consumers. A way to contrast this phenomenon is to analyse the psychological factors underlying consumers’ vulnerabilities. This paper is situated in this research context: first, we study the correlation between psycho-linguistic patterns in user’s posts and the tendency to spread false information. Moreover, since online contents exploit multimedia information, a methodology aimed at profiling the authors based on the images they share is employed. The reported experiments show that the proposed method, which considers both text-related and image-related features, outperforms the results of state-of-the-art approaches.

With the advent of contextualized embeddings, attention towards neural ranking approaches for Information Retrieval increased considerably. However, two aspects have remained largely neglected: i) queries usually consist of few keywords only, which increases ambiguity and makes their contextualization harder, and ii) performing neural ranking on non-English documents is still cumbersome due to shortage of labeled datasets. In this paper we present SIR (Sense-enhanced Information Retrieval) to mitigate both problems by leveraging word sense information. At the core of our approach lies a novel multilingual query expansion mechanism based on Word Sense Disambiguation that provides sense definitions as additional semantic information for the query. Importantly, we use senses as a bridge across languages, thus allowing our model to perform considerably better than its supervised and unsupervised alternatives across French, German, Italian and Spanish languages on several CLEF benchmarks, while being trained on English Robust04 data only. We release SIR at https://github.com/SapienzaNLP/sir.

Social media allow to fulfill perceived social needs such as connecting with friends or other individuals with similar interests into virtual communities; they have also become essential as news sources, microblogging platforms, in particular, in a variety of contexts including that of health. However, due to the homophily property and selective exposure to information, social media have the tendency to create distinct groups of individuals whose ideas are highly polarized around certain topics. In these groups, a.k.a. echo chambers, people only "hear their own voice,” and divergent visions are no longer taken into account. This article focuses on the study of the echo chamber phenomenon in the context of the COVID-19 pandemic, by considering both the relationships connecting individuals and semantic aspects related to the content they share over Twitter. To this aim, we propose an approach based on the application of a community detection strategy to distinct topology- and content-aware representations of the COVID-19 conversation graph. Then, we assess and analyze the controversy and homogeneity among the different polarized groups obtained. The evaluations of the approach are carried out on a dataset of tweets related to COVID-19 collected between January and March 2020.

Faced with the problem of widespread health misinformation, in recent years credibility is being considered as one of the important dimensions for health-related information access and retrieval. To encourage research in this field, a couple of evaluation labs have been recently set up to provide large test collections, baselines, and evaluation metrics to interested researchers. The purpose of this article is to provide an overview of such evaluation labs, discussing their characteristics and open issues.

Maintaining reviews of scientific publications as soon as new relevant publications are available is a typical challenge to many research communities. We address this challenge as a content-based recommendation problem, where the publications already selected for a review drive the recommendation of the new publications. In addition, resources such as domain databases, ontologies and academic graphs provide structured information about publications (e.g., authors, journals, conferences). Our experiments show that a simple model based on that structured information to represent publications achieve high precision and recall, and outperform models that use more sophisticated representations based on embeddings.

In recent years, deep learning models have been successfully applied to Information Retrieval (IR), mainly for assessing the topical relevance of documents with respect to queries. However, relevance is a multidimensional concept, which can be assessed based on several criteria, depending on the document type, the domain considered, the search task performed, etc. Given that recent advancements in deep neural networks enable several learning tasks to be solved simultaneously, in this paper we examine the possibility of modeling multidimensional relevance by jointly solving a retrieval task, to learn topical relevance, and a classification task, to learn additional relevance dimensions. To instantiate and evaluate the proposed model, we consider three query-independent relevance dimensions beyond topicality, i.e., readability, trustworthiness, and credibility. The reported findings show that the proposed joint modeling can improve the performance of the retrieval task.

Motivated by the ever increasing difficulties faced by laypeople in retrieving and digesting valid and relevant information to make health-centred decisions, the CLEF eHealth lab series has offered shared tasks to the community in the fields of Information Extraction (IE), management, and Information Retrieval (IR) since 2013. These tasks have attracted large participation and led to statistically significant improvements in processing quality. In 2021, CLEF eHealth is calling for participants to contribute to the following two tasks: Task 1 on IE focuses on IE from noisy text. Participants will identify and classify Named Entities in written ultrasonography reports, containing misspellings and inconsistencies, from a major public hospital in Argentina. Identified entities will then have to be classified, which can be very challenging as it requires to handle lexical variations. Task 2 is a novel extension of the most popular and established task on consumer health search (CHS), aiming at retrieving relevant, understandable, and credible information for patients and their next-of-kins. In this paper we describe recent advances in the fields of IE and IR, and the subsequent offerings of this years CLEF eHealth lab challenges.

This paper details materials, methods, results, and analyses of the Consumer Health Search Task of the CLEF eHealth 2021 Evaluation Lab. This task investigates the effectiveness of information retrieval (IR) approaches in providing access to medical information to laypeople. For this a TREC-style evaluation methodology was applied: a shared collection of documents and queries is distributed, participants' runs received, relevance assessments generated, and participants' submissions evaluated. The task generated a new representative web corpus including web pages acquired from a 2021 CommonCrawl and social media content from Twitter and Reddit, along with a new collection of 55 manually generated layperson medical queries and their respective credibility, understandability, and topicality assessments for returned documents. This year's task focused on three subtask: (i) ad-hoc IR, (ii) weakly supervised IR, and (iii) document credibility prediction. In total, 15 runs were submitted to the three subtasks: eight addressed the ad-hoc IR task, three the weakly supervised IR challenge, and 4 the document credibility prediction challenge. As in previous years, the organizers have made data and tools associated with the task available for future research and development.

In this paper, an Information Foraging based approach that offers social media’s users the ability to get relevant and credible information is proposed. A Social Media Information Foraging System is developed in order to operate on social graphs taking into account the users’ interests and their social relations and interactions. To evaluate the performance of the system, a dataset was built using real data extracted from the information sharing network Twitter. The results consist in surfing paths leading to relevant information taking into consideration the user’s information needs and the credibility of the information.

Recommender systems have become a popular and effective means to manage the ever-increasing amount of multimedia content available today and to help users discover interesting new items. Today's recommender systems suggest items of various media types, including audio, text, visual (images), and videos. In fact, scientific research related to the analysis of multimedia content has made possible effective content-based recommender systems capable of suggesting items based on an analysis of the features extracted from the item itself. The aim of this survey is to present a thorough review of the state-of-the-art of recommender systems that leverage multimedia content, by classifying the reviewed papers with respect to their media type, the techniques employed to extract and represent their content features, and the recommendation algorithm. Moreover, for each media type, we discuss various domains in which multimedia content plays a key role in human decision-making and is therefore considered in the recommendation process. Examples of the identified domains include fashion, tourism, food, media streaming, and e-commerce.

Book search is a challenging task due to discrepancies between the content and description of books, on one side, and the ways in which people query for books, on the other. However, online reviewers provide an opinionated description of the book, with alternative features that describe the emotional and experiential aspects of the book. Therefore, locating emotional sentences within reviews, could provide a rich alternative source of evidence to help improve book recommendations. Specifically, sentiment analysis (SA) could be employed to identify salient emotional terms, which could then be used for query expansion? This paper explores the employment ofSA based query expansion, in the book search domain. We introduce a sentiment-oriented method for the selection of sentences from the reviews of top rated book. From these sentences, we extract the terms to be employed in the query formulation. The sentence selection process is based on a semi-supervised SA method, which makes use of adapted word embeddings and lexicon seed-words.Using the CLEF 2016 Social Book Search (SBS) Suggestion TrackCollection, an exploratory comparison between standard pseudo-relevance feedback and the proposed sentiment-based approach is performed. The experiments show that the proposed approach obtains 24%-57% improvement over the baselines, whilst the classic technique actually degrades the performance by 14%-51%.

Fuzzy DL-Lite has been studied as a means to answer queries w.r.t. vague or imprecise knowledge and facts. Existing approaches consider only the Zadeh semantics or are limited to precise terminological knowledge, where only facts are graded. We study the problem of answering conjunctive queries over fuzzy DL-Lite ontologies which allow also graded axioms, and whose semantics is based on mathematical fuzzy logic. We show that for the Gödel t-norm, the degree of an answer is computable through repeated calls to a classical query answering engine. For non-idempotent t-norms, we show the difficulty in dealing with degrees, and provide some partial solutions.

In this paper, we provide an overview of the CLEF eHealth Task 2 on Information Retrieval (IR), organized as part of the eighth annual edition of the CLEF eHealth evaluation lab by the Conference and Labs of the Evaluation Forum. Its aim was to address laypeople’s difficulties in retrieving and digesting valid and relevant information, in their preferred language, to make health-centred decisions. The task was a novel extension of the most popular and established task in CLEF eHealth on Consumer Health Search (CHS), which makes responses to spoken ad-hoc queries. In total, five submissions were made to its two subtasks; three addressed the ad-hoc IR task on text data and two considered the spoken queries. Herein, we describe the resources created for the task and evaluation methodology adopted. We also summarize lab submissions and results. As in previous years, organizers have made data, methods, and tools associated with the lab tasks available for future research and development.

The Third WEPIR 2020 workshop builds on the success of the first two WEPIR meetings held at CHIIR 2018 and CHIIR 2019. WEPIR 2020 again brings together researchers from different backgrounds interested in continuing to explore and advance the evaluation of personalisation in information retrieval. Similar to the first two workshops, WEPIR 2020 has a strong emphasis on active participation by workshop attendees. This was very successfully achieved in the first two workshops by the use of workshop breakout groups exploring topics related to personalisation and information retrieval, and the evaluation of personalisation in information retrieval in general, with subsequent report back to the workshop as a whole. However, a key difference for WEPIR 2020 is that while the first two workshops focused on developing and articulating principles and ideas relating general topics relating to these topics, identified as interesting and important by the attendees at the workshops, WEPIR 2020 focuses breakout discussion on a number of relevant specific use cases of the evaluation of personalisation in information retrieval. A use case is assigned to each breakout group with the plan being to have more than one group work on each use case. The task for each group is to identify specific relevant factors relating to the use case in terms of user activities, data to be collected, ethical issues, and evaluation metrics. Groups will make reports of their discussions to the assembled workshop in the final session with discussion of the alternative solutions relating to the same use case, and contrasting the issues raised by the different use cases. The overall goal of the workshop is to work towards developing a general set of principles and guidelines for addressing the evaluation of specific instances of the use of personalisation in information retrieval tasks. This is consistent with the activities and goals of the first two workshops, but represents a significant progression of th

This paper describes our contribution to the Answer Localization track of the MedVidQA 2022 Shared Task. We propose two answer localization approaches that use only textual information extracted from the video. In particular, our approaches exploit the text extracted from the video's transcripts along with the text displayed in the video's frames to create a set of features. Having created a set of features that represents a video's textual information, we employ four different models to measure the similarity between a video's segment and a corresponding question. Then, we employ two different methods to obtain the start and end times of the identified answer. One of them is based on a random forest regressor, whereas the other one uses an unsupervised peak detection model to detect the answer's start time. Our findings suggest that for this task, leveraging only text-related features (transmitted either verbally or visually) and using a small amount of training data, lead to significant improvements over the benchmark Video Span Localization model that is based on deep neural networks.