1. Impact-Based Ranking of Scientific Publications: A Survey and Experimental Evaluation
- Author
-
Thanasis Vergoulis, Theodore Dalamagas, Yannis Vassiliou, Ilias Kanellos, and Dimitris Sacharidis
- Subjects
business.industry ,Computer science ,Rank (computer programming) ,02 engineering and technology ,Data science ,Popularity ,Computer Science Applications ,Ranking (information retrieval) ,Market research ,Computational Theory and Mathematics ,Ranking ,020204 information systems ,0202 electrical engineering, electronic engineering, information engineering ,Benchmark (computing) ,business ,Scientific disciplines ,Information Systems - Abstract
As the rate at which scientific work is published continues to increase, so does the need to discern high-impact publications. In recent years, there have been several approaches that seek to rank publications based on their expected citation-based impact. Despite this level of attention, this research area has not been systematically studied. Past literature often fails to distinguish between short-term impact, the current popularity of an article, and long-term impact, the overall influence of an article. Moreover, the evaluation methodologies applied vary widely and are inconsistent. In this work, we aim to fill these gaps, studying impact-based ranking theoretically and experimentally. First, we provide explicit definitions for short-term and long-term impact, and introduce the associated ranking problems. Then, we identify and classify the most important ideas employed by state-of-the-art methods. After studying various evaluation methodologies of the literature, we propose a specific benchmark framework that can help us better differentiate effectiveness across impact aspects. Using this framework we investigate: (1) the practical difference between ranking by short- and long-term impact, and (2) the effectiveness and efficiency of ranking methods in different settings. To avoid reporting results that are discipline-dependent, we perform our experiments using four datasets from different scientific disciplines.
- Published
- 2021
- Full Text
- View/download PDF