1. A SIMULATION OF DISAGREEMENT FOR CONTROL OF RATIONAL CHEATING IN PEER REVIEW.
- Author
-
GRIMALDO, FRANCISCO and PAOLUCCI, MARIO
- Subjects
MULTIAGENT systems ,PROFESSIONAL peer review ,COMPUTER simulation ,CROWDSOURCING ,REPUTATION ,RELIABILITY (Personality trait) - Abstract
Understanding the peer review process could help research and shed light on the mechanisms that underlie crowdsourcing. In this paper, we present an agent-based model of peer review built on three entities - the paper, the scientist and the conference. The system is implemented on a BDI platform (Jason) that allows to define a rich model of scoring, evaluating and selecting papers for conferences. Then, we propose a programme committee update mechanism based on disagreement control that is able to remove reviewers applying a strategy aimed to prevent papers better than their own to be accepted ("rational cheating"). We analyze a homogeneous scenario, where all conferences aim to the same level of quality, and a heterogeneous scenario, in which conferences request different qualities, showing how this affects the update mechanism proposed. We also present a first step toward an empirical validation of our model that compares the amount of disagreements found in real conferences with that obtained in our simulations. [ABSTRACT FROM AUTHOR]
- Published
- 2013
- Full Text
- View/download PDF