1. Does preregistration improve the credibility of research findings?
- Author
-
Mark Rubin
- Subjects
FOS: Computer and information sciences ,bepress|Physical Sciences and Mathematics ,bepress|Social and Behavioral Sciences|Psychology|Other Psychology ,MetaArXiv|Physical Sciences and Mathematics|Statistics and Probability|Other Statistics and Probability ,lcsh:BF1-990 ,Public access ,Credibility ,MetaArXiv|Physical Sciences and Mathematics|Statistics and Probability ,Research data ,harking ,bepress|Social and Behavioral Sciences|Psychology ,MetaArXiv|Social and Behavioral Sciences ,Data collection ,business.industry ,Other Statistics (stat.OT) ,multiple testing ,Publication bias ,bepress|Medicine and Health Sciences ,Public relations ,$p$-hacking ,Research findings ,MetaArXiv|Medicine and Health Sciences ,Transparency (behavior) ,MetaArXiv|Physical Sciences and Mathematics ,forking paths ,optional stopping ,Statistics - Other Statistics ,lcsh:Psychology ,bepress|Physical Sciences and Mathematics|Statistics and Probability|Other Statistics and Probability ,bepress|Social and Behavioral Sciences ,bepress|Physical Sciences and Mathematics|Statistics and Probability ,SocArXiv|Social and Behavioral Sciences ,SocArXiv|Social and Behavioral Sciences|Psychology ,Psychology ,business ,SocArXiv|Social and Behavioral Sciences|Psychology|Other Psychology - Abstract
Preregistration entails researchers registering their planned research hypotheses, methods, and analyses in a time-stamped document before they undertake their data collection and analyses. This document is then made available with the published research report to allow readers to identify discrepancies between what the researchers originally planned to do and what they actually ended up doing. This historical transparency is supposed to facilitate judgments about the credibility of the research findings. The present article provides a critical review of 17 of the reasons behind this argument. The article covers issues such as HARKing, multiple testing, p-hacking, forking paths, optional stopping, researchers’ biases, selective reporting, test severity, publication bias, and replication rates. It is concluded that preregistration’s historical transparency does not facilitate judgments about the credibility of research findings when researchers provide contemporary transparency in the form of (a) clear rationales for current hypotheses and analytical approaches, (b) public access to research data, materials, and code, and (c) demonstrations of the robustness of research conclusions to alternative interpretations and analytical approaches.
- Published
- 2020