1. Degrees of Freedom in Planning, Running, Analyzing, and Reporting Psychological Studies: A Checklist to Avoid p-Hacking
- Author
-
Wicherts, Jelte M., Veldkamp, Coosje L. S., Augusteijn, Hilde E. M., Bakker, Marjan, Van Aert, Robbie C. M., Van Assen, Marcel A. L. M., Leerstoel Buskens, Social Networks, Solidarity and Inequality, Department of Methodology and Statistics, Leerstoel Buskens, and Social Networks, Solidarity and Inequality
- Subjects
experimental design ,media_common.quotation_subject ,Applied psychology ,lcsh:BF1-990 ,Review ,Experimental design (study designs) ,050105 experimental psychology ,03 medical and health sciences ,0302 clinical medicine ,Bias ,Significance testing ,0502 economics and business ,P-hacking ,Psychology ,0501 psychology and cognitive sciences ,Psychological testing ,Quality (business) ,Statistical analysis ,050207 economics ,General Psychology ,media_common ,050208 finance ,Psychological research ,significance chasing ,Degrees of freedom ,05 social sciences ,Research methods education ,questionable research practices ,Checklist ,lcsh:Psychology ,p-hacking ,Social psychology ,030217 neurology & neurosurgery - Abstract
The designing, collecting, analyzing, and reporting of psychological studies entail many choices that are often arbitrary. The opportunistic use of these so-called researcher degrees of freedom aimed at obtaining statistically significant results is problematic because it enhances the chances of false positive results and may inflate effect size estimates. In this review article, we present an extensive list of 34 degrees of freedom that researchers have in formulating hypotheses, and in designing, running, analyzing, and reporting of psychological research. The list can be used in research methods education, and as a checklist to assess the quality of preregistrations and to determine the potential for bias due to (arbitrary) choices in unregistered studies.
- Published
- 2016
- Full Text
- View/download PDF