Assessing replication success via skeptical mixture priors

There is a growing interest in the analysis of replication studies of original findings across many disciplines. When testing a hypothesis for an effect size, two Bayesian approaches stand out for their principled use of the Bayes factor (BF), namely the replication BF and the skeptical BF. In particular, the latter BF is based on the skeptical prior, which represents the opinion of an investigator who is unconvinced by the original findings and wants to challenge them. We embrace the skeptical perspective, and elaborate a novel mixture prior which incorporates skepticism while at the same time controlling for prior-data conflict within the original data. Consistency properties of the resulting skeptical mixture BF are provided together with an extensive analysis of the main features of our proposal. Finally, we apply our methodology to data from the Social Sciences Replication Project. In particular we show that, for some case studies where prior-data conflict is an issue, our method uses a more realistic prior and leads to evidence-classification for replication success which differs from the standard skeptical approach.
Comment: 35 pages, 6 figures