Up or Out: Resetting Norms for Peer Reviewed Publishing in the Social Sciences

The peer review process typically has multiple evaluators. Too often editors rely on the Union Heuristic, which requires authors to perform all requested tests and extensions suggested by referees and editors, with results confirming the paper’s message. The Union Heuristic is easy, but has dysfunctional consequences—wasteful efforts on excessive empirical and theoretical robustness checks and extensions, slow publication times, bloated papers, and heavy use of referee time. We contrast this with the polar opposite (also not our recommendation), the Intersection Heuristic—requiring authors to perform only extensions requested by all evaluators. We propose that editors reset norms by making clear that as a matter of procedure, at editor discretion, excellent referee or coeditor suggestions will often not be mandated for authors. Instead, there should be a presumption of Up or Out (accept or reject typically one round of revisions or less) for submissions, with the extent of suggested changes held firmly in check. This will improve referee and author incentives, and streamline the publication process. It may also encourage authors to take the risks inherent in producing more innovative papers.