Search

To what extent are research results influenced by subjective decisions that scientists make as they design studies? Fifteen research teams independently designed studies to answer five original research questions related to moral judgments, negotiations, and implicit cognition. Participants from 2 separate large samples (total N > 15,000) were then randomly assigned to complete 1 version of each study. Effect sizes varied dramatically across different sets of materials designed to test the same hypothesis: Materials from different teams rendered statistically significant effects in opposite directions for 4 of 5 hypotheses, with the narrowest range in estimates being d = -0.37 to + 0.26. Meta-analysis and a Bayesian perspective on the results revealed overall support for 2 hypotheses and a lack of support for 3 hypotheses. Overall, practically none of the variability in effect sizes was attributable to the skill of the research team in designing materials, whereas considerable variability was attributable to the hypothesis being tested. In a forecasting survey, predictions of other scientists were significantly correlated with study results, both across and within hypotheses. Crowdsourced testing of research hypotheses helps reveal the true consistency of empirical support for a scientific claim. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Replication studies in psychological science sometimes fail to reproduce prior findings. If these studies use methods that are unfaithful to the original study or ineffective in eliciting the phenomenon of interest, then a failure to replicate may be a failure of the protocol rather than a challenge to the original finding. Formal pre-data-collection peer review by experts may address shortcomings and increase replicability rates. We selected 10 replication studies from the Reproducibility Project: Psychology (RP:P; Open Science Collaboration, 2015) for which the original authors had expressed concerns about the replication designs before data collection; only one of these studies had yielded a statistically significant effect (p

According to the social heuristics hypothesis, people intuitively cooperate or defect depending on which behavior is beneficial in their interactions. If cooperation is beneficial, people intuitively cooperate, but if defection is beneficial, they intuitively defect. However, deliberation promotes defection. Here, we tested two novel predictions regarding the role of trust in the social heuristics hypothesis. First, whether trust promotes intuitive cooperation. Second, whether preferring to think intuitively or deliberatively moderates the effect of trust on cooperation. In addition, we examined whether deciding intuitively promotes cooperation, compared to deciding deliberatively. To evaluate these predictions, we conducted a lab study in Colombia and an online study in the United Kingdom (N = 1,066; one study was pre-registered). Unexpectedly, higher trust failed to promote intuitive cooperation, though higher trust promoted cooperation. In addition, preferring to think intuitively or deliberatively failed to moderate the effect of trust on cooperation, although preferring to think intuitively increased cooperation. Moreover, deciding intuitively failed to promote cooperation, and equivalence testing confirmed that this null result was explained by the absence of an effect, rather than a lack of statistical power (equivalence bounds: d = -0.26 and 0.26). An intuitive cooperation effect emerged when non-compliant participants were excluded, but this effect could be due to selection biases. Taken together, most results failed to support the social heuristics hypothesis. We conclude by discussing implications, future directions, and limitations. The materials, data, and code are available on the Open Science Framework (). [ABSTRACT FROM AUTHOR]