Search

This paper reports from a large-scale experiment conducted to study the effects of social norms on distributive decision-making. In an incentivized spectator experiment, subjects chose how to divide bonus earnings between a pair of stakeholders. Before choosing a distribution, our spectators stated their beliefs about, and received a signal about, the share of payoff-equalizing spectators in a reference group, randomly drawn from a previous study with the same distributive setting (Almås et al., 2020). This draw gives random variation in the intensity of the signal about the norm that applies in the current setting. We find a statistically significant but small effect of the number of payoff-equalizing spectators in the reference group on the probability of equalizing payoffs in the current setting. The redistribution choice is strongly correlated with spectators’ initial beliefs about the reference group. The effect of the signal about redistribution is primarily driven by the subgroup of participants who receive a large shock to their initial beliefs.

Many researchers rely on meta-analysis to summarize research evidence. However, there is a concern that publication bias and selective reporting may lead to biased meta-analytic effect sizes. We compare the results of meta-analyses to large-scale preregistered replications in psychology carried out at multiple laboratories. The multiple-laboratory replications provide precisely estimated effect sizes that do not suffer from publication bias or selective reporting. We searched the literature and identified 15 meta-analyses on the same topics as multiple-laboratory replications. We find that meta-analytic effect sizes are significantly different from replication effect sizes for 12 out of the 15 meta-replication pairs. These differences are systematic and, on average, meta-analytic effect sizes are almost three times as large as replication effect sizes. We also implement three methods of correcting meta-analysis for bias, but these methods do not substantively improve the meta-analytic results. Kvarven, Stromland and Johannesson compare meta-analyses to multiple-laboratory replication projects and find that meta-analyses overestimate effect sizes by a factor of almost three. Commonly used methods of adjusting for publication bias do not substantively improve results.

This paper argues that some of the discussion around meta-scientific issues can be viewed as an argument over different “meta-hypotheses” – assumptions made about how different hypotheses in a scientific literature relate to each other. I argue that, currently, such meta-hypotheses are typically left unstated except in methodological papers and that the consequence of this practice is that it is hard to determine what can be learned from a direct replication study. I argue in favor of a procedure dubbed the “limited homogeneity assumption” – assuming very little heterogeneity of effect sizes when a literature is initiated but switching to an assumption of heterogeneity once an initial finding has been successfully replicated in a direct replication study. Until that has happened, we do not allow the literature to proceed to a mature stage. This procedure will elevate the scientific status of direct replication studies in science. Following this procedure, a well-designed direct replication study is a means of falsifying an overall claim in an early phase of a literature and thus sets up a hurdle against the canonization of false facts in the behavioral sciences.

The hypothesis that intuition promotes cooperation has attracted considerable attention. Although key results in this literature have failed to replicate in pre-registered studies, recent meta-analyses report an overall effect of intuition on cooperation. We address the question with a meta-analysis of 82 cooperation experiments, spanning four different types of intuition manipulations—time pressure, cognitive load, depletion, and induction—including 29,315 participants in total. We obtain a positive overall effect of intuition on cooperation, though substantially weaker than that reported in prior meta-analyses, and between studies the effect exhibits a high degree of systematic variation. We find that this overall effect depends exclusively on the inclusion of six experiments featuring emotion-induction manipulations, which prompt participants to rely on emotion over reason when making allocation decisions. Upon excluding from the total data set experiments featuring this class of manipulations, between-study variation in the meta-analysis is reduced substantially—and we observed no statistically discernable effect of intuition on cooperation. Overall, we fail to obtain compelling evidence for the intuitive cooperation hypothesis.

Contains fulltext : 228242.pdf (Publisher’s version ) (Open Access) Drawing on the concept of a gale of creative destruction in a capitalistic economy, we argue that initiatives to assess the robustness of findings in the organizational literature should aim to simultaneously test competing ideas operating in the same theoretical space. In other words, replication efforts should seek not just to support or question the original findings, but also to replace them with revised, stronger theories with greater explanatory power. Achieving this will typically require adding new measures, conditions, and subject populations to research designs, in order to carry out conceptual tests of multiple theories in addition to directly replicating the original findings. To illustrate the value of the creative destruction approach for theory pruning in organizational scholarship, we describe recent replication initiatives re-examining culture and work morality, working parents’ reasoning about day care options, and gender discrimination in hiring decisions. Significance statement It is becoming increasingly clear that many, if not most, published research findings across scientific fields are not readily replicable when the same method is repeated. Although extremely valuable, failed replications risk leaving a theoretical void - reducing confidence the original theoretical prediction is true, but not replacing it with positive evidence in favor of an alternative theory. We introduce the creative destruction approach to replication, which combines theory pruning methods from the field of management with emerging best practices from the open science movement, with the aim of making replications as generative as possible. In effect, we advocate for a Replication 2.0 movement in which the goal shifts from checking on the reliability of past findings to actively engaging in competitive theory testing and theory building. Scientific transparency statement The materials, code, and data for this article are posted publicly on the Open Science Framework, with links provided in the article. 19 p.

Heterogenous treatment effects make it difficult to extrapolate from one research setting to another. However, what appears to be differences in effects across subpopulations may simply be false positives. This paper uses a representative sample of the Norwegian population (N = 1390) to systematically test for several proposed sources of heterogeneity in the literature on intuitive prosociality – a literature with large variation in results, which some researchers claim results from heterogeneity in the underlying effect. We use time pressure to induce intuitive decision making, and exogenously vary participants’ experience with the game. We find no overall effect of time constraints on dictator game for inexperienced subjects, and there is no evidence for an interaction effect between subject experience and the effect of time pressure. As a more general test of treatment effect heterogeneity, we consider the full distribution of treatment effects conditional on various proposed moderators in the literature. The distribution of conditional effects is consistent with no causal effect of time pressure on giving and no systematic heterogeneity in the underlying effect across subpopulations.

Many view preregistration as a promising way to improve research credibility. However, scholars have argued that using pre-analysis plans in Experimental Economics has limited benefits. This paper argues that preregistration of studies is likely to improve research credibility. I show that in a setting with selective reporting and low statistical power, effect sizes are highly inflated, and this translates into low reproducibility. Preregistering the original studies could avoid such inflation of effect sizes—through increasing the share of “frequentist” researchers—and would lead to more credible power analyses for replication studies. Numerical applications of the model indicate that the inflation bias could be very large in practice, and available empirical evidence is in line with the central assumptions of the model. publishedVersion

Many markets resemble repeated prisoner's dilemma situations with the possibilities for mutual partner choice. In this paper, we show experimentally that partner choice by mutual consent has a strong positive effect on cooperation. Mutual partner choice makes it possible to form long-lasting reciprocal partnerships. To understand partnership formation we also add a treatment where the participants could communicate with each other in a common chat room. Chat transcript reveals that promises are important in forming and sustaining a partnership. acceptedVersion

Are humans intuitively cooperative, or does reflection foster cooperation? The Social Heuristics Hypothesis (SHH) proposes that intuition promotes cooperation, but the evidence is inconsistent across studies. We combine a lab experiment with a re-analysis of existing studies to test for an effect of time-pressure on cooperation. We suggest – based on a meta-study of about 12.000 subjects from lab and online experiments across the world - that time pressure only influences cooperation if subjects understand they are engaged in a social dilemma. This finding increases the relevance of the SHH for experiments in behavioral economics as such experiments typically take great care to ensure subject understanding. Moreover, while our findings are intuitively consistent with the Social Heuristics Hypothesis, our evidence suggests that future theoretical research should aim to explicitly incorporate the link between understanding, cognitive processes and cooperation.

We contribute to the experimental literature by examining the causal effect of partner choice opportunities on the earnings of different cooperative types. We first elicit cooperative types and then randomly assign subjects to a repeated prisoner's dilemma game, with either mutual partner choice or random matching. In each period, the individual who fails to attain a partner is excluded from the group. The results from three experiments show that mutual partner choice enables cooperators to outperform free riders; cooperators tend to earn more than free riders and are less frequently excluded. Our findings are robust with respect to varying group size and whether subjects are reminded about their entire partner and earnings history or only their recent history. publishedVersion