Search

We propose that the epistemic functions of replication in science are best understood by relating them to kinds of experimental error/uncertainty. One kind of replication, which we call "direct replications," principally serves to assess the reliability of an experiment through its precision: the presence and degree of random error/statistical uncertainty. The other kind of replication, which we call "conceptual replications," principally serves to assess the validity of an experiment through its accuracy: the presence and degree of systematic errors/uncertainties. To illustrate the aptness of this general view, we examine the Hubble constant controversy in astronomy, showing how astronomers have responded to the concordances and discordances in their results by carrying out the different kinds of replication that we identify, with the aim of establishing a precise, accurate value for the Hubble constant. We contrast our view with Machery's "re-sampling" account of replication, which maintains that replications only assess reliability. [ABSTRACT FROM AUTHOR]

Explainable Artificial Intelligence (XAI) enables Artificial Intelligence (AI) to explain its decisions. This holds the promise of making AI more understandable to users, improving interaction, and establishing an adequate level of trust. We tested this claim in the high-risk task of AI-assisted mushroom hunting, where people had to decide whether a mushroom was edible or poisonous. In a between-subjects experiment, 328 visitors of an Austrian media art festival played a tablet-based mushroom hunting game while walking through a highly immersive artificial indoor forest. As part of the game, an artificially intelligent app analyzed photos of the mushrooms they found and recommended classifications. One group saw the AI's decisions only, while a second group additionally received attribution-based and example-based visual explanations of the AI's recommendation. The results show that participants with visual explanations outperformed participants without explanations in correct edibility assessments and pick-up decisions. This exhibition-based experiment thus replicated the decision-making results of a previous online study. However, unlike in the previous study, the visual explanations did not significantly affect levels of trust or acceptance measures. In a direct comparison, we consequently discuss the findings in terms of generalizability. Besides the scientific contribution, we discuss the direct impact of conducting XAI experiments in immersive art- and game-based environments in exhibition contexts on visitors and local communities by triggering reflection and awareness for psychological issues of human–AI interaction. [ABSTRACT FROM AUTHOR]

Reproducibility is a fundamental principle in science, ensuring reliable and valid findings. However, replication studies are scarce, particularly in ecology, due to the emphasis on novelty for publication. We explored the possibility of replicating original findings in the field of microbial and chemical ecology by conducting a conceptual replication of a previous study analysing the sex-specific differences in the microbial communities inhabiting the wing sacs, a scent organ with crucial functions in olfactory communication, of greater sac-winged bat (Saccopteryx bilineata). In the original study, the skin swabs from the antebrachial wing sacs of the males and wing sac rudiments of the females were analysed using culture-dependent methods to test sex-specific differences. The authors demonstrated that males have lower microbial richness and different microbial composition than females. We attempted to reproduce these findings using 16S rRNA sequencing, which offers improved accuracy in pinpointing microbial members than culture-dependent methods because of advanced statistical methods. Our study validated the original study’s findings: Males had a lower microbial richness, and the community composition differed between the sexes. Furthermore, in the current study, males had an increased abundance of bacteria that might potentially be involved in odour production and degradation of malodorous substances and antimicrobial production. Our conceptual replication study corroborated that microbes can play a role in shaping their host’s olfactory phenotype and consequently influence sexual selection. Furthermore, the current study emphasises the importance of replication efforts and hopefully encourages a culture that values replication studies in scientific practice. [ABSTRACT FROM AUTHOR]

In this article, we review the ways in which replication has been and could be featured in consumer behavior, using Journal of Consumer Research as a specific setting. We present a framework for thinking about the generalizability of research findings and differentiate various potential benefits that replication can have for understanding variability in consumer research findings. We then define four different types of replications, describe how researchers can use these approaches to produce distinct benefits, and give guidance regarding conducting, interpreting, and the potential contributions of these different types of replications. We conclude with a discussion of various ways in which replication could be more fully integrated into different phases of the scientific research process, taking into account the contribution necessary for publication. In particular, we identify opportunities to incorporate independent replication into original papers, to increase the replication-based contribution in papers that build on prior work, and to use systematic replication in conjunction with meta-analysis to synthesize and confirm conclusions from a mature research literature. More fully integrating replication into scientific practice can yield a new equilibrium, in which replication is routine, typically consistent with previous results, and recognized as necessary for establishing an empirical generalization. [ABSTRACT FROM AUTHOR]

In their study, Nishimura et al. (2022) examined the relationships between parental self-esteem and the levels of depression, autonomy, and career consciousness among children. They specifically focused on the mediating effect of parenting behaviors using triad data involving junior high school students and their parents. They proposed that these associations are mediated through parents’ and children’s perceptions of respecting their children’s thoughts, as well as fathers’ and children’s perceptions of positive responsiveness. However, the models developed by Nishimura et al. (2022) include several paths that have been explored during their investigation. This study examined the external validity of these models using a triad of data (N=642). This study did not replicate the models of Nishimura et al. (2022). However, the relationship between fathers’ self-esteem and children’s career consciousness was mediated by fathers’ and children’s perceptions of respecting children’s thoughts, and fathers’ and children’s perceptions of positive responsiveness. These results support the assertion of Nishimura et al. (2022) that parenting behaviors are essential for career consciousness among children. [ABSTRACT FROM AUTHOR]

Previous randomized-controlled trials have shown that a peer learning group intervention for career preparation can have positive effects on adolescent career management and the transition to post-comprehensive education in secondary schools. However, this earlier evidence of efficacy has been found by intervention studies that have implemented the method intensively. Earlier studies suggest that tailoring implementation strategies to fit organizational processes may offer opportunities to improve the sustainability and scalability of evidence-based practices. In this study, the implementation of the career preparation intervention was integrated into the school curriculum as a part of regular career counselling, lengthening the delivery timeframe of the intervention programme and using the schools' own guidance counsellors as intervention providers. We conducted a conceptual replication study to investigate how effectively the adapted intervention programme enhanced perceived career preparation skills by carrying out a new cluster randomized-controlled trial among 1068 pupils. Linear mixed models showed that the effect of peer learning sessions on career choice preparedness was positive in comparison to that in the control group. The result suggests that the adapted intervention programme preserved its identifiable positive effects, although dismissing intensive delivery features somewhat reduced the effect size. These study results have implications for school-based career counselling interventions as schools seek to identify evidence-based methods that are cost-efficient and may be integrated into regular school routines. [ABSTRACT FROM AUTHOR]

This study is a conceptual replication of a widely cited study by Moffitt et al. (2011) which found that attention and behavior problems in childhood (a composite of impulsive hyperactive, inattentive, and impulsive-aggressive behaviors labeled "self-control") predicted adult financial status, health, and criminal activity. Using data from longitudinal cohort studies in the United States (n = 1,168) and the United Kingdom (n = 16,506), we largely reproduced their pattern of findings that attention and behavior problems measured across the course of childhood predicted a range of adult outcomes including educational attainment (βU.S. = −0.22, βU.K. = −0.13) and spending time in jail (ORU.S. = 1.74, ORU.K. = 1.48). We found that associations with outcomes in education, work, and finances diminished in the presence of additional covariates for children's home environment and achievement but associations for other outcomes were more robust. We also found that attention and behavior problems across distinct periods of childhood were associated with adult outcomes. Specific attention and behavior problems showed some differences in predicting outcomes in the U.S. cohort, with attention problems predicting lower educational attainment and hyperactivity/impulsivity predicting ever spending time in jail. Together with the findings from Moffitt et al., our study makes clear that childhood attention and behavior problems are associated with a range of outcomes in adulthood for cohorts born in the 1950s, 1970s, and 1990s across three countries. Public Significance Statement: This study reveals that findings from Moffitt et al. (2011) that children exhibiting attention and behavior problems had greater financial struggles, poorer health, and a greater likelihood of criminality once they reached adulthood, generalized to cohorts born in different decades in the United States and the United Kingdom. Given the consistency of findings across cohorts, attention, and behavior problems in childhood appear to make it difficult to meet the demands of one's environment throughout development. [ABSTRACT FROM AUTHOR]

We provide a conceptual replication of an experimental study that uncovered a robust correlation between the strength of individuals' family ties and their distrust of strangers, striving to establish whether the link is causal. Using a different subjects pool and an online setting, we repeat the binary trust-game experiment from Ermisch and Gambetta and enrich it by manipulating the payoffs to create a low-trust and high-trust environment. The key finding is corroborated, but as expected, only in the high-trust environment. The two environments further allow us to impose a diff-and-diff design on the data, which rules out selection of low-trusting individuals into strong-tied families and gives us indirect evidence of causation, namely, that having strong family ties stunts the development of trust in strangers. Our findings support the emancipatory theory of trust proposed by Toshio Yamagishi and could be interpreted as uncovering the micro foundations of classic ethnographic studies, such as that by Edward Banfield, which described how subcultures fostering tight bonds within families or small groups make cooperation harder to be achieved. [ABSTRACT FROM AUTHOR]

The central core theory establishes that the organization of a social representation depends on a distinction between central and peripheral cognitions. Only the first ones are considered as particularly linked to the group identity. Previous works showed that, contrary to peripheral cognitions, central cognitions are subject to social valorization within the group. The present contribution aimed to conceptually replicate this finding, and to explore new hypotheses. We implemented two self-presentation paradigms (N = 161 and N = 183) using two different measures of adherence to social representation's cognitions. The previous results related to the social value of central cognitions were replicated twice in a row. However, both studies also highlighted the social value of peripheral cognitions and of the adherence to the social representation in its entirety. These results constitute interesting theoretical advancements, especially in the study of the articulation between social representations and social identity. [ABSTRACT FROM AUTHOR]

This paper describes an independently evaluated randomised controlled trial of a self-affirmation intervention, replicating earlier studies, mostly conducted in the US with ethnic minority students. Self-affirmation theory suggests that some stigmatised groups, such as those from ethnic minority or poor families, face stereotype threats which undermine their academic performance. Engaging in value affirmation writing activities when such threats are most salient can give individuals a positive sense of value, negating harmful feelings, and fostering academic learning. The present study, involving 10,807 pupils aged 14 to 16 in England showed that the intervention can be successfully replicated with children from low socioeconomic backgrounds in England. The analysis showed positive effects for the intervention group. Pupils who completed more exercises also performed better. The findings are worth consideration given that it costs virtually nothing and does no harm. [ABSTRACT FROM AUTHOR]

The present study attempts to replicate and extend Snyder et al. (2015, JPSP). The original study examined the latent factor structure of the Early Adolescent Temperament Questionnaire–Revised (EATQ-R), a commonly used measure of adolescent temperament, and showed that the resulting latent factors (i.e., effortful control, negative emotionality, and positive emotionality) had theoretically meaningful concurrent associations with several measures of adolescent functioning (depression, anxiety, attention-deficit hyperactivity disorder [ADHD], relational aggression, and school performance and behavior). We performed these same analyses using data from a large sample of Mexican-origin youth (N = 674), and also examined prospective associations between the three EATQ-R factors and measures of adolescent functioning assessed two years later. We found some evidence supporting the bifactor models reported in the original study but poor replication of the correlations among latent factors. Additionally, model comparisons demonstrated that correlated factors models produced more interpretable factors than the bifactor models. In contrast, we replicated most of the concurrent correlations (and extended the findings to prospective associations) between the EATQ-R factors and measures of adolescent functioning, supporting the construct validity of the EATQ-R as a measure of adolescent temperament. Thus, these findings raise concerns about the generalizability of the factor structure identified by Snyder et al. (2015), but bolster claims about the generalizability of the concurrent and predictive validity of the EATQ-R. Overall, differences between the present findings and those of Snyder et al. (2015) highlight the importance of ongoing construct validation in youth temperament research, especially with participants from groups traditionally underrepresented in psychological research. [ABSTRACT FROM AUTHOR]

• This study successfully replicated an earlier randomized control trial. • We examined parent and teacher interventions with an online, remote delivery approach. • Both the original and remote study increased targeted teacher and parent behaviors. • The magnitude of adult effects was larger in the remote study than the original study. • Both studies found larger social-behavioral effects than academic effects for children. This replication study examined remote delivery of 2 evidence-based prekindergarten (pre-k) interventions to evaluate whether the combination of interventions, one in Head Start classrooms (online The Early Education Model, eTEEM) and one in the home (online Play and Learning Strategies, ePALS) resulted in enhanced effects on at-risk 3- to 5-year-old children's school readiness skills when compared to either of these interventions alone. The aligned interventions trained adults – teachers and parents – to use responsiveness and guided learning approaches within routine activities with the aim of increasing children's social-behavioral and academic skills. Both the original in-person study (Landry et al., 2017) and the present remote study used a four-group factorial design. The present study first randomized classrooms to eTEEM (n = 36) or control (n = 33). Next, within classrooms, parent-child dyads were randomized to ePALS or No ePALS, resulting in four conditions: n = 186 eTEEM/ePALS; n = 151 eTEEM/No ePALS; n = 160 No eTEEM/ePALS; and n = 133 No eTEEM/No ePALS control. The original study found small to moderate effects on parent and teacher behaviors (d = 0.18 to 0.55). In this remote study, there were generally larger parent and teacher effects for responsiveness (g = 0.63 to 0.70) and for guided learning (g = 0.37 to 1.05), behaviors targeted by the interventions. Results of the original study showed few effects on children's academic outcomes and the strongest effect on children's social-behavioral skills for the parent intervention (d = 0.15 to 0.19). The present study produced similar patterns of results for child outcomes with larger social-behavioral effects (g = 0.19 to 0.27) than academic (g = -0.23 to 0.30). We discuss the importance of these replication findings when delivering interventions remotely to improve scalability. [ABSTRACT FROM AUTHOR]

Previous research has found that thinking about the presence of a partner in one's life can, at least temporarily, increase relationship satisfaction. However, other studies have found that doing the opposite—thinking about the absence of a partner from one's life (i.e., mentally subtracting a partner)—can boost relationship satisfaction. The current study is a conceptual replication and extension of a previous study demonstrating that people who imagined never meeting their current partner reported greater satisfaction than people who thought about the presence of a partner in their life or a neutral control. We expected that thinking about the absence of a partner could boost gratitude and, in turn, promote greater relationship satisfaction. However, we did not find support for the mental subtraction effect on relationship satisfaction or gratitude. That is, participants who mentally subtracted their romantic partners from their lives did not report greater relationship satisfaction or gratitude compared to participants who thought about the presence of their partner in their lives (or those in a control condition). Our study suggests that mentally subtracting positive life events may not always boost feelings of satisfaction. [ABSTRACT FROM AUTHOR]

Recent large‐scale projects in other disciplines have shown that results often fail to replicate when studies are repeated. The conditions contributing to this problem are also present in ecology, but there have not been any equivalent replication projects. Here, we survey ecologists' understanding of and opinions about replication studies. The majority of ecologists in our sample considered replication studies to be important (97%), not prevalent enough (91%), worth funding even given limited resources (61%), and suitable for publication in all journals (62%). However, there is a disconnect between this enthusiasm and the prevalence of direct replication studies in the literature which is much lower (0.023%: Kelly 2019) than our participants' median estimate of 10%. This may be explained by the obstacles our participants identified including the difficulty of conducting replication studies and of funding and publishing them. We conclude by offering suggestions for how replications could be better integrated into ecological research. [ABSTRACT FROM AUTHOR]

Does a more or less sympathetic protagonist influence transportation of the reader? Two new replication studies Three previous studies into presenting a protagonist in a story as more or less sympathetic have not provided a clear picture of the effects that the portrayal of the protagonist may have on transportation, and via transportation on story-consistent beliefs. Results from a first study (N = 83) by De Graaf and Hustinx (2015) suggest that the way the protagonist is portrayed – as sympathetic, unsympathetic or neutral – influences the extent to which readers are transported into a story. No significant effects on beliefs of the readers were found, however. In a direct replication study (N = 79) and in a conceptual replication study (N = 81), Jansen, Nederhoff, and Ooms (2017) found results that supported the hypotheses from the original study to a limited extent. In view of the relatively small numbers of participants in these three studies and the resulting limited power of the statistical tests two new, larger-scaled replication studies were conducted. A direct replication study was performed (N = 238) with the same versions of the story as used in the original study, and also a conceptual replication study (N = 248) with three versions of a new story. Again, the hypotheses from the original study were supported to a limited extent. A meta-analysis of all five studies revealed a large indirect positive effect of story version on transportation via empathy, when comparing the versions with a sympathetic protagonist with the versions with an unsympathetic protagonist. When comparing the neutral story versions with the versions with an unsympathetic protagonist, the meta-analytic indirect effect was medium sized. Other than what the Affective Disposition Theory (Raney, 2004; Zillmann, 1994; 2006) claims, the story versions with a neutral protagonist did not lead to an absence of emotional responses. Furthermore, the outcomes add to the Transportation-Imagery Model (Green & Brock, 2002; Van Laer, De Ruyter, Visconti, & Wetzels, 2014). While this model does not include concrete suggestions of story characteristics that lead to transportation, our studies show that a protagonist who is portrayed as sympathetic may contribute to the level of transportation that readers experience, be it indirectly through empathy. [ABSTRACT FROM AUTHOR]

Shimizu, Nakashima, and Morinaga (2016) reported that tendencies consistent with defensive pessimism (DP) are positively associated with considerate and respectful behavioral intentions toward strangers. However, two limitations hinder the generalizability of their findings: (1) their participants were exclusively female students of a women’s junior college, and (2) the cognitive strategy scale used in their study did not take all four types of cognitive strategy; i.e., DP, strategic optimism (SO), realistic pessimism (RP), and unjustified optimism (UO), into consideration. We replicated Shimizu et al. (2016) with adult respondents and used a different scale to enhance the generalizability of the results. Japanese adults (N=337) participated in an online survey. Path analysis of their responses indicated that a relationship exists between DP and behavioral intentions, which was consistent with the findings reported by Shimizu et al. (2016). The study also produced exploratory evidence that individuals that exhibit UO show less considerate and respectful behavioral intentions in interpersonal contexts than those who display SO or RP. [ABSTRACT FROM AUTHOR]

The present paper tackles a problem in the candidate reactions literature: the replicability of results that emerge when using in vivo and in vitro designs. The current study was designed to provide a conceptual replication of a dynamic model of change in fairness perceptions during the hiring process, using a different research design, different measures, and a different sample than the original paper. A sample of 209 participants in a simulated selection process reported their level of perceived fairness three times: before the testing, after the testing, and after the results were communicated. These results support the results of the original that the construction of fairness perceptions declines in a nonlinear way over time, with high initial levels of fairness perception corresponding to a lower rate of decline, and vice versa. Further analysis revealed no change in the individuals' conceptualization of perceived fairness nor any shift in the respondents' use of the measurement scale (i.e., no beta and gamma change were detected). [ABSTRACT FROM AUTHOR]

The purpose of this study was to examine whether the early and late start typology of criminal behaviour in major mental disorder can be generalized to different populations than evaluated to date and is replicable in two different countries. A Dutch forensic sample (Sample 1) consisted of 260 reports to the court with early start offenders (n= 62) and late start offenders (n= 198). A Canadian civil psychiatric sample (Sample 2) consisted of file information collected from 78 involuntarily hospitalized civil psychiatric patients with an early start group (n= 38) and a late start group (n= 40). In both samples, early and late starters were compared on different domains. Results showed that in general, early starters have a higher risk of having problems in different domains. There were also differences in the early and late start typology between the Dutch and Canadian samples. Our results partially support the early and late starter typology within two different samples. This study showed that early starters typically have a higher risk of problems in different domains. This highlights the importance of different risk management and treatment strategies for both the early start and the late start group. [ABSTRACT FROM PUBLISHER]

Abstract Kammermeier and Paulus (2018) report a partial replication of the results of Rubio-Fernández and Geurts (2013) but present their study as a failed replication. Paulus and Kammermeier (2018) insist on a negative interpretation of their findings, discrediting the Duplo task against their own empirical evidence. Here I argue that Paulus and Kammermeier may try to make an impactful contribution to the field by adding to the growing skepticism towards early Theory of Mind studies, but fail to make any significant contribution to our understanding of young children's Theory of Mind abilities. [ABSTRACT FROM AUTHOR]

Although many researchers have discussed replication as a means to facilitate self-correcting science, in this article, we identify meta-analyses and evaluating the validity of correlational and causal inferences as additional processes crucial to self-correction. We argue that researchers have a duty to describe sampling decisions they make; without such descriptions, self-correction becomes difficult, if not impossible. We developed the Replicability and Meta-Analytic Suitability Inventory (RAMSI) to evaluate the descriptive adequacy of a sample of studies taken from current psychological literature. Authors described only about 30% of the sampling decisions necessary for self-correcting science. We suggest that a modified RAMSI can be used by authors to guide their written reports and by reviewers to inform editorial recommendations. Finally, we claim that when researchers do not describe their sampling decisions, both readers and reviewers may assume that those decisions do not matter to the outcome of the study, do not affect inferences made from the research findings, do not inhibit inclusion in meta-analyses, and do not inhibit replicability of the study. If these assumptions are in error, as they often are, and the neglected decisions are relevant, then the neglect may create a good deal of mischief in the field. [ABSTRACT FROM AUTHOR]

Reproducibility is the cornerstone of science. If an effect is reliable, any competent researcher should be able to obtain it when using the same procedures with adequate statistical power. Two of the articles in this special section question the value of direct replication by other laboratories. In this commentary, I discuss the problematic implications of some of their assumptions and argue that direct replication by multiple laboratories is the only way to verify the reliability of an effect. [ABSTRACT FROM PUBLISHER]

The standard visual search task is integral to the study of selective attention and in search tasks target present slopes are the primary index of attentional demand. However, there are times when similarities in slopes may obscure important differences between conditions. To demonstrate this point, we used the case of line-ending illusory contours, building on a study by Li, Cave, and Wolfe (2008) where orientation-based search for figures defined by line-ending illusory contours was compared to that for the corresponding real-contour controls. Consistent with Li et al. (2008), we found search to be efficient for both illusory contour figures and the corresponding real-contour controls, with no significant differences between them. However, major differences between illusory contours and the real-contour controls emerged in selective enumeration, a task where participants enumerated targets in a display of distractors, with the number of targets and distractors manipulated. When looking at the distractor slopes, the increase in RT to enumerate a single target as a function of the number of distractors (a direct analogue to target present trials, with identical displays), we found distractor costs for illusory contour figures to be over 100 ms/distractor higher than for the corresponding real-contour controls. Furthermore, the discrepancies in RT slope between 1–3 and 6–8 targets associated with subitizing were only seen in the real-contour controls. These results show that similarities in RT slopes in search may mask important differences between conditions that emerge in other tasks. • Search equivalent for line-ending illusory contours and real-contour controls • Differences emerge in selective enumeration. • No subitizing and high distractor costs with illusory contour figures • Subitizing and low distractor costs with real-contour controls [ABSTRACT FROM AUTHOR]