Search

SummaryFor many, declining biodiversity represents an emotionally and psychologically distant ‘cost’ – similar to how a number of people perceive climate change. Using an expectancy-value theory framework, we showed participants photographs that visibly illustrated the threat of biodiversity loss. Specifically, we tested a combination of preregistered and exploratory hypotheses through an online experiment (n = 843) to understand whether viewing photographs of plants and animals (with and without captions) bolstered people’s valuing of biodiversity and willingness to donate to a nature-focused charity relative to a control group. Participants who viewed photographs (without captions) valued biodiversity more and donated more to the nature-focused charity; those who viewed photographs with captions showed similar though more muted (non-statistically significant) effects. Follow-up mediation analyses on the photographs-only participants suggested that the photographs may have catalysed negative emotions that increased valuing of biodiversity and, in turn, increased donations. This study provides preregistered evidence that thoughtfully selected photographs boost people’s valuing of biodiversity and exploratory evidence that the pathway through which that might occur is more likely via negative emotions than through reduced psychological distance. Educators, conservationists, journalists and others may find these results informative as they develop strategies for addressing the acute problem of biodiversity loss.

Recently, scholars have noted how several “old school” practices—a host of well-regarded, long-standing scientific norms—in combination, sometimes compromise the credibility of research. In response, other scholarly fields have developed several “open science” norms and practices to address these credibility issues. Against this backdrop, this special issue explores the extent to which and how these norms should be adopted and adapted for educational psychology and education more broadly. Our introductory article contextualizes the special issue’s goals by: overviewing the historical context that led to open science norms (particularly in medicine and psychology); providing a conceptual map to illustrate the interrelationships between various old school as well as open science practices; and then describing educational psychologists’ opportunity to benefit from and contribute to the translation of these norms to novel research contexts. We conclude by previewing the articles in the special issue.

Many states mandate districts or schools notify parents when students have missed multiple unexcused days of school. We report a randomized experiment ( N = 131,312) evaluating the impact of sending parents truancy notifications modified to target behavioral barriers that can hinder effective parental engagement. Modified truancy notifications that used simplified language, emphasized parental efficacy, and highlighted the negative incremental effects of missing school reduced absences by 0.07 days in the 1 month following compared to the standard, legalistic, and punitively worded notification—an estimated 40% improvement over the standard truancy notification. This work illustrates how behavioral insights and randomized experiments can be used to improve administrative communications in education.

It is common for organizations to offer awards to motivate individual behavior, yet few empirical studies evaluate their effectiveness in the field. We report a randomized field experiment (N = 15,329) that tests the impact of two common types of symbolic awards: pre-announced awards (prospective) and surprise awards (retrospective). The context is U.S. schools, where we explore how awards motivate student attendance. Contrary to our pre-registered hypotheses and organizational leaders’ expectations, the prospective awards did not on average improve behavior, and the retrospective awards decreased subsequent attendance. Moreover, we find a significant negative effect on attendance after prospective incentives were removed, which points to a crowding-out effect. Survey experiments probing the mechanisms suggest that awards may cause these unintended effects by inadvertently signaling that the target behavior (perfect attendance) is neither the social norm nor institutionally expected. In addition, receiving the retrospective award suggests to recipients that they have already outperformed the norm and what was expected of them, hence licensing them to miss school. Exploratory analyses shed further light on differential effects of awards by age and performance.

As an educational problem, the accelerating loss of biodiversity—one of the planet’s most pressing issues—represents particularly intriguing challenge. First, although biodiversity loss unfolds around on a daily basis, many individuals struggle to see, comprehend, and care about it. In addition, educators addressing biodiversity loss must address multiple outcomes simultaneously. For people to value biodiversity, educators must attend to emotional, motivational, and behavioral dimensions as well as learners’ understanding of key concepts. Focusing on the valuing component of expectancy-value theory, we evaluated the potential of photographs to affect participants’ emotional reactions, valuing of biodiversity, pro-environmental behaviors and content-relevant learning. Through a preliminary, exploratory experiment ( N = 340 adults) and a preregistered, confirmatory experiment ( N = 1870 secondary school students), we found broadly consistent evidence that strategically selected photographs induced negative emotions, increased participants’ valuing of biodiversity, and motivated pro-environmental behavior. Meanwhile, we saw no evidence of deleterious (or positive) effects on participants’ learning when the photographs accompanied an informational text. We conclude by discussing how costs are conceptualized within expectancy-value theory, as well as the potential of photographs as a useful pedagogical strategy for broad array of environmental educators.

Pre-registration and registered reports are two of the most promising open science practices for increasing transparency in the scientific process. Pre-registration involves publishing a timestamped record of a study design, ideally before data collection and analysis, so that research consumers can discern which analytic decisions were set a priori and which were changed after seeing data. Registered reports take the idea of pre-registration one step further, and provide peer review at the pre-registration stage. Researchers submit a Phase I manuscript that contains the introduction, background and context, and methods section of a study, and these Phase I manuscripts are peer reviewed. If reviewed positively, manuscripts are given in-principle acceptance, where the editors agree that if the researchers conduct the study as pre-registered--or document the deviations from their plan--the study will be published without regard for the direction or magnitude of findings. In this manner, studies are judged by whether they address important questions and use well-designed methods, not on the basis of reaching specific benchmarks for significance or effect size. This article illustrates the emerging range of approaches to pre-registration and registered reports with examples from a variety of studies and from the first special issue in educational research devoted to Registered Reports.PLEASE DO NOT CITE YET:This article is part of a forthcoming journal Special Issue on Open Science in Education and currently under review. Carly Robinson is NOT the correct author, so please do not cite this article until it is updated with the correct authors' names. If you are interested in citing this work please either (a) check back at this url later -- we anticipated that the correct authors' names will be included no later than February 2021, or (b) contact Carly Robinson (carly_robinson@brown.edu) directly to see if the paper might be cited on an earlier time frame.

Many educational interventions encourage parents to engage in their child’s education as if parental time and attention is limitless. Sadly, though, it is not. Successfully encouraging certain parental investments may crowd out other productive behaviors. A randomized field experiment (N = 2,212) assessed the impact of an intervention in which parents of middle and high school students received multiple text messages per week encouraging them to ask their children specific questions tied to their science curriculum. The intervention increased parent–child at-home conversations about science but did not detectably impact science test scores. However, the intervention decreased parent engagement in other, potentially productive, parent behaviors. These findings illustrate that parent engagement interventions are not costless: There are opportunity costs to shifting parental effort.

Like performance-enhancing drugs inflating apparent athletic achievements, several common social science practices contribute to the production of illusory results. In this article, we examine the processes that lead to illusory findings and describe their consequences. We borrow from an approach used increasingly by other disciplines—the norm of preregistering studies. Specifically, we examine how this practice of publicly posting documentation of one's prespecified hypotheses and other key decisions of a study prior to study implementation or data analysis could improve scientific integrity within education. In an attempt to develop initial guidelines to facilitate preregistrations in education, we discuss the types of studies that ought to be preregistered and the logistics of how educational researchers might execute preregistrations. We conclude with ideas for how researchers, reviewers, and the field of education more broadly might speed the adoption of this new norm.

Administrators often struggle in getting teachers to trust their school’s evaluation practices – a necessity if teachers are to learn from the feedback they receive. We attempted to bolster teacher...

The purpose of education research is to better understand educational phenomena to inform policy and improve practice. Forward progress within any field is based on the validity and credibility of that field’s research base - educators cannot make informed decisions based on anecdotal evidence, opaque research practices, or on studies that cannot be replicated. Educational research is plagued by the same challenges as a number of fields - publication bias, barriers to research access, and seminal theories that have failed to replicate. This symposium will share four research practices that can increase the replicability and credibility of educational science. Each will be presented in terms of benefits and challenges for the researcher, the journal editor, and the educational knowledge base in general.

Competing in today’s workforce increasingly requires earning a college degree, yet almost half of all enrolled undergraduates do not graduate. As the costs of dropping out of college continue to rise, instructor-student relationships may be a critical yet underexplored avenue for improving college student outcomes. The present study attempts to replicate and extend a prior study that improved teacher-student relationships at the high school level in a college setting. In this registered report, we test whether an intervention that highlights instructor-student commonalities improves similarity, instructor-student relationships, academic achievement, and persistence for undergraduate students in a large, diverse public university. We found that the intervention increased perceptions of similarity but not downstream relational or academic outcomes. Our exploratory analyses provide one of the first investigations suggesting that instructor-student relationships predict an array of consequential student outcomes in college. These findings show a notable relationship gap: instructors perceived less positive relationships with certain student groups, but on average, students perceived equally positive relationships with their instructors. American Educational Research Association

Many states mandate districts or schools notify parents when students have missed multiple unexcused days of school. We report a randomized experiment (N = 131,312) evaluating the impact of sending parents truancy notifications modified to target behavioral barriers that can hinder effective parental engagement. Modified truancy notifications that used simplified language, emphasized parental efficacy, and highlighted the negative incremental effects of missing school reduced absences by 0.07 days compared to the standard, legalistic, and punitively-worded notification — an estimated 40% improvement. This work illustrates how behavioral insights and randomized experiments can be used to improve administrative communications in education.

It is common for organizations to offer awards to motivate individual behavior, yet few empirical studies evaluate their effectiveness in the field. We report a randomized field experiment (N = 15,329) that tests the impact of two common types of symbolic awards: pre-announced awards (prospective) and surprise awards (retrospective). The context is U.S. schools, where we explore how awards motivate student attendance. Contrary to our pre-registered hypotheses and organizational leaders’ expectations, the prospective awards did not on average improve behavior, and the retrospective awards decreased subsequent attendance. Moreover, we find a significant negative effect on attendance after prospective incentives were removed, which points to a crowding-out effect. Survey experiments probing the mechanisms suggest that awards may cause these unintended effects by inadvertently signaling that the target behavior (perfect attendance) is neither the social norm nor institutionally expected. In addition, receiving the retrospective award suggests to recipients that they have already outperformed the norm and what was expected of them, hence licensing them to miss school. Exploratory analyses shed further light on differential effects of awards by age and performance.

Attendance in kindergarten and elementary school robustly predicts student outcomes. Despite this well-documented association, there is little experimental research on how to reduce absenteeism in the early grades. This paper presents results from a randomized field experiment in 10 school districts evaluating the impact of a low-cost, parent-focused intervention on student attendance in grades K–5. The intervention targeted commonly held parental misbeliefs undervaluing the importance of regular K–5 attendance as well as the number of school days their child had missed. The intervention decreased chronic absenteeism by 15%. This study presents the first experimental evidence on how to improve student attendance in grades K–5 at scale and has implications for increasing parental involvement in education.

Like performance enhancing drugs inflating apparent athletic achievements, several common social science practices contribute to the production of illusory results. In this article, we examine the processes that lead to illusory findings and describe their consequences. Borrowing from an approach used increasingly by other disciplines—the norm of pre-registering studies—we examine how this practice could improve scientific integrity within education. As an initial attempt to develop guidelines that facilitate pre-registrations in education, we discuss the types of studies that ought to be pre-registered and the logistics of how educational researchers might execute pre-registrations. We conclude with ideas for how researchers, reviewers, and the field of education more broadly might speed the adoption of this new norm.

Student intention and motivation are among the strongest predictors of persistence and completion in Massive Open Online Courses (MOOCs), but these factors are typically measured through fixed-response items that constrain student expression. We use natural language processing techniques to evaluate whether text analysis of open responses questions about motivation and utility value can offer additional capacity to predict persistence and completion over and above information obtained from fixed-response items. Compared to simple benchmarks based on demographics, we find that a machine learning prediction model can learn from unstructured text to predict which students will complete an online course. We show that the model performs well out-of-sample, compared to a standard array of demographics. These results demonstrate the potential for natural language processing to contribute to predicting student success in MOOCs and other forms of open online learning.