Search

Registered reports engender a review process in which reviewers evaluate the merits of the research question rather than the magnitude or direction of the results. In this process, authors first submit introduction, methods, and prespecified analyses sections of an article. Next, an initial peer-review process assesses the merits of the research question, in the absence of knowing the results. Worthy articles are granted an "in-principle acceptance." The authors, then conduct their study, analyze their data, write up the remainder of their article, and resubmit it. The in-principle acceptance means that if the authors adhere to the research design as originally submitted, then the article will be accepted. Building from seven of the earliest registered reports in education research, our introduction describes the benefits, opportunities, and challenges of registered reports as a means to increasing transparency in research and reducing the number of illusory results.

As the world reckons with the existential threat posed by climate change, the US remains deeply divided about the need for action. The solution to this problem, many have argued, begins with teaching environmentalism in primary and secondary schools and fostering receptivity to environmental issues and environmental science among the next generation. This strategy, of course, comes with its own set of challenges. Where does climate change fit in the school curriculum? What can teachers do in their classrooms to mitigate the influence of partisan politics? How can an abstract global phenomenon be made real at the local level? If K-12 education has a role to play in the broader project of environmental conservation, such questions need to be answered. For this Policy Dialogue, the "HEQ" editors asked Zeke Baker and Hunter Gehlbach to explore the challenges and opportunities inherent in educating the next generation about climate change, drawing on analogues from the past and scholarship from the present to help us better understand the future. Gehlbach is a professor and vice dean at the Johns Hopkins University School of Education. Thanks to a mid-career fellowship from the Spencer Foundation, much of his research focus has shifted toward investigating how social psychological approaches might improve environmental education. Baker is an assistant professor of sociology at Sonoma State University. His research explores the development and use of climate science, especially insofar as climate knowledge is embedded in social relationships of power. "HEQ" Policy Dialogues are, by design, intended to promote an informal, free exchange of ideas between scholars. At the end of the exchange, we offer references for readers who wish to follow up on sources relevant to the discussion.

Education researchers use surveys widely. Yet, critics question respondents' ability to provide high-quality responses. As schools increasingly use student surveys to drive local policy making, respondents' (lack of) motivation to provide quality responses may threaten the wisdom of using questionnaires for data-based decision making. To better understand student satisficing--the practice of suboptimal responding on surveys--and its impact on data quality, we examined its pervasiveness and impact on a large-scale social-emotional learning survey administered to 409,721 elementary and secondary students. Findings indicated that despite the prevalence of satisficing, its impact on data quality appeared more modest than anticipated. We conclude by outlining an accessible approach for defining and calculating satisficing for researchers, practitioners, and policymakers.

Our field experiment extends prior work on college matriculation by testing the extent to which an artificially intelligent (AI) chatbot's outreach and support to college students (N = 4442) reduced summer melt and improved first-year college enrollment at a 4-year university. Specifically, we investigate which students the intervention proves most effective for. We find that the AI chatbot increased overall success with navigating financial aid processes, such that student take up of educational loans increased by four percentage points. This financial aid effect was concentrated among would-be first-generation college goers, for whom loan acceptances increased by eight percentage points. In addition, the outreach increased first-generation students' success with course registration and fall semester enrollment each by three percentage points. Our findings suggest that proactive chatbot outreach to students is likely to be most successful in reducing summer melt among those who may need the chatbot support the most.

Deep reinforcement learning using convolutional neural networks is the technology behind autonomous vehicles. Could this same technology facilitate the road to college? During the summer between high school and college, college-related tasks that students must navigate can hinder successful matriculation. We employ conversational artificial intelligence (AI) to efficiently support thousands of would-be college freshmen by providing personalized, text message-based outreach and guidance for each task where they needed support. We implemented and tested this system through a field experiment with Georgia State University (GSU). GSU-committed students assigned to treatment exhibited greater success with pre-enrollment requirements and were 3.3 percentage points more likely to enroll on time. Enrollment impacts are comparable to those in prior interventions but with substantially reduced burden on university staff. Given the capacity for AI to learn over time, this intervention has promise for scaling personalized college transition guidance.

Given family-school engagement is correlated with student academic achievement, educational researchers have long been interested in the construct. However, measuring parental engagement is deceptively challenging. The concept includes school-based engagement, home-based learning support, and family-school communication. In this article we focus exclusively on measuring school-based family engagement. Examining this topic requires measuring parents' perceptions of their engagement with schools alongside the barriers that may limit their involvement. However, it is not obvious that barriers items should function like a traditional survey scale. We addressed these measurement challenges through a survey design process that synthesized academic theory with empirical findings from parent respondents, resulting in three survey tools: an engagement scale, a school invitational barriers sub-scale, and a non-school barriers composite measure. Three studies (n = 385; n = 266; n = 589) provide evidence that the school-based engagement tools effectively measure engagement patterns. We conclude by describing the potential uses of the tools for educators and researchers.

Teacher evaluation reform has been among the most controversial education reforms in recent years. It also is one of the costliest in terms of the time teachers and principals must spend on classroom observations. We conducted a randomized field trial at four sites to evaluate whether substituting teacher-collected videos for in-person observations could improve the value of teacher observations for teachers, administrators, or students. Relative to teachers in the control group who participated in standard in-person observations, teachers in the video-based treatment group reported that post-observation meetings were more "supportive" and they were more able to identify a specific practice they changed afterward. Treatment principals were able to shift their observation work to noninstructional times. The program also substantially increased teacher retention. Nevertheless, the intervention did not improve students' academic achievement or self-reported classroom experiences, either in the year of the intervention or for the next cohort of students. Following from the literature on observation and feedback cycles in low-stakes settings, we hypothesize that to improve student outcomes schools may need to pair video feedback with more specific supports for desired changes in practice.

For a host of reasons, training students and teachers to more accurately discern the thoughts, feelings, and motivations of others is more important than ever. One approach to improving social perspective taking entails teaching students to understand when and how bias impedes their efforts to "read" others. To facilitate this educational goal, we propose a theory of meta-biases that unifies and distills those biases that frequently derail perspective taking attempts. Specifically, we argue that, as a default, people strive to accurately read themselves and others. However, two competing motives--meta-biases--often derail this default goal: a desire for cognitive efficiency and a need to protect or enhance one's sense of self. We hope the proposed theory proves to be practical for educators trying to facilitate interpersonal understanding and relationships, stimulating for researchers interested in testing theory, and generative for anyone developing interventions to mitigate social cognitive biases and improve person perception attempts.

Administrators often struggle in getting teachers to trust their school's evaluation practices--a necessity if teachers are to learn from the feedback they receive. We attempted to bolster teachers' support for receiving evaluative feedback from a particularly controversial source: student-perception surveys. For our intervention, we took one of two approaches to asking 309 teachers how they felt about students evaluating their teaching practice. Control participants responded only to core questions regarding their attitudes towards student-perception surveys. Meanwhile, treatment participants were first asked whether "teachers should evaluate administrators" in performance reviews and were then asked the core items about student-perception surveys. Congruent with cognitive dissonance theory, this juxtaposition of questions bolstered treatment teachers' support for using student surveys in teacher evaluations relative to the control group. We discuss the implications of these findings with respect to increasing teacher openness to alternative evaluation approaches, and consider whether surveys show promise as a vehicle for delivering interventions.

Like performance-enhancing drugs inflating apparent athletic achievements, several common social science practices contribute to the production of illusory results. In this article, we examine the processes that lead to illusory findings and describe their consequences. We borrow from an approach used increasingly by other disciplines--the norm of preregistering studies. Specifically, we examine how this practice of publicly posting documentation of one's prespecified hypotheses and other key decisions of a study prior to study implementation or data analysis could improve scientific integrity within education. In an attempt to develop initial guidelines to facilitate preregistrations in education, we discuss the types of studies that ought to be preregistered and the logistics of how educational researchers might execute preregistrations. We conclude with ideas for how researchers, reviewers, and the field of education more broadly might speed the adoption of this new norm.

When it comes to measuring the quality of a school, policy leaders tend to embrace standardized tests as the go-to indicator, whereas parents and community members tend to rely on reputation, word-of-mouth, and what they perceive with their own eyes. The authors suggest a better approach: a new framework that looks at three categories of inputs (the teaching environment, school culture, and resources) as well as two categories of outputs (academic learning and cultivation of student character and well-being). Parents and community members who used the new interactive tool came away with significantly more positive perceptions of unfamiliar schools.

Despite the enthusiasm around social-emotional learning, the vast number of skills, dispositions, and attitudes we hope to infuse into students will overwhelm even the most ambitious schools. However, a single core capacity underlies a great many social-emotional learning outcomes: social perspective taking. Recent research on this process of "reading" others--figuring out their thoughts, feelings, and motivations--suggests that we now know enough to teach this capacity to students. By helping youth engage in this process regularly as "detectives" rather than "judges" and providing them with feedback, a constellation of social-emotional learnings will blossom.

When people perceive themselves as similar to others, greater liking and closer relationships typically result. In the first randomized field experiment that leverages actual similarities to improve real-world relationships, we examined the affiliations between 315 9th grade students and their 25 teachers. Students in the treatment condition received feedback on 5 similarities that they shared with their teachers; each teacher received parallel feedback regarding about half of his or her 9th grade students. Five weeks after our intervention, those in the treatment conditions perceived greater similarity with their counterparts. Furthermore, when teachers received feedback about their similarities with specific students, they perceived better relationships with those students, and those students earned higher course grades. Exploratory analyses suggest that these effects are concentrated within relationships between teachers and their "underserved" students. This brief intervention appears to close the achievement gap at this school by over 60%.

Students whose school environment fits their developmental needs also typically experience academic success and increased motivation. Most investigations of school fit, which focus on teachers' and students' perceptions, have found a general decline in fit across the transition from elementary to middle school. However, little research has examined the school-child fit from the parent perspective. In this article, we first detail the development process behind a new survey measure of parent perceptions of school fit. Second, using three online panel-based samples of parents from across the country (n[subscript 1] = 323, n[subscript 2] = 188, n[subscript 3] = 1,033), we evaluate the scale's measurement properties and conduct exploratory analyses examining grade-level and income-based differences on reported school fit. Finally, in line with previous research, we find that parents of middle school children perceived statistically significantly worse fit than parents of elementary school children. Among parents of high school students, we found that, on average, high-income parents perceive statistically significantly better fit than low-income parents.

As pressure builds to assess students, teachers, and schools, educational practitioners and policy makers are increasingly looking toward student perception surveys as a promising means to collect high-quality, useful data. For instance, the widely cited Measures of Effective Teaching study lists student perception surveys as one of the three key measures of teachers' efficacy and describes these measures as a source of potentially valuable feedback for teachers (Cantrell & Kane, 2013). When one factors in the low cost with their prospective utility in assessing teachers and fostering teaching effectiveness, it seems clear that surveying students will increase dramatically in the coming years. Within this context, researchers who survey early adolescents must navigate the confluence of three tensions. First, responding to survey items requires an array of cognitive skills that early adolescents are still mastering (Downer, Stuhlman, Schweig, Martínez, & Ruzek, 2014). Second, unlike national, public opinion surveys, school-based surveys need to provide accurate data for small samples. For instance, a middle school teacher may have 100 students divided between her sixth, seventh, and eighth-grade science classes; an elementary school teacher may only have 20 students total. Third, for the full promise of surveys to be realized as a tool for improving schools, the survey scales need to be practitioner friendly: short, easy to administer, and straightforward to interpret (Hamre & Cappella, 2015, Kosovich, Hulleman, Barron, & Getty, 2014). In most cases, it should be relatively straightforward to adhere to the scientifically grounded best practices described in this article. In the coming years, surveys administered to early adolescents are likely to proliferate. Thus, there is an urgent need for research on the best approaches to designing surveys for this unique and important population to keep pace. In the meantime, the sins described here and the ideas for redressing them provide researchers with some low-cost approaches to improving their measures in most situations.

Background: Although teacher-student relationships lie at the heart of students' schooling experience, fundamental questions regarding these relationships remain unanswered. Aims: This study investigates three related questions about these relationships: To what extent do they change from the beginning to the end of a school year? Are any emergent changes associated with shifts in students' academic or motivational outcomes? Are certain "upstream" factors associated with improvements or declines in teacher-student relationships? Sample: We investigate these questions with a sample of middle school students (N = 119) and their teachers (N = 30). Methods: Through a novel approach which accounts for both perspectives within teacher-student relationships, we assess these relationships at the beginning and end of the school year. Using multi-level models, we examine how changes in these relationships are associated with changes in students' grades, homework completion rates, self-efficacy, and effort. In addition, we examine associations with two potential precursors to teacher-student relationships: students' accuracy in taking their teachers' perspective and their perceptions of similarity to their teachers. Results: We find that substantial changes occur in these relationships from the beginning to the end of the year; these changes are associated with shifts in important student outcomes; and changes in students' social perspective taking accuracy and perceived similarity to their teachers correspond with changes in teacher-student relationships. Conclusions: Given the malleability of teacher-student relationships and their importance for key achievement and motivational outcomes, we advocate for researchers to conduct field experiments to inform how to improve these critical relationships. (Contains 3 footnotes and 4 tables.)

Purpose/Objective: Under the reauthorized Every Student Succeeds Act (ESSA), states and districts retain greater discretion over the measures included in school quality report cards. Moreover, ESSA now requires states to expand their measurement efforts to address factors like school climate. This shift toward more comprehensive measures of school quality provides an opportunity for states and districts to think intentionally about a basic question: What specific information should schools collect and report to their communities? Setting: This study took place in the community surrounding a small, highly diverse urban school district. Population/Participants: Forty-five local residents representing a range of demographic backgrounds participated in a modified deliberative poll with an experimental treatment. Intervention/Program/Practice: We randomly assigned participants into two conditions. In the first, participants accessed the state web portal, which houses all publicly available educational data about districts in the state. In the second condition, participants accessed a customized portal that contained a wider array of school performance information collected by the research team. Research Design: This mixed-methods study used a modified deliberative polling format, in conjunction with a randomized controlled field trial. Data Collection and Analysis: Participants in both conditions completed a battery of survey items that were analyzed through multiple regressions. Findings/Results: When users of a more holistic and comprehensive data system evaluated unfamiliar schools, they not only valued the information more highly but also expressed more confidence in the quality of the schools. Conclusions/Recommendations: We doubt that more comprehensive information will inevitably lead to higher ratings of school quality. However, it appears--both from prior research, from theory, and from this project--that deeper familiarity with a school often fosters more positive perceptions. This may be because those unfamiliar with particular schools rely on a limited range of data, which fail to adequately capture the full range of performance variables, particularly in the case of urban schools. We encourage future exploration of this topic, which may have implications for school choice, parental engagement, and accountability policy.