Search

It is a healthy exercise to debate the merits of using effect-size benchmarks to interpret research findings. However, these debates obscure a more central insight that emerges from empirical distributions of effect-size estimates in the literature: Efforts to improve education often fail to move the needle. I find that 36% of effect sizes from randomized control trials of education interventions with standardized achievement outcomes are less than 0.05 "SD." Publication bias surely masks many more failed efforts from our view. Recognizing the frequency of these failures should be at the core of any approach to interpreting the policy relevance of effect sizes. We can aim high without dismissing as trivial those effects sizes that represent more incremental improvement.

A core motivation for the widespread teacher evaluation reforms of the past decade was the belief that these new systems would promote teacher development through high-quality feedback. We examine this theory by studying teachers' perceptions of evaluation feedback in Boston Public Schools and evaluating the district's efforts to improve feedback through an administrator training program. Teachers generally reported that evaluators were trustworthy, fair, and accurate but that they struggled to provide high-quality feedback. We find little evidence that the training program improved perceived feedback quality, classroom instruction, teacher self-efficacy, or student achievement. Our results illustrate the challenges of using evaluation systems as engines for professional growth when administrators lack the time and skill necessary to provide frequent, high-quality feedback.

Numerous high-profile efforts have sought to "turn around" low-performing schools. Evidence on the effectiveness of school turnarounds, however, is mixed, and research offers little guidance on which models are more likely to succeed. We present a mixed-methods case study of turnaround efforts led by the Blueprint Schools Network in three schools in Boston. Using a difference-in-differences framework, we find that Blueprint raised student achievement in ELA by at least a quarter of a standard deviation, with suggestive evidence of comparably large effects in math. We document qualitatively how differential impacts across the three Blueprint schools relate to contextual and implementation factors. In particular, Blueprint's role as a turnaround partner (in two schools) versus school operator (in one school) shaped its ability to implement its model. As a partner, Blueprint provided expertise and guidance but had limited ability to fully implement its model. In its role as an operator, Blueprint had full authority to implement its turnaround model but was also responsible for managing the day-to-day operations of the school, a role for which it had limited prior experience.

Economic downturns can cause major funding shortfalls for U.S. public schools, often forcing districts to make difficult budget cuts, including teacher layoffs. In this brief, we synthesize the empirical literature on the widespread teacher layoffs caused by the Great Recession. Studies find that teacher layoffs harmed student achievement and were inequitably distributed across schools, teachers, and students. Research suggests that specific elements of the layoff process can exacerbate these negative effects. Seniority-based policies disproportionately concentrate layoffs among teachers of color, who are more likely to be early career teachers. These "last-in first-out" policies also disproportionately affect disadvantaged students because these students are more likely to be taught by early career teachers. The common practice of widely distributing pink slips warning about a potential job loss also appears to increase teacher churn and negatively impact teacher performance. Drawing on this evidence, we outline a set of policy recommendations to minimize the need for teacher layoffs during economic downturns and ensure that the burden of any unavoidable job cuts does not continue to be borne by students of color and students from low-income backgrounds.

Researchers commonly interpret effect sizes by applying benchmarks proposed by Jacob Cohen over a half century ago. However, effects that are small by Cohen's standards are large relative to the impacts of most field-based interventions. These benchmarks also fail to consider important differences in study features, program costs, and scalability. In this article, I present five broad guidelines for interpreting effect sizes that are applicable across the social sciences. I then propose a more structured schema with new empirical benchmarks for interpreting a specific class of studies: causal research on education interventions with standardized achievement outcomes. Together, these tools provide a practical approach for incorporating study features, costs, and scalability into the process of interpreting the policy importance of effect sizes.

Teacher coaching has emerged as a promising alternative to traditional models of professional development. We review the empirical literature on teacher coaching and conduct meta-analyses to estimate the mean effect of coaching programs on teachers' instructional practice and students' academic achievement. Combining results across 60 studies that employ causal research designs, we find pooled effect sizes of 0.49 standard deviations (SD) on instruction and 0.18 SD on achievement. Much of this evidence comes from literacy coaching programs for prekindergarten and elementary school teachers in the United States. Although these findings affirm the potential of coaching as a development tool, further analyses illustrate the challenges of taking coaching programs to scale while maintaining effectiveness. Average effects from effectiveness trials of larger programs are only a fraction of the effects found in efficacy trials of smaller programs. We conclude by discussing ways to address scale-up implementation challenges and providing guidance for future causal studies.

In recent years, states and districts have responded to federal incentives and pressure to institute major reforms to their teacher evaluation systems. The passage of the Every Student Succeeds Act in 2015 now provides state policymakers with even greater autonomy to redesign existing evaluation systems. Yet, little evidence exists to inform decisions about two key system design features: teacher performance measure weights and performance ratings thresholds. Using data from the Measures of Effective Teaching study, we conduct simulation-based analyses that illustrate the critical role that performance measure weights and ratings thresholds play in determining teachers' summative evaluation ratings and the distribution of teacher proficiency rates. These findings offer insights to policymakers and administrators as they refine and possibly remake teacher evaluation systems.

Matthew A. Kraft, an assistant professor of education and economics at Brown University, highlights new research showing that frequent, personalized outreach to parents can boost parent engagement and student achievement. He offers tips on how schools can create infrastructures, including digital technology tools, to better support such communication.

We study the relationship between school organizational contexts, teacher turnover, and student achievement in New York City (NYC) middle schools. Using factor analysis, we construct measures of four distinct dimensions of school climate captured on the annual NYC School Survey. We identify credible estimates by isolating variation in organizational contexts within schools over time. We find that improvements in school leadership especially, as well as in academic expectations, teacher relationships, and school safety are all independently associated with corresponding reductions in teacher turnover. Increases in school safety and academic expectations also correspond with student achievement gains. These results are robust to a range of threats to validity suggesting that our findings are consistent with an underlying causal relationship.

We use matched employee-employer records from the teacher labor market to explore the effects of late teacher hiring on student achievement. Hiring teachers after the school year starts reduces student achievement by 0.042 SD in mathematics and 0.026 SD in reading. This reflects, in part, a temporary disruption effect in the first year. In mathematics, but not in reading, late-hired teachers remain persistently less effective, evidence of negative selection in the teacher labor market. Late hiring concentrates in schools that disproportionately serve disadvantaged student populations, contributing to challenges in ensuring an equitable distribution of educational resources for all students.

Most teacher layoffs during the Great Recession were implemented following inverse-seniority policies. In this paper, I examine the implementation of a discretionary layoff policy in Charlotte Mecklenburg Schools. Administrators did not uniformly lay off the most or least senior teachers but instead selected teachers who were previously retired, late-hired, unlicensed, low-performing, or nontenured. Using quasi-experimental variation within schools across grades, I then estimate the differential effects of teacher layoffs on student achievement based on teacher seniority and effectiveness. Mathematics achievement in grades that lost an effective teacher, as measured by principal evaluations or value-added scores, decreased 0.05 to 0.11 standard deviations more than in grades that lost an ineffective teacher. In contrast, teacher seniority has limited predictive power on the effects of layoffs. Simulation analyses show that the district selected teachers who were, on average, less effective than those teachers identified under an inverse-seniority policy, and also reduced job losses.

Research has been showing that the most important development in K-12 education over the past decade has been the emergence of a growing number of urban schools that have been convincingly shown to have dramatic positive effects on the achievement of disadvantaged students. Those with the strongest evidence of success are oversubscribed charter schools. These schools hold admissions lotteries, which enable researchers to compare the subsequent test-score performance of students who enroll to that of similar students not given the same opportunity. Through careful study of the most effective of these charter schools, researchers have identified common practices--a longer school day and year, regular coaching to improve teacher performance, routine use of data to inform instruction, a culture of high expectations--that have yielded promising results when replicated in district schools.

Context: In the past two years, states have implemented sweeping reforms to their teacher evaluation systems in response to Race to the Top legislation and, more recently, NCLB waivers. With these new systems, policymakers hope to make teacher evaluation both more rigorous and more grounded in specific job performance domains such as teaching quality and contributions to student outcomes. Attaching high stakes to teacher scores has prompted an increased focus on the reliability and validity of these scores. Teachers unions have expressed strong concerns about the reliability and validity of using student achievement data to evaluate teachers and the potential for subjective ratings by classroom observers to be biased. The legislation enacted by many states also requires scores derived from teacher observations and the overall systems of teacher evaluation to be valid and reliable. Focus of the study: In this paper, we explore how state education officials and their district and local partners plan to implement and evaluate their teacher evaluation systems, focusing in particular on states' efforts to investigate the reliability and validity of scores emerging from the observational component of these systems. Research design: Through document analysis and interviews with state education officials, we explore several issues that arise in observational systems, including the overall generalizability of teacher scores; the training, certification, and reliability of observers; and specifications regarding the sampling and number of lessons observed per teacher. Findings: Respondents' reports suggest that states are attending to the reliability and validity of scores, but inconsistently; in only a few states does there appear to be a coherent strategy regarding reliability and validity in place. Conclusions: There remain a variety of system design and implementation decisions that states can optimize to increase the reliability and validity of their teacher evaluation scores. While a state may engage in auditing scores, for instance, it may miss the gains to reliability and validity that would accrue from periodic rater retraining and recertification, a stiff program of rater monitoring, and the use of multiple raters per teacher. Most troublesome are decisions about which and how many lessons to sample, which are either mandated legislatively, result from practical concerns or negotiations between stakeholders, or, at best case, rest on broad research not directly related to the state context. This suggests that states should more actively investigate the number of lessons and lesson sampling designs required to yield high-quality scores.

Background/Context: Educational policy makers have begun to recognize the challenges posed by teacher turnover. Schools and students pay a price when new teachers leave the profession after only 2 or 3 years, just when they have acquired valuable teaching experience. Persistent turnover also disrupts efforts to build a strong organizational culture and to sustain coordinated instructional programs throughout the school. Retaining effective teachers is a particular challenge for schools that serve high proportions of low-income and minority students. Although some interpret these turnover patterns as evidence of teachers' discontent with their students, recent large-scale quantitative studies provide evidence that teachers choose to leave schools with poor work environments and that these conditions are most common in schools that minority and low-income students typically attend. Thus, mounting evidence suggests that the seeming relationship between student demographics and teacher turnover is driven not by teachers' responses to their students, but by the conditions in which they must teach and their students are obliged to learn. Purpose/Objective/Research Question/Focus of Study: We build on this body of work by further examining how working conditions predict both teachers' job satisfaction and their career plans. We use a broad conception of the context of teachers' work, paying attention not only to narrowly defined working conditions but also to the interpersonal and organizational contexts in which teachers work. We also extend Ladd's analysis describing the relationship between the work context and student achievement. Advancing our understanding of this relationship is particularly important, given the increasing emphasis legislators place on evidence of student achievement when evaluating education policy. Specifically, we ask three research questions: (1) Do the conditions of work in Massachusetts public schools affect teachers' satisfaction with their jobs and their career plans? (2) Are schools with better conditions of work more successful in raising student performance than schools with less supportive working conditions? (3) If the conditions of work are important, what elements of the work environment matter the most? Research Design: In this article, we combine a statewide survey of school working conditions (MassTeLLS) with demographic and student achievement data from Massachusetts. We examine three primary outcomes: teacher satisfaction, teacher career intentions, and student achievement growth. From different items on the MassTeLLS, we construct a set of nine key elements that reflect the broad-based conditions in which teachers work. We fit standard regression models that describe the relationship between each outcome and both overall conditions of work and each element separately, modeling this relationship according to the properties of our outcome variables. Findings/Results: We found that measures of the school environment explain away much of the apparent relationship between teacher satisfaction and student demographic characteristics. The conditions in which teachers work matter a great deal to them and, ultimately, to their students. Teachers are more satisfied and plan to stay longer in schools that have a positive work context, independent of the school's student demographic characteristics. Furthermore, although a wide range of working conditions matter to teachers, the specific elements of the work environment that matter the most to teachers are not narrowly conceived working conditions such as clean and well-maintained facilities or access to modern instructional technology. Instead, it is the social conditions--the school's culture, the principal's leadership, and relationships among colleagues--that predominate in predicting teachers' job satisfaction and career plans. More important, providing a supportive context in which teachers can work appears to contribute to improved student achievement. We found that favorable conditions of work predict higher rates of student academic growth, even when we compare schools serving demographically similar groups of students. Conclusions/Recommendations: In short, we found that the conditions of teachers' work matter a great deal. These results align with a growing body of work examining the organizational characteristics of the schools in which teachers work. Together, these studies suggest strongly that the high turnover rates of teachers in schools with substantial populations of low-income and minority students are driven largely by teachers fleeing the dysfunctional and unsupportive work environments in the schools to which low-income and minority students are most likely to be assigned. If public education is to provide effective teachers for all students, then the schools those students attend must become places that support effective teaching and learning across all classrooms. (Contains 7 tables and 1 figure.)

We examine the fundamental and complex role that time plays in the learning process. We begin by developing a conceptual framework to elucidate the multiple obstacles schools face in converting total time in school into active learning time. We then synthesize the causal research and document a clear positive effect of additional time on student achievement typically of small to medium magnitude depending on dosage, use, and context. Further descriptive analyses reveal how large differences in the length of the school day and year across public schools are an underappreciated dimension of educational inequality in the United States. Finally, our case study of time loss in one urban district demonstrates the potential to substantially increase instructional time within existing constraints. [ABSTRACT FROM AUTHOR]

Parental involvement is correlated with student performance, though the causal relationship is less well established. This experiment examined an intervention that delivered weekly one-sentence individualized messages from teachers to the parents of high school students in a credit recovery program. Messages decreased the percentage of students who failed to earn course credit from 15.8% to 9.3%—a 41% reduction. This reduction resulted primarily from preventing drop-outs, rather than from reducing failure or dismissal rates. The intervention shaped the content of parent–child conversations with messages emphasizing what students could improve, versus what students were doing well, producing the largest effects. We estimate the cost of this intervention per additional student credit earned to be less than one-tenth the typical cost per credit earned for the district. These findings underscore the value of educational policies that encourage and facilitate teacher-to-parent communication to empower parental involvement in their children's education. [ABSTRACT FROM AUTHOR]