281 results on '"KRAFT, MATTHEW A."'
Search Results
2. The Rise and Fall of the Teaching Profession: Prestige, Interest, Preparation, and Satisfaction over the Last Half Century. EdWorkingPaper No. 22-679
- Author
-
Annenberg Institute for School Reform at Brown University, Kraft, Matthew A., and Lyon, Melissa Arnold
- Abstract
We examine the state of the U.S. K-12 teaching profession over the last half century by compiling nationally representative time-series data on four interrelated constructs: professional prestige, interest among students, preparation for entry, and job satisfaction. We find a consistent and dynamic pattern across every measure: a rapid decline in the 1970s, a swift rise in the 1980s, relative stability for two decades, and a sustained drop beginning around 2010. The current state of the teaching profession is at or near its lowest levels in 50 years. We identify and explore a range of factors that might explain these historical patterns including education funding, teacher pay, outside opportunities, unionism, barriers to entry, working conditions, accountability, autonomy, and school shootings.
- Published
- 2022
3. Teacher Shortages: A Unifying Framework for Understanding and Predicting Vacancies. EdWorkingPaper No. 22-684
- Author
-
Annenberg Institute for School Reform at Brown University, Edwards, Danielle Sanderson, Kraft, Matthew A., Christian, Alvin, and Candelaria, Christopher A.
- Abstract
We develop a unifying conceptual framework for understanding and predicting teacher shortages at the state, region, district, and school levels. We then generate and test hypotheses about geographic, grade level, and subject variation in teacher shortages using data on teaching vacancies in Tennessee during the fall of 2019. We find that teacher staffing challenges are highly localized, causing shortages and surpluses to coexist. Aggregate descriptions of staffing challenges mask considerable variation between schools and subjects within districts. Schools with fewer local early-career teachers, smaller district salary increases, worse working conditions, and higher historical attrition rates have higher vacancy rates. Our findings illustrate why viewpoints about, and solutions to, shortages depend critically on whether one takes an aggregate or local perspective. [The Tennessee Education Research Alliance provided financial and technical support of this work.]
- Published
- 2022
4. Preferences, Inequities, and Incentives in the Substitute Teacher Labor Market. EdWorkingPaper No. 22-680
- Author
-
Annenberg Institute for School Reform at Brown University, Kraft, Matthew A., Conklin, Megan, and Falken, Grace T.
- Abstract
We examine the labor supply decisions of substitute teachers -- a large, on-demand market with broad shortages and inequitable supply. In 2018, Chicago Public Schools implemented a targeted bonus program designed to reduce unfilled teacher absences in largely segregated Black schools with historically low substitute coverage rates. Using a regression discontinuity design, we find that incentive pay substantially improved coverage equity and raised student achievement. Changes in labor supply were concentrated among Black and Hispanic substitutes from nearby neighborhoods with experience in incentive schools. Wage elasticity estimates suggest incentives would need to be 50% of daily wages to close fill-rate gaps.
- Published
- 2022
5. Instructional Time in U.S. Public Schools: Wide Variation, Causal Effects, and Lost Hours. EdWorkingPaper No. 22-653
- Author
-
Annenberg Institute for School Reform at Brown University, Kraft, Matthew A., and Novicoff, Sarah
- Abstract
Policymakers have renewed calls for expanding instructional time in the wake of the COVID-19 pandemic. We establish a set of empirical facts about time in school, synthesize the literature on the causal effects of instructional time, and conduct a case study of time use in an urban district. On average, instructional time in U.S. public schools is comparable to most high-income countries, with longer days but shorter years. However, instructional time varies widely across U.S. public schools with a 90th-10th percentile difference of 190 total hours. Empirical literature confirms that additional time can increase student achievement, but how this time is structured matters. Our case study suggests schools might also recover substantial lost learning time within the existing school day. [The Providence Public School District provided support for this study.]
- Published
- 2022
6. What Happened to the K-12 Education Labor Market during COVID? The Acute Need for Better Data Systems. EdWorkingPaper No. 22-544
- Author
-
Annenberg Institute for School Reform at Brown University, Bleiberg, Joshua F., and Kraft, Matthew A.
- Abstract
The COVID-19 pandemic upended the U.S. education system and the economy in ways that dramatically affected the jobs of K-12 educators. However, data limitations have led to considerable uncertainty and conflicting reports about the nature of staffing challenges in schools. We draw on education employment data from the Bureau of Labor Statistics (BLS) and State Education Agencies (SEA) to describe patterns in K-12 education employment and to highlight the limitations of available data. Data from the BLS suggest overall employment in the K-12 labor market declined by 9.3 percent at the onset of the pandemic and remains well below pre-pandemic levels. SEA data suggest that teachers have not (yet) left the profession in mass as many predicted, but that turnover decreased in the summer of 2020. We explore possible explanations for these patterns including (1) weak hiring through the summer of 2020 and (2) high attrition among K-12 instructional support staff. State vacancy data also suggest that schools are facing substantial challenges filling open positions during the 2021-22 academic year. Our analyses illustrate the imperative to build more timely, detailed, and nationally representative data systems on the K-12 education labor market to better inform policy.
- Published
- 2022
7. The Effect of Teacher Evaluation on Achievement and Attainment: Evidence from Statewide Reforms. EdWorkingPaper No. 21-496
- Author
-
Annenberg Institute for School Reform at Brown University, Bleiberg, Joshua, Brunner, Eric, Harbatkin, Erica, Kraft, Matthew A., and Springer, Matthew G.
- Abstract
Starting in 2009, the U.S. public education system undertook a massive effort to institute new high-stakes teacher evaluation systems. We examine the effects of these reforms on student achievement and attainment at a national scale by exploiting the staggered timing of implementation across states. We find precisely estimated null effects, on average, that rule out impacts as small as 1.5 percent of a standard deviation for achievement and 1 percentage point for high school graduation and college enrollment. We also find little evidence of heterogeneous effects across an index measuring system design rigor, specific design features, and district characteristics.
- Published
- 2021
8. The Inequitable Effects of Teacher Layoffs: What We Know and Can Do. EdWorkingPaper No. 21-487
- Author
-
Annenberg Institute for School Reform at Brown University, Kraft, Matthew A., and Bleiberg, Joshua F.
- Abstract
Economic downturns can cause major funding shortfalls for U.S. public schools, often forcing districts to make difficult budget cuts including teacher layoffs. In this brief, we synthesize the empirical literature on the widespread teacher layoffs caused by the Great Recession. Studies find that teacher layoffs harmed student achievement and were inequitably distributed across schools, teachers, and students. Research suggests that specific elements of the layoff process can exacerbate these negative effects. Seniority-based policies disproportionately concentrate layoffs among teachers of color who are more likely to be early career teachers. These "last-in first-out" policies also disproportionately affect disadvantaged students because these students are more likely to be taught by early career teachers. The common practice of widely distributing pink slips warning about a potential job loss also appears to increase teacher churn and negatively impact teacher performance. Drawing on this evidence, we outline a set of policy recommendations to minimize the need for teacher layoffs during economic downturns and ensure that the burden of any unavoidable job cuts does not continue to be borne by students of color and students from low-income backgrounds.
- Published
- 2021
9. Elevating Education in Politics: How Teacher Strikes Shape Congressional Election Campaigns. EdWorkingPaper No. 21-482
- Author
-
Annenberg Institute for School Reform at Brown University, Lyon, Melissa Arnold, and Kraft, Matthew A.
- Abstract
Teacher strikes have gained national attention with the "#RedforEd" movement. Such strikes are polarizing events that could serve to elevate education as a political priority or cast education politics in a negative light. We investigate this empirically by collecting original panel data on U.S. teacher strikes, which we link to congressional election campaign advertisements. Election ads provide a useful window into political discourse because they are costly to sponsors, consequential for voter behavior, and predictive of future legislative agendas. Using a differences-in-differences framework, we find that teacher strikes dramatically increase education issue salience, with impacts concentrated among positively-framed ads. Effects are driven by strikes lasting only a few days and occurring in battleground areas with highly-contested elections.
- Published
- 2021
10. The Effect-Size Benchmark That Matters Most: Education Interventions Often Fail
- Author
-
Kraft, Matthew A.
- Abstract
It is a healthy exercise to debate the merits of using effect-size benchmarks to interpret research findings. However, these debates obscure a more central insight that emerges from empirical distributions of effect-size estimates in the literature: Efforts to improve education often fail to move the needle. I find that 36% of effect sizes from randomized control trials of education interventions with standardized achievement outcomes are less than 0.05 "SD." Publication bias surely masks many more failed efforts from our view. Recognizing the frequency of these failures should be at the core of any approach to interpreting the policy relevance of effect sizes. We can aim high without dismissing as trivial those effects sizes that represent more incremental improvement.
- Published
- 2023
- Full Text
- View/download PDF
11. Accelerating Student Learning with High-Dosage Tutoring. EdResearch for Recovery Design Principles Series
- Author
-
EdResearch for Recovery Project, Annenberg Institute for School Reform at Brown University, Results for America, Robinson, Carly D., Kraft, Matthew A., Loeb, Susanna, and Schueler, Beth E.
- Abstract
This brief is one in a series aimed at providing K-12 education decision-makers and advocates with an evidence base to ground discussions about how to best serve students during and following the novel coronavirus pandemic. Rigorous research provides strong evidence that high-dosage tutoring can produce large learning gains for a wide range of students, including those who have fallen behind academically. This brief provides information on design principles for effective tutoring including: frequency, group size, personnel, focus, measurement, relationships, curriculum, scheduling, delivery mode, and prioritization. [This brief was prepared in collaboration with the University of Virginia, School of Education and Human Development.]
- Published
- 2021
12. The Big Problem with Little Interruptions to Classroom Learning
- Author
-
Kraft, Matthew A. and Monti-Nussbaum, Manuel
- Abstract
Narrative accounts of classroom instruction suggest that external interruptions, such as intercom announcements and visits from staff, are a regular occurrence in U.S. public schools. We study the frequency, nature, duration, and consequences of external interruptions in the Providence Public School District (PPSD) using original data from a district-wide survey and classroom observations. We estimate that a typical classroom in the PPSD is interrupted more than 2,000 times per year and that these interruptions and the disruptions they cause result in the loss of between 10 and 20 days of instructional time. Several findings suggest that there exists substantial scope for reducing interruptions. Administrators appear to systematically underestimate the frequency and negative consequences of interruptions. Furthermore, interruptions vary widely across schools and are largely caused by school staff. Schools might reduce disruptions to the learning environment by creating a culture that prioritizes instructional time, instituting better communication protocols, and addressing the challenges posed by student tardiness.
- Published
- 2021
13. A Blueprint for Scaling Tutoring and Mentoring across Public Schools
- Author
-
Kraft, Matthew A. and Falken, Grace T.
- Abstract
In this thought experiment, we explore how to make access to individualized instruction and academic mentoring more equitable by taking tutoring to scale as a permanent feature of the U.S. public education system. We first synthesize the tutoring and mentoring literature and characterize the landscape of existing tutoring programs. We then outline a blueprint for integrating federally funded and locally delivered tutoring into the school day. High school students would serve as tutors/mentors in elementary schools via an elective class, college students in middle schools via federal work-study, and 2- and 4-year college graduates in high schools via AmeriCorps. We envision an incremental, demand-driven expansion process with priority given to high-needs schools. Our blueprint highlights a range of design tradeoffs, implementation challenges, and program costs. We estimate that targeted approaches to scaling school-wide tutoring nationally, such as focusing on K-8 Title I schools, would cost between $5 and $16 billion annually.
- Published
- 2021
14. What Happened to the K-12 Education Labor Market during COVID? The Acute Need for Better Data Systems
- Author
-
Bleiberg, Joshua F. and Kraft, Matthew A.
- Abstract
The COVID-19 pandemic upended the U.S. education system in ways that dramatically affected the jobs of K-12 employees. However, there remains considerable uncertainty about the nature and degree of staffing challenges during the pandemic. We draw on data from the Bureau of Labor Statistics (BLS) and State Education Agencies (SEA) to describe patterns in K-12 education employment and to highlight the limitations of available data. Data from the BLS suggest overall employment in the K-12 labor market declined by 9 percent at the onset of the pandemic and remained well below pre-pandemic levels more than two years later. SEA data suggest that teachers did not leave the profession en masse as many predicted, with turnover decreasing in the summer of 2020 and then increasing modestly in 2021 back to pre-pandemic levels. We explore possible explanations for these patterns including weak hiring through the summer of 2020 and high attrition among K-12 instructional support and noninstructional staff. State vacancy data also suggest that schools faced substantial challenges filling open positions during the 2021-22 academic year. Our analyses illustrate the imperative to build nationally representative, detailed, and timely data systems on the K-12 education labor market to better inform policy.
- Published
- 2023
- Full Text
- View/download PDF
15. Local Supply, Temporal Dynamics, and Unrealized Potential in Teacher Hiring
- Author
-
James, Jessalynn, Kraft, Matthew A., and Papay, John P.
- Abstract
We explore the dynamics of competitive search in the K-12 public education sector. Using detailed panel data on teacher hiring from Boston Public Schools, we document how teacher labor supply varies substantially across vacancies even within a single district depending on position type, school characteristics, and the timing of job postings. We find that early-posted positions are more likely to be filled and end up securing new hires that are better qualified, more effective, and more likely to remain at a school. In contrast, the number of applicants to a position is largely unassociated with hire quality, suggesting that schools may struggle to identify and select the best candidates even when there is a large pool of qualified applicants. These patterns persist even when we restrict comparisons to only positions within an individual school using school fixed effects. Our findings point to substantial unrealized potential for improving teacher hiring.
- Published
- 2023
- Full Text
- View/download PDF
16. A Blueprint for Scaling Tutoring across Public Schools. EdWorkingPaper No. 20-335
- Author
-
Annenberg Institute for School Reform at Brown University, Kraft, Matthew A., and Falken, Grace T.
- Abstract
In this paper, we explore how tutoring could become a permanent feature of the U.S. public education system. We outline a blueprint for taking tutoring to scale nationally and estimate its costs, while highlighting a range of design and implementation challenges. Our blueprint is centered on ten core principles and a federal architecture to support adoption, while providing for local ownership over key implementation features. High school students would tutor in elementary schools via an elective class, college students in middle schools via federal work-study, and full time 2- and 4-year college graduates in high schools via AmeriCorps. We envision an incremental, demand-driven expansion process with priority given to high-needs schools. Our estimates suggest that a range of targeted approaches to scaling school-wide tutoring nationally, such as focusing on K-8 Title I schools, would cost between $5 and $15 billion annually. These costs are comparable to existing federal programs such as Title I, the National School Lunch Program, and Head Start and equate to roughly 1% to 2% of total expenditures on U.S. public education. Attempts to scale tutoring to address COVID-19 learning loss might be most successful and sustainable if they are part of an effort to incrementally integrate tutoring services within the public school system.
- Published
- 2020
17. Sustaining a Sense of Success: The Importance of Teacher Working Conditions during the COVID-19 Pandemic. EdWorkingPaper No. 20-279
- Author
-
Annenberg Institute for School Reform at Brown University, Kraft, Matthew A., Simon, Nicole S., and Lyon, Melissa Arnold
- Abstract
COVID-19 shuttered schools across the United States, upending traditional approaches to education. We examine teachers' experiences during emergency remote teaching in the spring of 2020 using responses to a working conditions survey from a sample of 7,841 teachers across 206 schools and 9 states. Teachers reported a range of challenges related to engaging students in remote learning and balancing their professional and personal responsibilities. Teachers in high-poverty and majority Black schools perceived these challenges to be the most severe, suggesting the pandemic further increased existing educational inequities. Using data from both pre-post and retrospective surveys, we find that the pandemic and pivot to emergency remote teaching resulted in a sudden, large drop in teachers' sense of success. We also demonstrate how supportive working conditions in schools played a critical role in helping teachers to sustain their sense of success. Teachers who could depend on their district and school-based leadership for strong communication, targeted training, meaningful collaboration, fair expectations, and recognition of their efforts were least likely to experience declines in their sense of success.
- Published
- 2020
18. Can Teacher Evaluation Systems Produce High-Quality Feedback? An Administrator Training Field Experiment
- Author
-
Kraft, Matthew A. and Christian, Alvin
- Abstract
A core motivation for the widespread teacher evaluation reforms of the past decade was the belief that these new systems would promote teacher development through high-quality feedback. We examine this theory by studying teachers' perceptions of evaluation feedback in Boston Public Schools and evaluating the district's efforts to improve feedback through an administrator training program. Teachers generally reported that evaluators were trustworthy, fair, and accurate but that they struggled to provide high-quality feedback. We find little evidence that the training program improved perceived feedback quality, classroom instruction, teacher self-efficacy, or student achievement. Our results illustrate the challenges of using evaluation systems as engines for professional growth when administrators lack the time and skill necessary to provide frequent, high-quality feedback.
- Published
- 2022
- Full Text
- View/download PDF
19. Developing Ambitious Mathematics Instruction through Web-Based Coaching: A Randomized Field Trial. EdWorkingPaper No. 19-119
- Author
-
Annenberg Institute for School Reform at Brown University, Kraft, Matthew A., and Hill, Heather C.
- Abstract
This paper describes and evaluates a web-based coaching program designed to support teachers in implementing Common Core-aligned math instruction. Web-based coaching programs can be operated at relatively lower costs, are scalable, and make it more feasible to pair teachers with coaches who have expertise in their content area and grade level. Results from our randomized field trial document sizable and sustained effects on both teachers' ability to analyze instruction and on their instructional practice, as measured the Mathematical Quality of Instruction (MQI) instrument and student surveys. However, these improvements in instruction did not result in corresponding increases in math test scores as measured by state standardized tests or interim assessments. We discuss several possible explanations for this pattern of results.
- Published
- 2019
20. Interpreting Effect Sizes of Education Interventions. EdWorkingPaper No. 19-10
- Author
-
Annenberg Institute for School Reform at Brown University and Kraft, Matthew A.
- Abstract
Researchers commonly interpret effect sizes by applying benchmarks proposed by Cohen over a half century ago. However, effects that are small by Cohen's standards are large relative to the impacts of most field-based interventions. These benchmarks also fail to consider important differences in study features, program costs, and scalability. In this paper, I present five broad guidelines for interpreting effect sizes that are applicable across the social sciences. I then propose a more structured schema with new empirical benchmarks for interpreting a specific class of studies: causal research on education interventions with standardized achievement outcomes. Together, these tools provide a practical approach for incorporating study features, cost, and scalability into the process of interpreting the policy importance of effect sizes.
- Published
- 2019
21. Balancing Rigor, Replication, and Relevance: A Case for Multiple-Cohort, Longitudinal Experiments
- Author
-
Blazar, David and Kraft, Matthew A.
- Abstract
Over the past 15 years, the education research community has advocated for rigorous research designs that support causal inferences, for research that provides more generalizable results across settings, and for the value of research-practice partnerships that inform the design of local programs and policies. We propose the multi-cohort, longitudinal experiment (MCLE) as one approach to balancing these three, sometimes competing goals in a single study. We describe our application of an MCLE design to evaluate a teacher coaching program, where we find that changes in program features related to personnel, content, and duration coincided with substantial differences in effectiveness across three cohorts of the experiment. Our analyses and corresponding recommendations can help researchers weigh the benefits and trade-offs of the MCLE design.
- Published
- 2019
22. Can Technology Transform Communication between Schools, Teachers, and Parents? Evidence from a Randomized Field Trial
- Author
-
Kraft, Matthew A. and Bolves, Alexander J.
- Abstract
We study the adoption and implementation of a new mobile communication application (app) among a sample of 132 New York City public schools. The app provides a platform for sharing general announcements and news, as well as engaging in personalized two-way communication with individual parents. We provide participating schools with free access to the app and randomize schools to receive intensive support (training, guidance, monitoring, and encouragement) for maximizing the efficacy of the app. Although user supports led to higher levels of communication within the app in the treatment year, overall usage remained low and declined in the following year when treatment schools no longer received intensive supports. We find few subsequent effects on perceptions of communication quality or student outcomes. We leverage rich internal user data to explore how take-up and usage patterns varied across staff and school characteristics. These analyses help to identify early adopters and reluctant users, revealing both opportunities and obstacles to engaging parents through new communication technology.
- Published
- 2022
- Full Text
- View/download PDF
23. Operator versus Partner: A Case Study of Blueprint School Network's Model for School Turnaround
- Author
-
Papay, John P., Kraft, Matthew A., and James, Jessalynn K.
- Abstract
Numerous high-profile efforts have sought to "turn around" low-performing schools. Evidence on the effectiveness of school turnarounds, however, is mixed, and research offers little guidance on which models are more likely to succeed. We present a mixed-methods case study of turnaround efforts led by the Blueprint Schools Network in three schools in Boston. Using a difference-in-differences framework, we find that Blueprint raised student achievement in ELA by at least a quarter of a standard deviation, with suggestive evidence of comparably large effects in math. We document qualitatively how differential impacts across the three Blueprint schools relate to contextual and implementation factors. In particular, Blueprint's role as a turnaround partner (in two schools) versus school operator (in one school) shaped its ability to implement its model. As a partner, Blueprint provided expertise and guidance but had limited ability to fully implement its model. In its role as an operator, Blueprint had full authority to implement its turnaround model but was also responsible for managing the day-to-day operations of the school, a role for which it had limited prior experience.
- Published
- 2022
- Full Text
- View/download PDF
24. The Inequitable Effects of Teacher Layoffs: What We Know and Can Do
- Author
-
Kraft, Matthew A. and Bleiberg, Joshua F.
- Abstract
Economic downturns can cause major funding shortfalls for U.S. public schools, often forcing districts to make difficult budget cuts, including teacher layoffs. In this brief, we synthesize the empirical literature on the widespread teacher layoffs caused by the Great Recession. Studies find that teacher layoffs harmed student achievement and were inequitably distributed across schools, teachers, and students. Research suggests that specific elements of the layoff process can exacerbate these negative effects. Seniority-based policies disproportionately concentrate layoffs among teachers of color, who are more likely to be early career teachers. These "last-in first-out" policies also disproportionately affect disadvantaged students because these students are more likely to be taught by early career teachers. The common practice of widely distributing pink slips warning about a potential job loss also appears to increase teacher churn and negatively impact teacher performance. Drawing on this evidence, we outline a set of policy recommendations to minimize the need for teacher layoffs during economic downturns and ensure that the burden of any unavoidable job cuts does not continue to be borne by students of color and students from low-income backgrounds.
- Published
- 2022
- Full Text
- View/download PDF
25. How informal mentoring by teachers, counselors, and coaches supports students' long-run academic success
- Author
-
Kraft, Matthew A., Bolves, Alexander J., and Hurd, Noelle M.
- Published
- 2023
- Full Text
- View/download PDF
26. Taking Teacher Evaluation to Scale: The Effect of State Reforms on Achievement and Attainment. Working Paper 30995
- Author
-
National Bureau of Economic Research (NBER), Bleiberg, Joshua, Brunner, Eric, Harbatkin, Erica, Kraft, Matthew A., and Springer, Matthew G.
- Abstract
Federal incentives and requirements under the Obama administration spurred states to adopt major reforms to their teacher evaluation systems. We examine the effects of these reforms on student achievement and attainment at a national scale by exploiting the staggered timing of implementation across states. We find precisely estimated null effects, on average, that rule out impacts as small as 0.015 standard deviation for achievement and 1 percentage point for high school graduation and college enrollment. We also find little evidence that the effect of teacher evaluation reforms varied by system design rigor, specific design features or student and district characteristics. We highlight five factors that may have undercut the efficacy of teacher evaluation reforms at scale: political opposition, the decentralized structure of U.S. public education, capacity constraints, limited generalizability, and the lack of increased teacher compensation to offset the non-pecuniary costs of lower job satisfaction and security.
- Published
- 2023
27. Teacher and Teaching Effects on Students' Attitudes and Behaviors
- Author
-
Blazar, David and Kraft, Matthew A.
- Abstract
Research has focused predominantly on how teachers affect students' achievement on standardized tests despite evidence that a broad range of attitudes and behaviors are equally important to their long-term success. We find that upper-elementary teachers have large effects on self-reported measures of students' self-efficacy in math, and happiness and behavior in class. Students' attitudes and behaviors are predicted by teaching practices most proximal to these measures, including teachers' emotional support and classroom organization. However, teachers who are effective at improving test scores often are not equally effective at improving students' attitudes and behaviors. These findings lend empirical evidence to well-established theory on the multidimensional nature of teaching and the need to identify strategies for improving the full range of teachers' skills.
- Published
- 2017
- Full Text
- View/download PDF
28. Preferences, Inequities, and Incentives in the Substitute Teacher Labor Market. Working Paper 30714
- Author
-
National Bureau of Economic Research (NBER), Kraft, Matthew A., Conklin, Megan Lane, and Falken, Grace T.
- Abstract
We examine the labor supply decisions of substitute teachers -- a large, on-demand market with broad shortages and inequitable supply. In 2018, Chicago Public Schools implemented a targeted bonus program designed to reduce unfilled teacher absences in largely segregated Black schools with historically low substitute coverage rates. Using a regression discontinuity design, we find that incentive pay substantially improved coverage equity and raised student achievement. Changes in labor supply were concentrated among Black and Hispanic substitutes from nearby neighborhoods with experience in incentive schools. Wage elasticity estimates suggest incentives would need to be 50% of daily wages to close fill-rate gaps.
- Published
- 2022
29. Sustaining a Sense of Success: The Protective Role of Teacher Working Conditions during the COVID-19 Pandemic
- Author
-
Kraft, Matthew A., Simon, Nicole S., and Lyon, Melissa Arnold
- Abstract
COVID-19 shuttered schools across the United States, upending traditional approaches to education. We examine teachers' experiences during emergency remote teaching in the spring of 2020 using responses to a working conditions survey from a sample of 7,841 teachers across 206 schools and 9 states. Teachers reported a range of challenges related to engaging students in remote learning and balancing their professional and personal responsibilities. Teachers in high-poverty and majority Black schools perceived these challenges to be the most severe, suggesting the pandemic further increased existing educational inequities. Using data from both pre-post and retrospective surveys, we find that the pandemic and pivot to emergency remote teaching resulted in a sudden, large drop in teachers' sense of success. We also demonstrate how supportive working conditions in schools played a critical role in helping teachers to sustain their sense of success. Teachers were less likely to experience declines in their sense of success when they worked in schools with strong communication, targeted training, meaningful collaboration, fair expectations, and authentic recognition during the pandemic.
- Published
- 2021
- Full Text
- View/download PDF
30. How Informal Mentoring by Teachers, Counselors, and Coaches Supports Students’ Long-Run Academic Success
- Author
-
Kraft, Matthew, primary, Bolves, Alexander, additional, and Hurd, Noelle, additional
- Published
- 2023
- Full Text
- View/download PDF
31. Taking Teacher Evaluation to Scale: The Effect of State Reforms on Achievement and Attainment
- Author
-
Bleiberg, Joshua, primary, Brunner, Eric, additional, Harbatkin, Erica, additional, Kraft, Matthew, additional, and Springer, Matthew, additional
- Published
- 2023
- Full Text
- View/download PDF
32. Developing Ambitious Mathematics Instruction through Web-Based Coaching: A Randomized Field Trial
- Author
-
Kraft, Matthew A. and Hill, Heather C.
- Abstract
This article describes and evaluates a web-based coaching program designed to support teachers in implementing Common Core-aligned math instruction. Web-based coaching programs can be operated at relatively lower costs, are scalable, and make it more feasible to pair teachers with coaches who have expertise in their content area and grade level. Results from our randomized field trial document sizable and sustained effects on both teachers' ability to analyze instruction and on their instructional practice, as measured by the Mathematical Quality of Instruction instrument and student surveys. However, these improvements in instruction did not result in corresponding increases in math test scores as measured by state standardized tests or interim assessments. We discuss several possible explanations for this pattern of results.
- Published
- 2020
- Full Text
- View/download PDF
33. Time in School: A Conceptual Framework, Synthesis of the Causal Research, and Empirical Exploration.
- Author
-
Kraft, Matthew A. and Novicoff, Sarah
- Subjects
SCHOOL day ,EDUCATIONAL equalization ,LEARNING ,ACADEMIC achievement ,PUBLIC schools - Abstract
We examine the fundamental and complex role that time plays in the learning process. We begin by developing a conceptual framework to elucidate the multiple obstacles schools face in converting total time in school into active learning time. We then synthesize the causal research and document a clear positive effect of additional time on student achievement typically of small to medium magnitude depending on dosage, use, and context. Further descriptive analyses reveal how large differences in the length of the school day and year across public schools are an underappreciated dimension of educational inequality in the United States. Finally, our case study of time loss in one urban district demonstrates the potential to substantially increase instructional time within existing constraints. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
34. Can Principals Promote Teacher Development as Evaluators?
- Author
-
Society for Research on Educational Effectiveness (SREE), Kraft, Matthew A., and Gilmour, Allison
- Abstract
District- and state-level efforts to remake teacher evaluation systems are among the most substantial and widely adopted reforms that U.S. public schools have experienced in decades. Research on these next generation of evaluation systems has focused overwhelmingly on policy goals, program designs, and performance measures. However, still very little is known about how these policies are interpreted and enacted by school leaders. In this study, the authors examine the perspectives and experiences of principals as evaluators in a large urban school district in the northeastern United States that recently implemented sweeping reforms to its teacher evaluation system. The study focuses on principals' perspectives and experiences with classroom observation and feedback because this process is a primary mechanism through which evaluation is intended to promote teacher development. Principals' abilities to rate teachers accurately, to facilitate teachers' own self-reflection, to make specific actionable recommendations, and to communicate this feedback effectively are central to any evaluation process intended to improve instruction. The district studied is an urban district in the northeast that serves a racially and linguistically diverse student population. Interviews were conducted with principals lasting 45 to 60 minutes in July and August of 2013, the summer after the first year the new evaluation system was implemented district-wide. It was found that the quality of feedback teachers receive through the evaluation process depends critically on the time and training evaluators have to provide individualized and actionable feedback. Districts that task principals with primary responsibility for conducting observation and feedback cycles must attend to the many implementation challenges associated with this approach in order for next-generation evaluation systems to successfully promote teacher development. A table is appended. [SREE documents are structured abstracts of SREE conference symposium, panel, and paper or poster submissions.]
- Published
- 2016
35. Interpreting Effect Sizes of Education Interventions
- Author
-
Kraft, Matthew A.
- Abstract
Researchers commonly interpret effect sizes by applying benchmarks proposed by Jacob Cohen over a half century ago. However, effects that are small by Cohen's standards are large relative to the impacts of most field-based interventions. These benchmarks also fail to consider important differences in study features, program costs, and scalability. In this article, I present five broad guidelines for interpreting effect sizes that are applicable across the social sciences. I then propose a more structured schema with new empirical benchmarks for interpreting a specific class of studies: causal research on education interventions with standardized achievement outcomes. Together, these tools provide a practical approach for incorporating study features, costs, and scalability into the process of interpreting the policy importance of effect sizes.
- Published
- 2020
- Full Text
- View/download PDF
36. Teacher Skill Development: Evidence from Performance Ratings by Principals
- Author
-
Kraft, Matthew A., Papay, John P., and Chi, Olivia L.
- Abstract
We examine the dynamic nature of teacher skill development using panel data on principals' subjective performance ratings of teachers. Past research on teacher productivity improvement has focused primarily on one important but narrow measure of performance: teachers' value-added to student achievement on standardized tests. Unlike value-added, subjective performance ratings provide detailed information about specific skill dimensions and are available for teachers in non-tested grades and subjects. Using a within-teacher returns-to-experience framework, we find, on average, large and rapid improvements in teachers' instructional practices throughout their first 10 years on the job as well as substantial differences in improvement rates across individual teachers. We also document that subjective performance ratings contain important information about teacher effectiveness. In the district we study, principals appear to differentiate teacher performance throughout the full distribution instead of just in the tails. Furthermore, prior performance ratings and gains in these ratings provide additional information about teachers' ability to improve test scores that is not captured by prior value-added scores. Taken together, our study provides new insights on teacher performance improvement and variation in teacher development across instructional skills and individual teachers.
- Published
- 2020
- Full Text
- View/download PDF
37. Preferences, Inequities, and Incentives in the Substitute Teacher Labor Market
- Author
-
Kraft, Matthew, primary, Conklin, Megan Lane, additional, and Falken, Grace, additional
- Published
- 2022
- Full Text
- View/download PDF
38. The Rise and Fall of the Teaching Profession: Prestige, Interest, Preparation, and Satisfaction over the Last Half Century.
- Author
-
Kraft, Matthew A. and Lyon, Melissa Arnold
- Published
- 2024
39. The Rise and Fall of the Teaching Profession: Prestige, Interest, Preparation, and Satisfaction Over the Last Half Century
- Author
-
Kraft, Matthew A., primary and Lyon, Melissa Arnold, additional
- Published
- 2024
- Full Text
- View/download PDF
40. School Organizational Contexts, Teacher Turnover, and Student Achievement: Evidence from Panel Data
- Author
-
Society for Research on Educational Effectiveness (SREE), Kraft, Matthew A., Marinell, William H., and Yee, Darrick
- Abstract
In education, there is ample evidence that some schools far outperform others at raising student achievement even when accounting for differences in the students they serve and the resources at their disposal. Differences in the human capital stock of teachers across schools cannot fully account for the differential productivity across schools. In teaching, as in any occupation where professionals perform their work in organizational contexts, productivity is influenced by both individual and organizational factors (Hackman & Oldman, 1980; Kanter, 1983; Johnson, 1990). A growing body of literature attempts to identify, measure, and evaluate the potential contribution of organizational practices and contexts to overall productivity in schools. This study is among the first to address the empirical limitations of prior studies on organizational contexts by leveraging one of the largest survey administration efforts ever conducted in the United States outside of the decennial population census. While it seems intuitively obvious that an individual's performance is affected by the environment in which he or she works, policymakers have largely overlooked the central role that schools play in influencing teachers' in their push to overhaul teacher evaluation systems. To systematically improve student performance, school and district leaders need robust evidence about the strengths and weaknesses of both individual teachers and the school organization as a whole. Equipped with this data, policymakers and practitioners can take steps to address individual as well as organizational strengths and deficiencies. Tables are appended.
- Published
- 2015
41. Teacher-to-Parent Communication: Experimental Evidence from a Low-Cost Communication Policy
- Author
-
Society for Research on Educational Effectiveness (SREE), Kraft, Matthew A., and Rogers, Todd
- Abstract
A wide body of literature documents the important role that parents play in supporting children's academic success in school (Houtenville & Conway, 2008; Barnard, 2004; Fan & Chen, 2001). Drawing on this literature, national taskforces and federal legislation consistently identify increased parental involvement as a central goal of educational reform initiatives (e.g. No Child Left Behind, Title I, Part A, Section 1118). Schools attempt to promote greater parental engagement though a variety of efforts centered on teacher-parent communication (Epstein, 2008). Cheung and Pomerantz (2012) found that children whose parents were more likely to be involved with their learning were more likely to be motivated to meet their parents' academic expectations, and received higher grades. Recent experimental research has documented how two-way teacher-parent communication can lead to greater parental involvement, improved student engagement and academic achievement (Authors, 2013; Bergman, 2012). This study examines the effect of delivering to parents weekly messages written by teachers about each child's performance in school, and the authors explore how these effects differ across different message types. This is accomplished by conducting a field experiment during a summer credit recovery program in a large urban school district. Researchers randomly assigned participating students and their parents to one of three experimental conditions. Some parents received information throughout the summer program about what their students were doing well and should continue doing; others received information about what their students needed to improve upon, while a third group received no information. The research sought to answer two questions: (1) What is the effect of teacher-to-parent communication on the probability a student earns course credit in a credit recovery program; and 2) Are positive or needs-improvement messages more effective at increasing a student's likelihood of earning course credit? This research contributes to a growing body of evidence on the beneficial impact that teachers providing parents with individualized messages and information about their children's schoolwork can have on student achievement and advancement in school. This study also points to the importance of further examining how teachers and schools can improve the content and quality of their communication with parents. Tables are appended.
- Published
- 2014
42. Teacher accountability reforms and the supply and quality of new teachers
- Author
-
Kraft, Matthew A., Brunner, Eric J., Dougherty, Shaun M., and Schwegman, David J.
- Published
- 2020
- Full Text
- View/download PDF
43. Improving Teacher Practice: Experimental Evidence on Individualized Teacher Coaching
- Author
-
Society for Research on Educational Effectiveness (SREE), Kraft, Matthew A., and Blazar, David L.
- Abstract
For over a century, school systems in the U.S. have attempted to improve instructional quality by investing in the education and training of their teachers. Today, over 90% of teachers report participating in some form of professional development (PD). Practitioners have responded to critiques of PD by re-envisioning it in the form of individualized and sustained teacher coaching programs that provide tailored feedback to teachers about their classroom practices. In this paper, the authors evaluate the effect of a time-intensive, individualized coaching program MATCH Teacher Coaching (MTC) that focuses on improving teachers' classroom management and instructional practices, both of which are cited as important mechanisms behind improvements in student outcomes. In May of 2011, the authors recruited 59 teachers, working in Recovery School District charter schools across New Orleans, to participate in a randomized trial of the year-long coaching program. Study findings reported that that MTC was implemented with a high degree of fidelity and had a profound effect on teachers' instructional practices compared to control teachers who had access to standard professional development opportunities. Teachers' emails and coaches' logs were analyzed to assess the content and methods used during coaching sessions. Results indicate that teachers who received coaching are more effective than those who participated in the standard PD activities provided by their schools. Building the necessary evidentiary base and shifting longstanding norms about PD practices will take time, but the evidence-to-date suggests doing so will be a valuable investment. Tables and figures are appended.
- Published
- 2013
44. Missed Opportunities in the Labor Market or Temporary Disruptions? How Late Teacher Hiring Affects Student Achievement
- Author
-
Society for Research on Educational Effectiveness (SREE), Papay, John P., Kraft, Matthew A., and Bloom, Julia
- Abstract
While there are many reasons why late hiring may affect student achievement, no empirical studies have documented this effect in practice. This paper presents the first estimates of the direct impact of late hiring on students' academic achievement, sheds light on two competing explanations for the struggles of late-hire teachers advanced in the literature (labor market effects and disruption effects), and examines broader consequences of late hiring on student achievement, including spillover effects that occur from increased teacher turnover. Researchers address the following questions: (1) Do the observable characteristics of teachers who are hired late and the schools that hire them differ from on-time hires?; (2) Does late hiring reduce student achievement? If so, are labor market effects or disruption effects to blame?; and (3) Are teachers who are hired late more likely to switch schools or leave the school district? The study reveals the following findings: (1) Late-hired teachers differ from their peers in the district in several ways--gender, ethnicity, age, professional route to teaching, and a tendency to work in different types of schools; (2) A classroom with a teacher hired after school starts in the fall reduces student achievement; and (3) Late-hired teachers are much less likely to stay in the district than standard-hired teachers, and those who remain are more likely to transfer schools. Tables and figures are appended.
- Published
- 2013
45. Teacher Layoffs, Teacher Quality and Student Achievement: The Implementation and Consequences of a Discretionary Reduction-in-Force Policy
- Author
-
Society for Research on Educational Effectiveness (SREE) and Kraft, Matthew A.
- Abstract
Research has shown that "last hired, first fired" policies maximize the number of teachers subject to reductions in force by eliminating those teachers that are lowest on the pay scale first. Until now, advocates of effectiveness-based reduction-in-force (RIF) policies could only point to simulated policy exercises as evidence of the potential benefits of a discretionary reduction-in-force policy. This study suggests that, while reductions in force negatively affect student achievement, districts have the potential to reduce these negative effects by concentrating layoffs among the lowest-performing teachers. The analyses of this study focuses on three main objectives: (1) comparing the relative weight that North Carolina's Charlotte-Mecklenburg Schools' (CMS) administrators and principals placed on a variety of RIF criteria when implementing layoffs; (2) estimating the causal effect of discretionary layoffs in CMS on student achievement in the following academic year; and (3) documenting how these negative effects are exacerbated when highly effective teachers are laid-off teachers. Results from descriptive regression analyses suggest that CMS used multiple RIF criteria when selecting teachers for layoffs including tenure status, licensure status and type, and job performance. Although replacing seniority with performance measures can minimize effects on student achievement, exchanging one inflexible criterion for another will not provide districts with any discretion in navigating a complex process aimed at preventing a variety of negative consequences. Tables are appended.
- Published
- 2013
46. The Effect of Teacher Coaching on Instruction and Achievement: A Meta-Analysis of the Causal Evidence
- Author
-
Kraft, Matthew A., Blazar, David, and Hogan, Dylan
- Abstract
Teacher coaching has emerged as a promising alternative to traditional models of professional development. We review the empirical literature on teacher coaching and conduct meta-analyses to estimate the mean effect of coaching programs on teachers' instructional practice and students' academic achievement. Combining results across 60 studies that employ causal research designs, we find pooled effect sizes of 0.49 standard deviations (SD) on instruction and 0.18 SD on achievement. Much of this evidence comes from literacy coaching programs for prekindergarten and elementary school teachers in the United States. Although these findings affirm the potential of coaching as a development tool, further analyses illustrate the challenges of taking coaching programs to scale while maintaining effectiveness. Average effects from effectiveness trials of larger programs are only a fraction of the effects found in efficacy trials of smaller programs. We conclude by discussing ways to address scale-up implementation challenges and providing guidance for future causal studies.
- Published
- 2018
- Full Text
- View/download PDF
47. Taking Teacher Coaching to Scale: Can Personalized Training Become Standard Practice?
- Author
-
Kraft, Matthew A. and Blazar, David
- Abstract
Traditional efforts to increase teacher quality through professional development (PD) have been largely ineffective. A growing body of research on teacher coaching provides strong evidence of its effectiveness as a PD tool, although it also raises difficult questions about whether and how to implement coaching programs at scale. Key considerations for scaling up include: (1) Finding expert coaches able to deliver PD services; (2) Dealing with financial constraints due to the large personnel costs of hiring coaches to meet with teachers on a regular basis; (3) Thinking critically about how to implement organizational structures and systems that provide scaffolded supports to individual coaches without restricting their judgment and flexibility; and (4) Creating a culture of trust andrespect among administrators and staff in order to ease teachers' concerns and increase their willingness to actively engage.
- Published
- 2018
48. Individualized Coaching to Improve Teacher Practice across Grades and Subjects: New Experimental Evidence
- Author
-
Kraft, Matthew A. and Blazar, David
- Abstract
This article analyzes a coaching model focused on classroom management skills and instructional practices across grade levels and subject areas. We describe the design and implementation of MATCH Teacher Coaching among an initial cohort of 59 teachers working in New Orleans charter schools. We evaluate the effect of the program on teachers' instructional practices using a block randomized trial and find that coached teachers scored 0.59 standard deviations higher on an index of effective teaching practices comprised of observation scores, principal evaluations, and student surveys. We discuss implementation challenges and make recommendations for researcher--practitioner partnerships to address key remaining questions.
- Published
- 2017
- Full Text
- View/download PDF
49. The Sensitivity of Teacher Performance Ratings to the Design of Teacher Evaluation Systems
- Author
-
Steinberg, Matthew P. and Kraft, Matthew A.
- Abstract
In recent years, states and districts have responded to federal incentives and pressure to institute major reforms to their teacher evaluation systems. The passage of the Every Student Succeeds Act in 2015 now provides state policymakers with even greater autonomy to redesign existing evaluation systems. Yet, little evidence exists to inform decisions about two key system design features: teacher performance measure weights and performance ratings thresholds. Using data from the Measures of Effective Teaching study, we conduct simulation-based analyses that illustrate the critical role that performance measure weights and ratings thresholds play in determining teachers' summative evaluation ratings and the distribution of teacher proficiency rates. These findings offer insights to policymakers and administrators as they refine and possibly remake teacher evaluation systems.
- Published
- 2017
- Full Text
- View/download PDF
50. Engaging Parents through Better Communication Systems
- Author
-
Kraft, Matthew A.
- Abstract
Matthew A. Kraft, an assistant professor of education and economics at Brown University, highlights new research showing that frequent, personalized outreach to parents can boost parent engagement and student achievement. He offers tips on how schools can create infrastructures, including digital technology tools, to better support such communication.
- Published
- 2017
Catalog
Discovery Service for Jio Institute Digital Library
For full access to our library's resources, please sign in.