Search

Anecdotal evidence points to the importance of school principals, but the limited existing research has neither provided consistent results nor indicated any set of essential characteristics of effective principals. This paper exploits extensive student-level panel data across six states to investigate both variations in principal performance and the relationship between effectiveness and key certification factors. While principal effectiveness varies widely across states, there is little indication that regulation of the background and training of principals yields consistently effective performance. Having prior teaching or management experience is not related to our estimates of principal value-added.

We use administrative panel data from seven states covering nearly 3 million students to document and explore variation in "academic mobility," a term we use to describe the extent to which students' ranks in the distribution of academic performance change during their public schooling careers. On average, we show that student ranks are highly persistent during elementary and secondary education--that is, academic mobility is limited in U.S. schools as a whole. Still, there is non-negligible variation in the degree of upward mobility across some student subgroups as well as individual school districts. On average, districts that exhibit the greatest upward academic mobility serve more socioeconomically advantaged populations and have higher value-added to student achievement.

Measures of student disadvantage--or risk--are critical components of equity-focused education policies. However, the risk measures used in contemporary policies have significant limitations, and despite continued advances in data infrastructure and analytic capacity, there has been little innovation in these measures for decades. We develop a new measure of student risk for use in education policies, which we call Predicted Academic Performance (PAP). PAP is a flexible, data-rich indicator that identifies students at risk of poor academic outcomes. It blends concepts from emerging "early warning" systems with principles of incentive design to balance the competing priorities of accurate risk measurement and suitability for policy use. PAP is more effective than common alternatives at identifying students who are at risk of poor academic outcomes and can be used to target resources toward these students--and students who belong to several other associated risk categories--more efficiently.

We develop a new framework for identifying at-risk students in public schools. Our framework has two fundamental advantages over "status quo" systems: (1) it is based on a clear definition of what it means for a student to be at risk and (2) it leverages states' rich administrative data systems to produce more informative risk measures. Our framework is more effective than common alternatives at identifying students who are at risk of low academic performance and we use policy simulations to show that it can be used to target resources toward these students more efficiently. It also offers several other benefits relative to "status quo" systems. We provide an alternative approach to risk measurement that states can use to inform funding, accountability, and other policies, rather than continuing to rely on broad categories tied to the nebulous concept of "disadvantage."

There is empirical evidence of substantial heterogeneity in economic mobility across geographic areas and the efficacy of schools has been suggested as an explanatory factor. Using administrative microdata from seven states covering nearly 3 million students, we explore the potential role of schools in promoting economic mobility by estimating cross-district variation in "academic mobility"--a term we use to describe the extent to which students' ranks in the distribution of academic performance change during their schooling careers. We show that there exists considerable heterogeneity in academic mobility across school districts. However, after aggregating our district-level measures of academic mobility to the commuting-zone level and merging them with geographically matched external estimates of economic mobility, we find little scope for geographic differences in academic mobility to meaningfully account for differences in economic mobility.

Free and reduced-price meal eligibility (FRM) is commonly used in education research and policy applications as an indicator of student poverty. However, using multiple data sources external to the school system, we show that FRM status is a poor proxy for poverty, with eligibility rates far exceeding what would be expected based on stated income thresholds for program participation. This is true even without accounting for community eligibility for free meals, although community eligibility has exacerbated the problem in recent years. Over the course of showing the limitations of using FRM data to measure poverty, we provide promising validity evidence for a new, publicly-available measure of school poverty based on local-area family incomes.

We evaluate the feasibility of estimating test-score growth with a gap year in testing data, informing the scenario when state testing resumes after the 2020 COVID-19-induced test stoppage. Our research design is to simulate a gap year in testing using pre-COVID-19 data--when a true test gap did not occur--which facilitates comparisons of district- and school-level growth measures that are estimated with and without a gap year. We find that growth estimates based on the full data and gap-year data are generally similar, and our results highlight an advantage of using comprehensive growth models with rich controls for student and schooling circumstances. With the caveat that there is looming uncertainty about which students will be tested in public schools when testing resumes, and how they will be tested, our findings establish the potential for estimating useful growth measures with a gap year in testing.

We use comprehensive administrative data from three states to document the relationships between geographic mobility and student outcomes during K-12 schooling. We focus specifically on "nonstructural" mobility events--which we define as school or district changes that do not occur naturally as the result of planned transitions between schools--and on longitudinal measures that capture these events cumulatively for students. We show that the number of nonstructural moves experienced by a student is a powerful indicator of low academic performance and graduation rates. Longitudinal information on student mobility is unlikely to be readily available to local practitioners--i.e., individual districts, schools, or teachers. However, due to recent investments in longitudinal data systems in most states, this information can be made available at low cost by state education agencies.

When the COVID-19 pandemic hit the U.S. last spring, schools nationwide shut their doors and states cancelled annual standardized tests. Now federal and state policymakers are debating whether to cancel testing again in 2021. One factor they should consider is whether a two-year gap in testing will make it impossible to measure student-level achievement growth during this historic period. Using administrative data from Missouri from the pre-pandemic years of 2016 to 2019, a team of researchers in the department of economics at the University of Missouri--Ishtiaque Fazlul, Cory Koedel, Eric Parsons, and Cheng Qian--evaluated the efficacy and implications of growth models that reflected one-year versus two-year gaps in testing. Among their key findings: (1) District and school growth estimates based on a single-year gap convey similar information to growth estimates based on data with no gap year; (2) Student subgroup growth estimates based on gap-year data can also be reliably estimated; and (3) However, just 27 percent of students attend schools that can generate growth measures if two consecutive years of test scores are missing. These findings suggest that calculating student growth measures from 2019 to 2021 is feasible even with the gap in testing last year, and that the results are likely to be quite accurate--so long as states test students this year. But if states cancel testing again in 2021, it will be extremely difficult to determine how effective individual schools were during this challenging period in American education. That's because few students will be in the same school in 2022 as they were in 2019 (elementary schoolers will have moved onto middle school, and middle schoolers will have moved onto high school). Still, the next few months are hard to predict. If spring testing is still not possible for some states in 2021, they might consider administering assessments when schools reopen in fall 2021, although such a scenario would pose its own logistical, analytic, and political challenges. [Foreword written by Michael J. Petrilli.]

The Community Eligibility Provision (CEP) is a policy change to the federally-administered National School Lunch Program that allows schools serving low-income populations to classify all students as eligible for free meals, regardless of individual circumstances. This has implications for the use of free and reduced-price meal (FRM) data to proxy for student disadvantage in education research and policy applications, which is a common practice. We document empirically how the CEP has affected the value of FRM eligibility as a proxy for student disadvantage. At the individual student level, we show that there is essentially no effect of the CEP. However, the CEP does meaningfully change the information conveyed by the share of FRM-eligible students in a school. It is this latter measure that is most relevant for policy uses of FRM data. [Portions of this paper were previously circulated under the title "Using Free Meal and Direct Certification Data to Proxy for Student Disadvantage in the Era of the Community Eligibility Provision" (ED595218). The authors have since split the original paper into two parts. This is the first part.]

Free and reduced-price meal (FRM) data are used ubiquitously to proxy for student disadvantage in education research and policy applications. The Community Eligibility Provision (CEP)--a recently-implemented policy change to the federally-administered National School Lunch Program--allows schools serving low-income populations to identify all students as FRM-eligible regardless of individual circumstances. We study the CEP's effect on FRM eligibility as a proxy for student disadvantage, and relatedly, we examine the viability of direct certification (DC) status as an alternative disadvantage measure. Our findings on whether the CEP degrades the informational content of FRM data are mixed. At the individual level there is essentially no effect, but the CEP does meaningfully change the information conveyed by the FRM-eligible share of students in a school. Our comparison of FRM and DC data in the post-CEP era shows that these measures are similarly informative as proxies for disadvantage, despite the CEP-induced information loss in FRM data. Using both measures together can improve the identification of disadvantaged students, but only marginally.

An increasing emphasis on principals as key to school improvement has contributed to efforts to elevate principal effectiveness that have taken various forms across the US. The primacy of the state as the focal point of educational reform elevates the value of understanding commonalities and differences among states in characteristics of principals, the distribution of principals among schools and ultimately the policies associated with more effective school leadership, particularly for disadvantaged children. This paper describes major state policies, the distribution of elementary school principals among schools along a several dimensions, and pathways to the principalship to illustrate similarities and differences among six states in the tenure and experience distributions and how these vary by student demographic characteristics and district size. Measurement of principal effectiveness and its relationship with principal characteristics and state policies would be ideal, but complications introduced by the dynamics of principal influences and confounding effects of other factors inhibit this effort. Nonetheless, school value added to achievement provides information on differences in principal effectiveness, and we report within-school variation value added across principal regimes and the associations between value added and principal characteristics. The analysis reveals many similarities and some differences among the states, some of which are related to differences in governance structures. Perhaps the most striking differences relate to the pathways to the principalship including the fraction of principals with experiences as assistant principals and teachers.

Improving public sector workforce quality is challenging in sectors such as education where worker productivity is difficult to assess and manager incentives are muted by political and bureaucratic constraints. In this paper, we study how providing improved information to principals about teacher effectiveness and encouraging them to use the information in personnel decisions affects the composition of teacher turnovers. Our setting is the Houston Independent School District, which recently implemented a rigorous teacher evaluation system. Prior to the new system, teacher effectiveness was negatively correlated with district exit and we show that the policy significantly strengthened this relationship, primarily by increasing the relative likelihood of exit for teachers in the bottom quintile of the quality distribution. Low-performing teachers working in low-achieving schools were especially likely to leave. However, despite the success, the implied change to the quality of the workforce overall is too small to have a detectable impact on student achievement.

Improving public sector workforce quality is challenging in sectors such as education where worker productivity is difficult to assess and manager incentives are muted by political and bureaucratic constraints. In this paper, we study how providing information to principals about teacher effectiveness and encouraging them to use the information in personnel decisions affects the composition of teacher turnovers. Our setting is the Houston Independent School District, which recently implemented a rigorous teacher evaluation system. Prior to the new system teacher effectiveness was negatively correlated with district exit and we show that the policy significantly strengthened this relationship, primarily by increasing the relative likelihood of exit for teachers in the bottom quintile of the quality distribution. Low-performing teachers working in low-achieving schools were especially likely to leave. However, despite the success, the implied change to the quality of the workforce overall is too small to have a detectable impact on student achievement. Supplementary tables are appended.

There is increased policy interest in extending test-based evaluations in K-12 education to include student achievement in high school. High school achievement is typically measured by performance on end-of-course exams (EOCs), which test course-specific standards in a variety of subjects. However, unlike standardized tests in the early grades, students take EOCs at different points in their schooling careers. The timing of the test is a choice variable presumably determined by input from administrators, students and parents. Recent research indicates that school and district policies that determine when students take particular courses can have important consequences for achievement and subsequent outcomes like advanced course taking. We develop an approach for modeling EOC test performance that disentangles the influence of school and district policies regarding the timing of course taking from other factors. After separating out the timing issue, better measures of the quality of instruction provided by districts, schools and teachers can be obtained. Our approach also offers diagnostic value because it separates out the influence of school and district course-timing policies from other factors that determine student achievement.

The specifics of how growth models should be constructed and used to evaluate schools and teachers is a topic of lively policy debate in states and school districts nationwide. In this paper we take up the question of model choice and examine three competing approaches. The first approach, reflected in the popular student growth percentiles (SGPs) framework, eschews all controls for student covariates and schooling environments. The second approach, typically associated with value-added models (VAMs), controls for student background characteristics and aims to identify the causal effects of schools and teachers. The third approach, also VAM-based, fully levels the playing field so that the correlation between school- and teacher-level growth measures and student demographics is essentially zero. We argue that the third approach is the most desirable for use in educational evaluation systems. Our case rests on personnel economics, incentive-design theory, and the potential role that growth measures can play in improving instruction in K-12 schools. Data Description is appended. (Contains 4 tables, 3 figures and 23 footnotes.)

We compare teacher preparation programs in Missouri based on the effectiveness of their graduates in the classroom. The differences in effectiveness between teachers from different preparation programs are very small. In fact, virtually all of the variation in teacher effectiveness comes from within-program differences between teachers. Prior research has overstated differences in teacher performance across preparation programs for several reasons, most notably because some sampling variability in the data has been incorrectly attributed to the preparation programs. (A supplementary table is appended.)