Search

Anecdotal evidence points to the importance of school principals, but the limited existing research has neither provided consistent results nor indicated any set of essential characteristics of effective principals. This paper exploits extensive student-level panel data across six states to investigate both variations in principal performance and the relationship between effectiveness and key certification factors. While principal effectiveness varies widely across states, there is little indication that regulation of the background and training of principals yields consistently effective performance. Having prior teaching or management experience is not related to our estimates of principal value-added.

We use administrative panel data from seven states covering nearly 3 million students to document and explore variation in "academic mobility," a term we use to describe the extent to which students' ranks in the distribution of academic performance change during their public schooling careers. On average, we show that student ranks are highly persistent during elementary and secondary education--that is, academic mobility is limited in U.S. schools as a whole. Still, there is non-negligible variation in the degree of upward mobility across some student subgroups as well as individual school districts. On average, districts that exhibit the greatest upward academic mobility serve more socioeconomically advantaged populations and have higher value-added to student achievement.

The COVID-19 pandemic initially resulted in an unanticipated and near-universal shift from in-person to virtual instruction in spring 2020. During the 2020-21 school year, schools began to re-open and families were faced with decisions regarding the instructional mode for their children. We leverage administrative, survey, and virtual-learning data to examine the determinants of family learning-mode choice and the effects of virtual education on student engagement and academic achievement. Family preference for virtual (versus face-to-face) instruction was most highly associated with school-level infection rates and appeared relatively uniform within schools. We find that students who were assigned a higher proportion of instructional days in virtual mode experienced higher rates of attendance, but also negative student achievement growth compared to students who were assigned a higher proportion of instructional days in face-to-face mode. Students belonging to marginalized groups experienced more positive associations with attendance but were also more likely to experience lower student achievement growth when assigned a greater proportion of instructional days in virtual mode. Insights from this study can be used to better understand family preference as well as to target and refine virtual learning in a post-COVID-19 society.

There is empirical evidence of substantial heterogeneity in economic mobility across geographic areas and the efficacy of schools has been suggested as an explanatory factor. Using administrative microdata from seven states covering nearly 3 million students, we explore the potential role of schools in promoting economic mobility by estimating cross-district variation in "academic mobility"--a term we use to describe the extent to which students' ranks in the distribution of academic performance change during their schooling careers. We show that there exists considerable heterogeneity in academic mobility across school districts. However, after aggregating our district-level measures of academic mobility to the commuting-zone level and merging them with geographically matched external estimates of economic mobility, we find little scope for geographic differences in academic mobility to meaningfully account for differences in economic mobility.

Teacher turnover has adverse consequences for student achievement and imposes large financial costs for schools. Some have argued that high-stakes testing may lower teachers' satisfaction with their jobs and could be a major contributor to teacher attrition. In this paper, we exploit changes in the tested grades and subjects in Georgia to study the effects of eliminating high-stakes testing on teacher turnover and the distribution of teachers across grades and schools. To measure the effect of testing pressures on teacher mobility choices we use a "difference-in-differences" approach, comparing changes in mobility over time in grades/subjects that discontinue testing vis-à-vis grades/subjects that are always tested. Our results show that eliminating testing did not have an impact on the likelihood of leaving teaching, changing schools within a district, or moving between districts. We only uncover small negative effects on the likelihood of grade switching. However, we do find relevant positive effects on retention of beginning teachers in the profession. In particular, the average probability of exit for teachers with 0-4 years of experience fell from 14 to 13 percentage points for teachers in grades 1 and 2 and from 14 to 11 percentage points in grades 6 and 7.

This report analyzes the effects of implementing a personalization for academic and social emotional learning (PASL) program in selected Broward County high schools. PASL was implemented with a continuous improvement model promoting deliberate adaptations to class- and school-level conditions. PASL practices include goal setting, use of data, educator teams, and points of contact between teachers and students. The implementation of PASL started at three high schools in 2014-2015 (the "innovation year") and expanded to an additional five high schools in the 2015-2016 "scale-out year." Because of the nature of the adaptive continuous improvement, there were many differences in implementation and treatment practices across schools. As a result of these differences, the authors analyze program effects both collectively and on a school-by-school basis. Overall, the effects of PASL were quite varied across schools and outcomes. Findings revealed the most consistent evidence of an impact of PASL is a reduction of major disciplinary incidents in the innovation schools, although there was still a good deal of variation. Additionally, the schools with the most consistent evidence of a positive impact of PASL had the highest levels of implementation integrity.

Traditionally, teacher salaries have been determined solely by experience and educational attainment. This has led to chronic shortages of teachers in particular subject areas, such as math, science and special education. We study the first long-running statewide program to differentiate teacher pay based on subject area, Georgia's bonus system for math and science teachers. Using a difference-in-differences strategy, we find the bonuses reduce teacher attrition by 18 to 28 percent. However, we find no evidence the program increases the probability that education majors become secondary math or science teachers upon graduation or alters specific major choices within the education field.

An increasing emphasis on principals as key to school improvement has contributed to efforts to elevate principal effectiveness that have taken various forms across the US. The primacy of the state as the focal point of educational reform elevates the value of understanding commonalities and differences among states in characteristics of principals, the distribution of principals among schools and ultimately the policies associated with more effective school leadership, particularly for disadvantaged children. This paper describes major state policies, the distribution of elementary school principals among schools along a several dimensions, and pathways to the principalship to illustrate similarities and differences among six states in the tenure and experience distributions and how these vary by student demographic characteristics and district size. Measurement of principal effectiveness and its relationship with principal characteristics and state policies would be ideal, but complications introduced by the dynamics of principal influences and confounding effects of other factors inhibit this effort. Nonetheless, school value added to achievement provides information on differences in principal effectiveness, and we report within-school variation value added across principal regimes and the associations between value added and principal characteristics. The analysis reveals many similarities and some differences among the states, some of which are related to differences in governance structures. Perhaps the most striking differences relate to the pathways to the principalship including the fraction of principals with experiences as assistant principals and teachers.

We investigated the effects of a statewide program designed to increase the supply of teachers in "hard-to-staff" areas. The Florida Critical Teacher Shortage Program (FCTSP) had three elements: (a) it provided loan forgiveness to teachers who were certified and taught in designated shortage areas; (b) it compensated teachers for the tuition cost of taking courses to become certified in a designated shortage area; and (c) for a single year, it gave bonuses to high school teachers who were certified and taught in a designated subject area. Employing a difference-in-difference estimator, we find that the loan forgiveness program decreased attrition of teachers in shortage areas, although the effects varied by subject. Allowing for variation in the size of payments, we find that the effects were more pronounced when loan-forgiveness payments were more generous. A triple-difference estimate indicated the bonus program also substantially reduced the likelihood of teachers leaving the public school sector. A panel probit analysis reveals that the tuition-reimbursement program had modest positive effects on the likelihood a teacher would become certified in a designated shortage area. We also present qualitative evidence that loan-forgiveness recipients were of higher quality (as measured by value added) than nonrecipients who taught in the same subject but were not certified and thus ineligible.

Staffing problems are pervasive in certain subject areas, such as secondary math and science and special education, where the combination of training requirements and relatively high alternative wages makes it difficult to attract and retain high-quality teachers. This project evaluated the impacts of the Florida Critical Teacher Shortage Program (FCTSP) and Teacher Recruitment and Retention Fund (TRRF) on the supply of new teachers and the retention of teachers in high-need areas such as special education, math, and science. The research: (1) addressed specific research questions pertaining to three programs (loan forgiveness, tuition reimbursement, and recruitment/retention bonuses); (2) investigated the causal effects of the programs; and (3) addressed questions related to the general characteristics of the program and participating teachers. The authors' initial descriptive analysis indicated that FCTSP teachers tend to be of lower quality (as measured by value-added) than non-participants. They also tend to be less experienced and teach more challenging (lower achieving, less well-behaved) students. Analysis of mobility patterns revealed that participants are more likely to switch schools than non-participants. However, participants are also less likely to exit public school teaching. The authors concluded that, while their preliminary analysis revealed some interesting patterns, much work remains to be done and that future work will include an analysis of the effects of recruitment and retention bonuses. Tables are appended.

I investigate the determinants of high school completion and college attendance, the likelihood of taking science, technology, engineering or math (STEM) courses in the first year of college and the probability of earning a degree in a STEM field. The focus is on women and minorities, who tend to be underrepresented in STEM fields. Tracking four cohorts of students throughout Florida, I find that large differences in math achievement across racial lines exist as early as elementary school and persist through high school. These achievement differences lead to higher drop-out rates in high school and a reduced probability of attending college for black students. However, conditional on immediately attending a four-year college after high school, black and Hispanic students are more likely than whites to take STEM courses during their first year in college. Increased exposure to Hispanic math and science teachers in middle and high school tends to increase the likelihood that Hispanic students take STEM courses during their first year in college, though pairing black students and black math/science teachers does not have the same positive effect. For all students, having high school math and science teachers with a degree in biology, chemistry or math (as opposed to education) is associated with a higher likelihood of taking STEM courses as college freshmen. When pre-college differences in income and math achievement are taken into account, black and Hispanic students are at least as likely as white students to successfully complete a STEM major. Racial/ethnic pairing of students and college instructors in first-year STEM courses does not increase the likelihood of majoring in a STEM field. In contrast to underrepresented minorities, women perform nearly as well as men on math achievement tests through high school and are more likely to finish high school and attend college than males. Among college students, however, women are less likely than men to take courses in the physical sciences in their first year and are less likely to earn a degree in physics or engineering, even after adjusting for pre-college test scores. Gender matching of students and math/science teachers in middle and high school tends to increase the likelihood that female college freshman will take at least one STEM course, However, conditional on first-year coursework, neither gender matching at the secondary or college levels appears to have any effect on the likelihood of completing a major in a STEM field. Tables are appended.

Teacher turnover has adverse consequences for student achievement and imposes large financial costs for schools. Some have argued that high-stakes testing may lower teachers' satisfaction with their jobs and could be a major contributor to teacher attrition. This study uses data on individual students and teachers from Georgia's statewide longitudinal data system from 2007/08-2017/18 to measure the effect of testing pressures on teacher mobility choices. A "difference-in-differences" approach is used to compare changes in mobility over time in grades/subjects that discontinue testing vis-à-vis grades/subjects that are always tested. [SREE documents are structured abstracts of SREE conference symposium, panel, and paper or poster submissions.]

The core work of the National Center on Scaling Up Effective Schools consists of four stages: identifying practices of highly effective high schools, designing interventions and transfer of practices, evaluating the intervention's implementation and effects, and evaluating implementation at scale. The purpose of this report is to describe the methodology used for selecting two high value-added and two low-value added schools in one of the partner districts for intensive field work in year one of the Center. Florida and Texas were selected for study of scaling up effective practices in high school because they have two of the most comprehensive student-level administrative and achievement data systems among the 50 states. While NCLB only requires states to test once in the high school grades, both Texas and Florida test English/language arts and mathematics in more than one high school grade, improving the ability to identify effective and ineffective schools. As part of the proposal for the Center, initial analyses using statewide data were conducted in both Florida and Texas. The goal was to identify districts that had both highly effective and low-performing high schools for students in traditionally low performing subpopulations to serve as both sites of research in which to identify effective practices and sites of intervention to which to transfer those practices. This report describes how Broward County Public Schools (BCPS) compare in value added to other schools in Florida, as well as how the four case study schools were identified. (Contains 2 figures, 9 tables, and 6 footnotes.)

Traditionally, states have required individuals complete a program of study in a university-based teacher preparation program in order to be licensed to teach. In recent years, however, various "alternative certification" programs have been developed and the number of teachers obtaining teaching certificates through routes other than completing a traditional teacher preparation program has skyrocketed. In this paper I use a rich longitudinal data base from Florida to compare the characteristics of alternatively certified teachers with their traditionally prepared colleagues. I then analyze the relative effectiveness of teachers who enter the profession through different pathways by estimating "value-added" models of student achievement. In general, alternatively certified teachers have stronger pre-service qualifications than do traditionally prepared teachers, with the least restrictive alternative attracting the most qualified perspective teachers. These differences are less pronounced when controlling for the grade level of teachers, however. On average, alternatively certified science teachers have also had much more coursework in science while in college than traditionally prepared science teachers. The same is not true for math teachers, where the hours of college coursework are approximately equal across pathways. Of the three alternative certification pathways studied, teachers who enter through the path requiring no coursework have substantially greater effects on student achievement than do either traditionally prepared teachers or alternative programs that require some formal coursework in education. These results suggest that the additional education coursework required in traditional teacher preparation programs either does little to boost the human capital of teachers or that whatever gains accrue from traditional teacher education training are offset by greater innate ability of individuals who enter teaching through routes requiring little formal training in education. (Contains 7 tables and 13 footnotes.)

Using matched student-teacher panel data from the state of Florida, the authors study the determinants of teacher job change and the impact of such mobility on the distribution of teacher quality. The probability a teacher stays at a school increases the more productive they are in their current school. The quality of teachers who exit teaching tends to be bimodal; high and low-quality teachers are more likely to leave than those in the middle of the distribution. In contrast, the relationship between teacher productivity and inter-school mobility is relatively weak. In addition to own quality, peer characteristics play a significant role in teachers' job decisions. In particular, teachers who rank above their faculty colleagues are more likely to transfer to a new school within a district and exit teaching. Additionally, as the share of peer teachers with more experience, advanced degrees or professional certification increase, the likelihood of moving within district decreases. We also find evidence of assortative matching among teachers. The most effective teachers who transfer tend to go to schools whose faculties are in the top quartile of teacher quality. As a result, teacher mobility exacerbates differences in teacher quality across schools. (Contains 7 tables, 2 figures and 9 footnotes.)

A vast literature has investigated the efficacy of school leadership. Most of the extant research has been based on teacher perceptions of school leadership, rather than on more direct measures of student performance. A detailed review of the literature on principals through the mid-1990s is provided by Hallinger and Heck (1998). Within the last couple of years, however, researchers have begun to exploit longitudinal data on student test scores to evaluate the impact of principals on student achievement and how that impact varies with the characteristics of principals. Rather than look at the relationship between personal characteristics and principal efficacy, the authors instead focus on the policies that are adopted by effective principals. In particular, they consider how school policies change when there is a change in school leadership and which policy changes are associated with new schools leaders who have been previously effective. While still very preliminary, this paper provides new evidence of the role that principals play in shaping school policies. Using survey data on school policies in three time periods matched with principal value-added measures based on student test scores, the authors are able to determine the relationship between principal effectiveness and the policies of the schools that principals lead. Principals with stronger prior performance appear to systematically reduce focus on teacher incentives and concentrate more on other types of policies aimed at improving the performance of low-performing teachers and students--and principals who are measured as strong independent of the quality of their teachers apparently also focus more resources on teachers. Although much more analysis needs to be done, the authors' initial findings highlight the role of principals in shaping school policy and suggest that policies aimed at recruiting and retaining effective principals are likely to have important effects on school policy. Appendix (1) contains references, and Appendix (2) contains 4 figures and 7 tables. Also included are 3 footnotes.)

Research on teacher productivity, and recently developed accountability systems for teachers, rely on value-added models to estimate the impact of teachers on student performance. The authors test many of the central assumptions required to derive value-added models from an underlying structural cumulative achievement model and reject nearly all of them. Moreover, they find that teacher value added and other key parameter estimates are highly sensitive to model specification. While estimates from commonly employed value-added models cannot be interpreted as causal teacher effects, employing richer models that impose fewer restrictions may reduce the bias in estimates of teacher productivity. (Contains 7 tables and 21 footnotes.)

This paper examines whether teachers in schools serving students from high-poverty backgrounds are as effective as teachers in schools with more advantaged students. The question is important. Teachers are recognized as the most important school factor affecting student achievement, and the achievement gap between disadvantaged students and their better off peers is large and persistent. Using student-level microdata from 2000-2001 to 2004-2005 from Florida and North Carolina, the authors compare the effectiveness of teachers in high-poverty elementary schools (greater than 70% FRL students) with that of teachers in lower-poverty elementary schools (less than 70% FRL students). The results show that the average effectiveness of teachers in high-poverty schools is in general less than teachers in other schools, but only slightly, and not in all comparisons. The authors also find differences in within-school-type variation in teacher effectiveness in nearly every comparison. These differences are largely driven by the longer tail at the bottom of the teacher effectiveness distribution in high-poverty schools. Teachers at the top of the effectiveness distribution are very similar across school settings. The observed differences in teacher quality between high-poverty and lower-poverty schools are not due to differences in the observed characteristics of teachers, such as experience, certification status and educational attainment. Rather, they appear to arise from differences in the marginal return or payoff from increases in a characteristic. In particular, the gain in productivity from increased experience is much stronger in lower-poverty schools. The lower return to experience in high-poverty schools does not appear to be a result of differences in the quality of teachers who leave teaching or who switch schools. Rather, it may be the case that the effect of experience on teacher productivity may depend on the setting in which the experience is acquired. If there are positive spillovers among teachers that depend on teacher quality (ie. teacher "peer effects") or if exposure to challenging student populations lessens the future productivity of teachers (i.e. leads to "burn out"), teachers in schools serving large proportions of low-income students may simply not improve much as time goes by. These findings suggest that solutions to the achievement gap between high and lower-poverty schools may be complex. Changing the quality of new recruits or importing teachers with good credentials into high-poverty schools may not be sufficient. Rather, the findings suggest that measures that induce highly effective teachers to move to high-poverty schools and which promote an environment in which teachers' skills will improve over time are more likely to be successful. An appendix is included. (Contains 8 figures, 15 tables and 14 footnotes.) [This is an updated version of Working Paper 41.]

Struggling schools that come under increased accountability pressure face a number of challenges, changing instructional policies and practices to facilitate student improvement. But what effect does school accountability have on teachers' mobility decisions? This study is the first to exploit policy variation within the same state to examine the effects of school accountability on teacher job changes. Using student-level data from Florida State the authors measure the degree to which schools and teachers were "surprised" by the change in the school grading system that took hold in the summer of 2002--what they refer to as an "accountability shock"--by observing the mobility decisions of teachers in the years before and after the school grading change. Several key findings emerge: (1) over half of all schools in the state experience an accountability "shock" due to this grading change; (2) teachers are more likely to leave schools facing increased accountability pressure--and even more likely to leave schools that face the highest accountability pressure (schools shocked downward to a grade of "F")--and they are less likely to leave schools facing decreased accountability pressure; (3) While the quality differential between stayers and leavers does not change as a result of increased accountability pressure, schools facing increased pressure experience an increase in the quality of teachers who leave or stay and schools with no accountability shock experience no significant change to the quality of teachers that leave or stay. The results suggest if these schools were able to retain more of their high-quality teachers, perhaps through increased incentives to remain in the school, the performance gains associated with school accountability pressure could be greater than those already observed. (Contains 9 tables, 1 figure and 16 footnotes.)

This paper contributes importantly to the growing literature on the training of special education teachers and how it translates into classroom practice and student achievement. The authors examine the impact of pre-service preparation and in-service formal and informal training on the ability of teachers to promote academic achievement among students with disabilities. Using student-level longitudinal data from Florida over a five-year span the authors estimate "value-added" models of student achievement. There is little support for the efficacy of in-service professional development courses focusing on special education. However, teachers with advanced degrees are more effective in boosting the math achievement of students with disabilities than are those with only a baccalaureate degree. Also pre-service preparation in special education has statistically significant and quantitatively substantial effects on the ability of teachers of special education courses to promote gains in achievement for students with disabilities, especially in reading. Certification in special education, an undergraduate major in special education, and the amount of special education coursework in college are all positively correlated with the performance of teachers in special education reading courses. (Contains 4 tables and 12 footnotes.)

Mounting pressure in the policy arena to improve teacher productivity either by improving signals that predict teacher performance or through creating incentive contracts based on performance--has spurred two related questions: Are there important determinants of teacher productivity that are not captured by teacher credentials but that can be measured by subjective assessments? And would evaluating teachers based on a combination of subjective assessments and student outcomes more accurately gauge teacher performance than student test scores alone? Using data from a midsize Florida school district, this paper explores both questions by calculating teachers' "value added" and comparing those outcomes with subjective ratings of teachers by school principals. Teacher value-added and principals' subjective ratings are positively correlated and principals' evaluations are better predictors of a teacher's value added than traditional approaches to teacher compensation focused on experience and formal education. In settings where schools are judged on student test scores, teachers' ability to raise those scores is important to principals, as reflected in their subjective teacher ratings. Also, teachers' subject knowledge, teaching skill, and intelligence are most closely associated with both the overall subjective teacher ratings and the teacher value added. Finally, while past teacher value added predicts future teacher value added the principals' subjective ratings can provide additional information and substantially increase predictive power. (Contains 9 tables and 15 footnotes.)

In this report, the authors examined the elimination of teacher tenure in Florida to better understand how job protections for teachers affect students. The authors focus on identifying the effect of removing tenure as part of Florida's 2011 Student Success Act on student outcomes, and in particular the effect if this legislation on student achievement. They take a quasi-experimental research approach by comparing the effect of teachers' relative exposure to tenure reform on changes in individual students' test scores after versus before the policy. The authors find limited and circumstantial evidence that Florida's tenure reform "slightly" increased student test achievement in math and reading, and that the gains were more prominent for the lowest-performing students. Specifically, the authors look across schools where faculties were most versus least vulnerable to Student Success Act provisions -- measured in two different ways -- and find that students in the most vulnerable schools show gains that compared favorably to students in the least vulnerable schools, but by a very small degree.

There is little doubt that teacher quality is a key determinant of student achievement, but finding ways to identify and reward the best teachers has proven illusive. This research brief considers the stability of value-added measures of teacher effectiveness over time and the resulting implications for the design and implementation of performance-based teacher compensation schemes. In a companion policy brief, Dan Goldhaber and Mike Hansen explore the long-run stability of value-added measures and the associated implications for tenure policies. In contrast, this brief focuses on the stability of value-added measures over shorter time spans, across school districts, and over test instruments and considers the implications of teacher effect stability for the design and implementation of performance-based teacher compensation schemes. (Contains 3 tables, 2 figures, and 6 notes.)

In this paper we analyze the impact of classroom peers on individual student performance with a unique longitudinal data set covering all Florida public school students in grades 3-10 over a five-year period. Unlike many previous data sets used to study peer effects in education, our data set allow us to identify each member of a given student's classroom peer group in elementary, middle, and high school as well as the classroom teacher responsible for instruction. As a result, we can control for individual student fixed effects simultaneously with individual teacher fixed effects, thereby alleviating biases due to endogenous assignment of both peers and teachers, including some dynamic aspects of such assignments. Our estimation strategy, which focuses on the influence of peers' fixed characteristics--both observed and unobserved--on individual test score gains, also alleviates potential biases due to error in measuring peer quality, simultaneity of peer outcomes, and mean reversion. Under linear-in-means specifications, estimated peer effects are small to non-existent, but we find some sizable and significant peer effects within non-linear models. For example, we find that peer effects depend on an individual student's own ability and on the ability level of the peers under consideration, results that suggest Pareto-improving redistributions of students across classrooms and/or schools. Estimated peer effects tend to be smaller when teacher fixed effects are included than when they are omitted, a result that suggests co-movement of peer and teacher quality effects within a student over time. We also find that peer effects tend to be stronger at the classroom level than at the grade level. (Appended to this document is "Comparison of Estimation Methods Using Simulated Data." Contains 28 footnotes and 10 tables.)

We study the effects of various types of education and training on the ability of teachers to promote student achievement. Previous studies on the subject have been hampered by inadequate measures of teacher training and difficulties addressing the non-random selection of teachers to students and of teachers to training. We address these issues by estimating models that include detailed measures of pre-service and in-service training, a rich set of time-varying covariates, and student, teacher, and school fixed effects. Our results suggest that only two of the forms of teacher training we study influence productivity. First, content-focused teacher professional development is positively associated with productivity in middle and high school math. Second, more experienced teachers appear more effective in teaching elementary math and reading and middle school math. There is no evidence that either pre-service (undergraduate) training or the scholastic aptitude of teachers influences their ability to increase student achievement. (Contains 19 tables and 23 footnotes.)