Comparison of Student-Level and School-Level Data in a National Impact Evaluation

One of the primary obstacles facing education researchers today is the struggle to obtain student-level data from states, districts, and schools. Researchers typically face one of two scenarios; they must either (1) work with contractors hired by the state or district to handle data requests who can be prohibitively expensive or (2) invest significant amounts of time making data requests from overworked school district staff that can take a year or longer. Due to budget cuts and waning resources, district staff juggle many competing demands and typically prioritize district-level needs above those of external evaluators. It is becoming increasingly clear the research community must look for methodological alternatives that allow the continuation of rigorous research about what works in education. This study explores the possibility of using publicly available school-level data instead of student-level data to conduct an impact analysis of a national education intervention. The authors use a national evaluation of an Expanded Learning Time (ELT) initiative in middle schools across the country as a case-study for these methods. One table is appended.