Estimating Impacts on Program-Related Subgroups Using Propensity Score Matching: Evidence from the Early College High School Study

This paper addresses methodological issues arising from an experimental study of North Carolina's Early College High School Initiative, a four-year longitudinal experimental study funded by Institute for Education Sciences. North Carolina implemented the Early College High School (ECHS) Initiative in response to low high school graduation rates. The goal of the initiative is to increase the number of students graduating from high school and who continue on and succeed in college. The study has three main goals: (1) Determine the impact of the model on selected student outcomes, including course-taking patterns, achievement, attitudes, and dropout and leaving rates; (2) Determine the extent to which outcomes differ by student characteristics; and (3) Examine the implementation of the model and the extent to which specific model components are associated with positive outcomes. Schools participating in the study identify an eligible pool of student applicants. The research team then randomly assigns students to either the treatment group (attending the ECHS) or the control group (business as usual). The outcomes for students in the two groups are then tracked and compared. The study follows an intent-to-treat model in that once a student is assigned to ECHS, he or she remains in the treatment group regardless of whether he or she ends up enrolling in ECHS, or leaves ECHS. Findings suggest that ECHS has the same impact on pass-rates for the Always-takers as the traditional high school. Where the ECHS are making difference is by increasing the number of students who are taking these courses. (Contains 5 figures, 6 tables and 4 footnotes.)