Promoting Adolescents' Comprehension of Text: A Randomized Control Trial of Its Effectiveness

Promoting Adolescents' Comprehension of Text (PACT) is a text- and discourse-based set of instructional practices that engage students with disciplinary texts as a means of building content knowledge and improving reading comprehension. PACT)s "efficacy" has been the subject of extensive previous trials. The purpose of this study was to evaluate its "effectiveness" in a school-randomized design using stratified balanced sampling to assemble a representative sample of schools from the population of middle schools that teach U.S. history. The population-level effect estimates on a posttest of knowledge acquisition were 0.45 using weighted mixed effects estimation and 0.37 using weighted ordinary least squares. At follow-up, the effects were 0.53 based on weighted mixed effects estimation and 0.33 based on robust variance estimation. Furthermore, although treatment)s effects on content area reading comprehension and broad reading comprehension were not statistically significant, the sample-based treatment effects (g = 0.15 for content area reading and 0.14 for broad reading) were not trivial when evaluated in the context of other studies of literacy instruction with older readers. Our findings represent the most reliable estimates of PACT)s average student-level effects when the program is implemented at the school level. In addition, these estimates are for the program when implemented in "real-world" settings, versus in the more controlled conditions typical of efficacy designs. Finally, the results further replicate the PACT)s instructional practices, in this sample and in the population of middle schools that teach U.S. history.