1. Prosocial and Active Learning (PAL) Classrooms Evaluation. Final Summative Report
- Author
-
American Institutes for Research (AIR), Ryan Williams, Bo Zhu, Max Pardo, Crystal Aguilera, and Tara Zuber
- Abstract
Prosocial and Active Learning (PAL) Classrooms is a year-long teacher professional development program designed to increase students' prosocial behavior and engagement in 5th grade mathematics and science classrooms that use active, team-based lessons by altering the way that teachers interact with students using research-based strategies. The project is funded by a 6-year Education Innovation and Research grant and implemented by the eMINTS National Center and the Prosocial Development and Education Research Lab (ProsocialEd Lab) at the University of Missouri. The American Institutes for Research® (AIR®) as an independent evaluator, has completed an implementation and impact study of PAL Classrooms. The evaluation involves a mixed-methods study to assess the implementation of the program and a multicohort, school-level randomized controlled trial (RCT) to examine the impact of the program. The evaluation took place in 41 districts in three states--Arkansas, Kansas, and Missouri. For the impact study, AIR randomly assigned two cohorts (2021-22 and 2022-23) of elementary schools to receive PAL Classrooms immediately (treatment) or conduct treatment as usual and receive PAL Classrooms a year later (control). A total of 41 schools (21 treatment and 20 control), with 65 teachers and their 1,399 students, participated in the RCT. This final report summarizes the PAL Classrooms program and AIR's evaluation methods; findings on the extent to which PAL Classrooms' key components were implemented with fidelity; and the impact of PAL Classrooms on teacher outcomes (i.e., student-reported use of strategies, instructional quality), proximal student outcomes (i.e., prosocial behavior, engagement, perceived classroom climate, collaboration, teacher-student relationship), and student math and science achievement. Results of implementation analyses indicated that PAL Classrooms was generally implemented with fidelity despite occurring during the COVID-19 pandemic The program delivered key professional development activities and teachers generally participated as intended. In addition, teachers implemented strategies that promoted student prosocial behavior. Results were consistent across both cohorts. Results of impact analyses indicated that teachers assigned to PAL Classrooms had higher levels of overall instructional quality, student engagement, emotional support, and instructional support, measured by classroom observation. However, most of the impact estimates were not statistically significant, except for the estimate of impact on emotional support (p = 0.05), with an effect size of 0.58. In contrast, impacts on teachers' use of strategies such as praise or induction measured by student report were smaller, but still positive. Results reveal that PAL Classrooms had statistically significantly positive impact on proximal student outcomes measured by self-report surveys. Students assigned to PAL Classrooms reported higher levels of prosocial behavior in their peers and teachers, with effect sizes of 0.23 and 0.20 standard deviations (SD), respectively, compared with students in control schools. Analyses of students' self-report of prosocial behavior, engagement, classroom climate, and student-teacher relationships also yielded positive results, although the impact estimates were not statistically significant. Impacts on student-reported collaboration and more distal student achievement in math and science were less conclusive, with the impact estimates being smaller and not statistically significant. Additionally, the impact results on student achievement did not vary by student background characteristics (i.e., free or reduced-price lunch eligibility, or receipt of academic services under an individualized education plan).
- Published
- 2024