University of Memphis, Center for Research in Educational Policy (CREP), Smithsonian Institution (SI), Smithsonian Science Education Center (SSEC), Todd Zoblotsky, Robert McKinney, Carolyn Kaldon, Mojtaba Khajeloo, Yu Wu, and Ivysmeralys Morales
The goal of Smithsonian Science for North and South Carolina Classrooms was to provide teachers with ongoing, differentiated professional development and research-based curricular materials to improve elementary student achievement in science, mathematics, and reading. This work used an inquiry-based science curriculum, Smithsonian Science for the Classroom, which is aligned with the Next Generation Science Standards (NGSS), and was implemented in Grades 3-5 within predominantly rural North and South Carolina school districts serving high-needs students. The Center for Research in Educational Policy (CREP) at the University of Memphis, a State of Tennessee Center of Excellence, was the independent third-party evaluator. CREP implemented a Randomized Controlled Trial (RCT) to collect and analyze data using a rigorous, mixed-methods approach, including quantitative analyses of student achievement data from the Stanford-10 and state assessments, and qualitative analyses of classroom observation data, as well as teacher feedback from professional development surveys, module logs, and focus groups. For this study, 36 schools (including one split cohort being treated as one school) in seven districts within North and South Carolina were randomly assigned to treatment and comparison groups. The study cohort, followed over three academic years (2020-21 through 2022-23), was composed of more than 1,600 third grade (2020-21) students in these schools. In the comparison condition, schools used their business-as-usual science curricula and teacher professional development and did not receive Smithsonian Science curricular materials or professional development. As the pretests for achievement outcomes, students took the Abbreviated Battery of the Stanford Achievement Test Series, Tenth Edition (Stanford-10) in Reading and Math in spring of 2021 (the cohort's third grade year). The main findings and posttests included the combined sample (i.e., both states combined) for (a) the Stanford-10 Science subtest, which was administered in the spring of 2023 (the cohort's fifth grade year), and (b) spring 2023 standardized state assessment scores in Reading and Math. Using a hierarchical linear model (HLM) with school as a random effect, the analysis of the effect of Smithsonian Science for the three main findings after several years of disruptions from the COVID-19 pandemic was (a) positive (g = 0.18) and statistically significant (p = 0.032) on the Stanford-10 science assessment and (b) positive for both Reading (g = 0.10) and Math (g = 0.16), meaning students in the treatment group performed better than the control group, but the effects were not statistically significant. Teachers were generally positive in terms of the professional development provided, and feedback on the module logs largely supported the effectiveness of the professional development and fidelity of implementation. Classroom observations further supported the effectiveness of the professional development and fidelity to inquiry-based practices in treatment classrooms (e.g., use of experiential hands-on learning and level of class time dedicated to inquiry-based science). With instructional time for science lagging other subject areas, and as learning recovery in reading and math after COVID-19 receive national attention, the results demonstrate that ongoing, differentiated, high-quality professional learning and high-quality instructional materials for science instruction can support student achievement in science, math, and reading.