Search

The purpose of this study was to examine the efficacy of Imagine Learning's MyPath program on student mathematics and reading achievement growth. This study examined achievement growth trajectories of kindergarten students in one suburban Midwestern district in the 2022-23 school year. NWEA MAP mathematics and reading scores served as the main outcome variables in quantitative impact analyses. As Imagine MyPath was used by all district kindergarten students, MAP mathematics and reading score growth of district kindergarten students was compared to that of matched comparison students obtained from a Similar Schools Report (SSR) provided to the district by NWEA. Additionally, a questionnaire was administered to district teachers that used Imagine MyPath to examine teacher perceptions of the program. A statistically significant positive impact of Imagine MyPath on mathematics achievement was observed, with treatment students outgaining matched comparison students by more than 2 points. Treatment students also outgained matched comparison students in reading achievement, but this difference was minimal and not statistically significant. Teacher perceptions of Imagine MyPath were largely positive, especially regarding perceived program impacts on student learning. The results of this study provide preliminary evidence regarding the efficacy of Imagine MyPath in relation to mathematics achievement. It is important to note that results from this study are derived from data in one unique district in which all students used the program. Future research may focus on contexts where some students are using Imagine MyPath while other students are participating in business-as-usual instruction.

Background: Since the COVID-19 pandemic, American students have demonstrated substantial losses in achievement (Amplify, 2021; Domingue et al., 2021; Dorn et al., 2020; Storey & Zhang, 2023), while chronic absenteeism doubled after the pandemic, with the largest increase among low-income students (Chang et al., 2022; Dorn et al., 2021). Among historically underserved students, the rate of academic failure is far higher than it is among other students. On the National Assessment of Educational Progress reading test (National Center for Education Statistics, 2022b), only 19% of fourth grade students qualifying for free- or reduced-price lunch (FRL) scored at or above "proficient," in comparison to 51% of students who did not qualify. While it is generally felt that higher levels of parental involvement may lead to higher academic achievement (Kim, 2022; Wilder, 2014), attendance (Epstein & Sheldon, 2010), and social-emotional development (Green et al., 2007), research on the effectiveness of family engagement on student outcomes remains mixed (Cosso et al., 2022; See et al., 2022). Schools must therefore search for solutions to address student learning, chronic absenteeism, and social-emotional development from various sources. The identification of approaches with demonstrated effectiveness through evidence appropriate to each school's unique context, population, and requirements is particularly important. This methodology has been shown to be effective in addressing student needs (Slavin et al., 2013; van Geel et al., 2016). Purpose: Given the urgent need to identify and implement effective models to improve student academic and social-emotional outcomes, researchers and practitioners need to develop a stronger understanding of the impact potential interventions may have. In this study, we will develop and provide a visual overview of the extent and distribution of rigorous, high-quality evidence on family engagement programs and interventions in support of K-12 students. Setting: The study will focus on family engagement programs and interventions implemented in elementary and secondary schools. Population: The study will focus on programs and interventions implemented with Kindergarten through 12th grade students. Intervention: The study will focus on school-based family engagement programs and interventions that could plausibly impact the following student outcomes: literacy, numeracy, attendance, and social-emotional learning. Following previous definitions of family involvement (Boonk et al., 2018), we consider school-based interventions to be those that take place in the school, either focused specifically on increasing family engagement, or addressing specific student outcomes of interest with family engagement components. Research Design: This study will utilize existing data on family engagement programs and interventions to produce an evidence gap map. Evidence gap maps (EGM) are considered to be "thematic collections of evidence on effects structured around a framework which schematically represents types of interventions and outcomes of relevance" (Snilstveit et al., 2016, p. 121). EGM illustrate both areas where there is stronger understanding, but also where there are potential areas for additional needed research or analysis (White, 2020). Data Collection and Analysis: Literature Search: A master list of family engagement programs was created, integrating lists of family engagement programs developed by the National PTA. Initial electronic searches were made of educational databases including ERIC, JSTOR, and PsycInfo, to identify studies meeting inclusion criteria for each program. To locate unpublished reports and other possible inclusions, Google Scholar, citations from identified studies and previous reviews, and each program's official websites were also searched. Inclusion Criteria: Accepted studies must: (1) Be published between 1990 and 2019; (2) Evaluate a family engagement program for preK-12 students; (3) Include a comparison group of children receiving "business as usual" services; (4) Take place in the U.S.A.; and (5) Use quantitative measures of student outcomes including reading, mathematics, attendance, or social-emotional learning. Review & Extraction: After a screening of the abstract and title for relevance, all full-text studies were read by at least two reviewers before being included in our set of included studies. For the full systematic review, a number of items were coded. For the present study, only the intervention, outcomes, and whether outcomes were statistically significant were required. The resulting data were cleaned and analyzed using the R statistical software (R Core Team, 2023). The visual evidence maps were produced using ggplot2 (Wickham, 2016) and shared via an online Shiny app. The resulting analytic code, datasets, and interactive Shiny app are publicly available (AUTHORS, 2023). Findings: A total of 48 studies of 20 Family Engagement programs were reviewed, with 34 meeting the initial inclusion criteria (Table 1). Across these included studies a total of 50 broad outcomes were collected. These outcomes were divided into 4 categories: Reading, Mathematics, Attendance, and Social-Emotional Learning. The results are summarized in screenshots of the evidence map in Figure 1. Each study is represented by one dot. A study may be represented in more than one column if it measured outcomes from different categories. The color is based on the outcome findings. A green dot means more than half of the outcomes in that study in that category were statistically significant. A red dot means the impacts in that study in that category were not statistically significant. The map illustrates that while some programs such as ParentCorps have multiple studies, others have a single study assessing their impact, while several programs have no studies of effectiveness. Additionally, outcomes were most often measures in the reading domain. Overall, there are few programs with multiple studies documenting multiple significant results across studies or domains. Conclusions: Our evidence map employs a visually intuitive system to represent the distribution and significance of outcomes offering an overview of where these programs stand in terms of empirical support. Our findings call for a concerted effort to expand the evidence base, not only by increasing the number of studies but also by ensuring a broader representation of program types and outcome measures. Future research should aim to address these gaps by focusing on under-studied programs and domains, employing rigorous methodologies to assess program effectiveness, and exploring the scalability and sustainability of successful interventions. While the evidence map provides valuable insights into the effectiveness of family engagement programs, it also underscores the critical need for a more extensive and diversified research agenda.

Background/Context: The importance of vision care begins at an early age, with doctors recommending screenings at multiple stages prior to reaching school age and throughout adolescence (Mathers et al., 2010; Solebo et al., 2015). Addressing unaddressed vision needs at an early age is particularly important due to the strong connection between student vision and student ability to access and engage in academic learning (Castanes, 2003; Glewwe et al., 2018; Pavithra et al., 2014). Yet the most disadvantaged students often lack eyeglasses, or the ability to access eyecare due to various barriers (Kemper et al., 2006; Slavin et al., 2018; Heslin et al., 2006; Qiu et al., 2014; Ruderman, 2016; Zhang, Cotch, et al., 2012; Zhang, Elliott, et al., 2012). Schools have been identified as sites for providing a host of non-school related services, including regular meals, dental screenings, asthma and chronic illness support, and vision care. For those students who cannot independently access vision care, school-based vision care for pre-school and early elementary students may provide a solution and a means of preventing later academic learning challenges. Objective/Research Question: It is established that the early grades are key to build a strong foundation for literacy. Interventions addressing student health conditions provide opportunities to examine effective methods of addressing the needs of students who otherwise would struggle to participate in academia. This study is designed to build on previous research linking school-based vision care to literacy learning through its focus on early grade students (prekindergarten - second). Setting: This study included a total of 102 schools serving preschool through second grade located in a mid-Atlantic urban district. Included elementary and elementary/middle schools enrolled predominantly black students (83.5%). Most students qualified as low-income (89.5%), but the sample included few English Language learners (EL) (4%) and a moderate number of students who qualified for special education (13.7%). Participants: The analytic sample included 1,383 students in preschool through second grade at baseline. Characteristics of these students were similar across the three cohorts (Table 1). Students were predominantly Black (76%), with 18% Latinx and 22% White. Fourteen percent of students were in special education and 12% were ELL. Students included in the analytic sample had to fail a vision screening, be prescribed glasses at the eye exam, have both baseline and post-intervention DIBELS composite test scores, and opt into the research study. Intervention: The school-based intervention included a vision screening conducted by the local health department for all students, an eye examination conducted by licensed optometrists in a mobile eye clinic at schools, and provision of eyeglasses for those diagnosed with refractive error. A consent form was sent to parents/legal guardians of students failing a screening, offering an eye examination. Eye examinations were subsequently conducted for students whose parents provided consent and diagnosed students could choose their preferred frames style and color. Students received their eyeglasses at school, and the intervention provided replacements as needed. Research Design: This cluster randomized study assigned elementary schools to one of three groups: intervention in the first, second, or third year starting in SY2016-17 (Cohort 1-3 respectively) (Figure 1). The treatment was a schoolwide intervention, and students' treatment status was determined by the school where they received school-based vision services. Data Collection and Analysis: Academic outcomes were English language arts (ELA) composite scores on the Amplify Dynamic Indicators of Basic Early Literacy Skills (DIBELS) test. DIBELS composite scores, based on initial sound, phoneme segmentation, nonsense word, and oral reading fluency subtests, vary from 13-287 points between kindergarten and second grade. Individual student test score data for SY 2015-16 through SY 2017-18 and student demographic data were obtained from the school district. Vision exam data were collected during the eye examinations. Vision data included the variables: prescription for eyeglasses, type of refractive error and severity of refractive error. To compare student achievement in English language arts for intervention and control students, we used hierarchical linear modeling following an intent to treat approach, where students were nested within the schools where they received their school-based vision services. The models analyzed program effects for each year separately, and controlled for student grade level, prior achievement, ethnicity, and blocking variables used in randomization (charter school status, school type, pilot study participation, school proportion of low-income and Black students, and whether the school served more than 25% Latinx students). Differential treatment effects for various student subgroups (gender, grade level, special education, and baseline achievement) were examined with interaction terms. Analysis was conducted using the R statistical software. Findings: The outcomes for all students are summarized in Table 2. For each cohort, there was a slightly negative non-significant impact, with effect sizes of -0.06 and -0.01 (p = 0.17, 0.89 respectively). Differential effects were explored for special education status, baseline achievement, grade level, and gender (Table 3). None of these four factors were found to moderate the treatment impact. Adjusted mean impacts and effect sizes for each of these categories of students are shown in Table 4. Discussion and Conclusions: The present study did not find a significant impact of the school-based vision care on DIBELS achievement scores. The study also did not identify differential effects by grade, gender, special education status, or baseline achievement. The lack of impacts may also be explained by poor consistency in the use of eyeglasses provided to students at such a young age. In addition, other research has pointed to inconsistent academic scores at pre-kindergarten and kindergarten levels being commonplace (Slavin et al., 2009). Future research should utilize additional tests that may be more sensitive to student growth, as well as further efforts to examine the degree to which the treatment received explains variations in outcomes. Limitations: This study was conducted in a real-world school-based program implementation context. The sample of students in the analysis were part of a larger study involving over 30,000 students over a wide age range. Consequently, variations in vision screenings and exams may have taken place in a manner that would not have occurred in a clinical trial setting.