Search

This report presents findings from the Access Evaluation, a study component that is designed to assess the potential impacts of direct certification-Medicaid (DC-M) on students' access to free school meals by conducting retrospective simulations of DC-M in school year 2011-2012, the year before the demonstration began. For the Access Evaluation, researchers at Mathematica collected: (1) student enrollment files for a sample of school districts in the demonstration; and (2) Medicaid data for school-age children. The authors matched these two types of files based on individual identifiers, such as name and date of birth, to simulate DC-M. If a student was found to be enrolled in Medicaid, they assessed whether the income information in the Medicaid file indicated that the student would be eligible for free meals and determined the certification status that each student in the school enrollment file would have if DC-M were used in addition to actual SY 2011-2012 certification procedures. Impacts are measured by comparing these simulated certification outcomes under DC-M to districts' actual certifications that year. Such comparisons reveal the extent to which DC-M could increase the number of students certified for free meals and decrease the reliance on applications. Separate simulations show the potential impact of DC-M under different matching algorithms and policies. Key findings from the Access Evaluation include: (1) DC-M could increase the direct certification rate by 12 percentage points in Access Evaluation districts; (2) The potential increase in the percentage of students certified for free meals is smaller--at 6 percentage points--because some of the students who could be directly certified under DC-M would be certified for free meals by application in the absence of DC-M.; (3) There is substantial variation in the effects of DC-M across districts; (4) The simulated impacts vary little under alternative matching procedures; and (5) The simulated impacts vary little under most alternative policy assumptions. The following are appended: (1) Methods; (2) Supplemental Tables Related to Simulations of DC-M as Currently Authorized; (3) Supplemental Tables Related to Simulations of DC-M under Alternative Policies; and (4) Supplemental Tables on Reasons for Match Failure and Indeterminate Income.

This last report from Mathematica's evaluation of Upward Bound analyzes data from the final round of survey and transcript data collection as well as administrative records from the National Student Clearinghouse and the federal Student Financial Aid files. It provides the first estimates of the effects of Upward Bound on postsecondary completion. The report also updates previous estimates of the program's effects on postsecondary enrollment and receipt of financial aid. Except for a statistically significant increase in the likelihood of earning a postsecondary certificate or license from a vocational school, the report finds no detectable effects on postsecondary outcomes, including enrollment, financial aid application or receipt, or the completion of bachelor's or associates degrees. In addition to these results, the report includes the findings from extensive subgroup and sensitivity analyses. Nine appendices are included: (1) Sample Design, Unit Nonresponse, and Weights; (2) Data Collection and Outcome Measures; (3) Sensitivity Analyses Pertaining to the Measurement of Outcomes; (4) Sample Sizes and Weighted Standard Deviations for all Outcome Variables; (5) Estimation of Impacts and Standard Errors; (6) Methods used to Estimate the Effects of Additional Upward Bound Participation; (7) Sensitivity Analyses Pertaining to Sample Weighting; (8) Other Supplemental Service Programs; and (9) Additional Subgroup Tables. (Contains 17 tables, 1 figure and 32 footnotes.)

This two-volume report, and Summary of Key Findings, presents findings from the congressionally mandated National Assessment of Title I on the implementation and impact of the program. Volume I [see ED499017] contains key findings on the implementation of the program under No Child Left Behind, and Volume II presents a report on follow-up findings from Closing the Reading Gap, an evaluation of the impact of supplemental remedial reading programs on achievement of 3rd and 5th grade students. According to the National Assessment of Educational Progress (U.S. Department of Education 2006), 36 percent of fourth graders read below the basic level. Such literacy problems can worsen as students advance through school and are exposed to progressively more complex concepts and courses. The nation's 16,000 school districts are spending hundreds of millions of dollars on educational products and services developed by textbook publishers, commercial providers, and nonprofit organizations, yet little is known about the effectiveness of these interventions. To help shed light on their effectiveness, the authors of the present volume initiated an evaluation of either parts or all of four widely used programs for elementary school students with reading problems. The programs are Corrective Reading, Failure Free Reading, Spell Read P.A.T., and Wilson Reading, all of which would be more intensive and skillfully delivered than the programs typically provided in public schools. The programs incorporate explicit and systematic instruction in the basic reading skills in which struggling readers are frequently deficient. This study is a large-scale, longitudinal evaluation comprising two main elements. The first element of the evaluation is an impact study designed to determine: (1) the impact of being in any of the four remedial reading interventions, considered as a group, relative to the instruction provided by the schools; (2) whether the impacts of the interventions vary across students with different baseline characteristics; and (3) the extent that the instruction provided in this study close the reading gap and bring struggling readers within the normal range, relative to the instruction provided by their schools. The second element of the evaluation is an implementation study that has two components: (1) an exploration of the similarities and differences in reading instruction offered in the four interventions; and (2) a description of the regular instruction that students in the control group received in the absence of the interventions, and of the regular instruction received by the treatment group beyond the interventions. The evaluation found that the interventions: improved some reading skills; did not improve Pennsylvania System of School Assessment (PSSA) scores; narrowed some reading gaps; and helped third-graders more than fifth graders, although the interventions did not consistently benefit any one subgroup more than another. The following are appended: (1) Details of Study Design and Implementation; (2) Data Collection; (3) Weighting Adjustments and Missing Data; (4) Details of Statistical Methods; (5) Intervention Impacts on Spelling and Calculation; (6) Instructional Group Clustering; (7) Baseline Characteristics of the PSSA Sample; (8) Impact Estimate Standard Errors, Test Statistics, and P-Values; (9) Estimated R-Squared Values and Intraclass Correlations; (10) Scientific Advisory Board; (11) PSSA Data Collection Form; (12) Sample Test Items; (13) Teacher Survey Form; (14) School Records Form. (Contains 54 footnotes, 28 tables, 1 exhibit, and 30 figures.) [This report was prepared for the National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences (IES) by the Corporation for the Advancement of Policy Evaluation. Additional funding of the interventions was provided by: the Barksdale Reading institute; the Haan Foundation for Children; and the Raymond Foundation.]

From July 1995 through September 2001, the U.S. Department of Labor (DOL) and the Ford Foundation (Ford) operated a demonstration of the Quantum Opportunity Program (QOP). QOP offered intensive and comprehensive services to help at-risk youth graduate from high school and enroll in postsecondary education or training. QOP was mainly an after-school program providing case management and mentoring, supplemental education, developmental activities, community service activities, supportive services, and financial incentives. These services were provided year-round for five years to enrollees who had not graduated from high school, and were designed to be comprehensive enough to address all barriers to success and to be intensive. To estimate QOP impacts on high school performance and graduation, postsecondary education or training, and risky behaviors, four surveys were conducted using a group of youth enrolled in QOP and a non-participating control group. The current document presents impacts on outcomes measured using data from the fourth survey. In addition to features of the QOP model, the impacts of the QOP demonstration are influenced by how well demonstration sites implemented the QOP model, how much they spent on the program, and the extent to which QOP enrollees participated in the program. It is noted that due to the overall comprehensiveness, intensity and complexity, sites generally did not meet enrollee needs for education or support services. It was also found that most enrollees attended fewer program activities than was stipulated by the participation goal. Primary objectives of QOP were to increase the likelihood of graduating from high school with a diploma, to increase engagement and persistence in postsecondary education, training, or military service, and to improve employment-related outcomes and earnings. Secondary objectives of the program included improving high school grades and achievement test scores, and reducing a broad range of targeted risky behaviors, including as binge drinking, illegal drug use, crime, and teen parenting. Impacts were found to vary significantly on a site-by-site basis but overall, it was found that QOP did not achieve its primary or secondary objectives, although that these shortfalls mask some suggestive evidence of promising effects for particular types of students, especially among younger enrollees. Eight appendixes are included: (A) Obtaining an Evaluation Sample and Conducting Random Assignment; (B) The Baseline Data; (C) Follow-Up Data from the Third Telephone Survey;(D) Outcomes and Subgroups; (E) Weighting, Impact Estimation, and Variance Estimation; (F) Sensitivity Analysis; (G) QOP and Control Group Means for Subgroups; and (H) QOP and Control Group Means for Sites.

According to the National Assessment of Educational Progress, nearly 4 in 10 fourth graders read below the basic level. These literacy problems get worse as students advance through school and are exposed to progressively more complex concepts and courses. The purpose of this study was to assess the effectiveness of four remedial reading programs in improving the reading skills of 3rd and 5th graders, whether the impacts of the programs vary across students with difference baseline characteristics, and to what extent can this instruction close the reading gap and bring struggling readers within the normal range--relative to the instruction normally provided by their schools. The study took place in elementary schools in 27 districts of the Allegheny Intermediate Unit outside Pittsburgh, PA during the 2003-04 school year. Within each of 50 schools, 3rd and 5th grade students were identified as struggling readers by their teachers. These students were tested and were eligible for the study if they scored at or below the 30th percentile on a word-level reading test and at or above the 5th percentile on a vocabulary test. The final sample contains a total of 742 students. There are 335 3rd graders ? 208 treatment and 127 control students. There are 407 5th graders ? 228 treatment and 179 control students. Four existing programs were used: Spell Read P.A.T., Corrective Reading, Wilson Reading, and Failure Free Reading. Corrective Reading and Wilson Reading were modified to focus only on word-level skills. Spell Read P.A.T. and Failure Free Reading were intended to focus equally on word-level skills and reading comprehension/vocabulary. Teachers received 70 hours of professional development and support during the year. Instruction was delivered in small groups of 3 students, 5 days a week, for a total of 90 hours. Seven measures of reading skill were administered at the beginning and end of the school year to assess student progress: Word Attack, Word Identification Comprehension (Woodcock Reading Mastery Test); Phonemic Decoding Efficiency and Sight Word Efficiency (Test of Word Reading Efficiency); Oral Reading Fluency (Edformation); and Passage Comprehension (Group Reading Assessment and Diagnostic Evaluation). After one year of instruction, there were significant impacts on phonemic decoding, word reading accuracy and fluency, and comprehension for 3rd graders, but not for 5th graders. For third graders in the reading programs, the gap in word attach skills between struggling readers and average readers was reduced by about two-thirds. It was found that reading skills of 3rd graders can be significantly improved through instruction in word-level skills, but not the reading skills of 5th graders. The following are appended: (1) Details of Study Design and Implementation; (2) Data Collection; (3) Weighting Adjustment and Missing Data; (4) Details of Statistical Methods; (5) Intervention Impacts on Spelling and Calculation; (6) Instructional Group Clustering; (7) Parent Survey; (8) Teacher Survey and Behavioral Rating Forms; (9) Instructional Group Clustering; (10) Videotape Coding Guidelines for Each Reading Program; (11) Supporting Tables; (12) Sample Test Items; (13) Impact Estimate Standard Errors and P-Values; (14) Association between Instructional Group Heterogeneity and The Outcome; (15) Teacher Rating Form; (16) School Survey; and (17) Scientific Advisory Board. [This report was produced by the Corporation for the Advancement of Policy Evaluation. Additional support provided by the Barksdale Reading Institute, and the Haan Foundation for Children.]

This report summarizes short-term impacts of the Quantum Opportunity Program (QOP) demonstration, which helped at-risk high school-age youth graduate and enroll in postsecondary education or training. QOP involved intensive case management and mentoring emphasizing after school supplemental academic education, developmental activities, and community service. It served a single cohort of youth from ninth grade in 1995 through fall 2000. A local community based organization in each of six inner city communities and one rural community teamed with a school to implement and operate a QOP program. The study examined how well the program was implemented at each site, how much it cost, how much time enrollees spent on program activities, and how QOP affected enrollees in the short term. Data from site visits, QOP conferences, conference calls with QOP staff, and student surveys indicated that QOP achieved short-term success in raising high school completion rates and enrollment in postsecondary education or other training. It did not improve grades and achievement test scores or reduce risky behaviors. QOP benefited enrollees in the middle of the eligible grade distribution more than enrollees at the top or bottom. Most enrollees attended very few program activities. Impact varied by site. Impacts for the whole QOP demonstration were substantially attributable to the impacts of two sites. (SM)

Upward Bound is a federal program designed to help disadvantaged students prepare to enter and succeed in college. Students typically enter the program in their freshman or sophomore year of high school and can remain in it through the summer following high school graduation. Upward Bound projects provide students with a variety of services, including instruction, tutoring, and counseling. This report presents findings from a national evaluation of Upward Bound that draws on a sample of 67 regular Upward Bound projects. About 1,500 students nationwide were in the study group, and about 1,300 other students formed a control group. Followup surveys of students and their academic records provided information about program effectiveness. Findings show that many students remain in Upward Bound only for a short time, and, in fact, the program has limited impact on students in high school, and no apparent impact on high school graduation. Upward Bound may have some impact on participants' postsecondary education experiences, but, since relatively few of the study group had reached college age, it is too soon to make definitive statements about program impact at the college level. Findings also show that Upward Bound has substantial impact on some groups of students, and not others. Positive effects were more evident for students who initially had low educational expectations and for students with poor performance as they entered the program. (SLD)

This report on the short-term effects of Upward Bound, a federal pre-college program designed to help economically disadvantaged students complete high school and gain access to post-secondary education, presents interim findings from the Longitudinal Effectiveness Study of Upward Bound based on data on approximately 2,800 students during the first year or two of high school. At present, there are more than 600 Upward Bound projects; they offer intensive instructional programs and are usually hosted by 2-year and 4-year colleges. The study found that: (1) Upward Bound has early positive impacts on students' educational expectations and academic course-taking; (2) students with lower educational expectations initially benefit more from Upward Bound; (3) Hispanic students initially benefit most from Upward Bound; and (4) many students (about 37 percent) who enter Upward Bound leave the program during the first year. After an executive summary and introductory chapter, Chapter 2 presents data on persistence in Upward Bound, and on the Upward Bound services offered. Chapter 3 details short-term impacts of Upward Bound, including the average impact of the program and groups benefitting most. The concluding chapter summarizes findings, compares them to previous findings, and draws implications for program improvement. Eight appendices provide additional detail on research methodology, data interpretation, and statistics. (Contains 18 references.) (DB)