Search

This report concludes a multiyear series of implementation studies intended to inform the California State University (CSU) system about the implementation of Executive Order 1110 (EO 1110). A major policy established in 2017, EO 1110 requires CSU campuses to eliminate noncredit developmental courses (often known as "remedial" courses) in Written Communication and Mathematics/Quantitative Reasoning, change the process for how students are placed into those courses, and improve how students are supported to succeed. Following up on earlier reports in the series which have described course models and instructional approaches adopted as a result of EO 1110, examined student progress since the policy has been implemented, and reported on initial outcomes of revamped summer Early Start programs, this fifth and final report of the series focuses on how students are placed into entry-level math courses. Specifically, it focuses on the use of multiple measures for placing students and examines the predictive power of the measures for anticipating students' success in their first college math course. The report discusses implications of its findings for policy issues such as determining the appropriate grade point average cutoffs for placing students into different levels of math courses, the role of standardized testing in determining college math placements, and whether additional measures should be used to inform student placement.

Offering "Early Start" programs in the summer before students start their first year of college is one of the strategies that the California State University (CSU) system has used to support incoming students who may not be ready for college-level work in mathematics or written communication. In 2017, the CSU system began implementing a major new policy, known as Executive Order 1110 (EO 1110), which called for changes to Early Start as part of a wider set of changes aimed at improving how the system serves students who may need extra support for success in college. As a result of EO 1110, Early Start offerings on several campuses, though not all, have been changed significantly from prior years to provide additional supports for students. This report, part of a series of studies on the implementation of EO 1110, looks at the impact of the updated Early Start offerings in preparing students for college-level mathematics. The report describes some initial positive outcomes that were found for students who enrolled in summer 2019 Early Start mathematics courses that were offered in response to the policy. [For the previous report in this series, "Student Progress before and after California State University's Executive Order 1110," see ED613407.]

This report is part of a series focusing on studies intended to inform the California State University (CSU) system about the implementation of Executive Order 1110 (EO 1110). A major policy adopted by the CSU Chancellor's Office in 2017, EO 1110 requires CSU campuses to eliminate noncredit developmental courses -- often known as "remedial" courses -- in Written Communication (WC) and Mathematics/Quantitative Reasoning (QR), change the process for how students are placed into WC and QR courses, and improve how students are supported to succeed. This report, the third in the series, compares outcomes for students who entered the CSU system in the first year under the new policy (2018) against the outcomes of students who entered in 2017, just prior to implementation. The study's comparisons indicate that, on a systemwide level, the EO 1110 policy had the intended impact of getting students to complete their General Education requirements and to earn more baccalaureate-level credits in their first year of enrollment. However, analyses also show very little change in the subsequent year's retention patterns for students under the new policy, and indicate that there are still important differences in outcomes for students of different racial and ethnic backgrounds. [For the previous report in this series, "Student Progress under California State University's Executive Order 1110: A Systemwide Analysis of Outcomes in 2018/19," see ED613408.]

WestEd is undertaking a multiyear series of implementation studies intended to inform the California State University (CSU) system about the implementation of Executive Order 1110 (EO 1110). A major policy adopted by the CSU Chancellor's Office in 2017, EO 1110 requires CSU campuses to eliminate non-credit developmental courses in Written Communication (WC) and Mathematics/Quantitative Reasoning (QR), change the process for how students are placed into WC and QR courses, and improve how students are supported to succeed. WestEd's first EO 1110 report, "College-Ready in the California State University System: Campus Experiences Implementing EO 1110," (ED595731) issued in 2019, illustrates the variation in course models and instructional approaches adopted by campuses in response to this major policy change. This second report looks at the progress of nearly 60,000 students during the first year of implementation of the policy, the 2018/19 academic year.

Executive Order (EO) 1110, a major policy adopted by the California State University System (CSU) Chancellor's Office in August 2017, requires CSU campuses to eliminate non-credit-bearing developmental courses in written communication and math/quantitative reasoning, and change how students are placed into entry-level courses in these areas. The order also requires campuses to design supportive course models to help students succeed in entry-level courses. This report, the first in a series, examines how CSU campuses are implementing EO 1110. Based on interviews with faculty, staff, and administrators at nine of the 23 CSU campuses, the report presents a snapshot of campus implementation from the perspective of those on the ground working to respond to the required changes. The report highlights findings from campus visits regarding the progress related to course redesign, professional development and course placement, and concludes with a discussion of the implementation challenges and implications for the work moving forward.

In this paper, the authors present qualitative data -- vignettes -- that describe how the professional learning systems and structures have changed for some of California's educators in recent years: organization of teachers within schools as professional learning hubs, management by school principals of the instructional mission, navigation by curriculum and instruction teams within districts, and models of support by providers including county offices of education (COEs). Key questions include: (1) How have school districts designed and implemented professional learning systems to support teachers through the transition to the new standards?; (2) What has been the role of support initiatives for school principals during these years of standards implementation? Are there particular models of support for school leaders that have surfaced that show promise?; and (3) How have the resources that the California Department of Education (CDE) has built around standards implementation found their way into the field? The authors have organized the paper around data reporting from several surveys, three case studies of field-based implementation, and the authors' reflections on a broad set of network structures and technical assistance strategies in California that support standards implementation initiatives. Their conclusions and recommendations are based on converging results from these different data sources.

In 2015, the Governor's Office of Planning and Research commissioned an analytic study to determine the number of California students who were eligible to attend college within the University of California (UC) and/or the California State University (CSU) systems. The study, "University Eligibility Study for the Public High School Class of 2015", conducted by RTI International, required the collection of individual transcripts from high schools across the state. This current commissioned report examines whether federal reporting data submitted to the California Department of Education by local education agencies could be utilized as a surrogate for individual transcript data in future UC/CSU eligibility studies. The report examines: (1) Data similarities and differences across four data sources: California Longitudinal Pupil Achievement Data System (CALPADS), UC admissions files, CSU admissions files, and raw transcript data files collected for the eligibility study; (2) Student-level records across the different sources for the four school years between the fall of 2011 and the spring of 2015 to examine the patterns of discrepant information; and (3) Both the availability and the alignment across sources of credits during this period. [The California Department of Education provided the data from the California Longitudinal Pupil Achievement Data System and reviewed the report for technical accuracy. Financial support for this study was provided by the Governor's Office of Planning and Research.]

In 2015, Fong, Finkelstein, Jaeger, Diaz, and Broek reported the findings from an independent evaluation of the Expository Reading and Writing Course (ERWC). The evaluation found positive and statistically significant effects of the ERWC on student achievement. Due to some concerns that the previously reported results in Fong et al. (2015) did not account for the possibility that students who were taught by the same teacher may have correlated error terms, an additional ordinary least squares (OLS) regression analysis was conducted using cluster-robust standard errors that allow for clustering on the teacher. The results of this additional analysis are reported in this addendum. The previously reported OLS regression results had used cluster-robust standard errors that allowed for clustering on the student. The newly reported regression analysis in this addendum only clusters on the teacher, and not on both the teacher and the student, since previous literature has recommended only clustering at the highest level (see, for instance, Cameron & Miller, 2015; Bertrand, Duflo, & Mullainathan, 2004). [For the full report, "Evaluation of the Expository Reading and Writing Course: Findings from the Investing in Innovation Development Grant," see ED559522.]

Middle school math placement and progression are topics that are part of an active policy and practice discussion in California and elsewhere. Beginning in the 2008/09 school year, California's State Board of Education recommended that students complete algebra I by the end of grade 8. Between 2003 and 2009 the proportion of grade 8 students taking algebra I rose from 32 percent to 54 percent (Williams et al., 2011). This increase resulted in a larger percentage of grade 8 students scoring "proficient" or "advanced" (achieving proficiency) on the algebra I California Standards Test (CST; see box 1 for definitions of key terms used in this report) and in a larger number of grade 8 students scoring "far below basic" or "below basic" on the test (Williams et al., 2011). Recent studies show that enrolling in algebra I in grade 8 works well for some students but backfires for others. The consequences of misplacement are most pronounced for students with weaknesses in key foundational areas that support algebra readiness, which frequently translates into difficulty reaching proficiency in higher level math in high school (Finkelstein et al., 2012). One study of California students concludes that placing all grade 8 students in algebra I, regardless of their preparation, sets up many students to fail (Williams et al., 2011). Recent longitudinal analysis of California statewide assessment data suggests that students who do not take algebra I in grade 8 but are successful in general math have a better chance of succeeding in algebra I when they wait until grade 9 (Liang, Heckman, & Abedi, 2012). Tables and figures are appended.

The information provided in this report shows how students perform when they repeat algebra I and how the level of improvement varies depending on initial course performance and the academic measure (course grades or CST scores). This information can help inform decisions and policies regarding whether and under what circumstances students should repeat the course. The study examined four research questions: (1) How many students repeat algebra I after taking it for the first time?; (2) How do student characteristics (such as race/ethnicity, gender, grade 7 math performance and initial algebra I performance) relate to the likelihood of repeating algebra I?; (3) How well do students perform when they repeat algebra I compared with the first time they took the course?; and (4) How does that difference in performance vary based on student characteristics? This study was conducted in the East Side Union High School District, located in Silicon Valley, California. The data for this study are from this district and five elementary school districts that feed into the East Side Union High School District: Alum Rock Union Elementary School District, Evergreen School District, Franklin-McKinley Elementary School District, Mt. Pleasant Elementary School District, and Oak Grove Elementary School District. The study examined algebra repetition rates and success rates among students based on their characteristics and initial performance. Data from the elementary feeder districts, which span from kindergarten through grade 8, were collected for the 2005/06-2008/09 school years, and data from the high school district, which spans from grade 9 through grade 12, were collected for the 2007/08-2011/12 school years. Student-level longitudinal data, collected from each district, include variables such as identification number, race/ethnicity, gender, math course name, final course letter grade received, math CST taken, CST scale score, and CST performance level. A logistic regression was used to identify the relationship between student characteristics and algebra repetition. Of the 3,400 students in the sample, 44.3 percent repeated algebra I. While grades and standardized test scores are the most common reasons to retake the course, other considerations may also factor in, such as parent preferences and teacher or counselor recommendations, depending on the school and district. The rates of repeating varied across student characteristics, with students in special education (69.6 percent), Hispanic students (61.1 percent), and English language learner students (56.7 percent) exhibiting the highest rates. Students' performance improved on average by approximately half a letter grade and a little less than a third of a CST performance level when they repeated algebra I, but the data showed variation in improvement levels among higher achieving students.

To raise math success rates in middle school, many schools and districts have implemented summer math programs designed to improve student preparation for algebra content in grade 8. However, little is known about the effectiveness of these programs. While students who participate typically experience learning gains, there is little rigorous evidence evaluating the effects of the programs on math achievement or readiness for algebra content. This study fills that void by rigorously examining the effects of one such summer program (the Elevate summer math program) on student achievement. In summer 2014, the Silicon Valley Education Foundation (SVEF), the research team, and several Silicon Valley school districts collaborated on a randomized controlled trial to assess the effects of the Elevate Math summer program on math achievement, algebra readiness, and attitudes toward math. The study focused on three primary questions: (1) What is the impact of the Elevate Math summer program on the math achievement and algebra readiness of rising grade 8 students?; (2) What is the impact of the Elevate Math summer program on math achievement in the math topic areas most closely aligned with the program's curriculum?; and (3) What is the impact of the Elevate Math summer program on the math interest and math self-efficacy of rising grade 8 students? The randomized controlled trial was conducted in summer 2014 at eight schools in six districts in California's Silicon Valley. Participating districts identified eligible students based on existing grade 6 California Standards Test (CST) data. The districts' enrollments range from 2,487 to 13,162, with an average of 9,426. The percentage of English learner students in each district ranges from 19% to 53%, with an average of 38%. Students were randomly assigned to a treatment group that received access to the program at the beginning of the summer or to a control group that received access to the program later in the summer. Math achievement was measured using the Mathematics Diagnostic Testing Project (MDTP) Algebra Readiness test, which was administered to the treatment and control groups on the first and last days of their participation in the summer program. The Elevate Math summer program significantly improved math achievement and algebra readiness among participating grade 7 students. The program improved the math achievement of the treatment group compared with the control group across several metrics. The Elevate Math summer program also had a positive, statistically significant effect on algebra readiness. Despite the Elevate Math summer program's effects, students' math achievement at the end of the program suggested that many students were still not ready for the algebra content in grade 8 math courses. There were no significant impacts on math interest or math self-efficacy. The estimated level of interest in math for the treatment group was higher than that of the control group, but the difference was not statistically significant. [The Silicon Valley Education Foundation and several Silicon Valley school districts collaborated on this study. For the Regional Educational Laboratory West report, "The Effects of the Elevate Math Summer Program on Math Achievement and Algebra Readiness. REL 2015-096" (2015), see ED558157.]

Math in Common® (MiC) is a five-year initiative that supports a formal network of 10 California school districts as they implement the Common Core State Standards in Mathematics (CCSS-M) across grades K-8. In spring 2015, WestEd administered surveys to understand the perspectives on Common Core State Standards-Mathematics (CCSS-M) implementation of teachers and administrators in eight California school districts participating in the Math in Common (MiC) initiative. Over 1,000 respondents replied to the survey, about some of the initial successes and challenges facing California educators attempting to put in place and support new--and what some consider revolutionary--ideas in U.S. mathematics education. The following are appended: (1) Research Methodology and Survey Sample, and (2) Sample Survey Item and Sub-Items.

The Expository Reading and Writing Course (ERWC) was developed by California State University (CSU) faculty and high school educators to improve the academic literacy of high school seniors, thereby reducing the need for students to enroll in remedial English courses upon entering college. This report, produced by Innovation Studies at WestEd, presents the findings of an independent evaluation of the ERWC funded by an Investing in Innovation (i3) development grant from the U.S. Department of Education. The study sample for the evaluation included more than 5,000 12th grade students in 24 high schools across nine California school districts in the 2013/14 school year. The authors of the report found that the ERWC has a statistically significant positive impact on student achievement. Results from an analysis of implementation fidelity are also presented, along with qualitative findings based on survey data from study participants. Appendixes include: (1) Statistical Power for Impact Estimates; (2) Data Collection Instruments to Measure Fidelity of Implementation; and (3) Rubric for Calculating Fidelity of Implementation for Each Component of the Expository Reading and Writing Course.

The Effects of the Elevate Math summer program on math achievement and algebra readiness: This randomized trial examined the effects of the Elevate Math summer program on math achievement and algebra readiness, as well as math interest and self-efficacy, among rising 8th grade students in California's Silicon Valley. The Elevate Math summer math program targets students who score in the range between "high basic" and "low proficient" on state math tests. It consists of 19 days of mathematics instruction, consisting of three hours per day in traditional classroom instruction and one hour per day using Khan Academy (a free online learning system). During summer 2014, students were randomly assigned to a treatment group that received access to the program at the beginning of the summer or to a control group that received access to the program later in the summer. End-of-program test scores and survey responses of students in the treatment group were compared with those of students in the control group prior to their exposure to the program. Treatment group students scored significantly higher than the control group (4 points or 0.7 standard deviation) on a test of algebra readiness. They were also significantly more likely (29 percent versus 12 percent) to reach achievement thresholds associated with success in algebra I. However, treatment and control groups did not show significant differences in terms of math interest or self-efficacy. The results show that the Elevate Math summer program can significantly improve student math achievement and algebra readiness; however, 70 percent of program participants were still not ready for algebra I content. This suggests that summer math programs such as Elevate Math's may be important tools for improving math achievement among rising eighth grade students, but most targeted students will need additional support in order to ensure success in algebra. The following are appended: (1) Data, outcomes, and methodology; and (2) Sensitivity analyses.

Through the Innovative Professional Development (iPD) Challenge, the Bill & Melinda Gates Foundation has invested in helping school districts and networks redesign their professional development systems to serve educators better and improve student performance. MDRC's evaluation of the iPD Challenge involves case studies and multiple rounds of surveys, document collection, and interviewing during the first three years of the initiative. This brief--the first in a series--introduces the case study component of the evaluation and presents some early findings from interviews with teachers regarding their beliefs about instructional improvement and useful learning opportunities.

Math in Common® (MiC) is a five-year initiative that supports a formal network of 10 California school districts as they implement the Common Core State Standards in Mathematics (CCSS-M) across grades K-8. This research brief explores how best to select or develop and use classroom observation systems in order to document instructional shifts and inform MiC school district improvement efforts. The report is organized into three main sections: (1) An exploration of what the research literature says about existing observation systems and several design considerations for successful observation systems; (2) A detailed discussion of several considerations of these findings for school districts as they implement observation systems in order to better track and understand how teachers are implementing the Common Core State Standards in Mathematics in their classrooms; (3) An annotated bibliography for recent publications on classroom observations that might be of interest to school districts interested in exploring these ideas in more depth. The following are appended: (1) Methods for Annotated Bibliography; and (2) Classroom Observation Analysis Tool. [For Volume 1, "Under Construction: Benchmark Assessments and Common Core Math Implementation in Grades K-8. Formative Evaluation Cycle Report for the Math in Common Initiative, Volume 1," see ED559581.]

Four years of math in high school, with a strong foundation in algebra that builds from middle school, is key to higher education access. Therefore, ensuring that middle and high school students succeed in math--and in algebra in particular--is an important issue for policy and practice. This research brief examines three recent Regional Educational Laboratory West (REL West) studies that shed important light on policies and practices that affect student success in algebra and preparation for higher-level math courses. Key findings included: (1) Middle school students who repeat algebra after initially failing the course have relatively low chances for becoming proficient in algebra; (2) The proportions of students who fail algebra and have to repeat the course are even higher among vulnerable populations, including low-income students, Hispanic students, and English language learners; and (3) Students must score well above the thresholds for proficiency in prior math courses in order to have even a 50-50 chance of success when placed into algebra.

Given the Common Core State Standards for Mathematics, California's history of math acceleration in the middle grades, and the concern for correct math course placement for all students, this brief examines patterns from the past to shed light on considerations for the future. The brief, written by WestEd's Tony Fong and Neal Finkelstein, presents the results of further analyses of data from 24 school districts in California that were previously analyzed in an earlier released report. The additional analysis focuses on the math experiences of minority students: When did minority students take algebra I, how often did they repeat the course, and what proportion of minority students reached calculus by grade 12? The answers to questions like these are critical for ensuring that all students are placed in appropriate courses to enable them to succeed in high school and college.

This REL West study explores the prevalence of students repeating Algebra I, who is most likely to repeat the course, and the level of improvement for students who repeat. Using six years of data from a cohort of 3,400 first-time seventh grade students in a California school district, authors found that 44 percent of students repeated algebra I. Overall, student performance improved on average by approximately one-half of a letter grade and a little less than one-third of a performance level on the CST when students repeated the course. But when the data was disaggregated based on initial performance in the class, higher-achieving students experienced variation in improvement levels. Repeating students who initially received average course grades of at least a "C" in Algebra I earned higher CST scores but lower course grades on average when they repeated the course. Students who initially scored Proficient on the Algebra I CST experienced increases in course grades but declines in CST scores on average when they repeated the course. Overall, these findings show that lower-performing students are likely to experience improvements in grades and CST scores when they repeat Algebra I, but that higher-performing students are likely to experience improvements on some measures of performance and declines on others when they repeat the course. The following are appended: (1) Detailed information about the dataset and analysis; and (2) The multivariate analysis, multiple repeaters, and summer repeaters.

Math in Common® (MiC) is a five-year initiative that supports a formal network of 10 California school districts as they implement the Common Core State Standards in mathematics (CCSS-M) across grades K-8. As the MiC initiative moves into its second year, one of the central activities that each of the districts is undergoing to support CCSS implementation involves putting in place new or revised student assessment processes or systems to better align with the new standards. This report examines the MiC districts' strategies and initial implementation efforts related to benchmark assessments used throughout the school year to assess student mastery of the CCSS-M and signal districts' progress. The report also highlights the range of approaches to developing and implementing benchmark assessments used across the MiC district community, in order to illuminate the challenges and headway that many districts in California, and perhaps beyond, are facing as they work to implement CCSS-M, specifically across grades K-8. [For Volume 2, see ED559582.]

Questions about how best to place students into appropriate middle grade math courses have been central to ongoing education policy and practice discussions in California and across the United States. Recent studies have shown that enrolling in algebra I in grade 8 works well for some students but backfires for others. This REL West report provides findings from a study of placements that were based on different test scores available for middle school students in California. Findings indicate that placement into grade 8 algebra I based solely on students' grade 6 California Standards Test (CST) proficiency status results in some students taking the course who have less than a 40 percent chance of succeeding. Using a higher cut point on the grade 6 CST scale score--as opposed to simply using CST proficiency status--would avoid placing students into grade 8 algebra I who have a lower than 50 percent chance of success, and would increase the overall success rate (from 69 to 75 percent) for students placed into grade 8 algebra I. Prediction accuracy is even higher using grade 7 CST scale scores (78 percent); however, grade 7 scale scores are typically not available until after initial algebra I placements are made. The study also finds that a newly available assessment of algebra readiness developed as part of the Math Diagnostic Testing Project (MDTP) makes a valuable contribution to decisions about algebra I placement. Placements based on the MDTP result in a success rate (77 percent) that is comparable to that of placements based on the grade 7 math CST. Furthermore, the MDTP test can be administered online at any time during the school year, and MDTP test results are available immediately after students take the test, whereas CST results are not available until the next school year. The following are appended: (1) Detailed study design; and (2) Model comparisons.

As California competes for jobs in an increasingly competitive global economy, the state faces a looming shortage of highly educated workers (PPIC, 2012). For a variety of reasons, the need for individuals with degrees in science, technology, engineering, and mathematics (STEM) is of particular concern. Nowhere is this more true than in the discipline of mathematics where understanding develops cumulatively, requiring that students master progressively more complex building-block concepts and skills in order to be successful in each next-higher-level course. Prior research confirms that success in high-level mathematics in high school is predictive of postsecondary success and careers in STEM fields. This study, funded by the S.D. Bechtel, Jr. Foundation and the Noyce Foundation, digs deeper into this middle- and high-school connection as it applies to STEM, in order to better understand the degree to which California students stay on the trajectory for STEM-related attendance eligibility at California's public universities and, if students veer off the trajectory, to better understand when and why. Thus, researchers examined math and science course-taking patterns for a representative cohort of some 24,000 California students who were enrolled in grade 7 in 2004/05 and stayed in their district through grade 12 in 2009/10. Although the study looked at students' science course-taking, this report focuses more tightly on the mathematics-related findings, partly because it turns out that course-taking patterns and performance in science are quite similar to, though less complex than, those in mathematics and partly because mathematical understanding, while not sufficient, is essential to student success in some key high school science courses, such as chemistry and physics. The math findings include: (1) Math performance in grade 7 is predictive of high-school math course-taking; (2) While the majority of students who achieved at least Proficient on their math CSTs are those who took algebra 1 in grade 8, geometry in grade 9, and algebra 2 in grade 10, in general this accelerated pathway does not support students who are not proficient in math in grade 7; (3) Many students repeat algebra, but few repeaters achieve proficiency on their second attempt; (4) Districts are keenly aware of poor student performance in mathematics but less aware of coursetaking patterns; and (5) Districts feel great urgency to improve algebra outcomes. Data Source is appended. (Contains 5 figures, 4 tables, and 24 footnotes.)

The primary purpose of this study is to assess student-level impacts of a problem-based instructional approach to high school economics. The curriculum approach examined here was designed to increase class participation and content knowledge for high school students who are learning economics. This study tests the effectiveness of Problem Based Economics, developed by the Buck Institute for Education, on student learning of economics content and problem-solving skills. Student achievement outcomes are of primary importance and are hypothesized to be mediated by changes in teacher knowledge and pedagogical practice. This study targeted high schools in both urban and rural areas and engaged teachers who committed to teach economics during the 2007/08 academic year. The study was implemented from summer 2007 through spring 2008 in high schools in Arizona and California. The findings of this study confirm that students benefited from the combination of the professional development program, ongoing support for teachers, and the Problem Based Economics curriculum. At the same time, teachers reported satisfaction with the Problem Based Economics materials. Teachers in the sample had, on average, fewer than three college-level economics courses; the opportunity to engage in a five-day workshop in economics instruction offers support to teachers who are interested in advancing their own professional development and increasing content knowledge. (Contains 1 table.) [For related report, "Effects of Problem Based Economics on High School Economics Instruction. Final Report. NCEE 2010-4002," see ED511228.]