Search

A new paradigm for machine translation has recently emerged: fine-tuning large language models (LLM) on parallel text has been shown to outperform dedicated translation systems trained in a supervised fashion on much larger amounts of parallel data (Xu et al., 2024a; Alves et al., 2024). However, it remains unclear whether this paradigm can enable massively multilingual machine translation or whether it requires fine-tuning dedicated models for a small number of language pairs. How does translation fine-tuning impact the MT capabilities of LLMs for zero-shot languages, zero-shot language pairs, and translation tasks that do not involve English? To address these questions, we conduct an extensive empirical evaluation of the translation quality of the TOWER family of language models (Alves et al., 2024) on 132 translation tasks from the multi-parallel FLORES-200 data. We find that translation fine-tuning improves translation quality even for zero-shot languages on average, but that the impact is uneven depending on the language pairs involved. These results call for further research to effectively enable massively multilingual translation with LLMs.

At its most effective, advising is a collaborative process between a student and an advisor designed to help the student realize their educational potential. Most postsecondary institutions have historically used advisors to help students select and register for courses, but postsecondary institutions are increasingly asking advisors to play an instrumental role in helping students progress through college. This summary introduces the recommendations and supporting evidence described in the full practice guide: (1) Intentionally design and deliver comprehensive, integrated advising that incorporates academic and non-academic supports to empower students to reach their educational goals; (2) Transform advising to focus on the development of sustained, personalized relationships with individual students throughout their college career; (3) Use mentoring and coaching to enhance comprehensive, integrated advising in ways that support students' achievement and progression; and (4) Embed positive incentives in intentionally designed advising structures to encourage student participation and continued engagement. [For the full practice guide, see ED615134.]

The COVID-19 pandemic brought the care crisis in the United States to the fore. Unprecedented closures of child care programs throughout 2020 placed a disproportionate burden on families, and mothers in particular. For parents enrolled in college or considering postsecondary enrollment at the time of the pandemic, the loss of child care services had an amplified effect-- threatening to derail their education goals. This brief draws on a series of interviews with campus child care directors and a review of data and relevant literature. It presents information for academic and institutional researchers, campus child care practitioners, college and university staff and administrators, and other stakeholders interested in understanding the availability and importance of campus child care services for student parent success. The brief makes the case for the importance of building rigorous evidence of the effect of campus child care access on student outcomes and discusses the factors related to campus child care center operations and institutional data collection that can, at times, undermine the ability to rigorously study the effectiveness of child care supports for student parents. The authors' findings are discussed in the context of evidence-based decision making to articulate the challenges and opportunities for conducting rigorous study. Given the timing of the feasibility study, the brief also provides a snapshot of campus child care providers' experiences during COVID-19. It concludes with recommendations to improve conditions for rigorous research on the role of campus child care in the outcomes of students with children.

In the quest to address income inequality and intergenerational poverty, it is critical to seek answers to the questions "What works?" and "For whom?" Education is a prime area in which to seek solutions, as one's level of education completed is positively correlated with income later in life. This connection contributes to a specific focus on schools to reduce inequality and a reliance on test scores as both a measure of intervention effectiveness and a proximal measure of young people's future outcomes. As Richard Murnane points out in a new essay, however, some efforts to improve and expand instruction have yielded improvements in test scores, but a narrow focus on this measure alone obscures a larger picture (2021). The findings suggest what many educators and parents know: Larger contexts affect school performance. To address income inequality and intergenerational poverty, education is just one instrument--and what is needed is a tool chest. Both research and data reflect the perspectives of the researcher and the data collector, as well as the societal context. Given the widespread fallacy that research and data are objective, this recognition will require intentional effort on the part of researchers. Ignoring these facts will only perpetuate recommendations for incomplete policy actions.

MDRC is pleased to have this opportunity to provide additional information for the consideration of Chairman Alexander, Ranking Member Murray, and members of the Committee on ways research can be used to improve the academic success of low-income college students. The goal of this additional testimony is to reiterate the second recommendation in the testimony of Dr. Lashawn Richburg-Hayes to "encourage innovation paired with research, especially rigorous evaluation." The testimony of Senators and witnesses at the August 5, 2015 hearing provided a number of promising ideas that are operationally feasible and whose outcomes seem encouraging, but that have not yet been subjected to rigorous evaluation based on WWC standards. In addition, other evaluated programs that were mentioned produced small effects at best on outcomes identified by the Institute of Education Sciences (IES) as key markers of collegiate progress. In a time of limited resources, building reliable evidence before making major investments is essential to the long-term goal of increasing college completion. This additional testimony is intended to emphasize the importance of building into the reauthorization of the Higher Education Act incentives for innovation, coupled with requirements for rigorous evaluation through randomized experiments whenever feasible. This testimony identifies opportunities in two areas--financial aid and student support services--and offers caution in the area of performance funding for higher education. A summary of recommendations is included.

One of the original purposes of student financial aid was to ensure fairer access to post-secondary education to those least able to afford it and to those traditionally underrepresented. Various federal and state programs were put in place to achieve this goal, including the federal Pell Grant and state aid programs. Yet policymakers and education leaders continue to grapple with how to boost college attendance and completion in an era when the number of college graduates is lagging behind demand and government resources are increasingly limited. Increasing the number of college graduates is particularly difficult given the continued rising cost of attending college and the failure of financial aid to keep pace. This report presents early findings from an evaluation of performance-based scholarships targeting college-bound high school seniors in California, referred to as the Cash for College Performance-Based Scholarship (CFC-PBS) Program. This program is one of six being studied as part of the Performance-Based Scholarship (PBS) Demonstration. Performance-based scholarships are need-based grants contingent on meeting certain academic benchmarks to receive payment--in this case, a half-time course load with a "C" or better grade point average (GPA). Unlike merit-based scholarships, there are no academic criteria to be eligible for the program at the outset. The CFC-PBS scholarship can be taken to any accredited, degree-granting college or university in the country. The goal of the CFC-PBS Program is to increase the amount of aid available for students while simultaneously providing an incentive for academic achievement. Using a random assignment design--the gold-standard methodology in program evaluation--MDRC assigned over 5,000 students to one of five program groups that were eligible for the incentive scholarship, to a group that was eligible for a scholarship without performance criteria, or to a control group that received their colleges' standard financial aid packages. This report analyzes three terms of follow-up data from the program in California. Overall, the findings in this report show the following: (1) The CFC-PBS Program was largely implemented as designed; (2) While few students received the entire amount of the scholarship for which they were eligible, most students received some funding. In this way, the design of the scholarship enabled more students to receive additional financial aid; (3) The CFC-PBS Program encouraged more students to matriculate, by about 5 percentage points above the control group rate of 84.4 percent. This increased matriculation largely occurred at community colleges. However, the program had only limited effects on persistence from semester to semester, and only for community college students.; (4) The program had positive impacts on academic success. These effects extend to numerous subgroups, such as males, females, and students of Latino ethnicity. There is strong evidence that the program affected students with lower high school GPAs more than students with higher high school GPAs; and (5) The cost to administer scholarships increased as performance requirements were added, but since on the whole, students received only a portion of the scholarship amount they were offered, the decrease in payments to students more than offset the increased cost of administration. All else being equal, scholarships with more performance requirements cost less than scholarships with fewer performance requirements. The following are appended: (1) Technical Appendix; (2) Supplementary Exhibits for Chapter 2; (3) Supplementary Exhibit for Chapter 3; (4) Supplementary Exhibits for Chapter 4; (5) Supplementary Information and Exhibits for Chapter 5; and (6) Supplementary Exhibits for Chapter 6. [This report was written with Drew McDermott.]

The Behavioral Interventions to Advance Self-Sufficiency (BIAS) project is the first major opportunity to use a behavioral economics lens to look at programs that serve poor and vulnerable people in the United States. Sponsored by the Office of Planning, Research and Evaluation of the Administration for Children and Families (ACF) within the U.S. Department of Health and Human Services, the BIAS project aims to learn how tools from behavioral economics can improve the well-being of individuals and families served by programs that ACF supports. Many human services programs require clients to make active decisions and follow a series of steps in order to reap a benefit--from deciding to apply, to completing forms, to arranging for child care. Program designers often assume that individuals will carefully consider options, make decisions that maximize their well-being, and diligently follow through. Behavioral economics, which combines insights from psychology and economics, may help explain why these assumptions are not always borne out. By describing work in three sites, this report illustrates how the BIAS project draws on the principles of behavioral economics to design solutions for ACF programs. In partnership with program administrators, the BIAS team uses a method called "behavioral diagnosis and design" to delve into problems that program administrators have identified, diagnose potential bottlenecks that may inhibit program performance, and identify areas where a relatively easy and low-cost, behaviorally informed change might improve outcomes. Working with the Texas Office of the Attorney General's Child Support Division, the BIAS team explored ways to increase the number of incarcerated noncustodial parents who apply for a modification of their child support order, with the goal of preventing further accrual of child support arrears. The project team also engaged with the Illinois Department of Human Services and one of its job search contractors to identify behavioral interventions that could help increase the rate of engagement in job search assistance among clients who receive Temporary Assistance for Needy Families. Finally, work with the National Domestic Violence Hotline (NDVH) focused on reducing the number of callers who reach a hold message but hang up before talking to an NDVH staffer. Working with the Texas Office of the Attorney General's Child Support Division, the BIAS team explored ways to increase the number of incarcerated noncustodial parents who apply for a modification of their child support order, with the goal of preventing further accrual of child support arrears. The project team also engaged with the Illinois Department of Human Services and one of its job search contractors to identify behavioral interventions that could help increase the rate of engagement in job search assistance among clients who receive Temporary Assistance for Needy Families. Finally, work with the National Domestic Violence Hotline (NDVH) focused on reducing the number of callers who reach a hold message but hang up before talking to an NDVH staffer. Key Findings include the following: (1) The behavioral diagnosis and design process provides a means of identifying and addressing key reasons that programs may not be performing to expectation. The process can uncover behavioral bottlenecks that are amenable to behavioral solutions. It may also identify structural issues, such as a need to hire more staff, which are often outside the scope of the BIAS project. (2) The diagnosis process encourages program designers to step back and examine multiple possible explanations for underperformance before embracing a particular theory or solution. This may improve the likelihood of success of any behavioral intervention. As the project moves forward, the BIAS team will continue to work with public officials to design and apply behavioral interventions in ACF program areas to generate new ways of tackling problems. Promising interventions will be tested using rigorous research designs, employing experimental methods to reliably determine the impact of an intervention. Future publications will report the impacts of these interventions. "Glossary of Select Behavioral Economics Terms" is appended.

The cost of attending college has risen sharply over the last 40 years. Although more credit and grant aid have been made available to students, there are still major gaps between aid and the cost of attendance for many students in the United States, all of whom are left to figure out whether they can afford the remaining costs associated with attending college. Given that people with similar financial means may make different decisions about what they can and cannot afford to pay for college, it is difficult to identify a universal definition of affordability. Nevertheless, examining college cost, student financial need, and progress toward degree completion trends in concert with one another can help illuminate patterns of when college is unaffordable. This paper will develop a definition of college affordability by examining the literature on student financial need and student aid interventions as well as student data collected by MDRC to identify important indicators of college affordability and the lack thereof, which may ultimately help or impede college persistence and graduation. This paper provides a review of the literature on the relationships between unmet financial need and progress toward degree completion as well as a review of the findings from studies that show the effects of financial aid intervention on student persistence and completion. The paper then examines MDRC's experimental data from the Performance-Based Scholarship (PBS) Demonstration, a study spanning six states around the United States. Ultimately, this paper offers several puzzle pieces that fit together to help identify the students for whom college may not be affordable. It also offers recommendations for how policymakers in various sectors can respond to the affordability issues faced by the low-income students they serve and want to help to achieve success in college. [This paper was written with Melvin Gutierrez and Jessica Gingrich.]

In today's economy, employers' demand for an educated workforce is steadily rising. Policymakers, education leaders, and communities across the country recognize the need to improve college attendance and success, but are constrained by the current budgetary environment. Meanwhile students themselves face mounting college costs, and financial aid has not been able to keep pace. Performance-based scholarships aim to help reduce the financial burdens on low-income college students while providing incentives for good academic progress. Students are generally paid at multiple points during the semester if they earn a certain number of credits with a "C" average or better. MDRC is currently evaluating performance-based scholarship programs in six states. Interim results suggest that these programs do improve students' performance and increase the number of credits they earn, and in some states where data are available, they also appear to reduce student debt. In one location, the program also increased the proportion of students earning a degree. They work in a wide range of institutions and for a wide variety of students, including those normally at risk of performing poorly.

Low-income students are at particular risk of not persisting to earn a certificate or degree, often because of competing priorities, financial pressures, and inadequate preparation for college. One form of financial assistance designed explicitly to reward students' academic success is a performance-based scholarship, paid contingent on attaining academic benchmarks. MDRC's 2009 Opening Doors study of performance-based scholarships in Louisiana showed a positive effect on students' credit accumulation, grades, and persistence in college. In 2008, MDRC launched the Performance-Based Scholarship Demonstration, a study in six states that is testing whether different configurations of these scholarships in different locations and among different types of students would be equally effective in improving their outcomes. This brief summarizes findings in Louisiana, Ohio, New York, and New Mexico, which show modest but positive effects on important markers of academic progress. (Contains 2 figures, 1 table, and 11 notes.)

The intention of the program is that conditioning additional financial aid on certain behaviors will lead students to increase their effort towards their studies, which in turn will lead them to perform better in their classes in the short-term and progress through their degree requirements at a quicker rate in the medium-term. The program is hypothesized to result in increased enrollment in subsequent terms, increased credits attempted and earned, and increased financial aid. Increases in these short-term education outcomes may then lead to changes in medium- and long-term educational outcomes, including year-to-year persistence, total cumulative credits earned, as well as graduation or transfer to another postsecondary institution. The proposed presentation will focus on the primary research question: What is the impact of performance-based scholarships (in various settings with different types of students) on short-term academic outcomes as well as medium- and long-term academic outcomes? The authors will present current findings on these outcomes, across the six sites in their demonstration. (Contains 4 tables and 1 footnote.)