Search

Adapting the classic line from the 1967 film Cool Hand Luke, the title is meant to convey that implementation science (IS), like other fields, has not been embracing replication studies, which is a key component to the open science movement. The purpose of this article is to review what is known about replication of implementation trials and identify the gaps and next steps to continue increasing the transparency, openness, and replicability of implementation research. After presenting an overview of study replication and how it is a key component of open science, the article will examine how replication of implementation studies has (or more accurately has not) been approached in IS. As will be discussed, replication in IS shares some challenges with studies that attempt to replicate interventions, but also presents unique challenges. This article discusses different types of replications (e.g., direct vs. conceptual) and how they can benefit the field of IS. The article then presents a specific example of an implementation strategy called Getting To Outcomes© to describe how to design a replication study and interpret the results. The article ends with multiple options implementation scientists could consider to improve the likelihood and quality of replication studies. The discussion also envisions how implementation science can enable researchers and practitioners to work together in real-world contexts to encourage wide replication of implementation studies and advance the goal of improving public health.

Implementation support can improve outcomes of evidence-based programs (EBP) for adolescents, but with a cost. To assist in determining whether this cost is worthwhile, this study estimated the cost of adding Getting To Outcomes© (GTO) implementation support to a teen pregnancy and sexually transmitted infection prevention EBP called Making Proud Choices (MPC) in 32 Boys and Girls Clubs (BGCs) in Alabama and Georgia. Enhancing Quality Interventions Promoting Healthy Sexuality (EQUIPS) was a 2-year, cluster-randomized controlled trial comparing MPC with MPC + GTO. We used micro-costing to estimate costs and captured MPC and GTO time from activity logs completed by GTO staff. Key resource use and cost components were compared between the randomized groups, years, and states (to capture different community site circumstances) using 2-sample t tests. There were no significant differences between randomized groups in attendees per site, resource use, or costs for either year. However, there were significant differences between states. Adding GTO to MPC increased the societal costs per attendee from $67 to $144 (2015 US dollars) in Georgia and from $106 to $314 in Alabama. The higher Alabama cost was due to longer travel distances and to more BGC staff time spent on GTO in that state. GTO also improved adherence, classroom delivery, and condom-use intentions more in Alabama youth. Thus, Alabama’s GTO-related BGC staff time costs may be better estimates of effective GTO. If teen childbearing costs taxpayers approximately $20,000 per teen birth, adding GTO to MPC would be worthwhile to society if it prevented one more teen birth per 140 attendees than MPC alone. Trial registration. ClinicalTrials.gov , NCT01818791. Registered March 26, 2013, https://clinicaltrials.gov/ct2/show/NCT01818791?term=NCT01818791&draw=2&rank=1

Problematic rates of alcohol, e-cigarette, and other drug use among US adolescents highlight the need for effective implementation of evidence-based programs (EBPs), yet schools and community organizations have great difficulty implementing and sustaining EBPs. Although a growing number of studies show that implementation support interventions can improve EBP implementation, the literature on how to improve sustainability through implementation support is limited. This randomized controlled trial advances the literature by testing the effects of one such implementation intervention—Getting To Outcomes (GTO)—on sustainability of CHOICE, an after-school EBP for preventing substance use among middle-school students. CHOICE implementation was tracked for 2 years after GTO support ended across 29 Boys and Girls Club sites in the greater Los Angeles area. Predictors of sustainability were identified for a set of key tasks targeted by the GTO approach (e.g., goal setting, evaluation, collectively called “GTO performance”) and for CHOICE fidelity using a series of path models. One year after GTO support ended, we found no differences between GTO and control sites on CHOICE fidelity. GTO performance was also similar between groups; however, GTO sites were superior in conducting evaluation. Better GTO performance predicted better CHOICE fidelity. Two years after GTO support ended, GTO sites were significantly more likely to sustain CHOICE implementation when compared with control sites. This study suggests that using an implementation support intervention like GTO can help low-resource settings continue to sustain their EBP implementation to help them get the most out of their investment. ClinicalTrials.gov Identifier: NCT02135991.

Costs of supporting prevention program implementation are not well known. This study estimates the societal costs of implementing CHOICE, a voluntary after-school alcohol and other drug prevention program for adolescents, in Boys and Girls Clubs (BGCs) across Southern California with and without an implementation support system called Getting To Outcomes© (GTO). This article uses micro-costing methods to estimate the cost of the CHOICE program and GTO support. Labor and expense data were obtained from logs kept by the BGC staff and by the GTO technical assistance (TA) staff, and staff time was valued based on Bureau of Labor Statistics estimates. From the societal perspective, the cost of implementing CHOICE at BGCs over the 2-year study period was $27 per attendee when CHOICE was offered by itself (all costs incurred by the BGCs) and $177 per attendee when CHOICE was offered with GTO implementation support ($67 cost to the BGCs; $110 to the entity funding GTO). These results were most sensitive to assumptions as to the number of times CHOICE was offered per year. Adding GTO implementation support to CHOICE increased the cost per attendee by approximately $150. For this additional cost, there was evidence that the CHOICE program was offered with more fidelity and offered more often after the 2-year intervention ended. If the long-term benefits of this better and continued implementation are found to exceed these additional costs, GTO could be an attractive structure to support evidence-based substance misuse prevention programs. Trial Registration. This project is registered at ClinicalTrials.gov with number NCT02135991 (URL: https://clinicaltrials.gov/show/NCT02135991). The trial was registered May 12, 2014.

Women military veterans are at increased risk of suicide compared to non-veterans, but little is known about the mental health service preferences and needs of women veterans in crisis. This study used qualitative, secondary source key informant interviews to ascertain the experiences of women veterans in crisis from 54 responders working at the Veterans Crisis Line. Responders indicated that women veterans reported different experiences with Veterans Administration (VA) and non-VA care, though drivers of satisfaction or dissatisfaction were similar. Availability of specialty care, sensitivity to veterans' issues or Military Sexual Trauma, strong provider relationships, and continuity of care contributed to satisfaction; lengthy appointment wait times, limited service options, and insensitivity to veterans' issues contributed to dissatisfaction. Responders suggested that barriers limiting VA access for women veterans are perceived as similar to non-VA care. Findings suggest that caller experiences with providers drive satisfaction with VA and non-VA mental health services.

Increasing attention is being paid to suicide among law enforcement officers, and how the agencies that employ these officers could prevent such deaths. This study presents the results of a national survey of U.S. law enforcement agencies’ strategies for preventing officer suicide. We invited 177 agencies from across the United States to be interviewed, and 110 agreed to participate in qualitative interviews. Agencies were grouped into one of four categories based on the services they offered. Agencies offered minimal (a municipal employee assistance program), basic (mental health, critical incident response procedures, and training), proactive (in-house mental health care, embedded chaplains, substance misuse programs, peer support, screening, or health and wellness programs), and integrated services (integration of services into day-to-day operations). The results indicate that many U.S. law enforcement agencies are engaged in efforts to promote officer wellness and prevent suicide. Officers’ perceptions of confidentiality may inhibit the use of in-house or contracted mental health services, while a weak or inconclusive evidence-base raises questions about common approaches like peer support or critical incident stress debriefing.

This study assesses how perceptions of school climate and four mediating factors (school connectedness, peer attachment, assertiveness, and empathy) influence reports of bullying behaviors among 2,834 students in 14 middle schools. Results revealed that students in positive school climates reported experiencing fewer physical, emotional, and cyberbullying behaviors. They also reported greater levels of school connectedness, peer attachment, assertiveness, and empathy, which in turn helped explain the influence of perceived school climate on bullying. In addition, the greater levels of empathy that students reported, the more likely they were to report being bullied. These results highlight the role that perceptions of school climate can play in influencing bullying and underscore the importance of mediating factors as schools work to track and improve school climate.

Abstract. Background: Crisis support services have historically been offered by phone-based suicide prevention hotlines, but are increasingly becoming available through alternative modalities, including Internet chat and text messaging. Aims: To better understand differences in the use of phone and chat/text services. Method: We conducted semistructured interviews with call responders at the Veterans Crisis Line who utilize multimodal methods to respond to veterans in crisis. Results: Responders indicated that veterans may access the chat/text service primarily for reasons that included a desire for anonymity and possible inability to use the phone. Responders were divided on whether callers and chatters presented with different issues or risk of suicide; however, they suggested that veterans frequently use chat/text to make their first contact with mental health services. Limitations: We spoke with call responders, not the veterans themselves. Additionally, as this is qualitative research, applicability to other settings may be limited. Conclusion: While new platforms offer promise, participants also indicated that chat services can supplement phone lines, but not replace them.

This content area module (CAM) for Air Force sleep health promotion is a companion volume to the Getting To Outcomes Operations Guide for U.S. Air Force Community Action Teams. It includes guidance, resources, and examples of completed tools for Community Action Teams that have an interest in planning, implementing, and evaluating sleep health promotion initiatives as part of a Community Action Plan.

BACKGROUND: Implementation support interventions have helped organizations implement programs with quality and obtain intended outcomes. For example, a recent randomized controlled trial called Preparing to Run Effective Programs (PREP) showed that an implementation support intervention called Getting To Outcomes (GTO) improved implementation of an evidence-based substance use prevention program (CHOICE) run in community-based settings. However, more information is needed on how these interventions affect organizational barriers and facilitators of implementation. This paper aims to identify differences in implementation facilitators and barriers in sites conducting a substance use prevention program with and without GTO. METHODS: PREP is a cluster-randomized controlled trial testing GTO, a two-year implementation support intervention, in Boys & Girls Clubs. The trial compares 15 Boys & Girls Club sites implementing CHOICE (Control group), a five-session evidence-based alcohol and drug prevention program, with 14 Boys & Girls Club sites implementing CHOICE supported by GTO (Intervention group). All sites received CHOICE training. Intervention sites also received GTO manuals, training, and onsite technical assistance to help practitioners complete implementation best practices specified by GTO (i.e., GTO Steps). During the first year, technical assistance providers helped the Intervention group adopt, plan, and deliver CHOICE, and then evaluate and make quality improvements to CHOICE implementation using feedback reports summarizing their data. Following the second year of CHOICE and GTO implementation, all sites participated in semi-structured interviews to identify barriers and facilitators to CHOICE implementation using the Consolidated Framework for Implementation Research (CFIR). This paper assesses the extent to which these facilitators and barriers differed between intervention and control group. RESULTS: Intervention sites had significantly higher average ratings than Control sites for two constructs from the CFIR process domain: Planning and Reflecting and Evaluating. At the same time, Intervention sites had significantly lower ratings on the Culture and Available Resources constructs. CONCLUSIONS: Findings suggest that strong planning, evaluation, and reflection—likely improved with GTO support—can facilitate implementation even in the face of a less desirable implementation climate. These findings highlight that implementation support, such as GTO, is likely to help low-resourced community-based organizations improve program delivery through a focus on implementation processes. TRIAL REGISTRATION. This project is registered at ClinicalTrials.gov with number (URL: https://clinicaltrials.gov/show/NCT02135991). The trial was first registered May 12, 2014.

Abstract. Background: Suicide hotlines are commonly used to prevent suicides, although centers vary with respect to their management and operations. Aims: To describe variability across suicide prevention hotlines. Method: Live monitoring of 241 calls was conducted at 10 suicide prevention hotlines in California. Results: Call centers are similar with respect to caller characteristics and the concerns callers raise during their calls. The proportion of callers at risk for suicide varied from 3 to 57%. Compliance with asking about current suicide risk, past ideation, and past attempts also ranged considerably. Callers to centers that were part of the National Suicide Prevention Lifeline (NSPL) were more likely to experience reduced distress than callers to centers that were not part of the NSPL. Conclusion: Because callers do not generally choose the center or responder that will take their call, it is critical to promote quality across call centers and minimize the variability that currently exists. Accrediting bodies, funders, and crisis centers should require that centers continuously monitor calls to ensure and improve call quality.

Restorative Practices in schools lack rigorous evaluation studies. As an example of rigorous school-based research, this paper describes the first randomized control trial of restorative practices to date, the Study of Restorative Practices. It is a 5-year, cluster-randomized controlled trial (RCT) of the Restorative Practices Intervention (RPI) in 14 middle schools in Maine to assess whether RPI impacts both positive developmental outcomes and problem behaviors and whether the effects persist during the transition from middle to high school. The two-year RPI intervention began in the 2014-2015 school year. The study's rationale and theoretical concerns are discussed along with methodological concerns including teacher professional development. The theoretical rationale and description of the methods from this study may be useful to others conducting rigorous research and evaluation in this area.

This study fills a gap in research on multi-level school-based approaches to promoting positive youth development and reducing bullying, in particular cyberbullying, among middle school youth. The study evaluates the Restorative Practices Intervention, a novel whole-school intervention designed to build a supportive environment through the use of 11 restorative practices (e.g., communication approaches that aim to build stronger bonds among leadership, staff, and students such as using “I” statements, encouraging students to express their feelings) that had only quasi-experimental evidence prior to this study. Studying multilevel (e.g., individual, peer group, school) approaches like the Restorative Practices Intervention is important because they are hypothesized to address a more complex interaction of risk factors than single level efforts, which are more common. Baseline and two-year post survey data was collected from 2,771 students at 13 middle schools evenly split between grades 6 (48 percent) and 7 (52 percent), and primarily ages 11 (38 percent) or 12 (41 percent). Gender was evenly split (51 percent male), and 92 percent of students were white. The intervention did not yield significant changes in the treatment schools. However, student self-reported experience with restorative practices significantly predicted improved school climate and connectedness, peer attachment, and social skills, and reduced cyberbullying victimization. While more work is needed on how interventions can reliably produce restorative experiences, this study suggests that the restorative model can be useful in promoting positive behaviors and addressing bullying.

In this chapter, we overview the current state of dissemination of evidence-based substance-use prevention programs (EBPs) in the substance-use prevention field from an implementation science perspective. Implementation science is defined as a scientific study of methods to promote the systematic uptake of research findings and other EBPs into routine practice to improve the quality and effectiveness of health services and care. To improve quality and effectiveness of EBP implementation a helpful set of models and frameworks are needed that are specific to supporting dissemination and quality implementation and that can be applied across the wide diversity of EBPs. We begin with an overview of how prevalent substance-use prevention EBPs are in real-world settings. Then we discuss factors that can impede and facilitate the dissemination and use of EBPs. That review will be anchored within a prominent implementation science framework called the Consolidated Framework for Implementation Research (CFIR), whose subdomains could be considered risk and protective factors for quality implementation. Then we review both researcher- and government-led efforts to improve the dissemination of EBPs guided by the Interactive Systems Framework (ISF) for dissemination and implementation of preventive innovations. The ISF is used because it is a meta-framework and takes a broad overarching and synthesis approach to implementation science—much like we are doing in this chapter. We end with a discussion of implications for the future dissemination and quality implementation of substance-use prevention EBPs, including presenting new ideas, such as social innovation bonds, for supporting implementation.

Background Community organizations can have difficulty implementing evidence-based prevention programs. More research is needed on implementation support interventions designed to help these organizations implement programs with quality. Methods Preparing to Run Effective Programs (PREP) is a randomized controlled trial testing Getting To Outcomes (GTO), a 2-year implementation support intervention. It compares 15 Boys and Girls Club sites implementing CHOICE (control group), a five-session evidence-based alcohol and drug prevention program, with 14 similar sites implementing CHOICE supported by GTO (intervention group). PREP replicates a previous GTO study that had the same design, but featured a teen pregnancy prevention program instead. All sites received typical CHOICE training. Fourteen intervention sites received GTO manuals, training, and onsite technical assistance to help practitioners complete implementation best practices specified by GTO (i.e., GTO steps). During the first year, technical assistance providers helped the intervention group adopt, plan, and deliver CHOICE. Then, this group was trained on evaluation and quality improvement steps of GTO using feedback reports summarizing their own data, which yielded revised plans for subsequent implementation of CHOICE. This paper presents results regarding GTO’s impact on CHOICE fidelity (adherence, quality of delivery, dosage) and the proximal outcomes of the youth participants (aged 10–14)—attitudes and intentions regarding cigarettes, alcohol, and marijuana use. Fidelity was assessed at all sites by observer ratings and attendance logs. Proximal outcomes were assessed via survey at baseline, 3, and 6 months. Results After 1 year, fidelity and proximal outcomes were similar between Intervention and control groups. After 2 years (which included GTO quality improvement activities that took place between years 1 and 2), intervention sites had higher ratings of CHOICE adherence and quality of delivery (dosage remained similar). Proximal outcomes did not differ between groups in either year, although there was universally high endorsement of prosocial responses to those outcomes from the start. Conclusions Findings suggest that systematic implementation support provided by GTO can help community organizations achieve better fidelity. Findings replicate the implementation results from a previous GTO study using the same design, but with a different evidence-based program and different fidelity measures. Although proximal outcomes did not change, in large part due to ceiling effects, the implementation findings suggest GTO can support a variety of programs. Trial registration This project is registered at ClinicalTrials.gov with number NCT02135991. The trial was first registered on May 12, 2014.

OBJECTIVES: The U.S. has high teen pregnancy rates compared to other developed nations. Many community-based organizations need assistance conducting evidence-based teen pregnancy prevention programs (EBPs) appropriately. This study evaluated the impact of an implementation support intervention called Getting To Outcomes (GTO) designed to help such organizations. METHODS: This cluster randomized controlled trial compared 16 Boys & Girls Clubs (BGCs) implementing a teen pregnancy prevention EBP called Making Proud Choices for two years, with 16 BGCs implementing MPC augmented with GTO training, tools, and technical assistance. Participating middle school youth were compared on proximal outcomes (knowledge, attitudes, and intentions about sex and condoms from Baseline to Post) and sexual behaviors (frequency of sex and condom use, from Baseline to 6-month Follow-up). RESULTS: In Year 1, there were no significant effects of GTO for any proximal outcome. After GTO-stimulated quality improvement in Year 2, the GTO group improved significantly more on condom attitudes and use intentions. Frequency of sex and condom use did not differ between the two groups in either year; however base rates of these behaviors in the sample were very low. CONCLUSIONS: Findings suggest that in typical community-based settings, detailed manuals and training common to structured EBPs may be sufficient to yield some improvement in key proximal outcomes, but that more systematic implementation support is needed to achieve greater improvement in these outcomes. Using GTO with many communities, as currently supported by various federal agencies, could yield public health impact via improvements in condom attitudes and use intentions.

The RAND Corporation and the Los Angeles County Department of Mental Health developed surveys to monitor prevention program outcomes. This webinar provides training guidance for data collectors and data entry personnel using these surveys to evaluate prevention program outcomes in compliance with the Mental Health Services Oversight and Accountability Commission regulations for all Mental Health Services Act prevention programs.

In this document, RAND researchers describe how they developed a tool to help the Los Angeles County Department of Mental Health comply with regulations to report on outcomes for prevention mental health programs.

Provides preliminary advice about which of the California Mental Health Services Authority's Prevention and Early Intervention activities seem most valuable to sustain or, in some cases, enhance.

There continues to be a gap in prevention outcomes achieved in research trials versus those achieved in "real-world" practice. This article reports interim findings from a randomized controlled trial evaluating Assets-Getting To Outcomes (AGTO), a two-year intervention designed to build prevention practitioners' capacity to implement positive youth development-oriented practices in 12 community coalitions in Maine. A survey of coalition members was used to assess change on individual practitioners' prevention capacity between baseline and one year later. Structured interviews with 32 program directors (16 in the intervention group and 16 in the control group) were used to assess changes in programs' prevention practices during the same time period. Change in prevention capacity over time did not differ significantly between the intervention and control groups. However, in secondary analyses of only those assigned to the AGTO intervention, users showed greater improvement in their self-efficacy to conduct Assets-based programming and increases in the frequency with which they engaged in AGTO behaviors, whereas among non-users, self-efficacy to conduct Assets-based programming declined. Interview ratings showed improvement in several key areas of performance among intervention programs. Improvement was associated with the number of technical assistance hours received. These results suggest that, after one year, AGTO is beginning to improve the capacity of community practitioners who make use of it.

Translational research is expanding, in part, because Evidence-Based Programs or Practices (EBPs) are not adopted in many medical domains. However, little translational research exists on EBPs that are prevention programs delivered in non-clinical, community-based settings. These organizations often have low capacity, which undermines implementation quality and outcomes. Rigorous translational research is needed in these settings so within a single study, capacity, implementation quality, and outcomes are measured and links between them tested. This paper overviews the study Enhancing Quality Interventions Promoting Healthy Sexuality (EQUIPS), which tests how well a community-based setting (Boys & Girls Clubs) conducts an EBP called Making Proud Choices that aims to prevent teen pregnancy and sexually transmitted infections, with and without an implementation support intervention called Getting To Outcomes®. The study design is novel as it assesses: Getting To Outcomes’ impact on capacity, implementation quality, and outcomes simultaneously and in both study conditions; will assess sustainability by measuring capacity and fidelity a year after the Getting To Outcomes support ends; and will operate on a large scale similar to many national initiatives. Many studies have not incorporated all these elements and thus EQUIPS could serve as a model for translational research in many domains.

This article describes the methods for developing and testing a silent monitoring protocol for California suicide crisis call centers to evaluate call content and quality, document staff strengths and weaknesses, and identify areas for further training.

This guide will lead the user through the ten steps of Getting To Outcomes (GTO) to assist with the planning, implementation, evaluation, continuous quality improvement, and sustainability of a community emergency preparedness program. Each chapter in the guide contains blank tools for that chapter's GTO step, completed examples of each tool, and detailed instructions.

This handbook provides guidance to practitioners and organizations interested in evaluating mental health prevention and early intervention programs.

Alcohol merchants (N=331) completed a cross-sectional survey assessing their attitudes and beliefs about underage drinking, its likely consequences, requirements for responsible beverage service (RBS) training, and performance of RBS practices and checking IDs. Merchants requiring more rigorous RBS training (i.e., state-approved versus in-house or none) have stronger beliefs that outlets who sell to minors will get cited and that their employees know RBS practices. Also, merchants who engage in more RBS practices require more rigorous RBS training, and believe more strongly that outlets who sell to minors are more likely to face, and deserve, stricter sanctions. Merchants who check IDs more strictly conduct more RBS practices and believe more strongly that underage drinking is serious and will result in stronger consequences if caught selling to minors. These findings about the attitudes, practices, and enforcement of alcohol merchants suggests ways communities can better target their limited resources to prevent underage drinking.

Background Research is needed to evaluate the impact of implementation support interventions over and above typical efforts by community settings to deploy evidence-based prevention programs. Methods Enhancing Quality Interventions Promoting Healthy Sexuality is a randomized controlled trial testing Getting To Outcomes (GTO), a 2-year implementation support intervention. It compares 16 Boys and Girls Club sites implementing Making Proud Choices (MPC, control group), a structured teen pregnancy prevention evidence-based program with 16 similar sites implementing MPC augmented with GTO (intervention group). All sites received training and manuals typical for MPC. GTO has its own manuals, training, and onsite technical assistance (TA) to help practitioners complete key programming practices specified by GTO. During the first year, TA providers helped the intervention group adopt, plan, and deliver MPC. This group then received training on the evaluation and quality improvement steps of GTO, including feedback reports summarizing their data, which were used in a TA-facilitated quality improvement process that yielded revised plans for the second MPC implementation. This paper presents results regarding GTO’s impact on performance of the sites (i.e., how well key programming practices were carried out), fidelity of MPC implementation, and the relationship between amount of TA support, performance, and fidelity. Performance was measured using ratings made from a standardized, structured interview conducted with participating staff at all 32 Boys and Girls Clubs sites after the first and second years of MPC implementation. Multiple elements of fidelity (adherence, classroom delivery, dosage) were assessed at all sites by observer ratings and attendance logs. Results After 2 years, the intervention sites had higher ratings of performance, adherence, and classroom delivery (dosage remained similar). Higher performance predicted greater adherence in both years. Conclusions These findings suggest that in typical community-based settings, manuals and training common to structured EBPs may be sufficient to yield low levels of performance and moderate levels of fidelity but that systematic implementation support is needed to achieve high levels of performance and fidelity. Trial registration ClinicalTrials.gov, NCT01818791

This article aims to describe continuous quality improvement (CQI) for substance abuse prevention and treatment programs in a community-based organization setting.CQI (e.g., plan-do-study-act cycles (PDSA)) applied in healthcare and industry was adapted for substance abuse prevention and treatment programs in a community setting. The authors assessed the resources needed, acceptability and CQI feasibility for ten programs by evaluating CQI training workshops with program staff and a series of three qualitative interviews over a nine-month implementation period with program participants. The CQI activities, PDSA cycle progress, effort, enthusiasm, benefits and challenges were examined.Results indicated that CQI was feasible and acceptable for community-based substance abuse prevention and treatment programs; however, some notable resource challenges remain. Future studies should examine CQI impact on service quality and intended program outcomes.The study was conducted on a small number of programs. It did not assess CQI impact on service quality and intended program outcomes. Practical implications- This project shows that it is feasible to adapt CQI techniques and processes for community-based programs substance abuse prevention and treatment programs. These techniques may help community-based program managers to improve service quality and achieve program outcomes.This is one of the first studies to adapt traditional CQI techniques for community-based settings delivering substance abuse prevention and treatment programs.

Prevention support systems (PSSs) are designed to help communities implement evidence-based practices (EBPs). Little is known about the factors that influence their adoption. In this article, we examined adoption of a PSS for substance abuse prevention called Getting To Outcomes (GTO) s among staff in two community coalitions with varying levels of exposure to it over a 2-year period. A multistage regression model was used to determine the relative predictive power of organizational, innovationrelated, and practitioner characteristics on adoption. Findings indicate that GTO adoption occurred through increased exposure, especially via staff training. Perceptions of the system’s complexity moderated this effect. Individuals with high GTO exposure were less likely to adopt it if they perceived it as complex. Other organizational, innovation-related, and practitioner characteristics did not predict adoption. Results from this study are useful for understanding the factors associated with PSS adoption and suggest ways to increase their use in typical delivery settings. � C 2009 Wiley

Demands on community-based prevention programs for performance accountability and positive outcomes are ever increasing in the face of constrained resources. Relatively little is known about how technical assistance (TA) should be structured to benefit community-based organizations and to lead to better outcomes. In this study, data from multiple sources were used to describe an effective TA model designed to improve the capacity of community-based organizations to plan, implement, and evaluate prevention programming. This article is the first of its kind to provide detailed analyses of the TA delivered to community-based organizations to build substance abuse prevention capacity. The results of this study describe the range of TA services provided and the importance of two-way communication between the TA provider and recipient. TA recipients reported high satisfaction and an improved understanding of targeted TA activities. However, achieving these benefits requires significant program staff time, and not all skills were successfully transferred. Results from this study suggest how TA may be structured to be effective in supporting quality prevention programming in community settings.

Substance abuse treatment research is complicated by the pervasive problem of non-ignorable missing data – i.e., the occurrence of the missing data is related to the unobserved outcomes. Missing data frequently arise due to early client departure from treatment. Pattern-mixture models (PMMs) are often employed in such situations to jointly model the outcome and the missing data mechanism. PMMs require non-testable assumptions to identify model parameters. Several approaches to parameter identification have therefore been explored for longitudinal modeling of continuous outcomes, and informative priors have been developed in other contexts. In this paper, we describe an expert interview conducted with five substance abuse treatment clinical experts who have familiarity with the Therapeutic Community modality of substance abuse treatment and with treatment process scores collected using the Dimensions of Change Instrument. The goal of the interviews was to obtain expert opinion about the rate of change in continuous client-level treatment process scores for clients who leave before completing two assessments and whose rate of change (slope) in treatment process scores is unidentified by the data. We find that the experts’ opinions differed dramatically from widely-utilized assumptions used to identify parameters in the PMM. Further, subjective prior assessment allows one to properly address the uncertainty inherent in the subjective decisions required to identify parameters in the PMM and to measure their effect on conclusions drawn from the analysis.

First-week Dimensions of Change Instrument (DCI) assessments from a cohort of 519 adults entering six therapeutic communities (TCs) were used to predict treatment retention and outcomes. A more positive first-week response to TC social processes (Community Responsibility; Resident Sharing, Support, and Enthusiasm; Group Process; and Clarity and Safety) and to one TC personal-development process (Positive Self-Attitude and Commitment to Abstinence) predicted retention for the first month. Improvement in Clarity and Safety and Resident Sharing, Support, and Enthusiasm scores at 30 days predicted retention in treatment for 3, 6, and 9 months. In multivariate analyses available to a subset of the entry cohort, a longer tenure in treatment was a robust predictor of posttreatment outcomes. First-week DCI scores on the community process scales predicted posttreatment alcohol-and-other-drug abstinence and functioning.

Getting To Outcomes (GTO) is a ten-step process that guides community organizations through the key tasks needed to make any prevention program a success. Based on the GTO ten steps, this guide is designed to help organizations assess the need for and select a teen pregnancy prevention program, identify specific goals and outcomes, create a detailed plan, identify and use evaluation measures, and apply data for program improvement.

This article evaluates the landscape in which suicide prevention hotlines operate, focusing on California but with nationwide implications, and provides recommendations to ensure that callers at risk of suicide seek and receive the help they need.

This study examined associations of a new multidimensional measure of therapeutic community (TC) treatment process, the Dimensions of Change Instrument (DCI), with treatment retention (n = 397) and posttreatment outcomes (n = 207) among adolescent residential clients. Adolescents who increased during the first 30 days in treatment on three of the eight DCI factors (Positive Self-Attitude and Commitment to Abstinence; Problem Recognition; Social Network) were more likely to stay in treatment for 90 days or more. Although change in DCI factor scores was not directly predictive of posttreatment outcomes, staying in treatment for 90 days or more increased the likelihood of attending 12-step meetings and having a 12-step sponsor after leaving treatment. Several demographic and pretreatment characteristics also predicted retention, posttreatment outcomes, or both. Much remains to be learned about how the TC treatment process produces positive outcomes for adolescents; however, results from this study highlight components of the treatment process that may be particularly important for practitioners to consider and monitor in their clients.

The Dimensions of Change Instrument (DCI) measures treatment process in residential therapeutic community (TC) settings. It summarizes eight factors of treatment process from a client perspective. We present evidence of the reliability of the DCI for assessing both adult (N = 519) and adolescent (N = 474) client perceptions of treatment process. The DCI factors significantly increase over time, with increases consistently seen across all DCI factors for adults. We highlight areas for future DCI modifications to broaden its applicability to adolescents. Our findings show that clinicians can use the DCI to evaluate adult client progress and target areas for improving quality of care.

Background Women veterans are at increasingly high risk of suicide, but little is known about the concerns and needs of this population. This is, in part, owing to the low base rate of suicide and the inability to conduct retrospective interviews with individuals who died. In this study, we used a qualitative approach to gain insight about the concerns and nature of comments regarding suicidal ideation and intent among women veterans calling the Veterans Crisis Line (VCL). Methods Fifty-four VCL call responders were interviewed in the spring of 2015. They were asked about the concerns and level of suicide risk of women veteran callers with whom they have spoken and about the ways in which women callers are similar to or different from men callers. Interviews were transcribed and thematic analyses were conducted to examine patterns or themes emerging from the data. Findings Military sexual trauma and non-suicidal self-harm were two commonly reported concerns of women veteran callers according to responders. VCL responders also noted differences between men and women veteran callers, including differences in clinical presentation, suicidal means, and protective factors. Conclusions Our findings shed light on potential avenues to prevent suicide among women veterans, although we spoke to VCL responders about their impressions, rather than to women veterans themselves. Efforts to 1) prevent and treat the consequences of military sexual trauma, 2) recognize, prevent, and treat non-suicidal self-harm, and 3) restrict access to lethal means most commonly reported among women veteran callers may be helpful to mitigate suicide risk in this vulnerable group of veterans.

This article summarizes key findings from the second year of an evaluation of the California Mental Health Services Authority's statewide prevention and early intervention programs.

Trauma exposure and related symptoms interfere with adult adherence to drug treatment. Whether these findings hold true for adolescents is unknown. We examined trauma exposure, PTSD symptoms, and psychosocial functioning among 212 adolescents upon admission to long-term residential drug treatment and examined retention in treatment at 6 months. Seventy-one percent reported lifetime trauma exposure, and 29% of the trauma-exposed met criteria for current PTSD. Trauma-exposed adolescents reported more behavioral problems, with gender differences apparent. We divided the sample into three groups: no trauma exposure (21%), trauma-exposed without PTSD (59%), and trauma-exposed with PTSD (20%). Survival analysis showed that trauma-exposed adolescents without PTSD left treatment sooner than the nonexposed. Need for attention to trauma in substance abuse treatment programs is discussed.

This article reports on the prevalence of probable mental health disorders among clients entering outpatient substance abuse treatment, their clinical characteristics, and past access to substance abuse and mental health care. Four hundred fifteen individuals (74% of those eligible) entering three publicly funded outpatient substance abuse treatment facilities in Los Angeles County were screened for a probable mental health disorder. Of the 210 with a positive screener (just over 50% of those screened), 195 (93%) were interviewed. Depression and anxiety were the most common disorders, and more than a third had two or more probable disorders. Close to 70% reported using alcohol, and almost half reported using crack or cocaine. Half had never received any mental health treatment, and for a third this was their first episode of addiction treatment; 22% were on psychotropic medications. Levels of physical and mental health functioning were lower than the 25th percentile of the U.S. population norms. Our results indicate high rates of co-occurring mental health disorders among individuals entering these outpatient substance abuse treatment clinics in Los Angeles. Identifying people with probable mental health disorders as they enter treatment has the potential to increase access to care among those with limited prior access.

Prior research has demonstrated that therapeutic communities (TCs) are effective at improving posttreatment outcomes for substance abusers. However, little is known about the in-treatment experience for clients with different backgrounds, experiences, and needs. The aim of this study is to examine the in-treatment experience for different clients by exploring the relationships between treatment process and client characteristics. A comprehensive measure of treatment process, operationalized as Community Environment and Personal Change and Development and change was administered to 447 adults and 148 adolescents receiving treatment at community-based TC programs in New York, California, and Texas. Data on demographic characteristics, substance use and treatment history, and client risk factors were extracted from intake interviews and analyzed separately for adolescent and adult residents. Multivariate general linear models were used to examine the effect of client variables on treatment process, after controlling for treatment duration and program effects. Within adult programs, clients who were 25 years or older, female, and had a prior drug treatment experience had higher Community Environment scores. Adolescents with one or more arrests within the past 2 years had lower scores on both process dimensions of Community Environment and Personal Development and Change. Our results indicate the need to understand why adult clients who are younger, male, and have no prior treatment history and adolescent clients with recent arrests reported lower ratings of treatment process. Future research should also examine the role of modifiable mediators so that appropriate strategies to enhance therapeutic engagement may be developed as necessary.

This guide offers practical steps for conducting continuous quality improvement (CQI) in community service organizations (CSOs). It provides tools that are of specific relevance to CSOs, such as for addressing outreach and referrals, admissions, service delivery, and outcomes aspects of a CSO program. The CQI process detailed in this guide is based on a broader program management tool called Getting To Outcomes®.

Underage drinking is a significant problem facing US communities. Several environmental alcohol prevention (EAP) strategies (laws, regulations, responsible beverage service training and practices) successfully address underage drinking. Communities, however, face challenges carrying out these EAP strategies effectively. This small-scale, three-year, randomized controlled trial assessed whether providing prevention coalitions with Getting To Outcomes-Underage Drinking (GTO-UD), a tool kit and implementation support intervention, helped improve implementation of two common EAP strategies, responsible beverage service training (RBS) and Compliance Checks. Three coalitions in South Carolina and their RBS and Compliance Check programs received the 16 month GTO-UD intervention, including the GTO-UD manual, training, and onsite technical assistance, while another three in South Carolina maintained routine operations. The measures, collected at baseline and after the intervention, were a structured interview assessing how well coalitions carried out their work and a survey of merchant attitudes and practices in the six counties served by the participating coalitions. Over time, the quality of some RBS and Compliance Check activities improved more in GTO-UD coalitions than in the control sites. No changes in merchant practices or attitudes significantly differed between the GTO-UD and control groups, although merchants in the GTO-UD counties did significantly improve on refusing sales to minors while control merchants did not.

Provides preliminary advice about which of the California Mental Health Services Authority's Prevention and Early Intervention activities seem most valuable to sustain or, in some cases, enhance.

This fact sheet summarizes the first-year evaluation of California's Suicide Prevention Initiative.

California's Mental Health Services Act of 2004 mandates the state and counties to develop mental health prevention and early intervention services and education. This document summarizes first-year findings from an ongoing evaluation of one element of that program, an initiative to prevent suicide.

Background Studies have shown that communities have not always been able to implement evidence-based prevention programs with quality and achieve outcomes demonstrated by prevention science. Implementation support interventions are needed to bridge this gap between science and practice. The purpose of this article is to present two-year outcomes from an evaluation of the Assets Getting To Outcomes (AGTO) intervention in 12 Maine communities engaged in promoting Developmental Assets, a positive youth development approach to prevention. AGTO is an implementation support intervention that consists of: a manual of text and tools; face-to-face training, and onsite technical assistance, focused on activities shown to be associated with obtaining positive results across any prevention program. Methods This study uses a nested and cross-sectional, cluster randomized controlled design. Participants were coalition members and program staff from 12 communities in Maine. Each coalition nominated up to five prevention programs to participate. At random, six coalitions and their respective 30 programs received the two-year AGTO intervention and the other six maintained routine operations. The study assessed prevention practitioner capacity (efficacy and behaviors), practitioner exposure to and use of AGTO, practitioner perceptions of AGTO, and prevention program performance. Capacity of coalition members and performance of their programs were compared between the two groups across the baseline, one-, and two-year time points. Results We found no significant differences between AGTO and control group’s prevention capacity. However, within the AGTO group, significant differences were found between those with greater exposure to and use of AGTO. Programs that received the highest number of technical assistance hours showed the most program improvement. Conclusions This study is the first of its kind to show that use of an implementation support intervention-AGTO -yielded improvements in practitioner capacity and consequently in program performance on a large sample of practitioners and programs using a randomized controlled design. ClinicalTrials.gov identifier NCT00780338

Similar to how research can be improved when community members are involved (i.e., Community-based participatory research, or CBPR), this brief report argues how preventive interventions can also be so improved (i.e., Community Based Participatory Interventions or CBPI). The paper describes how to facilitate CBPI, RAND and University of South Carolina researchers developed a prevention process and corresponding technical assistance package, called Getting To Outcomes (GTO).

A multimedia presentation with e-learning modules that introduces the steps of the Getting to Outcomes with Developmental Assets (AGTO) process. Each module provides information on how each step could strengthen planning, implementation, and evaluation efforts, whether for a youth-serving program or a community-based coalition or initiative.