Search

Abstract Objective Learning Health Systems (LHSs) have not directly informed evidence‐based policymaking. The Translation‐to‐Policy (T2P) Learning Cycle aligns scientists, end‐users, and policymakers to support a repeatable roadmap of innovation and quality improvement to optimize effective policies toward a common public health goal. We describe T2P learning cycle components and provide examples of their application. Methods The T2P Learning Cycle is based on the U.S. Department of Veterans Affairs (VA) Office of Research and Development and Quality Enhancement Research Initiative (QUERI), which supports research and quality improvement addressing national public health priorities to inform policy and ensure programs are evidence‐based and work for end‐users. Incorporating LHS infrastructure, the T2P Learning Cycle is responsive to the Foundations for Evidence‐based Policymaking Act, which requires U.S. government agencies to justify budgets using evidence. Results The learning community (patients, providers, clinical/policy leaders, and investigators) drives the T2P Learning Cycle, working toward one or more specific, shared priority goals, and supports a repeatable cycle of evidence‐building and evaluation. Core T2P Learning Cycle functions observed in the examples from housing/economic security, precision oncology, and aging include governance and standard operating procedures to promote effective priority‐setting; complementary research and quality improvement initiatives, which inform ongoing data curation at the learning system level; and sustainment of continuous improvement and evidence‐based policymaking. Conclusions The T2P Learning Cycle integrates research translation with evidence‐based policymaking, ensuring that scientific innovations address public health priorities and serve end‐users through a repeatable process of research and quality improvement that ensures policies are scientifically based, effective, and sustainable.

Abstract Background Although research on the implementation of evidence-based psychological treatments (EBPTs) has advanced rapidly, research on the sustainment of implemented EBPTs remains limited. This is concerning, given that EBPT activities and benefits regularly decline post-implementation. To advance research on sustainment, the present protocol focuses on the third and final phase—the Sustainment Phase—of a hybrid type 2 cluster-randomized controlled trial investigating the implementation and sustainment of the Transdiagnostic Intervention for Sleep and Circadian Dysfunction (TranS-C) for patients with serious mental illness and sleep and circadian problems in community mental health centers (CMHCs). Prior to the first two phases of the trial—the Implementation Phase and Train-the-Trainer Phase—TranS-C was adapted to fit the CMHC context. Then, 10 CMHCs were cluster-randomized to implement Standard or Adapted TranS-C via facilitation and train-the-trainer. The primary goal of the Sustainment Phase is to investigate whether adapting TranS-C to fit the CMHC context predicts improved sustainment outcomes. Methods Data collection for the Sustainment Phase will commence at least three months after implementation efforts in partnering CMHCs have ended and may continue for up to one year. CMHC providers will be recruited to complete surveys (N = 154) and a semi-structured interview (N = 40) on sustainment outcomes and mechanisms. Aim 1 is to report the sustainment outcomes of TranS-C. Aim 2 is to evaluate whether manipulating EBPT fit to context (i.e., Standard versus Adapted TranS-C) predicts sustainment outcomes. Aim 3 is to test whether provider perceptions of fit mediate the relation between treatment condition (i.e., Standard versus Adapted TranS-C) and sustainment outcomes. Mixed methods will be used to analyze the data. Discussion The present study seeks to advance our understanding of sustainment predictors, mechanisms, and outcomes by investigating (a) whether the implementation strategy of adapting an EBPT (i.e., TranS-C) to the CMHC context predicts improved sustainment outcomes and (b) whether this relation is mediated by improved provider perceptions of treatment fit. Together, the findings may help inform more precise implementation efforts that contribute to lasting change. Trial registration ClinicalTrials.gov identifier: NCT05956678 . Registered on July 21, 2023.

Abstract Background Obtaining information on implementation strategy costs and local budget impacts from multiple perspectives is essential to data-driven decision-making about resource allocation for successful evidence-based intervention delivery. This mixed methods study determines the costs and priorities of deploying Enhanced Replicating Effective Programs (REP) to implement the Michigan Model for Health™, a universal school-based prevention intervention, from key shareholder perspectives. Methods Our study included teachers in 8 high schools across 3 Michigan counties as part of a pilot cluster randomized trial. We used activity-based costing, mapping key Enhanced REP activities across implementation phases. We included multiple perspectives, including state agencies, regional education service agencies, lead organization, and implementers. We also conducted a budget impact analysis (BIA, assessing the potential financial impact of adopting Enhanced REP) and a scenario analysis to estimate replication and account for cost variability. We used an experimental embedded mixed methods approach, conducting semi-structured interviews and collecting field notes during the trial to expand and explain the cost data and the implications of costs across relevant perspectives. Results Based on trial results, we estimate costs for deploying Enhanced REP are $11,903/school, with an estimated range between $8263/school and $15,201/school. We estimate that adding four additional schools, consistent with the pilot, would cost $8659/school. Qualitative results indicated misalignment in school and teacher priorities in some cases. Implementation activities, including training and implementation facilitation with the health coordinator, were sometimes in addition to regular teaching responsibilities. The extent to which this occurred was partly due to leadership priorities (e.g., sticking to the district PD schedule) and organizational priorities (e.g., budget). Conclusions Previous research findings indicate that, from a societal perspective, universal prevention is an excellent return on investment. However, notable misalignment in cost burden and priorities exists across shareholder groups. Our results indicate significant personal time costs by teachers when engaging in implementation strategy activities that impose an opportunity cost. Additional strategies are needed to improve the alignment of costs and benefits to enhance the success and sustainability of implementation. We focus on those perspectives informed by the analysis and discuss opportunities to expand a multi-level focus and create greater alignment across perspectives. Trial registration ClinicalTrials.gov NCT04752189. Registered on 12 February 2021.

Abstract Background Train-the-trainer (TTT) is a promising method for implementing evidence-based psychological treatments (EBPTs) in community mental health centers (CMHCs). In TTT, expert trainers train locally embedded individuals (i.e., Generation 1 providers) to deliver an EBPT, who then train others (i.e., Generation 2 providers). The present study will evaluate implementation and effectiveness outcomes of an EBPT for sleep and circadian dysfunction—the Transdiagnostic Intervention for Sleep and Circadian Dysfunction (TranS-C)—delivered to CMHC patients with serious mental illness by Generation 2 providers (i.e., trained and supervised within CMHCs via TTT). Specifically, we will investigate whether adapting TranS-C to fit CMHC contexts improves Generation 2 (a) patient outcomes and (b) providers’ perceptions of fit. Methods TTT will be implemented in nine CMHCs in California, USA (N = 60 providers; N = 130 patients) via facilitation. CMHCs are cluster-randomized by county to Adapted TranS-C or Standard TranS-C. Within each CMHC, patients are randomized to immediate TranS-C or usual care followed by delayed treatment with TranS-C (UC-DT). Aim 1 will assess the effectiveness of TranS-C (combined Adapted and Standard), compared to UC-DT, on improvements in sleep and circadian problems, functional impairment, and psychiatric symptoms for Generation 2 patients. Aim 2 will evaluate whether Adapted TranS-C is superior to Standard TranS-C with respect to Generation 2 providers’ perceptions of fit. Aim 3 will evaluate whether Generation 2 providers’ perceived fit mediates the relation between TranS-C treatment condition and patient outcomes. Exploratory analyses will (1) evaluate whether the effectiveness of TranS-C for patient outcomes is moderated by generation, (2) compare Adapted and Standard TranS-C on patient perceptions of credibility/improvement and PhenX Toolkit outcomes (e.g., substance use, suicidality), and (3) evaluate other possible moderators. Discussion This trial has potential to (a) inform the process of embedding local trainers and supervisors to expand delivery of a promising transdiagnostic treatment for sleep and circadian dysfunction, (b) add to the growing body of TTT literature by evaluating TTT outcomes with a novel treatment and population, and (c) advance our understanding of providers’ perceptions of EBPT “fit” across TTT generations. Trial registration ClinicalTrials.gov identifier NCT05805657 . Registered on April 10, 2023.

Abstract Background Healthcare facilitation, an implementation strategy designed to improve the uptake of effective clinical innovations in routine practice, has produced promising yet mixed results in randomized implementation trials and has not been fully researched across different contexts. Objective Using mechanism mapping, which applies directed acyclic graphs that decompose an effect of interest into hypothesized causal steps and mechanisms, we propose a more concrete description of how healthcare facilitation works to inform its further study as a meta-implementation strategy. Methods Using a modified Delphi consensus process, co-authors developed the mechanistic map based on a three-step process. First, they developed an initial logic model by collectively reviewing the literature and identifying the most relevant studies of healthcare facilitation components and mechanisms to date. Second, they applied the logic model to write vignettes describing how facilitation worked (or did not) based on recent empirical trials that were selected via consensus for inclusion and diversity in contextual settings (US, international sites). Finally, the mechanistic map was created based on the collective findings from the vignettes. Findings Theory-based healthcare facilitation components informing the mechanistic map included staff engagement, role clarification, coalition-building through peer experiences and identifying champions, capacity-building through problem solving barriers, and organizational ownership of the implementation process. Across the vignettes, engagement of leaders and practitioners led to increased socialization of the facilitator’s role in the organization. This in turn led to clarifying of roles and responsibilities among practitioners and identifying peer experiences led to increased coherence and sense-making of the value of adopting effective innovations. Increased trust develops across leadership and practitioners through expanded capacity in adoption of the effective innovation by identifying opportunities that mitigated barriers to practice change. Finally, these mechanisms led to eventual normalization and ownership of the effective innovation and healthcare facilitation process. Impact Mapping methodology provides a novel perspective of mechanisms of healthcare facilitation, notably how sensemaking, trust, and normalization contribute to quality improvement. This method may also enable more efficient and impactful hypothesis-testing and application of complex implementation strategies, with high relevance for lower-resourced settings, to inform effective innovation uptake.

Abstract Background Serious mental illness (SMI) can have devastating consequences. Unfortunately, many patients with SMI do not receive evidence-based psychological treatment (EBPTs) in routine practice settings. One barrier is poor “fit” between EBPTs and contexts in which they are implemented. The present study will evaluate implementation and effectiveness outcomes of the Transdiagnostic Intervention for Sleep and Circadian Dysfunction (TranS-C) implemented in community mental health centers (CMHCs). TranS-C was designed to target a range of SMI diagnoses by addressing a probable mechanism and predictor of SMI: sleep and circadian problems. We will investigate whether adapting TranS-C to fit CMHC contexts improves providers’ perceptions of fit and patient outcomes. Methods TranS-C will be implemented in at least ten counties in California, USA (N = 96 providers; N = 576 clients), via facilitation. CMHC sites are cluster-randomized by county to Adapted TranS-C or Standard TranS-C. Within each county, patients are randomized to immediate TranS-C or usual care followed by delayed treatment with TranS-C (UC-DT). Aim 1 will compare TranS-C (combined Adapted and Standard) with UC-DT on improvements in sleep and circadian problems, functional impairment, and psychiatric symptoms. Sleep and circadian problems will also be tested as a mediator between treatment condition (combined TranS-C versus UC-DT) and functional impairment/psychiatric symptoms. Aim 2 will evaluate whether Adapted TranS-C is superior to Standard TranS-C with respect to provider perceptions of fit. Aim 3 will evaluate whether the relation between TranS-C treatment condition (Adapted versus Standard) and patient outcomes is mediated by better provider perceptions of fit in the Adapted condition. Exploratory analyses will (1) compare Adapted versus Standard TranS-C on patient perceptions of credibility/improvement and select PhenX Toolkit outcomes and (2) evaluate possible moderators. Discussion This trial has the potential to (a) expand support for TranS-C, a promising transdiagnostic treatment delivered to patients with SMI in CMHCs; (b) take steps toward addressing challenges faced by providers in delivering EBPTs (i.e., high caseloads, complex patients, poor fit); and (c) advance evidence on causal strategies (i.e., adapting treatments to fit context) in implementation science. Trial registration Clinicaltrials.gov NCT04154631. Registered on 6 November 2019. https://clinicaltrials.gov/ct2/show/NCT04154631

Abstract Background School-based drug use prevention programs have demonstrated notable potential to reduce the onset and escalation of drug use, including among youth at risk of poor outcomes such as those exposed to trauma. Researchers have found a robust relationship between intervention fidelity and participant (i.e., student) outcomes. Effective implementation of evidence-based interventions, such as the Michigan Model for HealthTM (MMH), is critical to achieving desired public health objectives. Yet, a persistent gap remains in what we know works and how to effectively translate these findings into routine practice. The objective of this study is to design and test a multi-component implementation strategy to tailor MMH to meet population needs (i.e., students exposed to trauma), and improve the population-context fit to enhance fidelity and effectiveness. Methods Using a 2-group, mixed-method randomized controlled trial design, this study will compare standard implementation versus Enhanced Replicating Effective Programs (REP) to deliver MMH. REP is a theoretically based implementation strategy that promotes evidence-based intervention (EBI) fidelity through a combination of EBI curriculum packaging, training, and as-needed technical assistance and is consistent with standard MMH implementation. Enhanced REP will tailor the intervention and training to integrate trauma-informed approaches and deploy customized implementation support (i.e., facilitation). The research will address the following specific aims: (1) design and test an implementation strategy (Enhanced REP) to deliver the MMH versus standard implementation and evaluate feasibility, acceptability, and appropriateness using mixed methods, (2) estimate the costs and cost-effectiveness of Enhanced REP to deliver MMH versus standard implementation. Discussion This research will design and test a multi-component implementation strategy focused on enhancing the fit between the intervention and population needs while maintaining fidelity to MMH core functions. We focus on the feasibility of deploying the implementation strategy bundle and costing methods and preliminary information on cost input distributions. The substantive focus on youth at heightened risk of drug use and its consequences due to trauma exposure is significant because of the public health impact of prevention. Pilot studies of implementation strategies are underutilized and can provide vital information on designing and testing effective strategies by addressing potential design and methods uncertainties and the effects of the implementation strategy on implementation and student outcomes. Trial registration NCT04752189—registered on 8 February 2021 on ClinicalTrials.gov PRS

Abstract Background Schools increasingly provide mental health services to students, but often lack access to implementation strategies to support school-based (and school professional [SP]) delivery of evidence-based practices. Given substantial heterogeneity in implementation barriers across schools, development of adaptive implementation strategies that guide which implementation strategies to provide to which schools and when may be necessary to support scale-up. Methods A clustered, sequential, multiple-assignment randomized trial (SMART) of high schools across Michigan was used to inform the development of a school-level adaptive implementation strategy for supporting SP-delivered cognitive behavioral therapy (CBT). All schools were first provided with implementation support informed by Replicating Effective Programs (REP) and then were randomized to add in-person Coaching or not (phase 1). After 8 weeks, schools were assessed for response based on SP-reported frequency of CBT delivered to students and/or barriers reported. Responder schools continued with phase 1 implementation strategies. Slower-responder schools (not providing ≥ 3 CBT components to ≥10 students or >2 organizational barriers identified) were re-randomized to add Facilitation to current support or not (phase 2). The primary aim hypothesis was that SPs at schools receiving the REP + Coaching + Facilitation adaptive implementation strategy would deliver more CBT sessions than SPs at schools receiving REP alone. Secondary aims compared four implementation strategies (Coaching vs no Coaching × Facilitation vs no Facilitation) on CBT sessions delivered, including by type (group, brief and full individual). Analyses used a marginal, weighted least squares approach developed for clustered SMARTs. Results SPs (n = 169) at 94 high schools entered the study. N = 83 schools (88%) were slower-responders after phase 1. Contrary to the primary aim hypothesis, there was no evidence of a significant difference in CBT sessions delivered between REP + Coaching + Facilitation and REP alone (111.4 vs. 121.1 average total CBT sessions; p = 0.63). In secondary analyses, the adaptive strategy that offered REP + Facilitation resulted in the highest average CBT delivery (154.1 sessions) and the non-adaptive strategy offering REP + Coaching the lowest (94.5 sessions). Conclusions The most effective strategy in terms of average SP-reported CBT delivery is the adaptive implementation strategy that (i) begins with REP, (ii) augments with Facilitation for slower-responder schools (schools where SPs identified organizational barriers or struggled to deliver CBT), and (iii) stays the course with REP for responder schools. Trial registration ClinicalTrials.gov, NCT03541317 , May 30, 2018.

Abstract Background Implementation facilitation is an effective strategy to support the implementation of evidence-based practices (EBPs), but our understanding of multilevel strategies and the mechanisms of change within the “black box” of implementation facilitation is limited. This implementation trial seeks to disentangle and evaluate the effects of facilitation strategies that separately target the care team and leadership levels on implementation of a collaborative care model in pediatric primary care. Strategies targeting the provider care team (TEAM) should engage team-level mechanisms, and strategies targeting leaders (LEAD) should engage organizational mechanisms. Methods We will conduct a hybrid type 3 effectiveness–implementation trial in a 2 × 2 factorial design to evaluate the main and interactive effects of TEAM and LEAD and test for mediation and moderation of effects. Twenty-four pediatric primary care practices will receive standard REP training to implement Doctor–Office Collaborative Care (DOCC) and then be randomized to (1) Standard REP only, (2) TEAM, (3) LEAD, or (4) TEAM + LEAD. Implementation outcomes are DOCC service delivery and change in practice-level care management competencies. Clinical outcomes are child symptom severity and quality of life. Discussion This statewide trial is one of the first to test the unique and synergistic effects of implementation strategies targeting care teams and practice leadership. It will advance our knowledge of effective care team and practice-level implementation strategies and mechanisms of change. Findings will support efforts to improve common child behavioral health conditions by optimizing scale-up and sustainment of CCMs in a pediatric patient-centered medical home. Trial registration ClinicalTrials.gov, NCT04946253 . Registered June 30, 2021.

Abstract The Veterans Health Administration (VHA), under the U.S. Department of Veterans Affairs (VA), is one of the largest single providers of health care in the U.S. VA supports an embedded research program that addresses VA clinical priorities in close partnership with operations leaders, which is a hallmark of a Learning Health System (LHS). Using the LHS framework, we describe current VA research initiatives in mental health and substance use disorders that rigorously evaluate national programs and policies designed to reduce the risk of suicide and opioid use disorder (data to knowledge); test implementation strategies to improve the spread of effective programs for Veterans at risk of suicide or opioid use disorder (knowledge to performance); and identify novel research directions in suicide prevention and opioid/pain treatments emanating from implementation and quality improvement research (performance to data). Lessons learned are encapsulated into best practices for building and sustaining an LHS within health systems, including the need for early engagement with clinical leaders; pragmatic research questions that focus on continuous improvement; multi‐level, ongoing input from regional and local stakeholders, and business case analyses to inform ongoing investment in sustainable infrastructure to maintain the research‐health system partnership. Essential ingredients for supporting VA as an LHS include data and information sharing capacity, protected time for researchers and leaders, and governance structures to enhance health system ownership of research findings. For researchers, incentives to work with health systems operations (e.g., retainer funding) are vital for LHS research to be recognized and valued by academic promotion committees.

Abstract Objectives The U.S. Department of Veterans Affairs (VA) has been a national leader in Learning Health System (LHS) implementation due to its combined mission of research, education, clinical care, and emergency preparedness. We describe the current VA LHS training ecosystem within the Veterans Health Administration's Office of Academic Affiliations (OAA), Office of Research and Development (ORD), ORD's Health Services Research and Development (HSR&D) program, and Innovation Ecosystem (IE), including lessons learned regarding their sustainment. Methods The VA LHS training ecosystem is based on the Learning Loop and HSR&D Quality Enhancement Research Initiative (QUERI) Roadmap, which describes VA learning opportunities, underlying infrastructures, and core competencies. Results VA‐focused LHS educational programs include data‐to‐knowledge initiatives in health sciences and analytics, for example, OAA/HSR&D health services and informatics research fellowships; knowledge‐to‐performance opportunities in implementation and quality improvement, for example, QUERI Learning Hubs and IEs' Diffusion of Excellence Initiative; and performance‐to‐data embedded opportunities, for example, IE's entrepreneur fellowship programs and QUERI's Advancing Diversity in Implementation Leadership. These training programs are supported by combined VA research and clinical operations investments in funding, informatics, governance, and processes. Lessons learned include ongoing alignment of research funding with operational priorities and capacity, relentless recruitment and retention of implementation, system, and information scientists especially from under‐represented groups, sustainment of data infrastructures suitable for research and quality improvement, and ensuring sustainable funding opportunities for researchers to work on system‐wide health care problems. Conclusions There is an urgent need to expand training opportunities in LHSs, especially as health care is increasingly driven by multiple interested parties, impacted by persistent health disparities exacerbated by emerging public health threats, and rapid technology growth. With ongoing alignment of research and clinical goals, foundational support through research funding, underlying clinical operations infrastructures, and active engagement interested parties, VA's LHS training ecosystem promotes a more LHS‐savvy, 21st century workforce.

This paper presents a methodological description of a randomized controlled trial (RCT) testing the effect of Raising Our Spirits Together (ROST), a technology-assisted cognitive behavioral therapy (T-CBT) for depression, tailored for the rural context and for delivery by clergy, compared to an enhanced control condition. Depression is among the most common mental health conditions; yet the majority of adults with depression do not receive needed treatment due to limited access to mental health professionals, treatment-associated costs, distance to care, and stigma. These barriers are particularly salient in rural areas of the United States. T-CBT with human support is an accessible and effective treatment for depression; however, currently available T-CBTs have poor completion rates due to the lack of tailoring and other features to support engagement. ROST is a T-CBT specifically tailored for the rural setting and delivery by clergy, who are preferred, informal providers. ROST also presents core CBT content in a simple, jargon-free manner that supports multiple learning preferences. ROST is delivered virtually in a small group format across 8 weekly sessions via videoconferencing software consistent with other clergy-based programs, such as Bible studies or self-help groups. In this study, adults with depressive symptoms recruited from two rural Michigan counties will be randomized to receive ROST versus an enhanced control condition (N = 84). Depressive symptoms post-treatment and at 3 months follow-up according to the Patient Health Questionnaire (PHQ-9) will be the primary outcome. Findings will determine whether ROST is effective for improving depression symptoms in underserved, under resourced rural communities.

Background: Effective implementation of evidence-based prevention interventions in schools is vital to reducing the burden of drug use and its consequences. Universal prevention interventions often fail to achieve desired public health outcomes due to poor implementation. One central reason for suboptimal implementation is the limited fit between the intervention and the setting. Research is needed to increase our understanding of how intervention characteristics and context influence intervention implementation in schools to design implementation strategies that will address barriers and improve public health impact. Methods Using a convergent mixed methods design we examined qualitative and quantitative data on implementation determinants for an evidence-based health curriculum, the Michigan Model for Health TM (MMH) from the perspective of health teachers delivering the curriculum in high schools across the state. We examined data strands independently and integrated them by investigating data alignment, expansion, and divergence. Results We identified three mixed methods domains: (1) Acceptability, (2) intervention-context fit, and (3) adaptability. We found alignment across data strands as teachers reporting low acceptability also reported low fidelity. The fit between student needs and the curriculum predicted fidelity (expansion). Teachers mentioned instances of poor intervention-context fit (discordance), including when meeting the needs of trauma-exposed youth and keeping updated on youth drug use trends. Teachers reported high adaptability (concordance) but also instances when adaptation was challenging (discordance). Conclusions This investigation advances implementation research by deepening our understanding of implementation determinants for an evidence-based universal prevention intervention in schools. This will support designing effective implementation strategies to address barriers and advance the public health impact of interventions that address important risk and protective factors for all youth. Plain Language Summary (1) What is Already Known About the Topic? While many evidence-based interventions (EBIs) exist to address key health issues among youth including substance use and mental health, few of these interventions are effectively implemented in community settings, such as schools. Notable multilevel barriers exist to implement universal prevention in schools. Researchers identify that misalignment between the intervention and the context is a key reason why many implementation efforts do not achieve desired outcomes. (2) What Does This Paper Add? This paper combines the strengths of qualitative and quantitative research methods to identify and understand challenges to intervention-context fit for a comprehensive health curriculum, the Michigan Model for Health TM (MMH) which is widely adopted throughout Michigan, from the perspective of end users. This paper also utilizes the consolidated framework for implementation research and implementation outcomes framework to guide our understanding of implementing complex interventions and key barriers to implementation in schools. This research provides a foundation to design effective strategies that will balance curriculum fidelity and adaptation to achieve public health objectives. (3) What are the Implications for Practice, Research, or Policy? We need implementation strategies that guide flexibility and fidelity in EBI delivery in schools. While overall teachers felt the curriculum was adaptable and met student needs, they also mentioned specific instances when they would benefit from additional implementation support, such as making adaptations to meet the needs of trauma-exposed youth and keeping up-to-date with emerging drugs. Implementation strategies designed to address these challenges can improve fidelity and ultimately student well-being.

Abstract Background Theory-based methods to support the uptake of evidence-based practices (EBPs) are critical to improving mental health outcomes. Implementation strategy costs can be substantial, and few have been rigorously evaluated. The purpose of this study is to conduct a cost-effectiveness analysis to identify the most cost-effective approach to deploying implementation strategies to enhance the uptake of Life Goals, a mental health EBP. Methods We used data from a previously conducted randomized trial to compare the cost-effectiveness of Replicating Effective Programs (REP) combined with external and/or internal facilitation among sites non-responsive to REP. REP is a low-level strategy that includes EBP packaging, training, and technical assistance. External facilitation (EF) involves external expert support, and internal facilitation (IF) augments EF with protected time for internal staff to support EBP implementation. We developed a decision tree to assess 1-year costs and outcomes for four implementation strategies: (1) REP only, (2) REP+EF, (3) REP+EF add IF if needed, (4) REP+EF/IF. The analysis used a 1-year time horizon and assumed a health payer perspective. Our outcome was quality-adjusted life years (QALYs). The economic outcome was the incremental cost-effectiveness ratio (ICER). We conducted deterministic and probabilistic sensitivity analysis (PSA). Results Our results indicate that REP+EF add IF is the most cost-effective option with an ICER of $593/QALY. The REP+EF/IF and REP+EF only conditions are dominated (i.e., more expensive and less effective than comparators). One-way sensitivity analyses indicate that results are sensitive to utilities for REP+EF and REP+EF add IF. The PSA results indicate that REP+EF, add IF is the optimal strategy in 30% of iterations at the threshold of $100,000/QALY. Conclusions Our results suggest that the most cost-effective implementation support begins with a less intensive, less costly strategy initially and increases as needed to enhance EBP uptake. Using this approach, implementation support resources can be judiciously allocated to those clinics that would most benefit. Our results were not robust to changes in the utility measure. Research is needed that incorporates robust and relevant utilities in implementation studies to determine the most cost-effective strategies. This study advances economic evaluation of implementation by assessing costs and utilities across multiple implementation strategy combinations. Trial registration ClinicalTrials.gov Identifier: NCT02151331 , 05/30/2014.

Translation of research to practice is challenging. In addition to the scientific challenges, there are additional hurdles in navigating the rapidly changing US health care system. There is a need for innovative health interventions that can be adopted in “real-world” settings. Barriers to translation involve misaligned timing of research funding and health system decision-making, lack of research questions aligned with health system and community priorities, and limited incentives in academia for health system and community-based research. We describe new programs from the US Department of Veterans Affairs Health Services Research and Development (HSR&D) and the National Center for Advancing Translational Sciences (NCATS) Clinical and Translational Science Award (CTSA) Programs that are building capacity for Learning Health System research. These programs help to incentivize adopting and adapting Learning Health System principles to ensure that, primarily in implementation science within academic/veterans affairs health systems, there is alignment of the research with the health system and community needs. Both HSR&D and NCATS CTSA Program encourage researchers to develop problem-focused research innovations in partnership with health systems and communities to ultimately facilitate design treatments that are feasible in “real-world” practice.

Abstract Embedding research and evaluation into organizations is one way to generate “practice‐based” evidence needed to accelerate implementation of evidence‐based innovations within learning health systems. Organizations and researchers/evaluators vary greatly in how they structure and operationalize these collaborations. One key aspect is the degree of embeddedness: from low embeddedness where researchers/evaluators are located outside organizations (eg, outside evaluation consultants) to high embeddedness where researchers/evaluators are employed by organizations and thus more deeply involved in program evolution and operations. Pros and cons related to the degree of embeddedness (low vs high) must be balanced when developing these relationships. We reflect on this process within the context of an embedded, mixed‐methods evaluation of the Veterans Health Administration (VHA) Diffusion of Excellence (DoE) program. Considerations that must be balanced include: (a) low vs high alignment of goals; (b) low vs high involvement in strategic planning; (c) observing what is happening vs being integrally involved with programmatic activities; (d) reporting findings at the project's end vs providing iterative findings and recommendations that contribute to program evolution; and (e) adhering to predetermined aims vs adapting aims in response to evolving partner needs.

School professionals (SPs) are increasingly tasked with providing mental health treatment for students. Successful implementation of school-based mental health evidence-based practices (EBPs) can reduce students’ depression and anxiety, but requires identification of appropriate students to benefit from these interventions. We examined the ability of SPs to identify students appropriate for cognitive behavioral therapy (CBT), using baseline data collected from a large-scale implementation trial. One hundred sixty-nine SPs from 94 Michigan high schools were asked to identify up to 10 students whom they believed could benefit from CBT. Identified students were asked to complete a baseline survey. We examined the proportion of consented students who screened positive for depression or anxiety, based on the Patient Health Questionnaire modified for teens and the Generalized Anxiety Disorder questionnaire, overall and by key school- and SP-level factors. Of the 1,347 students identified by 149 SPs, 899 students completed the survey. Of those students, 87% exhibited any depression or anxiety symptoms (score of five or higher) and 66% showed moderate/severe symptoms (score of ten or higher). Per SP, the proportion of consented students who screened positive for any depression or anxiety symptoms was, on average, 0.88. The effectiveness of SPs’ student identification did not vary significantly by school-specific role, prior CBT training, or geography. Results suggest that SPs can effectively identify students appropriate for CBT given minimal training and resources, an important first step in increasing student access to school-based mental healthcare.

Implementation assessment plans are crucial for clinical trials to achieve their full potential. Without a proactive plan to implement trial results, it can take decades for one-fifth of effective interventions to be adopted into routine care settings. The Veterans Health Administration Office of Research and Development is undergoing a systematic transformation to embed implementation planning in research protocols through the Cooperative Studies Program, its flagship clinical research program. This manuscript has two objectives: 1) to introduce an Implementation Planning Assessment (IPA) Tool that any clinical trialist may use to facilitate post-trial implementation of interventions found to be effective and 2) to provide a case study demonstrating the IPA Tool’s use. The IPA Tool encourages study designers to initially consider rigorous data collection to maximize acceptability of the intervention by end-users. It also helps identify and prepare potential interested parties at local and national leadership levels to ensure, upon trial completion, interventions can be integrated into programs, technologies, and policies in a sustainable way. The IPA Tool can alleviate some of the overwhelming nature of implementation science by providing a practical guide based on implementation science principles for researchers desiring to scale up and spread effective, clinical trial-tested interventions to benefit patients.

Abstract Background School safety is fundamental to fostering positive outcomes for children. Violence remains a critical public health issue with 8.1% of elementary and 21.8% of middle school students reporting daily or weekly bullying in 2015–16. Similarly, over half of lifetime mental health concerns become evident before age 14. Thus, elementary school is a key time for comprehensive school safety interventions. Yet, interventions are rarely delivered with fidelity in community settings. Evidence-based interventions must be complemented by implementation strategies to achieve desired public health outcomes. Methods We develop and test an intervention focused on promoting a positive school climate guided by a school-based 3-person leadership team (3-PLT) using a hybrid Type II design. The 3-PLT includes a School Resource Officer, (SRO), administrator and mental health services professional as a newly appointed climate specialist (CS). The interventions to be delivered include 1) Restorative justice, 2) Mental Health First Aid and 3) Crime Prevention Through Environmental Design. The CS will lead the team and coordinate implementation through a process of interactive problem solving and supports, consistent with the implementation facilitation strategy. We will conduct a cluster randomized controlled trial with staged entry over two school years in Genesee County, Michigan (n = 20 elementary schools, with 10 participating per school year). We will use a combination of data sources including data collected by schools (e.g., discipline data), a student survey, and a teacher survey. We will also conduct a process evaluation and assess implementation and sustainability through focus groups with key stakeholders, teachers, and students. Finally, we will conduct a cost-benefit analysis. Discussion Results from both the behavioral outcome and implementation strategy evaluations are expected to have significant implications for school safety and student well-being. This study adopts a unique approach by integrating three evidence-based programs and incorporating implementation facilitation led by the CS as part of the 3-PLT to support intervention delivery and enhance public health impact among schools in disadvantaged communities with students at risk of poor health outcomes. This study aims to create a comprehensive, well-integrated model intervention that is sustainable and can be translated to similar high-risk settings. Trial registration Trial was retrospectively registered, registration ISRCTN1226421, May 16, 2019.

Background: Despite increasing calls for further spread of evidence-based collaborative care interventions (EBIs) in community-based settings, practitioner-driven efforts are often stymied by a lack of experience in addressing barriers to community-based implementation, especially for those not familiar with implementation science. The Michigan Mental Health Integration Partnership (MIP) is a statewide initiative that funds projects that support implementation and uptake of EBIs in community-based settings. MIP also provides an in situ implementation laboratory for understanding barriers to the uptake of EBIs across a variety of settings. We report findings from a statewide qualitative study of practitioners involved in MIP projects to garner their perspectives of best practices in the implementation of EBIs.Methods: Twenty-eight semi-structured interviews of practitioners and researchers from six MIP Projects were conducted with individuals implementing various MIP EBI projects across Michigan, including stakeholders from project teams, implementation sites, and the State of Michigan, to identify common barriers, challenges, and implementation strategies deployed by the project teams, with the purpose of informing a set of implementation steps and milestones.Results: Stakeholders identified a number of barriers to and strategies for success, including the need for tailoring program deployment and implementation to specific site needs, development of web-based tools for facilitating program implementation, and the importance of upper-level administration buy-in. Findings informed our resultant community-based Implementation Roadmap, which identifies critical steps across three implementation phases—pre-implementation, implementation, and sustainability—for implementation practitioners to use in their EBI implementation efforts.Conclusion: Implementation practitioners interested in community-based EBI implementation often lack access to operationalized implementation “steps” or “best practices” that can facilitate successful uptake and evaluation. Our community-informed MIP Implementation Roadmap, offering generalized steps for reaching successful implementation, uses experiences from a diverse set of MIP teams to guide practitioners through the practices necessary for scaling up EBIs in community-based settings over pre-implementation, implementation and sustainability phases.

Abstract Background Depressive and anxiety disorders affect 20–30% of school-age youth, most of whom do not receive adequate services, contributing to poor developmental and academic outcomes. Evidence-based practices (EBPs) such as cognitive behavioral therapy (CBT) can improve outcomes, but numerous barriers limit access among affected youth. Many youth try to access mental health services in schools, but school professionals (SPs: counselors, psychologists, social workers) are rarely trained adequately in CBT methods. Further, SPs face organizational barriers to providing CBT, such as lack of administrative support. Three promising implementation strategies to address barriers to school-based CBT delivery include (1) Replicating Effective Programs (REP), which deploys customized CBT packaging, didactic training in CBT, and technical assistance; (2) coaching, which extends training via live supervision to improve SP competence in CBT delivery; and (3) facilitation, which employs an organizational expert who mentors SPs in strategic thinking to promote self-efficacy in garnering administrative support. REP is a relatively low-intensity/low-cost strategy, whereas coaching and facilitation require additional resources. However, not all schools will require all three strategies. The primary aim of this study is to compare the effectiveness of a school-level adaptive implementation intervention involving REP, coaching, and facilitation versus REP alone on the frequency of CBT delivered to students by SPs and student mental health outcomes. Secondary and exploratory aims examine cost-effectiveness, moderators, and mechanisms of implementation strategies. Methods Using a clustered, sequential multiple-assignment, randomized trial (SMART) design, ≥ 200 SPs from 100 schools across Michigan will be randomized initially to receive REP vs. REP+coaching. After 8 weeks, schools that do not meet a pre-specified implementation benchmark are re-randomized to continue with the initial strategy or to augment with facilitation. Discussion EBPs need to be implemented successfully and efficiently in settings where individuals are most likely to seek care in order to gain large-scale impact on public health. Adaptive implementation interventions hold the promise of providing cost-effective implementation support. This is the first study to test an adaptive implementation of CBT for school-age youth, at a statewide level, delivered by school staff, taking an EBP to large populations with limited mental health care access. Trial registration NCT03541317—Registered on 29 May 2018 on ClinicalTrials.gov PRS

Abstract Background Organizational culture and climate are considered key factors in implementation efforts but have not been examined as moderators of implementation strategy comparative effectiveness. We investigated organizational culture and climate as moderators of comparative effectiveness of two sequences of implementation strategies (Immediate vs. Delayed Enhanced Replicating Effective Programs [REP]) combining Standard REP and REP enhanced with facilitation on implementation of an outreach program for Veterans with serious mental illness lost to care at Veterans Health Administration (VA) facilities nationwide. Methods This study is a secondary analysis of the cluster-randomized Re-Engage implementation trial that assigned 3075 patients at 89 VA facilities to either the Immediate or Delayed Enhanced REP sequences. We hypothesized that sites with stronger entrepreneurial culture, task, or relational climate would benefit more from Enhanced REP than Standard REP. Veteran- and site-level data from the Re-Engage trial were combined with site-aggregated measures of entrepreneurial culture and task and relational climate from the 2012 VA All Employee Survey. Longitudinal mixed-effects logistic models examined whether the comparative effectiveness of the Immediate vs. Delayed Enhanced REP sequences were moderated by culture or climate measures at 6 and 12 months post-randomization. Three Veteran-level outcomes related to the engagement with the VA system were assessed: updated documentation, attempted contact by coordinator, and completed contact. Results For updated documentation and attempted contact, Veterans at sites with higher entrepreneurial culture and task climate scores benefitted more from Enhanced REP compared to Standard REP than Veterans at sites with lower scores. Few culture or climate moderation effects were detected for the comparative effectiveness of the full sequences of implementation strategies. Conclusions Implementation strategy effectiveness is highly intertwined with contextual factors, and implementation practitioners may use knowledge of contextual moderation to tailor strategy deployment. We found that facilitation strategies provided with Enhanced REP were more effective at improving uptake of a mental health outreach program at sites with stronger entrepreneurial culture and task climate; Veterans at sites with lower levels of these measures saw more similar improvement under Standard and Enhanced REP. Within resource-constrained systems, practitioners may choose to target more intensive implementation strategies to sites that will most benefit from them. Trial registration ISRCTN: ISRCTN21059161. Date registered: April 11, 2013.

This paper provides a methodological description of a multi-site, randomized controlled trial (RCT) of a cognitive-behavioral intervention for enhancing employment success among unemployed persons whose employment efforts have been undermined by social anxiety disorder (SAD). SAD is a common and impairing condition, with negative impacts on occupational functioning. In response to these documented employment-related impairments, in a previous project, we produced and tested an eight-session work-related group cognitive-behavioral therapy provided alongside vocational services as usual (WCBT + VSAU). WCBT is delivered by vocational service professionals and is designed in a context and style that overcomes accessibility and stigma-related obstacles with special focus on employment-related targets. Our previous project found that WCBT + VSAU significantly improved social anxiety, depression, and a range of employment-related outcomes compared to a control group of socially anxious job-seekers who received vocational services as usual without WCBT (VSAU-alone). Participants in this study were all homeless, primarily African American job-seekers with high levels of psychiatric comorbidity and limited education and employment histories. The present, two-region study addresses whether WCBT + VSAU enhances job placement, job retention and mental health outcomes in a larger sample assessed over an extended follow-up period. In addition, this trial evaluates whether the effects of WCBT + VSAU generalize to a new population of urban-based, racially diverse job-seekers with vocational and educational histories that differ from our original sample. This study also investigates the system-effects of WCBT + VSAU in a new site that will be informative for broad implementation of WCBT + VSAU. Finally, this project involves a refined, technology-assisted form of WCBT + VSAU designed to be delivered more easily by vocational services professionals. Keywords: Social anxiety, Unemployment, Cognitive-behavioral therapy, Minorities

Objectives. Persons with mental disorders experience functional impairments and premature mortality. Limited continuity of care may contribute to disparities in this group. We describe the replication of an evidence-based outreach program (Re-Engage) to reconnect Veterans with mental disorders into care who have dropped out of services. Methods. Using the Enhanced Replicating Effective Programs framework, population-based registries were used to identify Veterans lost-to-care, and providers used this information to determine Veteran disposition and need for care. Providers recorded Veteran preferences, health status, and care utilization, and formative process data was collected to document implementation efforts. Results. Among Veterans who dropped out of care (n=126), the mean age was 49 years, 10% were women, and 29% were African-American. Providers determined that 39% of Veterans identified for re-engagement were deceased, hospitalized, or ineligible for care. Of the remaining 68 Veterans, outreach efforts resulted in contact with 20, with 7 returning to care. Providers averaged 14.2 hours over 4 months conducting re-engagement services and reported that gaining facility leadership support and having service agreements for referrals were essential for program implementation. Conclusions. Population-level, panel management strategies to re-engage Veterans with mental disorders are potentially feasible if practices are identified to facilitate national rollout.