Search

Abstract Background Implementation science scholars have made significant progress identifying factors that enable or obstruct the implementation of evidence-based interventions, and testing strategies that may modify those factors. However, little research sheds light on how or why strategies work, in what contexts, and for whom. Studying implementation mechanisms—the processes responsible for change—is crucial for advancing the field of implementation science and enhancing its value in facilitating equitable policy and practice change. The Agency for Healthcare Research and Quality funded a conference series to achieve two aims: (1) develop a research agenda on implementation mechanisms, and (2) actively disseminate the research agenda to research, policy, and practice audiences. This article presents the resulting research agenda, including priorities and actions to encourage its execution. Method Building on prior concept mapping work, in a semi-structured, 3-day, in-person working meeting, 23 US-based researchers used a modified nominal group process to generate priorities and actions for addressing challenges to studying implementation mechanisms. During each of the three 120-min sessions, small groups responded to the prompt: “What actions need to be taken to move this research forward?” The groups brainstormed actions, which were then shared with the full group and discussed with the support of facilitators trained in structured group processes. Facilitators grouped critical and novel ideas into themes. Attendees voted on six themes they prioritized to discuss in a fourth, 120-min session, during which small groups operationalized prioritized actions. Subsequently, all ideas were collated, combined, and revised for clarity by a subset of the authorship team. Results From this multistep process, 150 actions emerged across 10 priority areas, which together constitute the research agenda. Actions included discrete activities, projects, or products, and ways to shift how research is conducted to strengthen the study of implementation mechanisms. Conclusions This research agenda elevates actions to guide the selection, design, and evaluation of implementation mechanisms. By delineating recommended actions to address the challenges of studying implementation mechanisms, this research agenda facilitates expanding the field of implementation science, beyond studying what works to how and why strategies work, in what contexts, for whom, and with which interventions.

Introduction: Social health is increasingly a focus of healthcare systems. Representative and intersectional analyses of individuals’ social risks such as food, housing, transportation, and financial insecurity and their interest in receiving assistance from the healthcare system (social needs) can provide healthcare organizations with more nuanced estimates that can lead to more effective interventions. Methods: The authors conducted cross-sectional survey of a representative sample of 43,936 Kaiser Permanente members in December 2019–September 2020. Study inclusion was based on membership, age, address, and absence of dementia. Modified Poisson regression models estimated respondents' social risks and needs overall and within domain (food, housing, finances, or transportation) and by intersectional strata on the basis of a combination of age, income, gender, and race and ethnicity. Results: Of 10,274 participants who completed the survey, 52% reported any social risk, and 32% had any social need. Financial strain was the most prevalent risk (44%), followed by food insecurity (31%), housing instability (17%), and transportation (7%). These intersectional analyses generated 74 intersectional groups. Across the intersectional groups, higher-income (>$50,000 per year) or older-age (>60 years) adults consistently had lower risk and need. However, this pattern varied by race and ethnicity. In particular, older and lower-income, Pacific Islander, African-American/Black, and multiracial adults had the highest estimates of social risk and need. Conclusions: Higher risk and need among particular intersectional groups suggest the importance of tailored interventions for social needs. The high aggregate prevalence of social risks and needs suggests that system and policy changes must compliment universal and population-based social health screening and assistance interventions in healthcare organizations.

BackgroundOpinion leadership, educational outreach visiting, and innovation championing are commonly used strategies to address barriers to implementing innovations and evidence-based practices in healthcare settings. Despite voluminous research, ambiguities persist in how these strategies work and under what conditions they work well, work poorly, or work at all. The current paper develops middle-range theories to address this gap.MethodsConceptual articles, systematic reviews, and empirical studies informed the development of causal pathway diagrams (CPDs). CPDs are visualization tools for depicting and theorizing about the causal process through which strategies operate, including the mechanisms they activate, the barriers they address, and the proximal and distal outcomes they produce. CPDs also clarify the contextual conditions (i.e., preconditions and moderators) that influence whether, and to what extent, the strategy's causal process unfolds successfully. Expert panels of implementation scientists and health professionals rated the plausibility of these preliminary CPDs and offered comments and suggestions on them.FindingsTheoretically, opinion leadership addresses potential adopters' uncertainty about likely consequences of innovation use (determinant) by promoting positive attitude formation about the innovation (mechanism), which results in an adoption decision (proximal outcome), which leads to innovation use (intermediate outcome). As this causal process repeats, penetration, or spread of innovation use, occurs (distal outcome). Educational outreach visiting addresses knowledge barriers, attitudinal barriers, and behavioral barriers (determinants) by promoting critical thinking and reflection about evidence and practice (mechanism), which results in behavioral intention (proximal outcome), behavior change (intermediate outcome), and fidelity, or guideline adherence (distal outcome). Innovation championing addresses organizational inertia, indifference, and resistance (determinants) by promoting buy-in to the vision, fostering a positive implementation climate, and increasing collective efficacy (mechanisms), which leads to participation in implementation activities (proximal outcome), initial use of the innovation with increasing skill (intermediate outcome) and, ultimately, greater penetration and fidelity (distal outcomes). Experts found the preliminary CPDs plausible or highly plausible and suggested additional mechanisms, moderators, and preconditions, which were used to amend the initial CPD.DiscussionThe middle-range theories depicted in the CPDs furnish testable propositions for implementation research and offer guidance for selecting, designing, and evaluating these social influence implementation strategies in both research studies and practice settings.

Abstract Background Although research on the implementation of evidence-based psychological treatments (EBPTs) has advanced rapidly, research on the sustainment of implemented EBPTs remains limited. This is concerning, given that EBPT activities and benefits regularly decline post-implementation. To advance research on sustainment, the present protocol focuses on the third and final phase—the Sustainment Phase—of a hybrid type 2 cluster-randomized controlled trial investigating the implementation and sustainment of the Transdiagnostic Intervention for Sleep and Circadian Dysfunction (TranS-C) for patients with serious mental illness and sleep and circadian problems in community mental health centers (CMHCs). Prior to the first two phases of the trial—the Implementation Phase and Train-the-Trainer Phase—TranS-C was adapted to fit the CMHC context. Then, 10 CMHCs were cluster-randomized to implement Standard or Adapted TranS-C via facilitation and train-the-trainer. The primary goal of the Sustainment Phase is to investigate whether adapting TranS-C to fit the CMHC context predicts improved sustainment outcomes. Methods Data collection for the Sustainment Phase will commence at least three months after implementation efforts in partnering CMHCs have ended and may continue for up to one year. CMHC providers will be recruited to complete surveys (N = 154) and a semi-structured interview (N = 40) on sustainment outcomes and mechanisms. Aim 1 is to report the sustainment outcomes of TranS-C. Aim 2 is to evaluate whether manipulating EBPT fit to context (i.e., Standard versus Adapted TranS-C) predicts sustainment outcomes. Aim 3 is to test whether provider perceptions of fit mediate the relation between treatment condition (i.e., Standard versus Adapted TranS-C) and sustainment outcomes. Mixed methods will be used to analyze the data. Discussion The present study seeks to advance our understanding of sustainment predictors, mechanisms, and outcomes by investigating (a) whether the implementation strategy of adapting an EBPT (i.e., TranS-C) to the CMHC context predicts improved sustainment outcomes and (b) whether this relation is mediated by improved provider perceptions of treatment fit. Together, the findings may help inform more precise implementation efforts that contribute to lasting change. Trial registration ClinicalTrials.gov identifier: NCT05956678 . Registered on July 21, 2023.

Abstract Background For youth receiving care in community mental health centers, comorbidities are the rule rather than the exception. Using measurement-based care (MBC), or the routine evaluation of symptoms to inform care decisions, as the foundation of treatment for youth with comorbid problems significantly improves the impact of psychotherapy by focusing care and building engagement and alliance. MBC increases the rate of symptom improvement, detects clients who would otherwise deteriorate, and alerts clinicians to non-responders. Despite its demonstrated utility, MBC is rarely implemented with fidelity; less than 15% of providers report using MBC per recommendations. Previous efforts to support MBC implementation have yielded suboptimal outcomes, in part, due to organizations’ challenges with identifying and prioritizing barriers and selecting and developing strategies to overcome them. New methods are needed for identifying and prioritizing barriers, and matching strategies to barriers to optimize MBC implementation and treatment quality to improve youth mental health outcomes in community settings. Methods Pragmatic implementation methods will be piloted in four diverse community mental health centers. Methods include (a) rapid evidence synthesis; (b) rapid ethnography; (c) design kits (e.g., kits with disposable cameras, journals, maps); (d) barrier prioritization, and (e) causal pathway diagramming. These activities will generate actionable barriers; subsequently, we will use facilitated group processes to prioritize barriers and develop causal pathway diagrams to match strategies to barriers to create implementation plans that optimize MBC fidelity (Aim 1). We will track strategy deployment for 6 months, then compare MBC fidelity for another 6 months post-implementation with data from 2 years of historical controls (Aim 2). Finally, we will co-design a toolkit for design kit methods with youth and the practice and scientific communities (Aim 3). Discussion Optimizing MBC implementation in community mental health centers could transform youth mental health care by ensuring the most pressing symptoms are targeted early in treatment. The discussion section highlights expected challenges and limits to using the five methods, including recruitment and engagement given the high pressure on community mental health settings. Trial registration Clinicaltrials.gov. NCT05644756 . Registered on 18 November 2022. This trial was retrospectively registered.

Abstract Background Audit and feedback (A&F) is an implementation strategy that can facilitate implementation tailoring by identifying gaps between desired and actual clinical care. While there are several theory-based propositions on which A&F components lead to better implementation outcomes, many have not been empirically investigated, and there is limited guidance for stakeholders when applying A&F in practice. The current study aims to illustrate A&F procedures in six community mental health clinics, with an emphasis on reporting A&F components that are relevant to theories of how feedback elicits behavior change. Methods Six clinics from a larger trial using a tailored approach to implement measurement-based care (MBC) were analyzed for feedback content, delivery mechanisms, barriers to feedback, and outcomes of feedback using archival data. Pattern analysis was conducted to examine relations between A&F components and changes in MBC use. Results Several sites utilized both aggregate and individualized data summaries, and data accuracy concerns were common. Feedback cycles featuring individual-level clinician data, data relevant to MBC barriers, and information requested by data recipients were related to patterns of increased MBC use. Conclusions These findings support extant theory, such as Feedback Intervention Theory. Mental health professionals wishing to apply A&F should consider establishing reciprocal feedback mechanisms on the quality and amount of data being received and adopting specific roles communicating and addressing data quality concerns. Trial registration ClinicalTrials.gov Identifier: NCT02266134.

Background Implementation strategies are theorized to work well when carefully matched to implementation determinants and when factors—preconditions, moderators, etc.—that influence strategy effectiveness are prospectively identified and addressed. Existing methods for strategy selection are either imprecise or require significant technical expertise and resources, undermining their utility. This article outlines refinements to causal pathway diagrams (CPDs), a method for articulating the causal process through which implementation strategies work and offers illustrations of their use. Method CPDs are a visualization tool to represent an implementation strategy, its mechanism(s) (i.e., the processes through which a strategy is thought to operate), determinants it is intended to address, factors that may impede or facilitate its effectiveness, and the series of outcomes that should be expected if the strategy is operating as intended. We offer principles for constructing CPDs and describe their key functions. Results Applications of the CPD method by study teams from two National Institute of Health-funded Implementation Science Centers and a research grant are presented. These include the use of CPDs to (a) match implementation strategies to determinants, (b) understand the conditions under which an implementation strategy works, and (c) develop causal theories of implementation strategies. Conclusions CPDs offer a novel method for implementers to select, understand, and improve the effectiveness of implementation strategies. They make explicit theoretical assumptions about strategy operation while supporting practical planning. Early applications have led to method refinements and guidance for the field.

Abstract Background Central nervous system (CNS) active medications have been consistently linked to falls in older people. However, few randomized trials have evaluated whether CNS-active medication reduction reduces falls and fall-related injuries. The objective of the Reducing CNS-active Medications to Prevent Falls and Injuries in Older Adults (STOP-FALLS) trial is to test the effectiveness of a health-system-embedded deprescribing intervention focused on CNS-active medications on the incidence of medically treated falls among community-dwelling older adults. Methods We will conduct a pragmatic, cluster-randomized, parallel-group, controlled clinical trial within Kaiser Permanente Washington to test the effectiveness of a 12-month deprescribing intervention consisting of (1) an educational brochure and self-care handouts mailed to older adults prescribed one or more CNS-active medications (aged 60 + : opioids, benzodiazepines and Z-drugs; aged 65 + : skeletal muscle relaxants, tricyclic antidepressants, and antihistamines) and (2) decision support for their primary health care providers. Outcomes are examined over 18–26 months post-intervention. The primary outcome is first incident (post-baseline) medically treated fall as determined from health plan data. Our sample size calculations ensure at least 80% power to detect a 20% reduction in the rate of medically treated falls for participants receiving care within the intervention (n = 9) versus usual care clinics (n = 9) assuming 18 months of follow-up. Secondary outcomes include medication discontinuation or dose reduction of any target medications. Safety outcomes include serious adverse drug withdrawal events, unintentional overdose, and death. We will also examine medication signetur fields for attempts to decrease medications. We will report factors affecting implementation of the intervention. Discussion The STOP-FALLS trial will provide new information about whether a health-system-embedded deprescribing intervention that targets older participants and their primary care providers reduces medically treated falls and CNS-active medication use. Insights into factors affecting implementation will inform future research and healthcare organizations that may be interested in replicating the intervention. Trial registration ClinicalTrial.gov NCT05689554. Registered on 18 January 2023, retrospectively registered.

Abstract Background Individuals who use a language other than English for medical care are at risk for disparities related to healthcare safety, patient-centered care, and quality. Professional interpreter use decreases these disparities but remains underutilized, despite widespread access and legal mandates. In this study, we compare two discrete implementation strategies for improving interpreter use: (1) enhanced education targeting intrapersonal barriers to use delivered in a scalable format (interactive web-based educational modules) and (2) a strategy targeting system barriers to use in which mobile video interpreting is enabled on providers’ own mobile devices. Methods We will conduct a type 3 hybrid implementation-effectiveness study in 3–5 primary care organizations, using a sequential multiple assignment randomized trial (SMART) design. Our primary implementation outcome is interpreter use, calculated by matching clinic visits to interpreter invoices. Our secondary effectiveness outcome is patient comprehension, determined by comparing patient-reported to provider-documented visit diagnosis. Enrolled providers (n = 55) will be randomized to mobile video interpreting or educational modules, plus standard interpreter access. After 9 months, providers with high interpreter use will continue as assigned; those with lower use will be randomized to continue as before or add the alternative strategy. After another 9 months, both strategies will be available to enrolled providers for 9 more months. Providers will complete 2 surveys (beginning and end) and 3 in-depth interviews (beginning, middle, and end) to understand barriers to interpreter use, based on the Theoretical Domains Framework. Patients who use a language other than English will be surveyed (n = 648) and interviewed (n = 75) following visits with enrolled providers to understand their experiences with communication. Visits will be video recorded (n = 100) to assess fidelity to assigned strategies. We will explore strategy mechanism activation to refine causal pathway models using a quantitative plus qualitative approach. We will also determine the incremental cost-effectiveness of each implementation strategy from a healthcare organization perspective, using administrative and provider survey data. Discussion Determining how these two scalable strategies, alone and in sequence, perform for improving interpreter use, the mechanisms by which they do so, and at what cost, will provide critical insights for addressing a persistent cause of healthcare disparities. Trial registration NCT05591586.

Abstract Background Sustainability is concerned with the long-term delivery and subsequent benefits of evidence-based interventions. To further this field, we require a strong understanding and thus measurement of sustainability and what impacts sustainability (i.e., sustainability determinants). This systematic review aimed to evaluate the quality and empirical application of measures of sustainability and sustainability determinants for use in clinical, public health, and community settings. Methods Seven electronic databases, reference lists of relevant reviews, online repositories of implementation measures, and the grey literature were searched. Publications were included if they reported on the development, psychometric evaluation, or empirical use of a multi-item, quantitative measure of sustainability, or sustainability determinants. Eligibility was not restricted by language or date. Eligibility screening and data extraction were conducted independently by two members of the research team. Content coverage of each measure was assessed by mapping measure items to relevant constructs of sustainability and sustainability determinants. The pragmatic and psychometric properties of included measures was assessed using the Psychometric and Pragmatic Evidence Rating Scale (PAPERS). The empirical use of each measure was descriptively analyzed. Results A total of 32,782 articles were screened from the database search, of which 37 were eligible. An additional 186 publications were identified from the grey literature search. The 223 included articles represented 28 individual measures, of which two assessed sustainability as an outcome, 25 covered sustainability determinants and one explicitly assessed both. The psychometric and pragmatic quality was variable, with PAPERS scores ranging from 14 to 35, out of a possible 56 points. The Provider Report of Sustainment Scale had the highest PAPERS score and measured sustainability as an outcome. The School-wide Universal Behaviour Sustainability Index-School Teams had the highest PAPERS score (score=29) of the measure of sustainability determinants. Conclusions This review can be used to guide selection of the most psychometrically robust, pragmatic, and relevant measure of sustainability and sustainability determinants. It also highlights that future research is needed to improve the psychometric and pragmatic quality of current measures in this field. Trial registration This review was prospectively registered with Research Registry (reviewregistry1097), March 2021.

Abstract Background Health systems are increasingly attempting to intervene on social adversity as a strategy to improve health care outcomes. To inform health system efforts to screen for social adversity, we sought to explore the stability of social risk and interest in assistance over time and to evaluate whether the social risk was associated with subsequent healthcare utilization. Methods We surveyed Kaiser Permanente members receiving subsidies from the healthcare exchange in Southern California to assess their social risk and desire for assistance using the Accountable Health Communities instrument. A subset of initial respondents was randomized to be re-surveyed at either three or six months later. Results A total of 228 participants completed the survey at both time points. Social risks were moderate to strongly stable across three and six months (Kappa range = .59-.89); however, social adversity profiles that included participants’ desire for assistance were more labile (3-month Kappa = .52; 95% CI = .41-.64 & 6-month Kappa = .48; 95% CI = .36-.6). Only housing-related social risks were associated with an increase in acute care (emergency, urgent care) six months after initial screening; no other associations between social risk and utilization were observed. Conclusions This study suggests that screening for social risk may be appropriate at intervals of six months, or perhaps longer, but that assessing desire for assistance may need to occur more frequently. Housing risks were associated with increases in acute care. Health systems may need to engage in screening and referral to resources to improve overall care and ultimately patient total health.

Introduction: Healthcare systems such as Kaiser Permanente are increasingly focusing on patients’ social health. However, there is limited evidence to guide social health integration strategy. The purpose of this study was to identify social health research opportunities using a stakeholder-driven process. Methods: A modified Concept Mapping approach was implemented from June 2021 to February 2022. Stakeholders (n=746) received the prompt, “One thing I wish we knew more about to advance my work addressing social health...” An inductive content analysis approach was used to assign topics and synthesize and refine research-focused statements into research questions. Questions were then rated on impact and priority by researcher stakeholders (n=16). Mean impact and priority scores and an overall combined score were calculated. Question rankings were generated using the combined score. Results: Brainstorming produced 148 research-focused statements. A final list of 59 research questions was generated for rating. Question topics were (1) Data, Measures, and Metrics; (2) Intervention Approach and Impact; (3) Technology; (4) Role of Healthcare Systems; (5) Community-Based Organizations; (6) Equity; (7) Funding; and (8) Social Health Integration. On a scale from 1 (low) to 10 (high), the mean impact score was 6.12 (range=4.14–7.79), and the mean priority score was 5.61 (range=3.07–8.64). Twenty-four statements were rated as both high impact (>6.12) and high priority (>5.61). Conclusions: The broad range of topics with high impact and priority scores reveals how nascent the evidence base is, with fundamental research on the nature of social risk and health system involvement still needed.

Abstract Background There is a fundamental gap in understanding the causal mechanisms by which strategies for implementing evidence-based practices address local barriers to effective, appropriate service delivery. Until this gap is addressed, scientific knowledge and practical guidance about which implementation strategies to use in which contexts will remain elusive. This research project aims to identify plausible strategy-mechanism linkages, develop causal models for mechanism evaluation, produce measures needed to evaluate such linkages, and make these models, methods, and measures available in a user-friendly website. The specific aims are as follows: (1) build a database of strategy-mechanism linkages and associated causal pathway diagrams, (2) develop psychometrically strong, pragmatic measures of mechanisms, and (3) develop and disseminate a website of implementation mechanisms knowledge for use by diverse stakeholders. Methods For the first aim, a combination of qualitative inquiry, expert panel methods, and causal pathway diagramming will be used to identify and confirm plausible strategy-mechanism linkages and articulate moderators, preconditions, and proximal and distal outcomes associated with those linkages. For the second aim, rapid-cycle measure development and testing methods will be employed to create reliable, valid, pragmatic measures of six mechanisms of common strategies for which no high-quality measures exist. For the third aim, we will develop a user-friendly website and searchable database that incorporates user-centered design, disseminating the final product using social marketing principles. Discussion Once strategy-mechanism linkages are identified using this multi-method approach, implementation scientists can use the searchable database to develop tailored implementation strategies and generate more robust evidence about which strategies work best in which contexts. Moreover, practitioners will be better able to select implementation strategies to address their specific implementation problems. New horizons in implementation strategy development, optimization, evaluation, and deployment are expected to be more attainable as a result of this research, which will lead to enhanced implementation of evidence-based interventions for cancer control, and ultimately improvements in patient outcomes.

Abstract Background Tailored implementation approaches are touted as superior to standardized ones with the reasoning that tailored approaches afford opportunities to select strategies to resolve determinants of the local context. However, results from implementation trials on this topic are equivocal. Therefore, it is important to explore relevant contextual factors that function as determinants to evaluate if they are improved by tailoring and subsequently associated with changes in implementation outcomes (i.e., via statistical mediation) to better understand how tailoring achieves (or does not achieve) its effects. The present study examined the association between a tailored and standardized implementation approach, contextual factors that might mediate change, and a target implementation outcome in an initiative to implement measurement-based care (specifically the clinical integration of the Patient Health Questionnaire [PHQ-9] for depression) in a community mental health organization. Methods Using a cluster randomized control design, twelve community-based mental health clinics were assigned to a tailored or standardized implementation group. Clinicians completed a self-report battery assessing contextual factors that served as candidate mediators informed by the Framework for Dissemination at three time points: baseline, 5 months after active implementation support, and 10 months after sustainment monitoring. A subset of clinicians also participated in focus groups at 5 months. The routine use of the PHQ-9 (implementation outcome) was monitored during the 10-month sustainment period. Multi-level mediation analyses assessed the association between the implementation group and contextual factors and the association between contextual factors and PHQ-9 completion. Quantitative results were then elaborated by analyzing qualitative data from exemplar sites. Results Although tailored clinics outperformed standard clinics in terms of PHQ-9 completion at the end of active implementation, these group differences disappeared post sustainment monitoring. Perhaps related to this, no significant mediators emerged from our quantitative analyses. Exploratory qualitative analyses of focus group content emphasized the importance of support from colleagues, supervisors, and leadership when implementing clinical innovations in practice. Conclusions Although rates of PHQ-9 completion improved across the study, their sustained levels were roughly equivalent across groups and low overall. No mediators were established using quantitative methods; however, several partial quantitative pathways, as well as themes from the qualitative data, reveal fruitful areas for future research. Trial registration Standardized versus tailored implementation of measurement-based care for depression. ClinicalTrials.gov NCT02266134, first posted on October 16, 2014

Abstract Background Chronic low back pain is a common and sometimes disabling condition, and mindfulness-based stress reduction is recommended as a first line of therapy. This study tested whether different descriptions of mindfulness training altered based on influential intervention characteristics increased adoption intentions. Methods People with chronic low back pain (n = 452) were randomized to review one of four mindfulness training descriptions in an online survey using a 2 × 2 factorial design. The first factor was evidence strength and quality with relative advantage (ER). The second factor was adaptability, trialability, complexity, and design quality and packaging (AD). Each factor had two levels: a description of standardized mindfulness training that described each intervention characteristic and a patient-centered description emphasizing flexibility and patient testimonials. The primary outcomes were intentions to try mindfulness training and practice mindfulness at home. Using structural equation modeling with a bootstrapped distribution, we tested six mediators, three of which are Theory of Planned Behavior predictors of intention—self-efficacy, norms, and attitudes— and the other three are predictors of adoption—feasibility, appropriateness, and acceptability. Results Overall, the mindfulness training descriptions were not associated with an increase in intentions compared to the classic vignette (11/12 p’s > 0.05). Most descriptions were unrelated to mediators except the classic ER with patient-centered AD was associated with higher self-efficacy/control and feasibility (p’s ≤ 0.05; standardized effect range: 0.111–0.125). Self-efficacy/control (training standardized coefficient: 0.531, home: 0.686), norms (training: 0.303, home: 0.256), and attitudes (training: 0.316, home: 0.293) were all positively associated with intentions to adopt mindfulness training and home practice. Feasibility (training: 0.185; home: 0.293) and acceptability (training: 0.639; home: 0.554) were positively related to intentions to adopt mindfulness training. Appropriateness was related to intentions to adopt home practice (0.187) but not mindfulness training (0.100). None of the indirect effects from experimental group to intentions was significant (all p’s > 0.05). Conclusions Self-efficacy/control and acceptability may be key mediators for increasing patient adoption of mindfulness. Because experimental manipulation did not substantially change intentions to adopt mindfulness, the presentation and delivery of MBSR may need to be tailored to the individual patient’s needs rather than a specific format for chronic low back pain.

Abstract Background Implementation science is at a sufficiently advanced stage that it is appropriate for the field to reflect on progress thus far in achieving its vision, with a goal of charting a path forward. In this debate, we offer such reflections and report on potential threats that might stymie progress, as well as opportunities to enhance the success and impact of the field, from the perspective of a group of US-based researchers. Main body Ten mid-career extramurally funded US-based researchers completed a “pre-mortem” or a group brainstorming exercise that leverages prospective hindsight to imagine that an event has already occurred and to generate an explanation for it — to reduce the likelihood of a poor outcome. We came to consensus on six key themes related to threats and opportunities for the field: (1) insufficient impact, (2) too much emphasis on being a “legitimate science,” (3) re-creation of the evidence-to-practice gap, (4) difficulty balancing accessibility and field coherence, (5) inability to align timelines and priorities with partners, and (6) overly complex implementation strategies and approaches. Conclusion We submit this debate piece to generate further discussion with other implementation partners as our field continues to develop and evolve. We hope the key opportunities identified will enhance the future of implementation research in the USA and spark discussion across international groups. We will continue to learn with humility about how best to implement with the goal of achieving equitable population health impact at scale.

Background: Tailoring strategies to target the salient barriers to and enablers of implementation is considered a critical step in supporting successful delivery of evidence based interventions in healthcare. Theory, evidence, and stakeholder engagement are considered key ingredients in the process however, these ingredients can be combined in different ways. There is no consensus on the definition of tailoring or single method for tailoring strategies to optimize impact, ensure transparency, and facilitate replication. Aim: The purpose of this scoping review is to describe how tailoring has been undertaken within healthcare to answer questions about how it has been conceptualised, described, and conducted in practice, and to identify research gaps. Methods: The review will be conducted in accordance with best practice guidelines and the Preferred Reporting Items for Systematic Reviews and Meta-analysis extension for scoping reviews (PRISMA-ScR) will be used to guide the reporting. Searches will be conducted of MEDLINE, Embase, Web of Science, Scopus, from 2005 to present. Reference lists of included articles will be searched. Grey literature will be searched on Google Scholar. Screening and data extraction will be conducted by two or more members of the research team, with any discrepancies resolved by consensus discussion with a third reviewer. Initial analysis will be quantitative involving a descriptive numerical summary of the characteristics of the studies and the tailoring process. Qualitative content analysis aligned to the research questions will also be conducted, and data managed using NVivo where applicable. This scoping review is pre-registered with the Open Science Framework. Conclusions: The findings will serve a resource for implementation researchers and practitioners to guide future research in this field and facilitate systematic, transparent, and replicable development of tailored implementation strategies.

Background Strategies to implement evidence-based interventions (EBIs) in children’s mental health services have complex direct and indirect causal impacts on multiple outcomes. Ripple effects are outcomes caused by EBI implementation efforts that are unplanned, unanticipated, and/or more salient to stakeholders other than researchers and implementers. The purpose of the current paper is to provide a compilation of possible ripple effects associated with EBI implementation strategies in children’s mental health services, to be used for implementation planning, research, and quality improvement. Methods Participants were identified via expert nomination and snowball sampling. Online surveys were completed by 81 participants, each representing one of five roles: providers of mental health services to children or youth, researchers, policy makers, caregivers, and youth. A partially directed conventional content analysis with consensus decision making was used to code ripple effects. Results Four hundred and four unique responses were coded into 66 ripple effects and 14 categories. Categories include general knowledge, skills, attitudes, and confidence about using EBIs; general job-related ripple effects; EBI treatment adherence, fidelity, and alignment; gaming the system; equity and stigma; shifting roles, role clarity, and task shifting; economic costs and benefits; EBI treatment availability, access, participation, attendance, barriers, and facilitators; clinical process and treatment quality; client engagement, therapeutic alliance, and client satisfaction; clinical organization structure, relationships in the organization, process, and functioning; youth client and caregiver outcomes; and use of EBI strategies and insights in one’s own life. Conclusions This research advances the field by providing children’s mental health implementers, researchers, funders, policy makers, and consumers with a menu of potential ripple effects. It can be a practical tool to ensure compliance with guidance from Quality Improvement/Quality Assurance, Complexity Science, and Diffusion of Innovation Theory. Future phases will match potential ripple effects with salient children’s mental health implementation strategies for each participant role. Plain Language Summary: This qualitative study of multiple stakeholders in children’s mental health services identifies several possible ripple effects of implementation strategies, opening a new area of study for implementation science. Ripple effects can be positive, negative, or neutral within the full balance of implementation quality and impact. The list of ripple effects will provide implementation scientists, developers, and others with a useful tool during implementation planning and evaluation. This expert-informed methodology can provide a model for other fields for exploring possible ripple effects within implementation science.

Abstract Background Evidence-based interventions (EBIs) could reduce cervical cancer deaths by 90%, colorectal cancer deaths by 70%, and lung cancer deaths by 95% if widely and effectively implemented in the USA. Yet, EBI implementation, when it occurs, is often suboptimal. This manuscript outlines the protocol for Optimizing Implementation in Cancer Control (OPTICC), a new implementation science center funded as part of the National Cancer Institute Implementation Science Consortium. OPTICC is designed to address three aims. Aim 1 is to develop a research program that supports developing, testing, and refining of innovative, efficient methods for optimizing EBI implementation in cancer control. Aim 2 is to support a diverse implementation laboratory of clinical and community partners to conduct rapid, implementation studies anywhere along the cancer care continuum for a wide range of cancers. Aim 3 is to build implementation science capacity in cancer control by training new investigators, engaging established investigators in cancer-focused implementation science, and contributing to the Implementation Science Consortium in Cancer. Methods Three cores serve as OPTICC’s foundation. The Administrative Core plans coordinates and evaluates the Center’s activities and leads its capacity-building efforts. The Implementation Laboratory Core (I-Lab) coordinates a network of diverse clinical and community sites, wherein studies are conducted to optimize EBI implementation, implement cancer control EBIs, and shape the Center’s agenda. The Research Program Core conducts innovative implementation studies, measurement and methods studies, and pilot studies that advance the Center’s theme. A three-stage approach to optimizing EBI implementation is taken—(I) identify and prioritize determinants, (II) match strategies, and (III) optimize strategies—that is informed by a transdisciplinary team of experts leveraging multiphase optimization strategies and criteria, user-centered design, and agile science. Discussion OPTICC will develop, test, and refine efficient and economical methods for optimizing EBI implementation by building implementation science capacity in cancer researchers through applications with our I-Lab partners. Once refined, OPTICC will disseminate its methods as toolkits accompanied by massive open online courses, and an interactive website, the latter of which seeks to simultaneously accumulate knowledge across OPTICC studies.

Abstract Background Public policy has tremendous impacts on population health. While policy development has been extensively studied, policy implementation research is newer and relies largely on qualitative methods. Quantitative measures are needed to disentangle differential impacts of policy implementation determinants (i.e., barriers and facilitators) and outcomes to ensure intended benefits are realized. Implementation outcomes include acceptability, adoption, appropriateness, compliance/fidelity, feasibility, penetration, sustainability, and costs. This systematic review identified quantitative measures that are used to assess health policy implementation determinants and outcomes and evaluated the quality of these measures. Methods Three frameworks guided the review: Implementation Outcomes Framework (Proctor et al.), Consolidated Framework for Implementation Research (Damschroder et al.), and Policy Implementation Determinants Framework (Bullock et al.). Six databases were searched: Medline, CINAHL Plus, PsycInfo, PAIS, ERIC, and Worldwide Political. Searches were limited to English language, peer-reviewed journal articles published January 1995 to April 2019. Search terms addressed four levels: health, public policy, implementation, and measurement. Empirical studies of public policies addressing physical or behavioral health with quantitative self-report or archival measures of policy implementation with at least two items assessing implementation outcomes or determinants were included. Consensus scoring of the Psychometric and Pragmatic Evidence Rating Scale assessed the quality of measures. Results Database searches yielded 8417 non-duplicate studies, with 870 (10.3%) undergoing full-text screening, yielding 66 studies. From the included studies, 70 unique measures were identified to quantitatively assess implementation outcomes and/or determinants. Acceptability, feasibility, appropriateness, and compliance were the most commonly measured implementation outcomes. Common determinants in the identified measures were organizational culture, implementation climate, and readiness for implementation, each aspects of the internal setting. Pragmatic quality ranged from adequate to good, with most measures freely available, brief, and at high school reading level. Few psychometric properties were reported. Conclusions Well-tested quantitative measures of implementation internal settings were under-utilized in policy studies. Further development and testing of external context measures are warranted. This review is intended to stimulate measure development and high-quality assessment of health policy implementation outcomes and determinants to help practitioners and researchers spread evidence-informed policies to improve population health. Registration Not registered

Abstract Background Understanding the mechanisms of implementation strategies (i.e., the processes by which strategies produce desired effects) is important for research to understand why a strategy did or did not achieve its intended effect, and it is important for practice to ensure strategies are designed and selected to directly target determinants or barriers. This study is a systematic review to characterize how mechanisms are conceptualized and measured, how they are studied and evaluated, and how much evidence exists for specific mechanisms. Methods We systematically searched PubMed and CINAHL Plus for implementation studies published between January 1990 and August 2018 that included the terms “mechanism,” “mediator,” or “moderator.” Two authors independently reviewed title and abstracts and then full texts for fit with our inclusion criteria of empirical studies of implementation in health care contexts. Authors extracted data regarding general study information, methods, results, and study design and mechanisms-specific information. Authors used the Mixed Methods Appraisal Tool to assess study quality. Results Search strategies produced 2277 articles, of which 183 were included for full text review. From these we included for data extraction 39 articles plus an additional seven articles were hand-entered from only other review of implementation mechanisms (total = 46 included articles). Most included studies employed quantitative methods (73.9%), while 10.9% were qualitative and 15.2% were mixed methods. Nine unique versions of models testing mechanisms emerged. Fifty-three percent of the studies met half or fewer of the quality indicators. The majority of studies (84.8%) only met three or fewer of the seven criteria stipulated for establishing mechanisms. Conclusions Researchers have undertaken a multitude of approaches to pursue mechanistic implementation research, but our review revealed substantive conceptual, methodological, and measurement issues that must be addressed in order to advance this critical research agenda. To move the field forward, there is need for greater precision to achieve conceptual clarity, attempts to generate testable hypotheses about how and why variables are related, and use of concrete behavioral indicators of proximal outcomes in the case of quantitative research and more directed inquiry in the case of qualitative research.

Abstract Background Implementing and sustaining evidence-based programs with fidelity may require multiple implementation strategies tailored to address multi-level, context-specific barriers and facilitators. Ideally, selecting and tailoring implementation strategies should be guided by theory, evidence, and input from relevant stakeholders; however, methods to guide the selection and tailoring of strategies are not well-developed. There is a need for more rigorous methods for assessing and prioritizing implementation determinants (barriers and facilitators) and linking implementation strategies to determinants. The Collaborative Organizational Approach to Selecting and Tailoring Implementation Strategies (COAST-IS) is an intervention designed to increase the effectiveness of evidence-based practice implementation and sustainment. COAST-IS will enable organizational leaders and clinicians to use Intervention Mapping to select and tailor implementation strategies to address their site-specific needs. Intervention Mapping is a multi-step process that incorporates theory, evidence, and stakeholder perspectives to ensure that implementation strategies effectively address key determinants of change. Methods COAST-IS will be piloted with community mental health organizations that are working to address the needs of children and youth who experience trauma-related emotional or behavioral difficulties by engaging in a learning collaborative to implement an evidence-based psychosocial intervention (trauma-focused cognitive behavioral therapy). Organizations will be matched and then randomized to participate in the learning collaborative only (control) or to receive additional support through COAST-IS. The primary aims of this study are to (1) assess the acceptability, appropriateness, feasibility, and perceived utility of COAST-IS; (2) evaluate the organizational stakeholders’ fidelity to the core elements of COAST-IS; and (3) demonstrate the feasibility of testing COAST-IS in a larger effectiveness trial. Discussion COAST-IS is a systematic method that integrates theory, evidence, and stakeholder perspectives to improve the effectiveness and precision of implementation strategies. If effective, COAST-IS has the potential to improve the implementation and sustainment of a wide range of evidence-based practices in mental health and other sectors. Trial registration This study was registered in ClinicalTrials.gov ( NCT03799432 ) on January 10, 2019 (last updated August 5, 2019).

Background: Brief educational trainings are often used for disseminating and implementing evidence-based practices (EBPs). However, many accessible trainings are ubiquitously standardized. Tailored training focused on modifying individual or contextual factors that may hinder EBP implementation is recommended, but there is a dearth of research comparing standardized versus tailored training. This study sought to: (a) assess the impact of MBC training on clinician intention to use measurement-based care (MBC); (b) compare the effect of standardized versus tailored training on clinician intention to MBC; and (c) identify clinician-level predictors of intention. Methods: Clinicians ( n = 152) treating adult clients with depression at 12 community mental health clinics were randomized to either tailored or standardized MBC training. Clinic-specific barriers and facilitators were used to inform training content and structure tailoring. Linear mixed modeling tested the association between training condition and post-training intention to use MBC, as well as hypothesized individual-level predictors of post-training intention (e.g., age, gender). Results: Clinician intention pre- and post-training increased across training conditions ( B = 0.38, t = −5.95, df = 36.99, p .05, Cohen's d = .15). Only baseline intention emerged as a predictor of post-training intention ( B = 0.39, SE = .05, p

Background Implementation mechanisms are defined as processes or events through which implementation strategies operate to affect one or more implementation outcomes. Understanding the mechanisms through which implementation strategies work is critical to understanding how and why implementation efforts are successful, and to matching, tailoring, and optimizing implementation strategies. This study examined the content of abstracts included in the program for the 2019 Society for Implementation Research Collaboration (SIRC) conference to characterize the presence of data related to implementation strategy mechanisms and their larger causal pathways. Methods Trained coders reviewed all 205 accepted abstracts and extracted information regarding discrete implementation strategies, determinants of implementation, implementation mechanisms, service outcomes, and implementation outcomes. Theoretical articles were omitted from further analyses due to their inability to offer data related to implementation mechanisms. Results Of the 151 empirical studies included, only 11 (7.28%) reported studying mechanisms. Mechanisms were examined in projects utilizing 14 different implementation strategies. We were able to construct implementation causal pathways for just two implementation strategies, “assess for readiness and identify determinants,” representing information pulled from four different abstracts, and “create a learning collaborative,” with data pulled from just one abstract. Conclusions These findings indicate that, at least based on SIRC conference abstracts, the empirical investigation of implementation mechanisms remains understudied, highlighting the need for focused research on the study of mechanisms. Plain Language Summary Understanding the mechanisms through which implementation strategies work is critical to understanding how and why implementation efforts are successful. The study of implementation mechanisms may be used to optimize implementation strategy decisions. Investigations to date have not established causal pathways linking implementation strategies, mechanisms, barriers, and outcomes. This study examined abstracts included in the program for the 2019 Society for Implementation Research Collaboration (SIRC) conference. Trained coders reviewed all 205 accepted abstracts and extracted information to characterize the presence of data related to implementation strategy causal pathways. A minority of abstracts reported studying mechanisms. We were able to construct implementation causal pathways for just two implementation strategies, representing information pulled from five different abstracts all together. This highlights the need for focused research on the study of mechanisms.

Background: Systematic measure reviews can facilitate advances in implementation research and practice by locating reliable, valid, pragmatic measures; identifying promising measures needing refinement and testing; and highlighting measurement gaps. This review identifies and evaluates the psychometric and pragmatic properties of measures of readiness for implementation and its sub-constructs as delineated in the Consolidated Framework for Implementation Research: leadership engagement, available resources, and access to knowledge and information. Methods: The systematic review methodology is described fully elsewhere. The review, which focused on measures used in mental or behavioral health, proceeded in three phases. Phase I, data collection, involved search string generation, title and abstract screening, full text review, construct assignment, and cited citation searches. Phase II, data extraction, involved coding relevant psychometric and pragmatic information. Phase III, data analysis, involved two trained specialists independently rating each measure using Psychometric and Pragmatic Evidence Rating Scales (PAPERS). Frequencies and central tendencies summarized information availability and PAPERS ratings. Results: Searches identified 9 measures of readiness for implementation, 24 measures of leadership engagement, 17 measures of available resources, and 6 measures of access to knowledge and information. Information about internal consistency was available for most measures. Information about other psychometric properties was often not available. Ratings for internal consistency were “adequate” or “good.” Ratings for other psychometric properties were less than “adequate.” Information on pragmatic properties was most often available regarding cost, language readability, and brevity. Information was less often available regarding training burden and interpretation burden. Cost and language readability generally exhibited “good” or “excellent” ratings, interpretation burden generally exhibiting “minimal” ratings, and training burden and brevity exhibiting mixed ratings across measures. Conclusion: Measures of readiness for implementation and its sub-constructs used in mental health and behavioral health care are unevenly distributed, exhibit unknown or low psychometric quality, and demonstrate mixed pragmatic properties. This review identified a few promising measures, but targeted efforts are needed to systematically develop and test measures that are useful for both research and practice. Plain language abstract: Successful implementation of effective mental health or behavioral health treatments in service delivery settings depends in part on the readiness of the service providers and administrators to implement the treatment; the engagement of organizational leaders in the implementation effort; the resources available to support implementation, such as time, money, space, and training; and the accessibility of knowledge and information among service providers about the treatment and how it works. It is important that the methods for measuring these factors are dependable, accurate, and practical; otherwise, we cannot assess their presence or strength with confidence or know whether efforts to increase their presence or strength have worked. This systematic review of published studies sought to identify and evaluate the quality of questionnaires (referred to as measures) that assess readiness for implementation, leadership engagement, available resources, and access to knowledge and information. We identified 56 measures of these factors and rated their quality in terms of how dependable, accurate, and practical they are. Our findings indicate there is much work to be done to improve the quality of available measures; we offer several recommendations for doing so.

Abstract Context Implementation science measures are rarely used by stakeholders to inform and enhance clinical program change. Little is known about what makes implementation measures pragmatic (i.e., practical) for use in community settings; thus, the present study’s objective was to generate a clinical stakeholder-driven operationalization of a pragmatic measures construct. Evidence acquisition The pragmatic measures construct was defined using: 1) a systematic literature review to identify dimensions of the construct using PsycINFO and PubMed databases, and 2) interviews with an international stakeholder panel (N = 7) who were asked about their perspectives of pragmatic measures. Evidence synthesis Combined results from the systematic literature review and stakeholder interviews revealed a final list of 47 short statements (e.g., feasible, low cost, brief) describing pragmatic measures, which will allow for the development of a rigorous, stakeholder-driven conceptualization of the pragmatic measures construct. Conclusions Results revealed significant overlap between terms related to the pragmatic construct in the existing literature and stakeholder interviews. However, a number of terms were unique to each methodology. This underscores the importance of understanding stakeholder perspectives of criteria measuring the pragmatic construct. These results will be used to inform future phases of the project where stakeholders will determine the relative importance and clarity of each dimension of the pragmatic construct, as well as their priorities for the pragmatic dimensions. Taken together, these results will be incorporated into a pragmatic rating system for existing implementation science measures to support implementation science and practice.

Abstract Background Tailored implementation approaches are touted as more likely to support the integration of evidence-based practices. However, to our knowledge, few methodologies for tailoring implementations exist. This manuscript will apply a model-driven, mixed methods approach to a needs assessment to identify the determinants of practice, and pilot a modified conjoint analysis method to generate an implementation blueprint using a case example of a cognitive behavioral therapy (CBT) implementation in a youth residential center. Methods Our proposed methodology contains five steps to address two goals: (1) identify the determinants of practice and (2) select and match implementation strategies to address the identified determinants (focusing on barriers). Participants in the case example included mental health therapists and operations staff in two programs of Wolverine Human Services. For step 1, the needs assessment, they completed surveys (clinician N = 10; operations staff N = 58; other N = 7) and participated in focus groups (clinician N = 15; operations staff N = 38) guided by the domains of the Framework for Diffusion [1]. For step 2, the research team conducted mixed methods analyses following the QUAN + QUAL structure for the purpose of convergence and expansion in a connecting process, revealing 76 unique barriers. Step 3 consisted of a modified conjoint analysis. For step 3a, agency administrators prioritized the identified barriers according to feasibility and importance. For step 3b, strategies were selected from a published compilation and rated for feasibility and likelihood of impacting CBT fidelity. For step 4, sociometric surveys informed implementation team member selection and a meeting was held to identify officers and clarify goals and responsibilities. For step 5, blueprints for each of pre-implementation, implementation, and sustainment phases were generated. Results Forty-five unique strategies were prioritized across the 5 years and three phases representing all nine categories. Conclusions Our novel methodology offers a relatively low burden collaborative approach to generating a plan for implementation that leverages advances in implementation science including measurement, models, strategy compilations, and methods from other fields.

Abstract Background Implementation science is the study of strategies used to integrate evidence-based practices into real-world settings (Eccles and Mittman, Implement Sci. 1(1):1, 2006). Central to the identification of replicable, feasible, and effective implementation strategies is the ability to assess the impact of contextual constructs and intervention characteristics that may influence implementation, but several measurement issues make this work quite difficult. For instance, it is unclear which constructs have no measures and which measures have any evidence of psychometric properties like reliability and validity. As part of a larger set of studies to advance implementation science measurement (Lewis et al., Implement Sci. 10:102, 2015), we will complete systematic reviews of measures that map onto the Consolidated Framework for Implementation Research (Damschroder et al., Implement Sci. 4:50, 2009) and the Implementation Outcomes Framework (Proctor et al., Adm Policy Ment Health. 38(2):65-76, 2011), the protocol for which is described in this manuscript. Methods Our primary databases will be PubMed and Embase. Our search strings will be comprised of five levels: (1) the outcome or construct term; (2) terms for measure; (3) terms for evidence-based practice; (4) terms for implementation; and (5) terms for mental health. Two trained research specialists will independently review all titles and abstracts followed by full-text review for inclusion. The research specialists will then conduct measure-forward searches using the “cited by” function to identify all published empirical studies using each measure. The measure and associated publications will be compiled in a packet for data extraction. Data relevant to our Psychometric and Pragmatic Evidence Rating Scale (PAPERS) will be independently extracted and then rated using a worst score counts methodology reflecting “poor” to “excellent” evidence. Discussion We will build a centralized, accessible, searchable repository through which researchers, practitioners, and other stakeholders can identify psychometrically and pragmatically strong measures of implementation contexts, processes, and outcomes. By facilitating the employment of psychometrically and pragmatically strong measures identified through this systematic review, the repository would enhance the cumulativeness, reproducibility, and applicability of research findings in the rapidly growing field of implementation science.

Abstract Objective Although tailored implementation methods are touted as superior to standardized, few researchers have directly compared the two and little guidance regarding the specific details of each method exist. Our study compares these methods in a dynamic cluster randomized trial seeking to optimize implementation of measurement based care (MBC) for depression in community behavioral health. This specific manuscript provides a detailed, replicable account of the components of each multi-faceted implementation method. Results The standardized best practice method includes training, consultation, a clinical guideline, and electronic health record enhancements with the goal to optimize the delivery of MBC with fidelity. Conversely, the tailored, customized and collaborative method is informed by recent implementation science advancements and begins with a needs assessment, followed by tailored training that feeds back barriers data to clinicians, the formation of an implementation team, a clinician-driven clinic-specific guideline, and the use of fidelity data to inform implementation team activities; the goal of the tailored condition is to ensure the intervention and implementation strategies address unique factors of the context. The description of these methods will inform others seeking to implement MBC, as well as those planning to use standardized or tailored implementation methods for interventions beyond behavioral health.

Abstract Background The recent growth in organized efforts to advance dissemination and implementation (D & I) science suggests a rapidly expanding community focused on the adoption and sustainment of evidence-based practices (EBPs). Although promising for the D & I of EBPs, the proliferation of initiatives is difficult for any one individual to navigate and summarize. Such proliferation may also result in redundant efforts or missed opportunities for participation and advancement. A review of existing D & I science resource initiatives and their unique merits would be a significant step for the field. The present study aimed to describe the global landscape of these organized efforts to advance D & I science. Methods We conducted a content analysis between October 2015 and March 2016 to examine resources and characteristics of D & I science resource initiatives using public, web-based information. Included resource initiatives must have engaged in multiple efforts to advance D & I science beyond conferences, offered D & I science resources, and provided content in English. The sampling method included an Internet search using D & I terms and inquiry among internationally representative D & I science experts. Using a coding scheme based on a priori and grounded approaches, two authors consensus coded website information including interactive and non-interactive resources and information regarding accessibility (membership, cost, competitive application, and location). Results The vast majority (83%) of resource initiatives offered at least one of seven interactive resources (consultation/technical assistance, mentorship, workshops, workgroups, networking, conferences, and social media) and one of six non-interactive resources (resource library, news and updates from the field, archived talks or slides, links pages, grant writing resources, and funding opportunities). Non-interactive resources were most common, with some appearing frequently across resource initiatives (e.g., news and updates from the field). Conclusion Findings generated by this study offer insight into what types of D & I science resources exist and what new resources may have the greatest potential to make a unique and needed contribution to the field. Additional interactive resources may benefit the field, particularly mentorship opportunities and resources that can be accessed virtually. Moving forward, it may be useful to consider strategic attention to the core tenets of D & I science put forth by Glasgow and colleagues to most efficiently and effectively advance the field.

Abstract Background Advancing implementation research and practice requires valid and reliable measures of implementation determinants, mechanisms, processes, strategies, and outcomes. However, researchers and implementation stakeholders are unlikely to use measures if they are not also pragmatic. The purpose of this study was to establish a stakeholder-driven conceptualization of the domains that comprise the pragmatic measure construct. It built upon a systematic review of the literature and semi-structured stakeholder interviews that generated 47 criteria for pragmatic measures, and aimed to further refine that set of criteria by identifying conceptually distinct categories of the pragmatic measure construct and providing quantitative ratings of the criteria’s clarity and importance. Methods Twenty-four stakeholders with expertise in implementation practice completed a concept mapping activity wherein they organized the initial list of 47 criteria into conceptually distinct categories and rated their clarity and importance. Multidimensional scaling, hierarchical cluster analysis, and descriptive statistics were used to analyze the data. Findings The 47 criteria were meaningfully grouped into four distinct categories: (1) acceptable, (2) compatible, (3) easy, and (4) useful. Average ratings of clarity and importance at the category and individual criteria level will be presented. Conclusions This study advances the field of implementation science and practice by providing clear and conceptually distinct domains of the pragmatic measure construct. Next steps will include a Delphi process to develop consensus on the most important criteria and the development of quantifiable pragmatic rating criteria that can be used to assess measures.

Abstract Background Implementation outcome measures are essential for monitoring and evaluating the success of implementation efforts. Yet, currently available measures lack conceptual clarity and have largely unknown reliability and validity. This study developed and psychometrically assessed three new measures: the Acceptability of Intervention Measure (AIM), Intervention Appropriateness Measure (IAM), and Feasibility of Intervention Measure (FIM). Methods Thirty-six implementation scientists and 27 mental health professionals assigned 31 items to the constructs and rated their confidence in their assignments. The Wilcoxon one-sample signed rank test was used to assess substantive and discriminant content validity. Exploratory and confirmatory factor analysis (EFA and CFA) and Cronbach alphas were used to assess the validity of the conceptual model. Three hundred twenty-six mental health counselors read one of six randomly assigned vignettes depicting a therapist contemplating adopting an evidence-based practice (EBP). Participants used 15 items to rate the therapist’s perceptions of the acceptability, appropriateness, and feasibility of adopting the EBP. CFA and Cronbach alphas were used to refine the scales, assess structural validity, and assess reliability. Analysis of variance (ANOVA) was used to assess known-groups validity. Finally, half of the counselors were randomly assigned to receive the same vignette and the other half the opposite vignette; and all were asked to re-rate acceptability, appropriateness, and feasibility. Pearson correlation coefficients were used to assess test-retest reliability and linear regression to assess sensitivity to change. Results All but five items exhibited substantive and discriminant content validity. A trimmed CFA with five items per construct exhibited acceptable model fit (CFI = 0.98, RMSEA = 0.08) and high factor loadings (0.79 to 0.94). The alphas for 5-item scales were between 0.87 and 0.89. Scale refinement based on measure-specific CFAs and Cronbach alphas using vignette data produced 4-item scales (α’s from 0.85 to 0.91). A three-factor CFA exhibited acceptable fit (CFI = 0.96, RMSEA = 0.08) and high factor loadings (0.75 to 0.89), indicating structural validity. ANOVA showed significant main effects, indicating known-groups validity. Test-retest reliability coefficients ranged from 0.73 to 0.88. Regression analysis indicated each measure was sensitive to change in both directions. Conclusions The AIM, IAM, and FIM demonstrate promising psychometric properties. Predictive validity assessment is planned.

Abstract Background Training is a core component in the implementation of empirically supported treatments, especially in the case of psychosocial interventions targeting mental illness. However, common forms of training are relatively ineffective in producing behavioral changes in providers. Trainers are in a strategic position to influence the success of training, but no research, to our knowledge, has explored whether personal characteristics of trainers (e.g., enthusiasm, charisma) increase effectiveness of training empirically supported treatments in the field of mental health. To address this gap, the current study created a measure of trainer characteristics (the Measure of Effective Attributes of Trainers (MEAT)) and assessed preliminary evidence for its reliability and validity by following gold standard measure development procedures. Methods Measure development consisted of three steps: (1) An initial pool of items was generated based on extant literature, input from the target population, and expert input; (2) target users of the measure interacted with the initial item pool to ensure face validity as well as clarity of measure instructions, response options, and items; and (3) a convenience sample viewed training videos and completed the measure resulting from step 2 to establish preliminary evidence of reliability and validity. An exploratory factor analysis was performed on the measure to determine whether latent factors (i.e., subscales of characteristics) underlie the data. Results The final solution consisted of two factors that demonstrated preliminary evidence for structural validity of the measure. The first factor, labeled “Charisma,” contained items related to characteristics that facilitate a positive personal relationship with the trainee (e.g., friendly, warm), and the second factor, labeled “Credibility,” contained items related to characteristics that emphasize the qualification of the trainer (e.g., professional, experienced). There was also evidence for face validity, content validity, reliability, and known groups validity of the measure. Conclusions The MEAT demonstrated preliminary evidence of key psychometric properties. Future research is needed to further explore and contribute to its psychometric evidence, which could be done in conjunction with measures of trainee knowledge, attitudes towards empirically supported treatments, and evaluations of trainee behavior change to delineate key characteristics of trainers to be leveraged for more effective training.

The field of implementation science was developed to better understand the factors that facilitate or impede implementation and generate evidence for implementation strategies. In this article, we briefly review progress in implementation science, and suggest five priorities for enhancing the impact of implementation strategies. Specifically, we suggest the need to: (1) enhance methods for designing and tailoring implementation strategies; (2) specify and test mechanisms of change; (3) conduct more effectiveness research on discrete, multi-faceted, and tailored implementation strategies; (4) increase economic evaluations of implementation strategies; and (5) improve the tracking and reporting of implementation strategies. We believe that pursuing these priorities will advance implementation science by helping us to understand when, where, why, and how implementation strategies improve implementation effectiveness and subsequent health outcomes.

BackgroundThe science of implementation has offered little toward understanding how different implementation strategies work. To improve outcomes of implementation efforts, the field needs precise, testable theories that describe the causal pathways through which implementation strategies function. In this perspective piece, we describe a four-step approach to developing causal pathway models for implementation strategies.Building causal modelsFirst, it is important to ensure that implementation strategies are appropriately specified. Some strategies in published compilations are well defined but may not be specified in terms of its core component that can have a reliable and measureable impact. Second, linkages between strategies and mechanisms need to be generated. Existing compilations do not offer mechanisms by which strategies act, or the processes or events through which an implementation strategy operates to affect desired implementation outcomes. Third, it is critical to identify proximal and distal outcomes the strategy is theorized to impact, with the former being direct, measurable products of the strategy and the latter being one of eight implementation outcomes (1). Finally, articulating effect modifiers, like preconditions and moderators, allow for an understanding of where, when, and why strategies have an effect on outcomes of interest.Future directionsWe argue for greater precision in use of terms for factors implicated in implementation processes; development of guidelines for selecting research design and study plans that account for practical constructs and allow for the study of mechanisms; psychometrically strong and pragmatic measures of mechanisms; and more robust curation of evidence for knowledge transfer and use.

Following publication of the original article [1] the authors reported an important acknowledgement was mistakenly omitted from the ‘Acknowledgements’ section. The full acknowledgement is included in this Correction article:

Mental health clinicians and administrators are increasingly asked to collect and report treatment outcome data despite numerous challenges to select and use instruments in routine practice. Measurement-based care (MBC) is an evidence-based practice for improving patient care. We propose that data collected from MBC processes with patients can be strategically leveraged by agencies to also support clinicians and respond to accountability requirements. MBC data elements are outlined using the Precision Mental Health Framework (Bickman et al. in Adm Policy Mental Health Mental Health Serv Res 43:271-276, 2016), practical guidance is provided for agency administrators, and conceptual examples illustrate strategic applications of one or more instruments to meet various needs throughout the organization.