Search

Abstract Background Implementation science scholars have made significant progress identifying factors that enable or obstruct the implementation of evidence-based interventions, and testing strategies that may modify those factors. However, little research sheds light on how or why strategies work, in what contexts, and for whom. Studying implementation mechanisms—the processes responsible for change—is crucial for advancing the field of implementation science and enhancing its value in facilitating equitable policy and practice change. The Agency for Healthcare Research and Quality funded a conference series to achieve two aims: (1) develop a research agenda on implementation mechanisms, and (2) actively disseminate the research agenda to research, policy, and practice audiences. This article presents the resulting research agenda, including priorities and actions to encourage its execution. Method Building on prior concept mapping work, in a semi-structured, 3-day, in-person working meeting, 23 US-based researchers used a modified nominal group process to generate priorities and actions for addressing challenges to studying implementation mechanisms. During each of the three 120-min sessions, small groups responded to the prompt: “What actions need to be taken to move this research forward?” The groups brainstormed actions, which were then shared with the full group and discussed with the support of facilitators trained in structured group processes. Facilitators grouped critical and novel ideas into themes. Attendees voted on six themes they prioritized to discuss in a fourth, 120-min session, during which small groups operationalized prioritized actions. Subsequently, all ideas were collated, combined, and revised for clarity by a subset of the authorship team. Results From this multistep process, 150 actions emerged across 10 priority areas, which together constitute the research agenda. Actions included discrete activities, projects, or products, and ways to shift how research is conducted to strengthen the study of implementation mechanisms. Conclusions This research agenda elevates actions to guide the selection, design, and evaluation of implementation mechanisms. By delineating recommended actions to address the challenges of studying implementation mechanisms, this research agenda facilitates expanding the field of implementation science, beyond studying what works to how and why strategies work, in what contexts, for whom, and with which interventions.

IntroductionLarge community-based public health programs, like mass drug administration (MDA), require coordination across many stakeholders. We used social network analysis (SNA) to systematically identify the network of stakeholders who influence delivery of school-based and community-wide MDA for soil-transmitted helminths (STH) in Benin, India, and Malawi and determine how network dynamics may impact implementation and scale-up across these delivery platforms.MethodsThis study was embedded within the implementation science research of the DeWorm3 Project, a hybrid clinical trial in Benin, India, & Malawi testing the feasibility of STH transmission interruption via community-wide MDA. Sites developed lists of stakeholders engaged in both MDA programs and indicated stakeholders’ attitudes towards the intervention and influence over intervention delivery. We developed digital sociograms for both MDA networks by site, comparing baseline vs. endline. We descriptively compared changes over time in stakeholder attitudes and influence and key SNA measures, including centrality, centralization, and density.ResultsAcross sites, we identified an expansive network of stakeholders involved in delivery of school-based (N= 139, 63, 58 vs. N=139, 162, 63) and community-wide MDA programs (N=52, 137, 54 vs. N=54, 136, 60) at baseline vs. endline in Benin, India, and Malawi, respectively. At both timepoints, a majority (>70%) of stakeholders held positive attitudes towards both programs. For both programs, stakeholders with the highest degree centrality scores (i.e., the most connected individuals) were those responsible for implementation such as community drug distributors or school teachers, while those with the highest betweenness centrality scores (i.e. those who controlled resource or information flow across networks) were responsible for policy-making & program leadership (e.g., NTD Program Managers). Low density scores indicated networks had poor overall connectedness due to minimal connectivity across administrative levels, while low centralization scores reflected stable networks where no single individual exhibited high control over resource flow.ConclusionDuring stages of innovation, redesign, or scale-up, analyzing the network of policymakers and implementers provides an opportunity to optimize effectiveness and efficiency of public health programs. Study findings provide useful insight for NTD policymakers and implementers in STH-endemic countries aiming to successfully interrupt STH transmission by transitioning from school-based to community-wide MDA.

Abstract Background Implementation science frameworks situate intervention implementation and sustainment within the context of the implementing organization and system. Aspects of organizational context such as leadership have been defined and measured largely within US health care settings characterized by decentralization and individual autonomy. The relevance of these constructs in other settings may be limited by differences like collectivist orientation, resource constraints, and hierarchical power structures. We aimed to adapt measures of organizational context in South African primary care clinics. Methods We convened a panel of South African experts in social science and HIV care delivery and presented implementation domains informed by existing frameworks and prior work in South Africa. Based on panel input, we selected contextual domains and adapted candidate items. We conducted cognitive interviews with 25 providers in KwaZulu-Natal Province to refine measures. We then conducted a cross-sectional survey of 16 clinics with 5–20 providers per clinic (N = 186). We assessed reliability using Cronbach’s alpha and calculated interrater agreement (awg) and intraclass correlation coefficient (ICC) at the clinic level. Within clinics with moderate agreement, we calculated correlation of clinic-level measures with each other and with hypothesized predictors – staff continuity and infrastructure – and a clinical outcome, patient retention on antiretroviral therapy. Results Panelists emphasized contextual factors; we therefore focused on elements of clinic leadership, stress, cohesion, and collective problem solving (critical consciousness). Cognitive interviews confirmed salience of the domains and improved item clarity. After excluding items related to leaders’ coordination abilities due to missingness and low agreement, all other scales demonstrated individual-level reliability and at least moderate interrater agreement in most facilities. ICC was low for most leadership measures and moderate for others. Measures tended to correlate within facility, and higher stress was significantly correlated with lower staff continuity. Organizational context was generally more positively rated in facilities that showed consistent agreement. Conclusions As theorized, organizational context is important in understanding program implementation within the South African health system. Most adapted measures show good reliability at individual and clinic levels. Additional revision of existing frameworks to suit this context and further testing in high and low performing clinics is warranted.

Background: People with HIV are both at elevated risk of lung cancer and at high risk of multimorbidity, which makes shared decision-making (SDM) for lung cancer screening (LCS) in people with HIV complex. Currently no known tools have been adapted for SDM in people with HIV. Research Question: Can an SDM decision aid be adapted to include HIV-specific measures with input from both people with HIV and their providers? Study Design and Methods: This study used qualitative methods including focus groups of people with HIV and interviews with HIV care providers to adapt and iterate an SDM tool for people with HIV. Eligible participants were those with HIV enrolled in an HIV primary care clinic who met age and smoking eligibility criteria for LCS and HIV care providers at the clinic. Both the focus groups and interviews included semistructured discussions of SDM and decision aid elements for people with HIV. We used a framework-guided thematic analysis, mapping themes onto the Health Equity Implementation framework. Results: Forty-three people with HIV participated in eight focus groups; 10 providers were interviewed. Key themes from patients included broad interest in adapting LCS SDM specifically for people with HIV, a preference for clear LCS recommendations, and the need for positive framing emphasizing survival. Providers were enthusiastic about personalized LCS risk assessments and point-of-care tools. Both patients and providers gave mixed views on the usefulness of HIV-specific risk measures in patient-facing tools. Themes were used to adapt a personalized and flexible SDM tool for LCS in people with HIV. Interpretation: People with HIV and providers were enthusiastic about specific tools for SDM that are personalized and tailored for people with HIV, that make recommendations, and that inform LCS decision-making. Divergent views on presenting patient-facing quantitative risk assessments suggests that these elements could be optional but available for review. This tool may have usefulness in complex decision-making for LCS in this population and currently is being evaluated in a pilot prospective trial.

BackgroundOpinion leadership, educational outreach visiting, and innovation championing are commonly used strategies to address barriers to implementing innovations and evidence-based practices in healthcare settings. Despite voluminous research, ambiguities persist in how these strategies work and under what conditions they work well, work poorly, or work at all. The current paper develops middle-range theories to address this gap.MethodsConceptual articles, systematic reviews, and empirical studies informed the development of causal pathway diagrams (CPDs). CPDs are visualization tools for depicting and theorizing about the causal process through which strategies operate, including the mechanisms they activate, the barriers they address, and the proximal and distal outcomes they produce. CPDs also clarify the contextual conditions (i.e., preconditions and moderators) that influence whether, and to what extent, the strategy's causal process unfolds successfully. Expert panels of implementation scientists and health professionals rated the plausibility of these preliminary CPDs and offered comments and suggestions on them.FindingsTheoretically, opinion leadership addresses potential adopters' uncertainty about likely consequences of innovation use (determinant) by promoting positive attitude formation about the innovation (mechanism), which results in an adoption decision (proximal outcome), which leads to innovation use (intermediate outcome). As this causal process repeats, penetration, or spread of innovation use, occurs (distal outcome). Educational outreach visiting addresses knowledge barriers, attitudinal barriers, and behavioral barriers (determinants) by promoting critical thinking and reflection about evidence and practice (mechanism), which results in behavioral intention (proximal outcome), behavior change (intermediate outcome), and fidelity, or guideline adherence (distal outcome). Innovation championing addresses organizational inertia, indifference, and resistance (determinants) by promoting buy-in to the vision, fostering a positive implementation climate, and increasing collective efficacy (mechanisms), which leads to participation in implementation activities (proximal outcome), initial use of the innovation with increasing skill (intermediate outcome) and, ultimately, greater penetration and fidelity (distal outcomes). Experts found the preliminary CPDs plausible or highly plausible and suggested additional mechanisms, moderators, and preconditions, which were used to amend the initial CPD.DiscussionThe middle-range theories depicted in the CPDs furnish testable propositions for implementation research and offer guidance for selecting, designing, and evaluating these social influence implementation strategies in both research studies and practice settings.

Abstract Background Implementation strategies are strategies to improve uptake of evidence-based practices or interventions and are essential to implementation science. Developing or tailoring implementation strategies may benefit from integrating approaches from other disciplines; yet current guidance on how to effectively incorporate methods from other disciplines to develop and refine innovative implementation strategies is limited. We describe an approach that combines community-engaged methods, human-centered design (HCD) methods, and causal pathway diagramming (CPD)—an implementation science tool to map an implementation strategy as it is intended to work—to develop innovative implementation strategies. Methods We use a case example of developing a conversational agent or chatbot to address racial inequities in breast cancer screening via mammography. With an interdisciplinary team including community members and operational leaders, we conducted a rapid evidence review and elicited qualitative data through interviews and focus groups using HCD methods to identify and prioritize key determinants (facilitators and barriers) of the evidence-based intervention (breast cancer screening) and the implementation strategy (chatbot). We developed a CPD using key determinants and proposed strategy mechanisms and proximal outcomes based in conceptual frameworks. Results We identified key determinants for breast cancer screening and for the chatbot implementation strategy. Mistrust was a key barrier to both completing breast cancer screening and using the chatbot. We focused design for the initial chatbot interaction to engender trust and developed a CPD to guide chatbot development. We used the persuasive health message framework and conceptual frameworks about trust from marketing and artificial intelligence disciplines. We developed a CPD for the initial interaction with the chatbot with engagement as a mechanism to use and trust as a proximal outcome leading to further engagement with the chatbot. Conclusions The use of interdisciplinary methods is core to implementation science. HCD is a particularly synergistic discipline with multiple existing applications of HCD to implementation research. We present an extension of this work and an example of the potential value in an integrated community-engaged approach of HCD and implementation science researchers and methods to combine strengths of both disciplines and develop human-centered implementation strategies rooted in causal perspective and healthcare equity.

Abstract Introduction Adolescent girls and young women (AGYW) who may benefit from HIV pre‐exposure prophylaxis (PrEP) face high levels of common mental disorders (e.g. depression, anxiety). Common mental disorders can reduce PrEP adherence and increase HIV risk, yet mental health interventions have not been well‐integrated into PrEP delivery. Methods We conducted a four‐phase human‐centred design process, from December 2020 to April 2022, to understand mental health challenges among AGYW in Johannesburg, South Africa and barriers to integrated mental health and PrEP services. In the “Discover” phase, we conducted in‐depth interviews with AGYW and key informants (KIs) in Johannesburg. We conducted a rapid qualitative analysis, informed by the Consolidated Framework for Implementation Research (CFIR), to identify facilitators and barriers of integrated mental health and PrEP services and mapped barriers to potential implementation strategies. In the “Design” and “Build” phases, we conducted stakeholder workshops to iteratively adapt an evidence‐based mental health intervention, the Friendship Bench, and refine implementation strategies for South African PrEP delivery settings. In the “Test” phase, we piloted our adapted Friendship Bench package. Results Interviews with 70 Discover phase participants (48 AGYW, 22 KIs) revealed the importance of integrated mental health and PrEP services for South African AGYW. Interviewees described barriers and implementation strategies for mental health and PrEP services around the CFIR domains: intervention characteristics (e.g. challenges with AGYW “opening up”); outer Johannesburg setting (e.g. community stigma); inner clinic setting (e.g. judgemental healthcare providers); characteristics of counsellors (e.g. training gaps); and the implementation process (e.g. need for demand creation). The Design and Build workshops included 13 AGYW and 15 KIs. Implementation barriers related to the quality and accessibility of public‐sector clinic services, lay counsellor training, and community education and demand creation activities were prioritized. This led to 12 key Friendship Bench adaptations and the specification of 10 implementation strategies that were acceptable and feasible in initial pilot testing with three AGYW. Conclusions Using a human‐centred approach, we identified determinants and potential solutions for integrating mental health interventions within PrEP services for South African AGYW. This design process centred stakeholders’ perspectives, enabling rapid development of an adapted Friendship Bench intervention implementation package.

Abstract Introduction Pharmacy‐delivered HIV prevention services might create more options for pregnant women to use HIV prevention tools earlier and more consistently during pregnancy. We quantified preferences for attributes of potential HIV prevention services among women of childbearing age in Western Kenya. Methods From June to November 2023, we administered a face‐to‐face discrete choice experiment survey to women aged 15–44 in Kenya's Homa Bay, Kisumu and Siaya counties. The survey evaluated preferences for HIV prevention services, described by seven attributes: service location, travel time, type of HIV test, sexually transmitted infection (STI) testing, partner HIV testing, pre‐exposure prophylaxis (PrEP) and service fee. Participants answered a series of 12‐choice questions. Each question asked them to select one of two service options or no services—an opt‐out option. We used hierarchical Bayesian modelling levels to estimate each attribute level's coefficient and understand how attributes influenced service choice. Results Overall, 599 participants completed the survey, among whom the median age was 23 years (IQR: 18–27); 33% were married, 20% had a job and worked regularly, and 52% had been pregnant before. Participants, on average, strongly preferred having any HIV prevention service option over none (opt‐out preference weight: −5.84 [95% CI: −5.97, −5.72]). The most important attributes were the availability of PrEP (relative importance 27.04% [95% CI: 25.98%, 28.11%]), followed by STI testing (relative importance 20.26% [95% CI: 19.52%, 21.01%]) and partner HIV testing (relative importance: 16.35% [95% CI: 15.79%, 16.90%]). While, on average, participants preferred obtaining services at the clinic more than pharmacies, women prioritized the availability of PrEP, STI testing and partner HIV testing more than the location or cost. Conclusions These findings suggest the importance of providing comprehensive HIV prevention services and ensuring PrEP, STI testing and partner HIV testing are available. If pharmacies can offer these services, women are likely to access those services at pharmacies even if they prefer clinics.

Abstract Introduction Implementation science (IS) offers methods to systematically achieve the Ending the HIV Epidemic goals in the United States, as well as the global UNAIDS targets. Federal funders such as the National Institutes of Mental Health (NIMH) have invested in implementation research to achieve these goals, including supporting the AIDS Research Centres (ARCs), which focus on high‐impact science in HIV and mental health (MH). To facilitate capacity building for the HIV/MH research workforce in IS, “grey areas,” or areas of IS that are confusing, particularly for new investigators, should be addressed in the context of HIV/MH research. Discussion A group of IS experts affiliated with NIMH‐funded ARCs convened to identify common and challenging grey areas. The group generated a preliminary list of 19 grey areas in HIV/MH‐related IS. From the list, the authors developed a survey which was distributed to all ARCs to prioritize grey areas to address in this paper. ARC members across the United States (N = 60) identified priority grey areas requiring clarification. This commentary discusses topics with 40% or more endorsement. The top grey areas that ARC members identified were: (1) Differentiating implementation strategies from interventions; (2) Determining when an intervention has sufficient evidence for adaptation; (3) Integrating recipient perspectives into HIV/MH implementation research; (4) Evaluating whether an implementation strategy is evidence‐based; (5) Identifying rigorous approaches for evaluating the impact of implementation strategies in the absence of a control group or randomization; and (6) Addressing innovation in HIV/MH IS grants. The commentary addresses each grey area by drawing from the existing literature (when available), providing expert guidance on addressing each in the context of HIV/MH research, and providing domestic and global HIV and HIV/MH case examples that address these grey areas. Conclusions HIV/MH IS is key to achieving domestic and international goals for ending HIV transmission and mitigating its impact. Guidance offered in this paper can help to overcome challenges to rigorous and high‐impact HIV/MH implementation research.

Abstract Background Unhealthy alcohol use (UAU) is particularly dangerous for people with chronic liver disease. Liver clinics may be an important setting in which to provide effective alcohol-related care by integrating evidence-based strategies, such as brief intervention and medications for alcohol use disorder. We conducted qualitative interviews with clinical stakeholders and patients at liver clinics in four Veterans Health Administration (VA) medical centers to understand barriers and facilitators of integrating alcohol-related care and to support tailoring of a practice facilitation implementation intervention. Methods Data collection and analysis were guided by the Consolidated Framework for Implementation Research (CFIR). Interviews were transcribed and qualitatively analyzed using a Rapid Assessment Process (RAP) guided by the CFIR. Results We interviewed 46 clinical stakeholders and 41 patient participants and analyzed findings based on the CFIR. Clinical stakeholders described barriers and facilitators that ranged from operations/clinic resource-based (e.g., time and capacity, desire for additional provider types, referral processes) to individual perspective and preference-based (e.g., supportiveness of leadership, individual experiences/beliefs). Patient participants shared barriers and facilitators that ranged from relationship-based (e.g., trusting the provider and feeling judged) to resource and education-based (e.g., connection to a range of treatment options, education about impact of alcohol). Many barriers and facilitators to integrating alcohol-related care in liver clinics were similar to those identified in other clinical settings (e.g., time, resources, role clarity, stigmatizing beliefs). However, some barriers (e.g., fellow-led care and lack of integration of liver clinics with addictions specialists) and facilitators (e.g., presence of quality improvement staff in clinics and integrated pharmacists and behavioral health specialists) were more unique to liver clinics. Conclusions These findings support the possibility of integrating alcohol-related care into liver clinics but highlight the importance of tailoring efforts to account for variation in provider beliefs and experiences and clinic resources. The barriers and facilitators identified in these interviews were used to tailor a practice facilitation implementation intervention in each clinic setting.

Abstract Background For youth receiving care in community mental health centers, comorbidities are the rule rather than the exception. Using measurement-based care (MBC), or the routine evaluation of symptoms to inform care decisions, as the foundation of treatment for youth with comorbid problems significantly improves the impact of psychotherapy by focusing care and building engagement and alliance. MBC increases the rate of symptom improvement, detects clients who would otherwise deteriorate, and alerts clinicians to non-responders. Despite its demonstrated utility, MBC is rarely implemented with fidelity; less than 15% of providers report using MBC per recommendations. Previous efforts to support MBC implementation have yielded suboptimal outcomes, in part, due to organizations’ challenges with identifying and prioritizing barriers and selecting and developing strategies to overcome them. New methods are needed for identifying and prioritizing barriers, and matching strategies to barriers to optimize MBC implementation and treatment quality to improve youth mental health outcomes in community settings. Methods Pragmatic implementation methods will be piloted in four diverse community mental health centers. Methods include (a) rapid evidence synthesis; (b) rapid ethnography; (c) design kits (e.g., kits with disposable cameras, journals, maps); (d) barrier prioritization, and (e) causal pathway diagramming. These activities will generate actionable barriers; subsequently, we will use facilitated group processes to prioritize barriers and develop causal pathway diagrams to match strategies to barriers to create implementation plans that optimize MBC fidelity (Aim 1). We will track strategy deployment for 6 months, then compare MBC fidelity for another 6 months post-implementation with data from 2 years of historical controls (Aim 2). Finally, we will co-design a toolkit for design kit methods with youth and the practice and scientific communities (Aim 3). Discussion Optimizing MBC implementation in community mental health centers could transform youth mental health care by ensuring the most pressing symptoms are targeted early in treatment. The discussion section highlights expected challenges and limits to using the five methods, including recruitment and engagement given the high pressure on community mental health settings. Trial registration Clinicaltrials.gov. NCT05644756 . Registered on 18 November 2022. This trial was retrospectively registered.

Abstract Background There is a higher risk for HIV acquisition during pregnancy and postpartum. Pre-exposure prophylaxis (PrEP) is recommended during this period for those at high risk of infection; integrated delivery in maternal and child health (MCH) clinics is feasible and acceptable but requires implementation optimization. Methods The PrEP in Pregnancy, Accelerating Reach and Efficiency study (PrEPARE; NCT04712994) engaged stakeholders to prioritize determinants of PrEP delivery (using Likert scores) and prioritize PrEP delivery implementation strategies. Using a sequential explanatory mixed methods design, we conducted quantitative surveys with healthcare workers at 55 facilities in Western Kenya and a stakeholder workshop (including nurses, pharmacists, counselors, and county and national policymakers), yielding visual plots of stakeholders’ perceived feasibility and effectiveness of the strategies. A stepwise elimination process was used to identify seven strategies for empirical testing. Facilitator debriefing reports from the workshop were used to qualitatively assess the decision-making process. Results Among 146 healthcare workers, the strongest reported barriers to PrEP delivery were insufficient providers and inadequate training, insufficient space, and high volume of patients. Sixteen strategies were assessed, 14 of which were included in the final analysis. Among rankings from 182 healthcare workers and 44 PrEP policymakers and implementers, seven strategies were eliminated based on low post-workshop ranking scores (bottom 50th percentile) or being perceived as low feasibility or low effectiveness for at least 50% of the workshop groups. The top seven strategies included delivering PrEP within MCH clinics instead of pharmacies, fast-tracking PrEP clients to reduce waiting time, delivering PrEP-related health talks in waiting bays, task shifting PrEP counseling, task shifting PrEP risk assessments, training different providers to deliver PrEP, and retraining providers on PrEP delivery. All top seven ranked strategies were grouped into bundles for subsequent testing. Facilitator debriefing reports generally aligned with rankings but noted how stakeholders’ decision-making changed when considering the impact of strategies on facility staff and non-PrEP clients. Conclusions The most impactful barriers to integrated PrEP delivery in MCH clinics were insufficient staffing and space. Implementation strategies prioritized through multiple methods of stakeholder input focused on co-location of services and increasing clinic efficiency. Future testing of these stakeholder-prioritized strategy bundles will be conducted to assess the effectiveness and implementation outcomes.

Abstract Background The COVID-19 pandemic resulted in disruptions to routine HIV services for youth living with HIV (YLH), provoking rapid adaptation to mitigate interruptions in care. The Adolescent Transition to Adult Care for HIV-infected adolescents (ATTACH) study (NCT03574129) was a hybrid I cluster randomized trial testing the effectiveness of a healthcare worker (HCW)-delivered disclosure and transition intervention — the Adolescent Transition Package (ATP). During the pandemic, HCWs leveraged phone delivery of the ATP and were supported to make adaptations. We characterized real-time, provider-driven adaptations made to support phone delivery of the ATP. Methods We conducted continuous quality improvement (CQI) meetings with HCWs involved in phone delivery of the ATP at 10 intervention sites. CQI meetings used plan-do-study-act (PDSA) cycles and were audio-recorded. Adaptations were coded by two-independent coders using the Framework for Reporting Adaptations and Modifications to Evidence-based Implementation Strategies (FRAME-IS). Adaptation testing outcomes (adopt, retest, or abandon) and provider experience implementing the adaptations were also recorded. We summarized adaptation characteristics, provider experience, and outcomes. Results We identified 72 adaptations, 32 were unique. Overall, adaptations included modification to context (53%, n = 38), content (49%, n = 35), and evaluation processes (13%, n = 9). Context adaptations primarily featured changes to personnel, format, and setting, while content and evaluation adaptations were frequently achieved by simple additions, repetition, and tailoring/refining of the phone delivery strategy. Nine adaptations involved abandoning, then returning to phone delivery. HCWs sought to increase reach, improve fidelity, and intervention fit within their context. Most adaptations (96%, n = 69) were perceived to increase the feasibility of phone delivery when compared to before the changes were introduced, and HCWs felt 83% (n = 60) of adaptations made phone delivery easier. Most adaptations were either incorporated into routine workflows (47%) or tested again (47%). Conclusion Adaptation of phone delivery was a feasible and effective way of addressing challenges with continuity of care for YLH during the COVID-19 pandemic. Adaptations were primarily context adaptions. While FRAME-IS was apt for characterizing adaptations, more use cases are needed to explore the range of its utility. Trial registration Trial registered on ClinicalTrial.gov as NCT03574129.

Background Implementation strategies are theorized to work well when carefully matched to implementation determinants and when factors—preconditions, moderators, etc.—that influence strategy effectiveness are prospectively identified and addressed. Existing methods for strategy selection are either imprecise or require significant technical expertise and resources, undermining their utility. This article outlines refinements to causal pathway diagrams (CPDs), a method for articulating the causal process through which implementation strategies work and offers illustrations of their use. Method CPDs are a visualization tool to represent an implementation strategy, its mechanism(s) (i.e., the processes through which a strategy is thought to operate), determinants it is intended to address, factors that may impede or facilitate its effectiveness, and the series of outcomes that should be expected if the strategy is operating as intended. We offer principles for constructing CPDs and describe their key functions. Results Applications of the CPD method by study teams from two National Institute of Health-funded Implementation Science Centers and a research grant are presented. These include the use of CPDs to (a) match implementation strategies to determinants, (b) understand the conditions under which an implementation strategy works, and (c) develop causal theories of implementation strategies. Conclusions CPDs offer a novel method for implementers to select, understand, and improve the effectiveness of implementation strategies. They make explicit theoretical assumptions about strategy operation while supporting practical planning. Early applications have led to method refinements and guidance for the field.

Human papillomavirus (HPV) self-collect shows promise to increase cervical cancer screening rates in underscreened populations, such as Somali patients, but little is known about how to integrate such an approach in primary care. In this study, primary care providers and staff who provide primary care services to Somali women were asked for their views on integrating HPV self-collect into routine care to address cervical cancer screening disparities. Thirty primary care providers and staff participated in semi-structured interviews exploring their views on HPV self-collect and their anticipated needs or barriers to implementing this approach into the clinic generally and with specific patient populations, such as Somali women. A thematic analysis using the constructivist version of grounded theory was undertaken. Providers and staff anticipate positive patient reaction to the option of HPV self-collect, and were interested in using this approach both for Somali patients and for all patients in general. HPV self-collect was viewed as straightforward to integrate into existing clinic workflows. Providers largely lacked awareness of the evidence supporting primary HPV testing and HPV self-collect specifically, sharing concerns about effectiveness of self-collect and the lack of a physical exam. Providers felt clinic-wide staff education and patient education, along with strategies to address disparities, such as cultural and linguistic tailoring would be needed for successful implementation. Integrating HPV self-collect as an option in the cervical cancer screening process in a primary care clinical encounter offers considerable opportunity to address health disparities and may benefit all patients.

Abstract Background There is a lack of consensus about how to prioritize potential implementation strategies for HIV pre-exposure prophylaxis (PrEP) delivery. We compared several prioritization methods for their agreement and pragmatism in practice in a resource-limited setting. Methods We engaged diverse stakeholders with clinical PrEP delivery and PrEP decision-making experience across 55 facilities in Kenya to prioritize 16 PrEP delivery strategies. We compared four strategy prioritization methods: (1) “past experience surveys” with experienced practitioners reflecting on implementation experience (N = 182); (2 and 3) “pre- and post-small-group ranking” surveys before and after group discussion (N = 44 and 40); (4) “go-zone” quadrant plots of perceived effectiveness vs feasibility. Kendall’s correlation analysis was used to compare strategy prioritization using the four methods. Additionally, participants were requested to group strategies into three bundles with up to four strategies/bundle by phone and online survey. Results The strategy ranking correlation was strongest between the pre- and post-small-group rankings (Tau: 0.648; p

Abstract Background Organizational readiness is a key factor for successful implementation of evidence-based interventions (EBIs), but a valid and reliable measure to assess readiness across contexts and settings is needed. The R = MC2 heuristic posits that organizational readiness stems from an organization’s motivation, capacity to implement a specific innovation, and its general capacity. This paper describes a process used to examine the face and content validity of items in a readiness survey developed to assess organizational readiness (based on R = MC2) among federally qualified health centers (FQHC) implementing colorectal cancer screening (CRCS) EBIs. Methods We conducted 20 cognitive interviews with FQHC staff (clinical and non-clinical) in South Carolina and Texas. Participants were provided a subset of items from the readiness survey to review. A semi-structured interview guide was developed to elicit feedback from participants using “think aloud” and probing techniques. Participants were recruited using a purposive sampling approach and interviews were conducted virtually using Zoom and WebEx. Participants were asked 1) about the relevancy of items, 2) how they interpreted the meaning of items or specific terms, 3) to identify items that were difficult to understand, and 4) how items could be improved. Interviews were transcribed verbatim and coded in ATLAS.ti. Findings were used to revise the readiness survey. Results Key recommendations included reducing the survey length and removing redundant or difficult to understand items. Additionally, participants recommended using consistent terms throughout (e.g., other units/teams vs. departments) the survey and changing pronouns (e.g., people, we) to be more specific (e.g., leadership, staff). Moreover, participants recommended specifying ambiguous terms (e.g., define what “better” means). Conclusion Use of cognitive interviews allowed for an engaged process to refine an existing measure of readiness. The improved and finalized readiness survey can be used to support and improve implementation of CRCS EBIs in the clinic setting and thus reduce the cancer burden and cancer-related health disparities.

Poor communication within healthcare contributes to inefficiencies, medical errors, conflict, and other adverse outcomes. A promising model to improve outcomes resulting from poor communication in the inpatient hospital setting is Interprofessional Patient- and Family-Centered rounds (IPFCR). IPFCR brings two or more health professions together with hospitalized patients and families as part of a consistent, team-based routine to share information and collaboratively arrive at a daily plan of care. A growing body of literature focuses on implementation and outcomes of IPFCR to improve healthcare quality and team and patient outcomes. Most studies report positive changes following IPFCR implementation. However, conceptual frameworks and theoretical models are lacking in the IPFCR literature and represent a major gap that needs to be addressed to move this field forward. The purpose of this two-part review is to propose a conceptual framework of how IPFCR works. The goal is to articulate a framework that can be tested in subsequent research studies. Published IPFCR literature and relevant theories and frameworks were examined and synthesized to explore how IPFCR works, to situate IPFCR in relation to existing models and frameworks, and to postulate core components and underlying causal mechanisms. A preliminary, context-specific, conceptual framework is proposed illustrating interrelationships between four core components of IPFCR (interprofessional approach, intentional patient and family engagement, rounding structure, shared development of a daily care plan), improvements in communication, and better outcomes.

BackgroundDelivery of PrEP to adolescent girls and young women (AGYW) and to pregnant women through maternal and child health (MCH) and family planning (FP) clinics is scaling up in Kenya. Evaluation of implementation challenges and strategies is critical to optimize delivery.MethodsWe conducted focus group discussions (FGDs) with healthcare workers (HCWs) in MCH and FP clinics offering PrEP in a large implementation project in Kisumu, Kenya. Discussion guides were based on the Consolidated Framework for Implementation Research (CFIR). FGDs were audio recorded and transcribed. Directed content analysis was used to identify implementation challenges and strategies to overcome them.ResultsFifty HCWs from 26 facilities participated in 8 FGDs. HCWs believed PrEP integration was appropriate because it met the needs of AGYW and pregnant women by providing a female-controlled prevention strategy and aligned with policy priorities of elimination of vertical HIV transmission. They were universally accepting of PrEP provision, especially through MCH clinics, noting the relative advantage of this approach because it: (1) enabled high coverage, (2) harmonized PrEP and MCH visits, and (3) minimized stigma compared to PrEP offered through HIV care clinics. However, HCWs noted implementation challenges affecting feasibility and adoption including: (1) increased workload and documentation burden amid workforce shortages, (2) insufficient health care worker knowledge (3) multiple implementing partners with competing priorities (4) drug and documentation form stockouts. HCWs employed various implementation strategies to overcome challenges, including task shifting from nurses to HIV testing providers, patient flow modifications (e.g., fast-tracking PrEP clients to reduce wait times), PrEP demand generation and myth clarification during health talks, provider education, dedicated PrEP delivery rooms, and coordination with adolescent-friendly services. Additional suggested strategies to improve PrEP integration included community education to increase broader PrEP awareness and enable shorter counseling sessions, and task-shifting data entry and client risk assessments.ConclusionsHCWs were enthusiastic about the appropriateness and acceptability of integrating PrEP services into MCH and FP clinics but noted challenges to adoption and feasibility. Strategies to address challenges focused on improving provider time and space constraints, and increasing provider and client knowledge.

Abstract Background There is a fundamental gap in understanding the causal mechanisms by which strategies for implementing evidence-based practices address local barriers to effective, appropriate service delivery. Until this gap is addressed, scientific knowledge and practical guidance about which implementation strategies to use in which contexts will remain elusive. This research project aims to identify plausible strategy-mechanism linkages, develop causal models for mechanism evaluation, produce measures needed to evaluate such linkages, and make these models, methods, and measures available in a user-friendly website. The specific aims are as follows: (1) build a database of strategy-mechanism linkages and associated causal pathway diagrams, (2) develop psychometrically strong, pragmatic measures of mechanisms, and (3) develop and disseminate a website of implementation mechanisms knowledge for use by diverse stakeholders. Methods For the first aim, a combination of qualitative inquiry, expert panel methods, and causal pathway diagramming will be used to identify and confirm plausible strategy-mechanism linkages and articulate moderators, preconditions, and proximal and distal outcomes associated with those linkages. For the second aim, rapid-cycle measure development and testing methods will be employed to create reliable, valid, pragmatic measures of six mechanisms of common strategies for which no high-quality measures exist. For the third aim, we will develop a user-friendly website and searchable database that incorporates user-centered design, disseminating the final product using social marketing principles. Discussion Once strategy-mechanism linkages are identified using this multi-method approach, implementation scientists can use the searchable database to develop tailored implementation strategies and generate more robust evidence about which strategies work best in which contexts. Moreover, practitioners will be better able to select implementation strategies to address their specific implementation problems. New horizons in implementation strategy development, optimization, evaluation, and deployment are expected to be more attainable as a result of this research, which will lead to enhanced implementation of evidence-based interventions for cancer control, and ultimately improvements in patient outcomes.

Abstract Background There is widespread agreement that the integration of cessation services in lung cancer screening (LCS) is essential for achieving the full benefits of LCS with low-dose computed tomography (LDCT). There is a formidable knowledge gap about how to best design feasible, effective, and scalable cessation services in LCS facilities. A collective of NCI-funded clinical trials addressing this gap is the Smoking Cessation at Lung Examination (SCALE) Collaboration. Methods The Cessation and Screening to Save Lives (CASTL) trial seeks to advance knowledge about the reach, effectiveness, and implementation of tobacco treatment in lung cancer screening. We describe the rationale, design, evaluation plan, and interventions tested in this multiphase optimization strategy trial (MOST). A total of 1152 screening-eligible current smokers are being recruited from 18 LCS sites (n = 64/site) in both academic and community settings across the USA. Participants receive enhanced standard care (cessation advice and referral to the national Quitline) and are randomized to receive additional tobacco treatment components (motivational counseling, nicotine replacement patches/lozenges, message framing). The primary outcome is biochemically validated, abstinence at 6 months follow-up. Secondary outcomes are self-reported smoking abstinence, quit attempts, and smoking reduction at 3 and 6 months. Guided by the Implementation Outcomes Framework (IOF), our evaluation includes measurement of implementation processes (reach, fidelity, acceptability, appropriateness, sustainability, and cost). Conclusion We will identify effective treatment components for delivery by LCS sites. The findings will guide the assembly of an optimized smoking cessation package that achieves superior cessation outcomes. Future trials can examine the strategies for wider implementation of tobacco treatment in LDCT-LCS sites. Trial registration ClinicalTrials.gov NCT03315910

Background Barriers and facilitators, collectively called determinants, of evidence-based practice implementation are key to identifying the best strategies for promoting implementation. Assessing determinants before implementation can help tailor strategies to those that would be most effective. Current measures of determinants are not comparable across implementation settings and implementation scientists and practitioners often have to create their own measures. This study was the first step in creating determinants item banks that are usable across settings and focused on intervention characteristics. We aimed to establish the content validity of the item bank. Method This study used a concurrent mixed methods approach. Items for assessing intervention characteristic determinants were first identified through systematic reviews. Implementation scientists then completed a survey where they provided both quantitative and qualitative feedback on the items. Finally, three experts with both clinical and implementation experience provided feedback on redundancy and representativeness. Results The systematic reviews identified over 1,959 items so subsequent steps were limited to focus on intervention characteristic determinants (271 items) such as adaptability of the practice. Based on feedback from thirty implementation scientists, the items were reduced to 92 but an additional 53 were added, most due to qualitative feedback. Items were also rewritten based on qualitative results. Three experts reviewed the remaining 145 items. Based on their feedback, the number of items was reduced to 109. Conclusions Creating a determinants item bank was feasible and the final items had content validity. The next steps include testing reliability and validity in a larger sample of clinicians implementing evidence-based practices.

Abstract Background Significant investments are being made to close the mental health (MH) treatment gap, which often exceeds 90% in many low- and middle-income countries (LMICs). However, limited attention has been paid to patient quality of care in nascent and evolving LMIC MH systems. In system assessments across sub-Saharan Africa, MH loss-to-follow-up often exceeds 50% and sub-optimal medication adherence often exceeds 60%. This study aims to fill a gap of evidence-based implementation strategies targeting the optimization of MH treatment cascades in LMICs by testing a low-cost multicomponent implementation strategy integrated into routine government MH care in Mozambique. Methods Using a cluster-randomized trial design, 16 clinics (8 intervention and 8 control) providing primary MH care will be randomized to the Systems Analysis and Improvement Approach for Mental Health (SAIA-MH) or an attentional placebo control. SAIA-MH is a multicomponent implementation strategy blending external facilitation, clinical consultation, and provider team meetings with system-engineering tools in an overall continuous quality improvement framework. Following a 6-month baseline period, intervention facilities will implement the SAIA-MH strategy for a 2-year intensive implementation period, followed by a 1-year sustainment phase. Primary outcomes will be the proportion of all patients diagnosed with a MH condition and receiving pharmaceutical-based treatment who achieve functional improvement, adherence to medication, and retention in MH care. The Consolidated Framework for Implementation Research (CFIR) will be used to assess determinants of implementation success. Specific Aim 1b will include the evaluation of mechanisms of the SAIA-MH strategy using longitudinal structural equation modeling as well as specific aim 2 estimating cost and cost-effectiveness of scaling-up SAIA-MH in Mozambique to provincial and national levels. Discussion This study is innovative in being the first, to our knowledge, to test a multicomponent implementation strategy for MH care cascade optimization in LMICs. By design, SAIA-MH is a low-cost strategy to generate contextually relevant solutions to barriers to effective primary MH care, and thus focuses on system improvements that can be sustained over the long term. Since SAIA-MH is integrated into routine government MH service delivery, this pragmatic trial has the potential to inform potential SAIA-MH scale-up in Mozambique and other similar LMICs. Trial registration ClinicalTrials.gov; NCT05103033 ; 11/2/2021.

Abstract Background Recent evidence suggests that community-wide mass drug administration (MDA) may interrupt the transmission of soil-transmitted helminths (STH), a group of intestinal worms that infect 1.5 billion individuals globally. Although current operational guidelines provide best practices for effective MDA delivery, they do not describe which activities are most essential for achieving high coverage or how they work together to produce effective intervention delivery. We aimed to identify the various packages of influential intervention delivery activities that result in high coverage of community-wide MDA for STH in Benin, India, and Malawi. Methods We applied coincidence analysis (CNA), a novel cross-case analytical method, to process mapping data as part of the implementation science research of the DeWorm3 Project, a Hybrid Type 1 cluster randomized controlled trial assessing the feasibility of interrupting the transmission of STH using bi-annual community-wide MDA in Benin, India, and Malawi. Our analysis aimed to identify any necessary and/or sufficient combinations of intervention delivery activities (i.e., implementation pathways) that resulted in high MDA coverage. Activities were related to drug supply chain, implementer training, community sensitization strategy, intervention duration, and implementation context. We used pooled implementation data from three sites and six intervention rounds, with study clusters serving as analytical cases (N = 360). Secondary analyses assessed differences in pathways across sites and over intervention rounds. Results Across all three sites and six intervention rounds, efficient duration of MDA delivery (within ten days) singularly emerged as a common and fundamental component for achieving high MDA coverage when combined with other particular activities, including a conducive implementation context, early arrival of albendazole before the planned start of MDA, or a flexible community sensitization strategy. No individual activity proved sufficient by itself for producing high MDA coverage. We observed four possible overall models that could explain effective MDA delivery strategies, all which included efficient duration of MDA delivery as an integral component. Conclusion Efficient duration of MDA delivery uniquely stood out as a highly influential implementation activity for producing high coverage of community-wide MDA for STH. Effective MDA delivery can be achieved with flexible implementation strategies that include various combinations of influential intervention components.

Abstract Background Assisted partner service (APS) is effective for increasing HIV testing services (HTS) uptake among sexual partners of people diagnosed with HIV with rare social harm. The acceptability of APS to HTS providers is important for the quality and effectiveness of APS delivery. Within a larger ongoing implementation science study of APS in western Kenya, we qualitatively evaluated the provider acceptability of APS. Methods From May–June 2020, we conducted virtual, semi-structured in-depth interviews with 14 HTS providers recruited from 8 of 31 study health facilities in Homa Bay and Kisumu counties. Participants were selected using criteria-based purposive sampling to maximize variation on patient volume (assessed by the number of index clients tested for HIV) and APS performance (assessed by sexual partners elicitation and enrollment). Interviews inquired providers’ experiences providing APS including challenges and facilitators and the impact of contextual factors. Data were analyzed using an inductive approach. Results Overall, HTS providers found APS acceptable. It was consistently reported that doing APS was a continuous process rather than a one-day job, which required building rapport and persistent efforts. Benefits of APS including efficiency in HIV case finding, expanded testing coverage in men, and increased HIV status awareness and linkage to care motivated the providers. Provider referral was perceived advantageous in terms of independent contact with partners on behalf of index clients and efficiency in partner tracing. Challenges of providing APS included protecting clients’ confidentiality, difficulty obtaining partners’ accurate contact information, logistic barriers of tracing, and clients’ refusal due to fear of being judged for multiple sexual partners, fear of breach of confidentiality, and HIV stigma. Building rapport with clients, communicating with patience and nonjudgmental attitude and assuring confidentiality were examples of facilitators. Working in rural areas and bigger facilities, training, supportive supervision, and community awareness of APS promoted APS delivery while low salaries, lack of equipment, and high workload undermined it. Conclusions HTS providers found APS acceptable. Delivering APS as a process was the key to success. Future scale-up of APS could consider encouraging provider referral instead of the other APS methods to improve efficiency and reduce potential harm to clients.

Abstract Background Symptoms in patients with advanced cancer are often inadequately captured during encounters with the healthcare team. Emerging evidence demonstrates that weekly electronic home-based patient-reported symptom monitoring with automated alerts to clinicians reduces healthcare utilization, improves health-related quality of life, and lengthens survival. However, oncology practices have lagged in adopting remote symptom monitoring into routine practice, where specific patient populations may have unique barriers. One approach to overcoming barriers is utilizing resources from value-based payment models, such as patient navigators who are ideally positioned to assume a leadership role in remote symptom monitoring implementation. This implementation approach has not been tested in standard of care, and thus optimal implementation strategies are needed for large-scale roll-out. Methods This hybrid type 2 study design evaluates the implementation and effectiveness of remote symptom monitoring for all patients and for diverse populations in two Southern academic medical centers from 2021 to 2026. This study will utilize a pragmatic approach, evaluating real-world data collected during routine care for quantitative implementation and patient outcomes. The Consolidated Framework for Implementation Research (CFIR) will be used to conduct a qualitative evaluation at key time points to assess barriers and facilitators, implementation strategies, fidelity to implementation strategies, and perceived utility of these strategies. We will use a mixed-methods approach for data interpretation to finalize a formal implementation blueprint. Discussion This pragmatic evaluation of real-world implementation of remote symptom monitoring will generate a blueprint for future efforts to scale interventions across health systems with diverse patient populations within value-based healthcare models. Trial registration NCT04809740 ; date of registration 3/22/2021.

Abstract Background HIV assisted partner services (aPS), or provider notification and testing for sexual and injecting partners of people diagnosed with HIV, is shown to be safe, effective, and cost-effective and was scaled up within the national HIV testing services (HTS) program in Kenya in 2016. We estimated the costs of integrating aPS into routine HTS within an ongoing aPS scale-up project in western Kenya. Methods We conducted microcosting using the payer perspective in 14 facilities offering aPS. Although aPS was offered to both males and females testing HIV-positive (index clients), we only collected data on female index clients and their male sex partners (MSP). We used activity-based costing to identify key aPS activities, inputs, resources, and estimated financial and economic costs of goods and services. We analyzed costs by start-up (August 2018), and recurrent costs one-year after aPS implementation (Kisumu: August 2019; Homa Bay: January 2020) and conducted time-and-motion observations of aPS activities. We estimated the incremental costs of aPS, average cost per MSP traced, tested, testing HIV-positive, and on antiretroviral therapy, cost shares, and costs disaggregated by facility. Results Overall, the number of MSPs traced, tested, testing HIV-positive, and on antiretroviral therapy was 1027, 869, 370, and 272 respectively. Average unit costs per MSP traced, tested, testing HIV-positive, and on antiretroviral therapy were $34.54, $42.50, $108.71 and $152.28, respectively, which varied by county and facility client volume. The weighted average incremental cost of integrating aPS was $7,485.97 per facility per year, with recurrent costs accounting for approximately 90% of costs. The largest cost drivers were personnel (49%) and transport (13%). Providers spent approximately 25% of the HTS visit obtaining MSP contact information (HIV-negative clients: 13 out of 54 min; HIV-positive clients: 20 out of 96 min), while the median time spent per MSP traced on phone and in-person was 6 min and 2.5 hours, respectively. Conclusion Average facility costs will increase when integrating aPS to HTS with incremental costs largely driven by personnel and transport. Strategies to efficiently utilize healthcare personnel will be critical for effective, affordable, and sustainable aPS.

Opportunities exist to leverage mobile phones to replace or supplement in-person supervision of lay counselors. However, contextual variables, such as network connectivity and provider preferences, must be considered. Using an iterative and mixed methods approach, we co-developed implementation guidelines to support the implementation of mobile phone supervision with lay counselors and supervisors delivering a culturally adapted trauma-focused cognitive behavioral therapy in Western Kenya. Guidelines were shared and discussed with lay counselors in educational outreach visits led by supervisors. We evaluated the impact of guidelines and outreach on the acceptability, feasibility, and usability of mobile phone supervision. Guidelines were associated with significant improvements in acceptability and usability of mobile phone supervision. There was no evidence of a significant difference in feasibility. Qualitative interviews with lay counselors and supervisors contextualized how guidelines impacted acceptability and feasibility – by setting expectations for mobile phone supervision, emphasizing importance, increasing comfort, and sharing strategies to improve mobile phone supervision. Introducing and discussing co-developed implementation guidelines significantly improved the acceptability and usability of mobile phone supervision. This approach may provide a flexible and scalable model to address challenges with implementing evidence-based practices and implementation strategies in lower-resourced areas.

Assisted partner service (aPS) augments HIV case-finding among sex partners to individuals newly diagnosed with HIV. In 2016, aPS was incorporated into the national HIV testing services (HTS) program in Kenya. We evaluated the extent of, barriers to, and facilitators of aPS integration into HTS. We conducted semi-structured in-depth interviews (IDIs) with 32 stakeholders selected using purposive sampling at national, county, facility, and community levels. IDIs were conducted at two timepoints, at baseline from August-September 2018 in Kisumu and January-June 2019 in Homa Bay, and at follow-up from May-August 2020 to understand changes in aPS integration over time. We defined integration as the creation of linkages between the new intervention (aPS) and the existing HTS program. Data were analyzed using thematic content analysis. We found varying degrees of aPS integration, highest in procurement/logistics and lowest in HTS provider recruitment/training. At baseline, aPS integration was low and activities were at an introductory phase. At follow-up, aPS was integrated in almost the entire HTS program with the exception of low community awareness, which was noted at both baseline and follow-up. There was increasing routinization with establishment of clear aPS cycles, e.g., quarterly data review meetings, annual budget cycles and work-plans. Major barriers included limited government funding, staff constraints, and inadequate community-level sensitization, while key facilitators included increased resources for aPS, and community health volunteer (CHV) facilitated awareness of aPS. Varying degrees of aPS integration across different units of the national HTS program highlights challenges in funding, human resource, and public awareness. Policymakers will need to address these barriers to ensure optimal provision of aPS.

In safety-net healthcare systems, colonoscopy completion within 1-year of an abnormal fecal immunochemical test (FIT) result rarely exceeds 50%. Understanding how electronic health records (EHR) documented reasons for missed colonoscopy match or differ from patient-reported reasons, is critical to optimize effective interventions to address this challenge. We conducted a convergent mixed-methods study which included a retrospective analysis of EHR data and semi-structured interviews of adults 50–75 years old, with abnormal FIT results between 2014 and 2020 in a large safety-net healthcare system. Of the 299 patients identified, 59.2% (n = 177) did not complete a colonoscopy within one year of their abnormal result. EHR abstraction revealed a documented reason for lack of follow-up colonoscopy in 49.2% (n = 87/177); patient-level (e.g., declined colonoscopy; 51.5%) and multi-factorial reasons (e.g., lost to follow-up; 37.9%) were most common. In 18 patient interviews, patient (e.g., fear of colonoscopy), provider (e.g., lack of result awareness), and system-level reasons (e.g., scheduling challenges) were most common. Only three reasons for lack of colonoscopy overlapped between EHR data and patient interviews (competing health issues, lack of transportation, and abnormal FIT result attributed to another cause). In a cohort of safety-net patients with abnormal FIT results, the most common reasons for lack of follow-up were patient-related. Our analysis revealed a discordance between EHR documented and patient-reported reasons for lack of colonoscopy after an abnormal FIT result. Mixed-methods analyses, as in the present study, may give us the greatest insight into modifiable determinants to develop effective interventions beyond quantitative and qualitative data analysis alone.

Abstract Background Current soil-transmitted helminth (STH) control programs target pre-school and school-age children with mass drug administration (MDA) of deworming medications, reducing morbidity without interrupting ongoing transmission. However, evidence suggests that STH elimination may be possible if MDA is delivered to all community members. Such a change to the STH standard-of-care would require substantial systems redesign. We measured baseline structural readiness to launch community-wide MDA for STH in Benin, India, and Malawi. Methods After field piloting and adaptation, the structural readiness survey included two constructs: Organizational Readiness for Implementing Change and Organizational Capacity for Change. Sub-constructs of organizational readiness include change commitment and change efficacy. Sub-constructs of organizational capacity include flexibility, organizational structure, and demonstrated capacity. Survey items were also separately organized into seven implementation domains. Surveys were administered to policymakers, mid-level managers, and implementers in each country using a five-point Likert scale. Item, sub-construct, construct, and domain-level medians and interquartile ranges were calculated for each stakeholder level within each country. Results Median organizational readiness for change scores were highest in Malawi (5.0 for all stakeholder groups). In India, scores were 5.0, 4.0, and 5.0 while in Benin, scores were 4.0, 3.0, and 4.0 for policymakers, mid-level managers, and implementers, respectively. Median change commitment was equal to or higher than median change efficacy across all countries and stakeholder groups. Median organizational capacity for change was highest in India, with a median of 4.5 for policymakers and mid-level managers and 5.0 for implementers. In Malawi, the median capacity was 4.0 for policymakers and implementers, and 3.5 for mid-level managers. In Benin, the median capacity was 4.0 for policymakers and 3.0 for mid-level managers and implementers. Median sub-construct scores varied by stakeholder and country. Across countries, items reflective of the implementation domain ‘policy environment’ were highest while items reflective of the ‘human resource’ domain were consistently lower. Conclusion Across all countries, stakeholders valued community-wide MDA for STH but had less confidence in their collective ability to effectively implement it. Perceived capacity varied by stakeholder group, highlighting the importance of accounting for multi-level stakeholder perspectives when determining organizational preparedness to launch new public health initiatives. Trial registration NCT03014167

Abstract Background Although research continues to support task-shifting as an effective model of delivering evidence-based practices (EBPs), little scholarship has focused how to scale up and sustain task-shifting in low- and middle-income countries, including how to sustainably supervise lay counselors. Ongoing supervision is critical to ensure EBPs are delivered with fidelity; however, the resources and expertise required to provide ongoing supervision may limit the potential to scale up and sustain task shifting. Opportunities may exist to leverage mobile technology to replace or supplement in-person supervision in low-resource contexts, but contextual variables, such as network connectivity and lay counselor preferences surrounding mobile technology, must be examined and considered when designing and implementing mobile technology supervision. Methods This study builds from an existing randomized trial in Kenya, wherein teachers and community health volunteers have been trained to provide trauma-focused cognitive behavioral therapy as lay counselors. The study will use an iterative and mixed methods approach, with qualitative interviews and a Human-Centered Design (HCD) workshop informing a non-randomized pilot trial. Semi-structured interviews will be conducted with lay counselors and supervisors to understand how mobile technology is currently being used for supervision and determine the barriers and facilitators to mobile technology supervision. Data from these interviews will inform an HCD workshop, where lay counselors and supervisors “re-design” supervision to most effectively leverage mobile technology. Workshop participants will also participate in focus groups to gather perceptions on the use of HCD techniques. The final outcome of the workshop will be a set of refined workflows, which will be tested in a mixed method, nonrandomized pilot with newly trained lay counselors and existing supervisors. The pilot trial will evaluate the acceptability, feasibility, and usability of mobile technology supervision through self-report questionnaires as well as perceptions of effectiveness through qualitative interviews with a subset of lay counselors and all supervisors. Discussion This study will provide a launching point for future research on supervision and methods to engage stakeholders to design and tailor interventions and implementation supports to fit low-resourced contexts. Trial registration The parent trial from which this study builds was registered on ClinicalTrials.gov on August 9, 2017 ( NCT03243396 ).

Abstract Background Practice facilitation is a promising strategy to enhance care processes and outcomes in primary care settings. It requires that practices and their facilitators engage as teams to drive improvement. In this analysis, we explored the practice and facilitator factors associated with greater team engagement at the mid-point of a 12-month practice facilitation intervention focused on implementing cardiovascular prevention activities in practice. Understanding factors associated with greater engagement with facilitators in practice-based quality improvement can assist practice facilitation programs with planning and resource allocation. Methods One hundred thirty-six ambulatory care small to medium sized primary care practices that participated in the EvidenceNow initiative’s NC Cooperative, named Heart Health Now (HHN), fit the eligibility criteria for this analysis. We explored the practice and facilitator factors associated with greater team engagement at the mid-point of a 12-month intervention using a retrospective cohort design that included baseline survey data, monthly practice activity implementation data and information about facilitator’s experience. Generalized linear mixed-effects models (GLMMs) identified variables associated with greater odds of team engagement using an ordinal scale for level of team engagement. Results Among our practice cohort, over half were clinician-owned and 27% were Federally Qualified Health Centers. The mean number of clinicians was 4.9 (SD 4.2) and approximately 40% of practices were in Medically Underserved Areas (MUA). GLMMs identified a best fit model. The Model presented as odd ratios and 95% confidence intervals suggests greater odds ratios of higher team engagement with greater practice QI leadership 17.31 (5.24–57.19), [0.00], and practice location in a MUA 7.25 (1.8–29.20), [0.005]. No facilitator characteristics were independently associated with greater engagement. Conclusions Our analysis provides information for practice facilitation stakeholders to consider when considering which practices may be more amendable to embracing facilitation services.

Abstract Background Alcohol-related care, including screening, brief intervention, and provision of/referral to medication or behavioral treatments for alcohol use disorder, could be delivered in liver clinics to better reach patients with chronic liver conditions. However, the provision of alcohol-related care in liver clinics is currently suboptimal. Practice facilitation is an evidence-based implementation strategy that may address barriers, harness facilitators, and optimize the implementation of alcohol-related care in liver clinic settings using a clinic-centered approach. We report the protocol of a study to test a practice facilitation intervention to implement alcohol-related care in four Veterans Health Administration liver clinics. Methods This study will employ a Hybrid Type 3 effectiveness-implementation design, in which implementation outcomes are considered primary and clinical outcomes secondary. Intervention and evaluation design were informed by the Consolidated Framework for Implementation Research. Qualitative data collected from clinical stakeholders and patients were used to tailor the intervention. The intervention involves a 6-month period of external practice facilitation, including regular meetings to identify clinic goals, challenges, and solutions; engagement of clinic champions; provision of training and development of educational materials for clinic staff and patients; and performance monitoring and feedback. Ongoing formative evaluation involves the collection of quantitative facilitator tracking data and qualitative data from meeting notes and patient interviews to describe intervention acceptability, feasibility, and adoption, and adjust implementation as needed. In the summative evaluation, implementation outcomes (clinic rates of screening, brief intervention, and treatment referral/receipt) and clinical outcomes (unhealthy alcohol use, liver health) will be assessed among patients in participating clinics using secondary electronic health record data and interrupted time series analysis. Discussion This will be the first study to our knowledge to test practice facilitation to implement alcohol-related care in liver clinic settings. Results from formative and summative evaluation will inform a framework for the successful implementation of effective alcohol-related care through practice facilitation in liver clinics, which may ultimately lead to better health outcomes for patients with chronic liver disease.

Abstract Background Understanding the mechanisms of implementation strategies (i.e., the processes by which strategies produce desired effects) is important for research to understand why a strategy did or did not achieve its intended effect, and it is important for practice to ensure strategies are designed and selected to directly target determinants or barriers. This study is a systematic review to characterize how mechanisms are conceptualized and measured, how they are studied and evaluated, and how much evidence exists for specific mechanisms. Methods We systematically searched PubMed and CINAHL Plus for implementation studies published between January 1990 and August 2018 that included the terms “mechanism,” “mediator,” or “moderator.” Two authors independently reviewed title and abstracts and then full texts for fit with our inclusion criteria of empirical studies of implementation in health care contexts. Authors extracted data regarding general study information, methods, results, and study design and mechanisms-specific information. Authors used the Mixed Methods Appraisal Tool to assess study quality. Results Search strategies produced 2277 articles, of which 183 were included for full text review. From these we included for data extraction 39 articles plus an additional seven articles were hand-entered from only other review of implementation mechanisms (total = 46 included articles). Most included studies employed quantitative methods (73.9%), while 10.9% were qualitative and 15.2% were mixed methods. Nine unique versions of models testing mechanisms emerged. Fifty-three percent of the studies met half or fewer of the quality indicators. The majority of studies (84.8%) only met three or fewer of the seven criteria stipulated for establishing mechanisms. Conclusions Researchers have undertaken a multitude of approaches to pursue mechanistic implementation research, but our review revealed substantive conceptual, methodological, and measurement issues that must be addressed in order to advance this critical research agenda. To move the field forward, there is need for greater precision to achieve conceptual clarity, attempts to generate testable hypotheses about how and why variables are related, and use of concrete behavioral indicators of proximal outcomes in the case of quantitative research and more directed inquiry in the case of qualitative research.

Abstract Background The Consolidated Framework for Implementation Research (CFIR) is a determinants framework that may require adaptation or contextualization to fit the needs of implementation scientists in low- and middle-income countries (LMICs). The purpose of this review is to characterize how the CFIR has been applied in LMIC contexts, to evaluate the utility of specific constructs to global implementation science research, and to identify opportunities to refine the CFIR to optimize utility in LMIC settings. Methods A systematic literature review was performed to evaluate the use of the CFIR in LMICs. Citation searches were conducted in Medline, CINAHL, PsycINFO, CINAHL, SCOPUS, and Web of Science. Data abstraction included study location, study design, phase of implementation, manner of implementation (ex., data analysis), domains and constructs used, and justifications for use, among other variables. A standardized questionnaire was sent to the corresponding authors of included studies to determine which CFIR domains and constructs authors found to be compatible with use in LMICs and to solicit feedback regarding ways in which CFIR performance could be improved for use in LMICs. Results Our database search yielded 504 articles, of which 34 met final inclusion criteria. The studies took place across 21 countries and focused on 18 different health topics. The studies primarily used qualitative study designs (68%). Over half (59%) of the studies applied the CFIR at study endline, primarily to guide data analysis or to contextualize study findings. Nineteen (59%) of the contacted authors participated in the survey. Authors unanimously identified culture and engaging as compatible with use in global implementation research. Only two constructs, patient needs and resources and individual stages of change were commonly identified as incompatible with use. Author feedback centered on team level influences on implementation, as well as systems characteristics, such as health system architecture. We propose a “Characteristics of Systems” domain and eleven novel constructs be added to the CFIR to increase its compatibility for use in LMICs. Conclusions These additions provide global implementation science practitioners opportunities to account for systems-level determinants operating independently of the implementing organization. Newly proposed constructs require further reliability and validity assessments. Trial registration PROSPERO, CRD42018095762

Abstract Background Across sub-Saharan Africa, evidence-based clinical guidelines to screen and manage hypertension exist; however, country level application is low due to lack of service readiness, uneven health worker motivation, weak accountability of health worker performance, and poor integration of hypertension screening and management with chronic care services. The systems analysis and improvement approach (SAIA) is an evidence-based implementation strategy that combines systems engineering tools into a five-step, facility-level package to improve understanding of gaps (cascade analysis), guide identification and prioritization of low-cost workflow modifications (process mapping), and iteratively test and redesign these modifications (continuous quality improvement). As hypertension screening and management are integrated into chronic care services in sub-Saharan Africa, an opportunity exists to test whether SAIA interventions shown to be effective in improving efficiency and coverage of HIV services can be effective when applied to the non-communicable disease services that leverage the same platform. We hypothesize that SAIA-hypertension (SAIA-HTN) will be effective as an adaptable, scalable model for broad implementation. Methods We will deploy a hybrid type III cluster randomized trial to evaluate the impact of SAIA-HTN on hypertension management in eight intervention and eight control facilities in central Mozambique. Effectiveness outcomes include hypertension cascade flow measures (screening, diagnosis, management, control), as well as hypertension and HIV clinical outcomes among people living with HIV. Cost-effectiveness will be estimated as the incremental costs per additional patient passing through the hypertension cascade steps and the cost per additional disability-adjusted life year averted, from the payer perspective (Ministry of Health). SAIA-HTN implementation fidelity will be measured, and the Consolidated Framework for Implementation Research will guide qualitative evaluation of the implementation process in high- and low-performing facilities to identify determinants of intervention success and failure, and define core and adaptable components of the SAIA-HTN intervention. The Organizational Readiness for Implementing Change scale will measure facility-level readiness for adopting SAIA-HTN. Discussion SAIA packages user-friendly systems engineering tools to guide decision-making by front-line health workers to identify low-cost, contextually appropriate chronic care improvement strategies. By integrating SAIA into routine hypertension screening and management structures, this pragmatic trial is designed to test a model for national scale-up. Trial registration ClinicalTrials.gov NCT04088656 (registered 09/13/2019; https://clinicaltrials.gov/ct2/show/NCT04088656 ).

Objective: Integration of mental health and chronic disease services in primary care could reduce the mental health treatment gap and improve associated health outcomes in low-resource settings. Low rates of nurse identification and referral of patients with depression limit the effectiveness of integrated mental health care; the barriers to and facilitators of identification and referral in South Africa and comparable settings remain undefined. This study explored barriers to and facilitators of nurse identification and referral of patients with depressive symptoms as part of integrated mental health service delivery in KwaZulu-Natal, South Africa. Design: Triangulation mixed methods study incorporating qualitative and quantitative data. Methods: Data collection, analysis, and interpretation were guided by the Consolidated Framework for Implementation Research (CFIR). Participants were professional nurses at ten primary health care facilities in Amajuba, KwaZulu-Natal, South Africa. Qualitative data collection involved semi-structured interviews targeting specific CFIR constructs with high- and low-referring nurses. Deductive and inductive coding were used to derive primary themes related to barriers and facilitators. Quantitative data collection involved a structured questionnaire assessing determinants explored in the interviews. Qualitative comparative analysis was used to identify the necessary or sufficient conditions for high and low nurse referral. Results: Twenty-two nurses were interviewed. Primary themes related to insufficient training, supervision, and competency; emotional burden; limited human and physical resources; perceived patient need for integrated services; and intervention acceptability. Sixty-eight nurses completed questionnaires. Quantitative results confirmed and expanded upon the qualitative findings. Low self-assessed competency was a consistent barrier to appropriate service delivery. Conclusions: To promote the success of integrated care in a context of severe staff shortages and over-burdened providers, implementation strategies including direct training, structured supervision, and routine behavioral health screening tools are warranted. Interventions to improve mental health literacy of patients as well as emotional support for nurses are also needed.

Abstract Background The mental health treatment gap—the difference between those with mental health need and those who receive treatment—is high in low- and middle-income countries. Task-shifting has been used to address the shortage of mental health professionals, with a growing body of research demonstrating the effectiveness of mental health interventions delivered through task-shifting. However, very little research has focused on how to embed, support, and sustain task-shifting in government-funded systems with potential for scale up. The goal of the Building and Sustaining Interventions for Children (BASIC) study is to examine implementation policies and practices that predict adoption, fidelity, and sustainment of a mental health intervention in the education sector via teacher delivery and the health sector via community health volunteer delivery. Methods BASIC is a Hybrid Type II Implementation-Effectiveness trial. The study design is a stepped wedge, cluster randomized trial involving 7 sequences of 40 schools and 40 communities surrounding the schools. Enrollment consists of 120 teachers, 120 community health volunteers, up to 80 site leaders, and up to 1280 youth and one of their primary guardians. The evidence-based mental health intervention is a locally adapted version of Trauma-focused Cognitive Behavioral Therapy, called Pamoja Tunaweza. Lay counselors are trained and supervised in Pamoja Tunaweza by local trainers who are experienced in delivering the intervention and who participated in a Train-the-Trainer model of skills transfer. After the first sequence completes implementation, in-depth interviews are conducted with initial implementing sites’ counselors and leaders. Findings are used to inform delivery of implementation facilitation for subsequent sequences’ sites. We use a mixed methods approach including qualitative comparative analysis to identify necessary and sufficient implementation policies and practices that predict 3 implementation outcomes of interest: adoption, fidelity, and sustainment. We also examine child mental health outcomes and cost of the intervention in both the education and health sectors. Discussion The BASIC study will provide knowledge about how implementation of task-shifted mental health care can be supported in government systems that already serve children and adolescents. Knowledge about implementation policies and practices from BASIC can advance the science of implementation in low-resource contexts. Trial registration Trial Registration: ClinicalTrials.gov Identifier: NCT03243396. Registered 9th August 2017, https://clinicaltrials.gov/ct2/show/NCT03243396

Abstract In the 13 years since the inception of Implementation Science, we have witnessed a continued rise in the number of submissions, reflecting the growing global interest in methods to enhance the uptake of research findings into healthcare practice and policy. We now receive over 800 submissions annually, and there is a large gap between what is submitted and what gets published. To better serve the needs of the research community, we announce our plans to introduce a new journal, Implementation Science Communications, which we believe will support publication of types of research reports currently not often published in Implementation Science. In this editorial, we state both journals’ scope and current boundaries and set out our expectations for the scientific reporting, quality, and transparency of the manuscripts we receive.

Abstract Background As lower-income countries look to develop a mature healthcare workforce and to improve quality and reduce costs, they are increasingly turning to quality improvement (QI), a widely-used strategy in higher-income countries. Although QI is an effective strategy for promoting evidence-based practices, QI interventions often fail to deliver desired results. This failure may reflect a problem with implementation. As the key implementing unit of QI, teams are critical for the success or failure of QI efforts. Thus, we used the model of work-team learning to identify factors related to the effectiveness of newly-formed hospital-based QI teams in Ghana. Methods This was a cross-sectional, observational study. We used structural equation modeling to estimate relationships between coaching-oriented team leadership, perceived support for teamwork, team psychological safety, team learning behavior, and QI implementation. We used an observer-rated measure of QI implementation, our outcome of interest. Team-level factors were measured using aggregated survey data from 490 QI team members, resulting in a sample size of 122 teams. We assessed model fit and tested significance of standardized parameters, including direct and indirect effects. Results Learning behavior mediated a positive relationship between psychological safety and QI implementation (β = 0.171, p = 0.001). Psychological safety mediated a positive relationship between team leadership and learning behavior (β = 0.384, p = 0.068). Perceived support for teamwork did not have a significant effect on psychological safety or learning behavior. Conclusions Psychological safety and learning behavior are key for the success of newly formed QI teams working in lower-income countries. Organizational leaders and implementation facilitators should consider these leverage points as they work to establish an environment where QI and other team-based activities are supported and encouraged.

Abstract Background The introduction of option B+—rapid initiation of lifelong antiretroviral therapy regardless of disease status for HIV-infected pregnant and breastfeeding women—can dramatically reduce HIV transmission during pregnancy, birth, and breastfeeding. Despite significant investments to scale-up Option B+, results have been mixed, with high rates of loss to follow-up, sub-optimal viral suppression, continued pediatric HIV transmission, and HIV-associated maternal morbidity. The Systems Analysis and Improvement Approach (SAIA) cluster randomized trial demonstrated that a package of systems engineering tools improved flow through the prevention of mother-to-child HIV transmission (PMTCT) cascade. This five-step, facility-level intervention is designed to improve understanding of gaps (cascade analysis), guide identification and prioritization of low-cost workflow modifications (process mapping), and iteratively test and redesign these modifications (continuous quality improvement). This protocol describes a novel model for SAIA delivery (SAIA-SCALE) led by district nurse supervisors (rather than research nurses), and evaluation procedures, to serve as a foundation for national scale-up. Methods The SAIA-SCALE stepped wedge trial includes three implementation waves, each 12 months in duration. Districts are the unit of assignment, with four districts randomly assigned per wave, covering all 12 districts in Manica province, Mozambique. In each district, the three highest volume health facilities will receive the SAIA-SCALE intervention (totaling 36 intervention facilities). The RE-AIM framework will guide SAIA-SCALE’s evaluation. Reach describes the proportion of clinics and population in Manica province reached, and sub-groups not reached. Effectiveness assesses impact on PMTCT process measures and patient-level outcomes. Adoption describes the proportion of districts/clinics adopting SAIA-SCALE, and determinants of adoption using the Organizational Readiness for Implementing Change (ORIC) tool. Implementation will identify SAIA-SCALE core elements and determinants of successful implementation using the Consolidated Framework for Implementation Research (CFIR). Maintenance describes the proportion of districts sustaining the intervention. We will also estimate the budget and program impact from the payer perspective for national scale-up. Discussion SAIA packages user-friendly systems engineering tools to guide decision-making by frontline health workers, and to identify low-cost, contextually appropriate PMTCT improvement strategies. By integrating SAIA delivery into routine management structures, this pragmatic trial is designed to test a model for national intervention scale-up. Trial registration ClinicalTrials.gov NCT03425136 (registered 02/06/2018).

The community health worker (CHW) asthma home-visiting model developed by Public Health-Seattle & King County (PHSKC) is an evidence-based approach proven to improve health outcomes and quality of life. In addition, it has been shown to be an effective and culturally appropriate approach to helping people with asthma understand the environmental and behavioral causes of uncontrolled asthma, while acquiring the skills they need to control their asthma. This paper describes the development and implementation of training curricula for CHWs and supervisors in the asthma home visiting program. To facilitate dissemination, this program took advantage of the current healthcare landscape in Washington State resulting from Centers for Medicare & Medicaid Services (CMS) approval of the 1115 Medicaid Waiver project. Key aspects of the training program development included: (1) Engagement: forming a Community Advisory Board with multiple stakeholders to help prioritize training content; (2) Curriculum Development: building the training on evidence-based home-visit protocols previously developed at PHSKC; (3) Implementation of the training program; (4) Evaluation of the training; and (5) Adaptation of the training based on lessons learned. We describe key factors in the training program's improvement including the use of a community-based participatory approach to engage stakeholders at multiple phases of the project and ensure regional adaption; combining in-person and online modules for delivery; and holding learning collaboratives for post-training and technical support. We also outline our training program evaluation plan and the planned evaluation of the home visit program which the trainees will deliver, both of which follow the RE-AIM framework. However, because the COVID-19 pandemic has curtailed training activities and prohibited the trainees from implementation of these CHW home visit practices, our evaluation is currently incomplete. Therefore, this case study provides insight into the adaptation of the training program, but not the delivery of the home visit program, the outcomes of which remain to be seen.

The COVID-19 pandemic continues to expand globally, requiring massive public health responses from national and local governments. These bodies have taken heterogeneous approaches to their responses, including when and how to introduce and enforce evidence-based interventions—such as social distancing, hand-washing, personal protective equipment (PPE), and testing. In this commentary, we reflect on opportunities for implementation science to contribute meaningfully to the COVID-19 pandemic response. We reflect backwards on missed opportunities in emergency preparedness planning, using the example of PPE stockpiling and supply management; this planning could have been strengthened through process mapping with consensus-building, microplanning with simulation, and stakeholder engagement. We propose current opportunities for action, focusing on enhancing the adoption, fidelity, and sustainment of hand washing and social distancing; we can combine qualitative data, policy analysis, and dissemination science to inform agile and rapid adjustment to social marketing strategies to enhance their penetration. We look to future opportunities to enhance the integration of new evidence in decision-making, focusing on serologic and virologic testing systems; we can leverage simulation and other systems engineering modeling to identify ideal system structures. Finally, we discuss the ways in which the COVID-19 pandemic challenges implementation science to become more rapid, rigorous, and nimble in its approach, and integrate with public health practice. In summary, we articulate the ways in which implementation science can inform, and be informed by, the COVID-19 pandemic, looking backwards, proposing actions for the moment, and approaches for the future.

Background: Systematic measure reviews can facilitate advances in implementation research and practice by locating reliable, valid, pragmatic measures; identifying promising measures needing refinement and testing; and highlighting measurement gaps. This review identifies and evaluates the psychometric and pragmatic properties of measures of readiness for implementation and its sub-constructs as delineated in the Consolidated Framework for Implementation Research: leadership engagement, available resources, and access to knowledge and information. Methods: The systematic review methodology is described fully elsewhere. The review, which focused on measures used in mental or behavioral health, proceeded in three phases. Phase I, data collection, involved search string generation, title and abstract screening, full text review, construct assignment, and cited citation searches. Phase II, data extraction, involved coding relevant psychometric and pragmatic information. Phase III, data analysis, involved two trained specialists independently rating each measure using Psychometric and Pragmatic Evidence Rating Scales (PAPERS). Frequencies and central tendencies summarized information availability and PAPERS ratings. Results: Searches identified 9 measures of readiness for implementation, 24 measures of leadership engagement, 17 measures of available resources, and 6 measures of access to knowledge and information. Information about internal consistency was available for most measures. Information about other psychometric properties was often not available. Ratings for internal consistency were “adequate” or “good.” Ratings for other psychometric properties were less than “adequate.” Information on pragmatic properties was most often available regarding cost, language readability, and brevity. Information was less often available regarding training burden and interpretation burden. Cost and language readability generally exhibited “good” or “excellent” ratings, interpretation burden generally exhibiting “minimal” ratings, and training burden and brevity exhibiting mixed ratings across measures. Conclusion: Measures of readiness for implementation and its sub-constructs used in mental health and behavioral health care are unevenly distributed, exhibit unknown or low psychometric quality, and demonstrate mixed pragmatic properties. This review identified a few promising measures, but targeted efforts are needed to systematically develop and test measures that are useful for both research and practice. Plain language abstract: Successful implementation of effective mental health or behavioral health treatments in service delivery settings depends in part on the readiness of the service providers and administrators to implement the treatment; the engagement of organizational leaders in the implementation effort; the resources available to support implementation, such as time, money, space, and training; and the accessibility of knowledge and information among service providers about the treatment and how it works. It is important that the methods for measuring these factors are dependable, accurate, and practical; otherwise, we cannot assess their presence or strength with confidence or know whether efforts to increase their presence or strength have worked. This systematic review of published studies sought to identify and evaluate the quality of questionnaires (referred to as measures) that assess readiness for implementation, leadership engagement, available resources, and access to knowledge and information. We identified 56 measures of these factors and rated their quality in terms of how dependable, accurate, and practical they are. Our findings indicate there is much work to be done to improve the quality of available measures; we offer several recommendations for doing so.

Abstract Context Implementation science measures are rarely used by stakeholders to inform and enhance clinical program change. Little is known about what makes implementation measures pragmatic (i.e., practical) for use in community settings; thus, the present study’s objective was to generate a clinical stakeholder-driven operationalization of a pragmatic measures construct. Evidence acquisition The pragmatic measures construct was defined using: 1) a systematic literature review to identify dimensions of the construct using PsycINFO and PubMed databases, and 2) interviews with an international stakeholder panel (N = 7) who were asked about their perspectives of pragmatic measures. Evidence synthesis Combined results from the systematic literature review and stakeholder interviews revealed a final list of 47 short statements (e.g., feasible, low cost, brief) describing pragmatic measures, which will allow for the development of a rigorous, stakeholder-driven conceptualization of the pragmatic measures construct. Conclusions Results revealed significant overlap between terms related to the pragmatic construct in the existing literature and stakeholder interviews. However, a number of terms were unique to each methodology. This underscores the importance of understanding stakeholder perspectives of criteria measuring the pragmatic construct. These results will be used to inform future phases of the project where stakeholders will determine the relative importance and clarity of each dimension of the pragmatic construct, as well as their priorities for the pragmatic dimensions. Taken together, these results will be incorporated into a pragmatic rating system for existing implementation science measures to support implementation science and practice.

Abstract Background Implementation science is the study of strategies used to integrate evidence-based practices into real-world settings (Eccles and Mittman, Implement Sci. 1(1):1, 2006). Central to the identification of replicable, feasible, and effective implementation strategies is the ability to assess the impact of contextual constructs and intervention characteristics that may influence implementation, but several measurement issues make this work quite difficult. For instance, it is unclear which constructs have no measures and which measures have any evidence of psychometric properties like reliability and validity. As part of a larger set of studies to advance implementation science measurement (Lewis et al., Implement Sci. 10:102, 2015), we will complete systematic reviews of measures that map onto the Consolidated Framework for Implementation Research (Damschroder et al., Implement Sci. 4:50, 2009) and the Implementation Outcomes Framework (Proctor et al., Adm Policy Ment Health. 38(2):65-76, 2011), the protocol for which is described in this manuscript. Methods Our primary databases will be PubMed and Embase. Our search strings will be comprised of five levels: (1) the outcome or construct term; (2) terms for measure; (3) terms for evidence-based practice; (4) terms for implementation; and (5) terms for mental health. Two trained research specialists will independently review all titles and abstracts followed by full-text review for inclusion. The research specialists will then conduct measure-forward searches using the “cited by” function to identify all published empirical studies using each measure. The measure and associated publications will be compiled in a packet for data extraction. Data relevant to our Psychometric and Pragmatic Evidence Rating Scale (PAPERS) will be independently extracted and then rated using a worst score counts methodology reflecting “poor” to “excellent” evidence. Discussion We will build a centralized, accessible, searchable repository through which researchers, practitioners, and other stakeholders can identify psychometrically and pragmatically strong measures of implementation contexts, processes, and outcomes. By facilitating the employment of psychometrically and pragmatically strong measures identified through this systematic review, the repository would enhance the cumulativeness, reproducibility, and applicability of research findings in the rapidly growing field of implementation science.