Search

Abstract Background Effective implementation strategies are needed to optimize advancements in the fields of cancer diagnosis, treatment, survivorship, and end-of-life care. We conducted a review of systematic reviews to better understand the evidentiary base of implementation strategies in cancer control. Methods Using three databases, we conducted a search and identified English-language systematic reviews published between 2005 and 2010 that targeted consumer, professional, organizational, regulatory, or financial interventions, tested exclusively or partially in a cancer context (primary focus); generic or non-cancer-specific reviews were also considered. Data were extracted, appraised, and analyzed by members of the research team, and research ideas to advance the field were proposed. Results Thirty-four systematic reviews providing 41 summaries of evidence on 19 unique interventions comprised the evidence base. AMSTAR quality ratings ranged between 2 and 10. Team members rated most of the interventions as promising and in need of further research, and 64 research ideas were identified. Conclusions While many interventions show promise of effectiveness in the cancer-control context, few reviews were able to conclude definitively in favor of or against a specific intervention. We discuss the complexity of implementation research and offer suggestions to advance the science in this area.

Abstract Background Practice guidelines (PGs) are systematically developed statements intended to assist in patient and practitioner decisions. The AGREE II is the revised tool for PG development, reporting, and evaluation, comprised of 23 items, two global rating scores, and a new User's Manual. In this study, we sought to develop, execute, and evaluate the impact of two internet interventions designed to accelerate the capacity of stakeholders to use the AGREE II. Methods Participants were randomized to one of three training conditions. 'Tutorial'--participants proceeded through the online tutorial with a virtual coach and reviewed a PDF copy of the AGREE II. 'Tutorial + Practice Exercise'--in addition to the Tutorial, participants also appraised a 'practice' PG. For the practice PG appraisal, participants received feedback on how their scores compared to expert norms and formative feedback if scores fell outside the predefined range. 'AGREE II User's Manual PDF (control condition)'--participants reviewed a PDF copy of the AGREE II only. All participants evaluated a test PG using the AGREE II. Outcomes of interest were learners' performance, satisfaction, self-efficacy, mental effort, time-on-task, and perceptions of AGREE II. Results No differences emerged between training conditions on any of the outcome measures. Conclusions We believe these results can be explained by better than anticipated performance of the AGREE II PDF materials (control condition) or the participants' level of health methodology and PG experience rather than the failure of the online training interventions. Some data suggest the online tools may be useful for trainees new to this field; however, this requires further study.

Abstract Background Self-administered surveys are an essential methodological tool for health services and knowledge translation research, and engaging end-users of the research is critical. However, few documented accounts of the efforts invested in recruitment of multiple different stakeholders to one health services research study exist. Here, we highlight the challenges of recruiting key stakeholders (policy-makers, clinicians, guideline developers) to a Canadian Institutes of Health Research (CIHR) funded health services research (HSR) study aimed to develop an updated and refined version of a guideline appraisal tool, the AGREE. Methods Using evidence-based methods of recruitment, our goal was to recruit 192 individuals: 80 international guideline developers, 80 Canadian clinicians and 32 Canadian policy/decision-makers. We calculated the participation rate and the recruitment efficiency. Results We mailed 873 invitation letters. Of 838 approached, our participation rate was 29%(240) and recruitment efficiency, 19%(156). One policy-maker manager did not allow policy staff to participate in the study. Conclusions Based on the results from this study, we suggest that future studies aiming to engage similar stakeholders in HSR over sample by at least 5 times to achieve their target sample size and allow for participant withdrawals. We need continued efforts to communicate the value of research between researchers and end-users of research (policy-makers, clinicians, and other researchers), integration of participatory research strategies, and promotion of the value of end-user involvement in research. Future research to understand methods of improving recruitment efficiency and engaging key stakeholders in HSR is warranted.

Abstract Background Practice guidelines (PGs) are systematically developed statements intended to assist in patient, practitioner, and policy decisions. The AGREE II is the revised and updated standard tool for guideline development, reporting and evaluation. It is comprised of 23 items and a user's Manual. The AGREE II is ready for use. Objectives To develop, execute, and evaluate the impact of two internet-based educational interventions designed to accelerate the capacity of stakeholders to use the AGREE II: a multimedia didactic tutorial with a virtual coach, and a higher intensity training program including both the didactic tutorial and an interactive practice exercise component. Methods Participants (clinicians, developers, and policy makers) will be randomly assigned to one of three conditions. Condition one, didactic tutorial -- participants will go through the on-line AGREE II tutorial supported by a virtual coach and review of the AGREE II prior to appraising the test PG. Condition two, tutorial + practice -- following the multimedia didactic tutorial with a virtual coach, participants will review the on-line AGREE II independently and use it to appraise a practice PG. Upon entering their AGREE II score for the practice PG, participants will be given immediate feedback on how their score compares to expert norms. If their score falls outside a predefined range, the participant will receive a series of hints to guide the appraisal process. Participants will receive an overall summary of their performance appraising the PG compared to expert norms. Condition three, control arm -- participants will receive a PDF copy of the AGREE II for review and to appraise the test PG on-line. All participants will then rate one of ten test PGs with the AGREE II. The outcomes of interest are learners' performance, satisfaction, self-efficacy, mental effort, and time-on-task; comparisons will be made across each of the test groups. Discussion Our research will test innovative educational interventions of various intensities and instructional design to promote the adoption of AGREE II and to identify those strategies that are most effective for training. The results will facilitate international capacity to apply the AGREE II accurately and with confidence and to enhance the overall guideline enterprise.

Background: The impact of social frailty on older adults is profound including mortality risk, functional decline, falls, and disability. However, effective strategies that respond to the needs of socially frail older adults are lacking and few studies have unpacked how social determinants operate or how interventions can be adapted during periods requiring social distancing and isolation such as the COVID-19 pandemic. To address these gaps, we conducted a scoping review using JBI methodology to identify interventions that have the best potential to help socially frail older adults (age ≥65 years)., Methods: We searched MEDLINE, CINAHL (EPSCO), EMBASE and COVID-19 databases and the grey literature. Eligibility criteria were developed using the PICOS framework. Our results were summarized descriptively according to study, patient, intervention and outcome characteristics. Data synthesis involved charting and categorizing identified interventions using a social frailty framework.  RESULTS: Of 263 included studies, we identified 495 interventions involving ~124,498 older adults who were mostly female. The largest proportion of older adults (40.5%) had a mean age range of 70-79 years. The 495 interventions were spread across four social frailty domains: social resource (40%), self-management (32%), social behavioural activity (28%), and general resource (0.4%). Of these, 189 interventions were effective for improving loneliness, social and health and wellbeing outcomes across psychological self-management, self-management education, leisure activity, physical activity, Information Communication Technology and socially assistive robot interventions. Sixty-three interventions were identified as feasible to be adapted during infectious disease outbreaks (e.g., COVID-19, flu) to help socially frail older adults., Conclusions: Our scoping review identified promising interventions with the best potential to help older adults living with social frailty., (© 2024. The Author(s).)

Objectives: The objective of this study was to conduct a formative evaluation of the Art Your Service (AYS) arts-based program to determine the program's potential for improving the social and physical well-being of older adults during the COVID-19 pandemic., Design, Settings and Participants: An online questionnaire was administered to the AYS members who consented to be invited to participate in the study. Questionnaire items consisted of a Likert scale and open-ended questions delivered using an online platform (SurveyMonkey). Participants provided feedback on their perceptions and experiences of the AYS program, such as its impact on their health and well-being during the COVID-19 pandemic, the benefits and challenges of participating, and any suggestions for program improvement., Outcome Measures and Analysis: Quantitative data were analyzed using descriptive statistics (frequencies, means with standard deviations), and open-ended questions (qualitative data) were analyzed using content analysis. Outcomes included participant demographics, perceptions about the program, usability (System Usability Scale [SUS]), eHealth literacy (eHealth Literacy Scale), and social isolation (Lubben Social Network Scale; LSNS-6)., Results: Program participants revealed consistent patterns of their perceptions and experiences about the program, including a high satisfaction rate (95%) and a perceived positive impact on participants' health and well-being. The program sessions were perceived to be a well-organized, convenient, and safe way to engage with one another socially during the COVID-19 pandemic. The program usability was also perceived to be high (SUS mean score 86.2). Participants felt a sense of connectedness and had reduced feelings of social isolation. Most participants (75%) reported that the program improved their physical health., Conclusions: Findings from this formative evaluation study identified key strengths and opportunities to improve the Art Your Service arts-based program, which can be used to help enhance the program's functioning and long-term sustainability potential., (© 2024. The Author(s).)

Objectives: To identify candidate quality indicators from existing tools that provide guidance on how to practice knowledge translation and implemenation science (KT practice tools) across KT domains (dissemination, implementation, sustainability, and scalability)., Study Design and Setting: We conducted a scoping review using the Joanna Briggs Institute Manual for Evidence Synthesis. We systematically searched multiple electronic databases and the gray literature. Documents were independently screened, selected, and extracted by pairs of reviewers. Data about the included articles, KT practice tools, and candidate quality indicators were analyzed, categorized, and summarized descriptively., Results: Of 43,060 titles and abstracts that were screened from electronic databases and gray literature, 850 potentially relevant full-text articles were identified, and 253 articles were included in the scoping review. Of these, we identified 232 unique KT practice tools from which 27 unique candidate quality indicators were generated. The identified candidate quality indicators were categorized according to the development (n = 17), evaluation (n = 5) and adaptation (n = 3) of the tools, and engagement of knowledge users (n = 2). No tools were identified that appraised the quality of KT practice tools., Conclusions: The development of a quality appraisal instrument of KT practice tools is needed. The results will be further refined and finalized in order to develop a quality appraisal instrument for KT practice tools., Competing Interests: Declaration of competing interest The authors declare no conflict of interest., (Copyright © 2023 The Authors. Published by Elsevier Inc. All rights reserved.)

Background. Despite local training opportunities for Cognitive Stimulation Therapy (CST) for occupational therapists, there has been limited evaluation of the feasibility of implementation in Canada. Purpose. This study explored the feasibility of CST delivery by an occupational therapist in an Ontario health care setting and the impact it may have on self-efficacy and hope measures of individuals with dementia. Methods. A mixed-methods experimental pre/post design was used. Survey measures included hope, self-efficacy, feasibility, satisfaction, and individual session evaluation. Semistructured focus groups were held for participants and facilitators. Findings. Quantitative findings are summarized descriptively for the 10 participants. Qualitative findings were grouped into themes: social connectedness, knowledge gained and shared, tailored implementation adjustments, and need for long-term programing. Implications. Occupational therapists are well-positioned to implement CST and should be a part of further research to test the intervention rigorously for applicability within a range of Canadian health care settings.

Objective: The objective of this scoping review is to identify and characterize relevant knowledge translation methods tools (those that provide guidance for optimized knowledge translation practice) to uncover candidate quality indicators to inform a future quality assessment tool for knowledge translation strategies., Introduction: Knowledge translation strategies (defined as including knowledge translation interventions, tools, and products) target various knowledge users, including patients, clinicians, researchers, and policy-makers. The development and use of strategies that support knowledge translation practice have been rapidly increasing, making it difficult for knowledge users to decide which to use. There is limited evidence-based guidance or measures to help assess the overall quality of knowledge translation strategies., Inclusion Criteria: Empirical and non-empirical documents will be considered if they explicitly describe a knowledge translation methods tool and its development, evaluation or validation, methodological strengths or limitations, and/or use over time. The review will consider a knowledge translation methods tool if it falls within at least one knowledge translation domain (ie, implementation, dissemination, sustainability, scalability, integrated knowledge translation) in the health field., Methods: We will conduct a systematic search of relevant electronic databases and gray literature. The search strategy will be developed iteratively by an experienced medical information specialist and peer-reviewed with the PRESS checklist. The search will be limited to English-only documents published from 2005 onward. Documents will be independently screened, selected, and extracted by 2 researchers. Data will be analyzed and summarized descriptively, including the characteristics of the included documents, knowledge translation methods tools, and candidate quality indicators., Scoping Review Registration: Open Science Framework ( https://osf.io/chxvq )., Competing Interests: The authors declare no conflict of interest., (Copyright © 2022 JBI.)

Background: The Diabetes Action Canada Training and Mentoring (DAC-TM) Program launched in June 2017, with the goal of building capacity in the next generation of diabetes researchers in Canada in patient-oriented research (POR)., Methods: We conducted a program evaluation of the DAC-TM program using a sequential, mixed-methods research design., Results: Our analysis of 82 surveys and 22 in-depth interviews from a wide range of DAC-TM Program stakeholders revealed consistent patterns in experience with the program. The training sessions were perceived to be well-organized, convenient educational opportunities to gain new knowledge about POR and become integrated into a community of practice of POR researchers who study diabetes and its complications in Canada. The content of the training was perceived to be useful and relevant to participants, although improvements could be made to help address the training needs of the broader DAC community. There is broad support for and appreciation of the mentorship awards, which were perceived to be appropriately targeted to early-career investigators. The mentor-mentee relationships were perceived to be positive, productive and career-advancing overall, but could benefit from a more strategic design and promote better connectivity to foster mentor-mentee relationships. In addition, feedback about opportunities to network and forge new connections was mixed and represents another opportunity for improvement to strengthen capacity building., Conclusions: Findings from this formative evaluation study show key strengths and opportunities to improve the DAC-TM Program, which can be used to help enhance its function and promote its long-term sustainability., (Copyright © 2022 Canadian Diabetes Association. Published by Elsevier Inc. All rights reserved.)

Objectives: To evaluate the sustainability potential of Choosing Wisely (CW) to address unnecessary medical care at Ontario community hospitals., Data Sources/study Setting: Ontario community hospitals and their affiliated family health teams (FHTs)., Study Design: A mixed-methods study involving the administration of a validated sustainability survey to CW implementation teams followed by their participation in focus groups., Data Collection/extraction Methods: Survey data were collected using an Excel file with an embedded, automated scoring system. We collated individual survey scores and generated aggregate team scores. We also performed descriptive statistics for quantitative data (frequencies, means). Qualitative data were triangulated with quantitative assessments to support data interpretations using the meta-matrix method., Principal Findings: Fifteen CW implementation teams across four Ontario community hospitals and six affiliated primary care FHTs participated. CW priority areas investigated were de-prescribing of proton pump inhibitors (PPIs) and reducing Pre-Op testing and BUN/Urea lab testing. Survey results showed steady improvements in sustainability scores from baseline to final follow-up among most implementation teams: 10% increase for PPI de-prescribing (six FHTs) and 2% increase (three hospital teams); 18% increase in BUN/Urea lab testing (three hospital teams). Regardless of site or CW priority area, common facilitators were fit with existing processes and workflows, leadership support, and optimized team communication; common challenges were lack of awareness and buy-in, leadership engagement or a champion, and lack of fit with existing workflow and culture. All teams identified at least one challenge for which they co-designed and implemented a plan to maximize the sustainability potential of their CW initiative., Conclusions: Evaluating the sustainability potential of an innovation such as Choosing Wisely is critical to ensuring that they have the best potential for impact. Our work highlights that implementation teams can be empowered to influence implementation efforts and to realize positive outcomes for their health care services and patients., (© 2021 Health Research and Educational Trust.)

Choosing Wisely is a medical stewardship and quality-improvement initiative led by the American Board of Internal Medicine Foundation in collaboration with leading medical societies in the United States. The American Society of Hematology (ASH) has been an active participant in the Choosing Wisely project. In 2019, ASH and the American Society of Pediatric Hematology/Oncology (ASPHO) formed a joint task force to solicit, evaluate, and select items for a pediatric-focused Choosing Wisely list. By using an iterative process and an evidence-based method, the ASH-ASPHO Task Force identified 5 hematologic tests and treatments that health care providers and patients should question because they are not supported by evidence, and/or they involve risks of medical and financial costs with low likelihood of benefit. The ASH-ASPHO Choosing Wisely recommendations are as follows: (1) avoid routine preoperative hemostatic testing in an otherwise healthy child with no previous personal or family history of bleeding, (2) avoid platelet transfusion in asymptomatic children with a platelet count >10 × 103/μL unless an invasive procedure is planned, (3) avoid thrombophilia testing in children with venous access-associated thrombosis and no positive family history, (4) avoid packed red blood cells transfusion for asymptomatic children with iron deficiency anemia and no active bleeding, and (5) avoid routine administration of granulocyte colony-stimulating factor for prophylaxis of children with asymptomatic autoimmune neutropenia and no history of recurrent or severe infections. We recommend that health care providers carefully consider the anticipated risks and benefits of these identified tests and treatments before performing them., (© 2022 by The American Society of Hematology. Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0), permitting only noncommercial, nonderivative use with attribution. All other rights reserved.)

Choosing Wisely is a medical stewardship and quality-improvement initiative led by the American Board of Internal Medicine Foundation in collaboration with leading medical societies in the United States. The American Society of Hematology (ASH) has been an active participant in the Choosing Wisely project. In 2019, ASH and the American Society of Pediatric Hematology/Oncology (ASPHO) formed a joint task force to solicit, evaluate, and select items for a pediatric-focused Choosing Wisely list. By using an iterative process and an evidence-based method, the ASH-ASPHO Task Force identified 5 hematologic tests and treatments that health care providers and patients should question because they are not supported by evidence, and/or they involve risks of medical and financial costs with low likelihood of benefit. The ASH-ASPHO Choosing Wisely recommendations are as follows: (1) avoid routine preoperative hemostatic testing in an otherwise healthy child with no previous personal or family history of bleeding, (2) avoid platelet transfusion in asymptomatic children with a platelet count 10 × 10 3 /μL unless an invasive procedure is planned, (3) avoid thrombophilia testing in children with venous access-associated thrombosis and no positive family history, (4) avoid packed red blood cells transfusion for asymptomatic children with iron deficiency anemia and no active bleeding, and (5) avoid routine administration of granulocyte colony-stimulating factor for prophylaxis of children with asymptomatic autoimmune neutropenia and no history of recurrent or severe infections. We recommend that health care providers carefully consider the anticipated risks and benefits of these identified tests and treatments before performing them., (© 2021 American Society of Hematology. All rights reserved. Pediatric Blood & Cancer published by Wiley Periodicals LLC.)

Introduction: In response to the burden of chronic disease among older adults, different chronic disease self-management tools have been created to optimise disease management. However, these seldom consider all aspects of disease management are not usually developed specifically for seniors or created for sustained use and are primarily focused on a single disease. We created an eHealth self-management application called 'KeepWell' that supports seniors with complex care needs in their homes. It incorporates the care for two or more chronic conditions from among the most prevalent high-burden chronic diseases., Methods and Analysis: We will evaluate the effectiveness, cost and uptake of KeepWell in a 6-month, pragmatic, hybrid effectiveness-implementation randomised controlled trial. Older adults age ≥65 years with one or more chronic conditions who are English speaking are able to consent and have access to a computer or tablet device, internet and an email address will be eligible. All consenting participants will be randomly assigned to KeepWell or control. The allocation sequence will be determined using a random number generator.Primary outcome is perceived self-efficacy at 6 months. Secondary outcomes include quality of life, health background/status, lifestyle (nutrition, physical activity, caffeine, alcohol, smoking and bladder health), social engagement and connections, eHealth literacy; all collected via a Health Risk Questionnaire embedded within KeepWell (intervention) or a survey platform (control). Implementation outcomes will include reach, effectiveness, adoption, fidelity, implementation cost and sustainability., Ethics and Dissemination: Ethics approval has been received from the North York General Hospital Research and Ethics Board. The study is funded by the Canadian Institutes of Health Research and the Ontario Ministry of Health. We will work with our team to develop a dissemination strategy which will include publications, presentations, plain language summaries and an end-of-grant meeting., Trial Registration Number: NCT04437238., Competing Interests: Competing interests: None declared., (© Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.)

Background: Positive impacts of quality improvement initiatives on health care and services have not been substantial. Knowledge translation (KT) strategies (tools, products and interventions) strive to facilitate the uptake of knowledge thereby the potential to improve care, but there is little guidance on how to develop them. Existing KT guidance or planning tools fall short in operationalizing all aspects of KT practice activities conducted by knowledge users (researchers, clinicians, patients, decision-makers), and most do not consider their variable needs or to deliver recommendations that are most relevant and useful for them., Methods: We conducted a 3-phase study. In phase 1, we used several sources to develop a conceptual framework for creating optimized Knowledge-activated Tools (KaT) (consultation with our integrated KT team, the use of existing KT models and frameworks, findings of a systematic review of multimorbidity interventions and a literature review and document analysis on existing KT guidance tools). In phase 2, we invited KT experts to participate in a Delphi study to refine and evaluate the conceptual KaT framework. In phase 3, we administered an online survey to knowledge users (researchers, clinicians, decision-makers, trainees) to evaluate the potential usefulness of an online mock-up version of the KaT framework., Results: We developed the conceptual KaT framework, and iteratively refined it with 35 KT experts in a 3-round Delphi study. The final framework represents the blueprint for what is needed to create KT strategies. Feedback from 201 researcher, clinician, decision-maker and trainee knowledge users on the potential need and usefulness of an online, interactive version of KaT indicated that they liked the idea of it (mean score 4.36 on a 5-point Likert scale) and its proposed features (mean score range 4.30-4.79)., Conclusions: Our findings suggest that mostly Canadian KT experts and knowledge users perceived the KaT framework and the future development of an online, interactive version to be important and needed. We anticipate that the KaT framework will provide clarity for knowledge users about how to identify their KT needs and what activities can address these needs, and to help streamline the process of these activities to facilitate efficient uptake of knowledge., Competing Interests: Competing interestsThe authors have no competing interests to declare., (© The Author(s) 2020.)

Objectives: To understand how and why effective multi-chronic disease management interventions influence health outcomes in older adults 65 years of age or older., Design: A realist review., Data Sources: Electronic databases including Medline and Embase (inception to December 2017); and the grey literature., Eligibility Criteria for Selecting Studies: We considered any studies (ie, experimental quasi-experimental, observational, qualitative and mixed-methods studies) as long as they provided data to explain our programme theories and effectiveness review (published elsewhere) findings. The population of interest was older adults (age ≥65 years) with two or more chronic conditions., Analysis: We used the Realist And MEta-narrative Evidence Syntheses: Evolving Standards (RAMESES) quality and publication criteria for our synthesis aimed at refining our programme theories such that they contained multiple context-mechanism-outcome configurations describing the ways different mechanisms fire to generate outcomes. We created a 3-step synthesis process grounded in meta-ethnography to separate units of data from articles, and to derive explanatory statements across them., Results: 106 articles contributed to the analysis. We refined our programme theories to explain multimorbidity management in older adults: (1) care coordination interventions with the best potential for impact are team-based strategies, disease management programmes and case management ; (2) optimised disease prioritisation involves ensuring that clinician work with patients to identify what symptoms are problematic and why, and to explore options that are acceptable to both clinicians and patients and (3) optimised patient self-management is dependent on patients' capacity for selfcare and to what extent, and establishing what patients need to enable selfcare., Conclusions: To optimise care, both clinical management and patient self-management need to be considered from multiple perspectives (patient, provider and system). To mitigate the complexities of multimorbidity management, patients focus on reducing symptoms and preserving quality of life while providers focus on the condition that most threaten morbidity and mortality., Prospero Registration Number: CRD42014014489., Competing Interests: Competing interests: None declared., (© Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.)

Background: Practice guidelines (PGs) can assist health care practitioners and patients to make decisions about health care options. A key component of high quality PGs is the consideration of patient values and preferences. A mixed methods study was conducted to understand optimal approaches to patient engagement in the development of cancer PGs., Methods: Cancer patients, survivors, family members and caregivers were recruited from cancer clinics, follow-up clinics, community support programs, a provincial patient and family advisory committee, and a provincial cancer PG development program. Participants attended a workshop, completed a survey, or participated in a telephone interview, to provide information about PG awareness, attitudes, information needs, training, engagement approaches and barriers and facilitators., Results: Forty-one participants (12 workshop attendees, 21 survey respondents and 8 interviewees) provided data. For those with no PG development experience, fewer than half were previously aware of PGs but perceived several benefits to the inclusion of this perspective. Common barriers to participation across the groups were time commitment, duration of the PG development process, and financial costs. Positive beliefs about the contributions that could be made and practical considerations (e.g., orientation and training, defined roles and expectations) were identified as key features in the successful integration of patients into the PG development process. There was no single model of engagement favored over another., Conclusions: Study results align with similar studies in other contexts and with international patient engagement efforts. Findings are being used to test new patient engagement models in a programmatic PG development initiative in Ontario, Canada.

Objectives: To identify factors associated with the implementability of clinical practice guidelines (CPGs) and to determine what characteristics improve their uptake., Study Design and Setting: We conducted a realist review, which involved searching multiple sources (eg, databases, experts) to determine what about guideline implementability works, for whom, and under what circumstances. Two sets of reviewers independently screened abstracts and extracted data from 278 included studies. Analysis involved the development of a codebook of definitions, validation of data, and development of hierarchical narratives to explain guideline implementability., Results: We found that guideline implementability is associated with two broad goals in guideline development: (1) creation of guideline content, which involves addressing the domains of stakeholder involvement in CPGs, evidence synthesis, considered judgment (eg, clinical applicability), and implementation feasibility and (2) the effective communication of this content, which involves domains related to fine-tuning the CPG's message (using simple, clear, and persuasive language) and format., Conclusion: Our work represents a comprehensive and interdisciplinary effort toward better understanding, which attributes of guidelines have the potential to improve uptake in clinical practice. We also created codebooks and narratives of key concepts, which can be used to create tools for developing better guidelines to promote better care., (Copyright © 2015 The Authors. Published by Elsevier Inc. All rights reserved.)

Background: Practice guideline (PG) implementability refers to PG features that promote their use. While there are tools and resources to promote PG implementability, none are based on an evidence-informed and multidisciplinary perspective. Our objectives were to (i) create a comprehensive and evidence-informed model of PG implementability, (ii) seek support for the model from the international PG community, (iii) map existing implementability tools on to the model, (iv) prioritize areas for further investigation, and (v) describe how the model can be used by PG developers, users, and researchers., Methods: A mixed methods approach was used. Using our completed realist review of the literature of seven different disciplines as the foundation, an iterative consensus process was used to create the beta version of the model. This was followed by (i) a survey of international stakeholders (guideline developers and users) to gather feedback and to refine the model, (ii) a content analysis comparing the model to existing PG tools, and (iii) a strategy to prioritize areas of the model for further research by members of the research team., Results: The Guideline Implementability for Decision Excellence Model (GUIDE-M) is comprised of 3 core tactics, 7 domains, 9 subdomains, 44 attributes, and 40 subattributes and elements. Feedback on the beta version was received from 248 stakeholders from 34 countries. The model was rated as logical, relevant, and appropriate. Seven PG tools were selected and compared to the GUIDE-M: very few tools targeted the Contextualization and Deliberations domain. Also, fewer of the tools addressed PG appraisal than PG development and reporting functions. These findings informed the research priorities identified by the team., Conclusions: The GUIDE-M provides an evidence-informed international and multidisciplinary conceptualization of PG implementability. The model can be used by PG developers to help them create more implementable recommendations, by clinicians and other users to help them be better consumers of PGs, and by the research community to identify priorities for further investigation.

Background: Consensus statements and clinical practice guidelines are widely available for enhancing the care of cancer patients. Despite subtle differences in their definition and purpose, these terms are often used interchangeably. We systematically assessed the methodological quality of consensus statements and clinical practice guidelines published in three commonly read, geographically diverse, cancer-specific journals. Methods Consensus statements and clinical practice guidelines published between January 2005 and September 2013 in Current Oncology, European Journal of Cancer and Journal of Clinical Oncology were evaluated. Each publication was assessed using the Appraisal of Guidelines for Research and Evaluation II (AGREE II) rigour of development and editorial independence domains. For assessment of transparency of document development, 7 additional items were taken from the Institute of Medicine's standards for practice guidelines and the Journal of Clinical Oncology guidelines for authors of guidance documents., Methods: Consensus statements and clinical practice guidelines published between January 2005 and September 2013 in Current Oncology, European Journal of Cancer and Journal of Clinical Oncology were evaluated. Each publication was assessed using the Appraisal of Guidelines for Research and Evaluation II (AGREE II) rigour of development and editorial independence domains. For assessment of transparency of document development, 7 additional items were taken from the Institute of Medicine's standards for practice guidelines and the Journal of Clinical Oncology guidelines for authors of guidance documents., Findings: Thirty-four consensus statements and 67 clinical practice guidelines were evaluated. The rigour of development score for consensus statements over the three journals was 32% lower than that of clinical practice guidelines. The editorial independence score was 15% lower for consensus statements than clinical practice guidelines. One journal scored consistently lower than the others over both domains. No journals adhered to all the items related to the transparency of document development. One journal's consensus statements endorsed a product made by the sponsoring pharmaceutical company in 64% of cases., Conclusion: Guidance documents are an essential part of oncology care and should be subjected to a rigorous and validated development process. Consensus statements had lower methodological quality than clinical practice guidelines using AGREE II. At a minimum, journals should ensure that that all consensus statements and clinical practice guidelines adhere to AGREE II criteria. Journals should consider explicitly requiring guidelines to declare pharmaceutical company sponsorship and to identify the sponsor's product to enhance transparency.

Background: The original AGREE (Appraisal of Guidelines for REsearch and Evaluation) Instrument was published in 2003, and its revision, the AGREE II, in 2009. Together, they filled an important gap in the guideline and quality of care fields. Ten years later, the AGREE Enterprise reflects on a trajectory of projects and international collaboration that have contributed to advancing the science and quality of practice guidelines and the uptake of AGREE/AGREE II., Findings: The AGREE Enterprise has undertaken activities to improve the tool and to develop resources to support its use. Since 2003, the uptake and adoption of AGREE by the international community has been swift and broad. A total of 33 language translations of the original AGREE Instrument and the current AGREE II are available and were initiated by the international community. A recent scan of the published literature identified over 600 articles that referenced the AGREE tools. The AGREE tools have been widely received and applied, with several organizations having incorporated the AGREE as part of their formal practice guideline programs. Since its redevelopment in 2010, the AGREE Enterprise website (www.agreetrust.org) continues to experience steady increases in visitors per month and currently has over 10,000 registered users., Conclusions: The AGREE Enterprise has contributed to the advancements of guidelines through research activities and international participation by scientific and user communities. As we enter a new decade, we look forward to ongoing collaborations and contributing to further advancements to improve quality of care and health care systems.

Introduction: Practice pattern data demonstrate regional variation and lower than expected rates of adherence to practice guideline (PG) recommendations for the treatment of stage II/IIIA resected and stage IIIA/IIIB unresected non-small cell lung cancer (NSCLC) patients in Ontario, Canada. This study sought to understand how clinical decisions are made for the treatment of these patients and the role of PGs., Methods: Surveys and key informant interviews were undertaken with clinicians and administrators., Results: Participants reported favorable ratings for PGs and the evidentiary bases underpinning them. The majority of participants agreed more patients should have received treatment and that regional variation is problematic. Participants estimated that up to 30% of patients are not good candidates for treatment and up to 20% of patients refuse treatment. The most common barrier to implementing PGs was the lack of organizational support by clinical administrative leadership. There was concern that the trial results underpinning the PG recommendations were not generalizable to the typical patients seen in clinic. The qualitative analysis yielded five themes related to physicians' decision making: the unique patient, the unique physician, the family, the clinical team, and the clinical evidence. A dynamic interplay between these factors exists., Conclusion: Our study demonstrates the challenges inherent in (i) the complexity of clinical decision making; (ii) how quality of care problems are perceived and operationalized; and (iii) the clinical appropriateness and utility of PG recommendations. We argue that systematic and rigorous methodologies to help decision makers mitigate or negotiate these challenges are warranted.

Background: Realist reviews offer a rigorous method to analyze heterogeneous data emerging from multiple disciplines as a means to develop new concepts, understand the relationships between them, and identify the evidentiary base underpinning them. However, emerging synthesis methods such as the Realist Review are not well operationalized and may be difficult for the novice researcher to grasp. The objective of this paper is to describe the development of an analytic process to organize and synthesize data from a realist review., Methods: Clinical practice guidelines have had an inconsistent and modest impact on clinical practice, which may in part be due to limitations in their design. This study illustrates the development of a transparent method for organizing and analyzing a complex data set informed by a Realist Review on guideline implementability to better understand the characteristics of guidelines that affect their uptake in practice (e.g., clarity, format). The data organization method consisted of 4 levels of refinement: 1) extraction and 2) organization of data; 3) creation of a conceptual map of guideline implementability; and 4) the development of a codebook of definitions., Results: This new method is comprised of four steps: data extraction, data organization, development of a conceptual map, and operationalization vis-a-vis a codebook. Applying this method, we extracted 1736 guideline attributes from 278 articles into a consensus-based set of categories, and collapsed them into 5 core conceptual domains for our guideline implementability map: Language, Format, Rigor of development, Feasibility, Decision-making., Conclusions: This study advances analysis methods by offering a systematic approach to analyzing complex data sets where the goals are to condense, organize and identify relationships.

Background: Practice guidelines (PGs) are systematically developed statements intended to assist in patient and practitioner decisions. The AGREE II is the revised tool for PG development, reporting, and evaluation, comprised of 23 items, two global rating scores, and a new User's Manual. In this study, we sought to develop, execute, and evaluate the impact of two internet interventions designed to accelerate the capacity of stakeholders to use the AGREE II., Methods: Participants were randomized to one of three training conditions. 'Tutorial'--participants proceeded through the online tutorial with a virtual coach and reviewed a PDF copy of the AGREE II. 'Tutorial + Practice Exercise'--in addition to the Tutorial, participants also appraised a 'practice' PG. For the practice PG appraisal, participants received feedback on how their scores compared to expert norms and formative feedback if scores fell outside the predefined range. 'AGREE II User's Manual PDF (control condition)'--participants reviewed a PDF copy of the AGREE II only. All participants evaluated a test PG using the AGREE II. Outcomes of interest were learners' performance, satisfaction, self-efficacy, mental effort, time-on-task, and perceptions of AGREE II., Results: No differences emerged between training conditions on any of the outcome measures., Conclusions: We believe these results can be explained by better than anticipated performance of the AGREE II PDF materials (control condition) or the participants' level of health methodology and PG experience rather than the failure of the online training interventions. Some data suggest the online tools may be useful for trainees new to this field; however, this requires further study.

Recent feedback from physical activity (PA) participants, fitness professionals, and physicians has indicated that there are limitations to the utility and effectiveness of the existing PAR-Q and PARmed-X screening tools for PA participation. The aim of this study was to have authorities in exercise and chronic disease management to work with an expert panel to increase the effectiveness of clearance for PA participation using an evidence-based consensus approach and the well-established Appraisal of Guidelines for Research and Evaluation (AGREE) Instrument. Systematic reviews were conducted to develop a new PA clearance protocol involving risk stratification and a decision-tree process. Evidence-based support was sought for enabling qualified exercise professionals to have a direct role in the PA participation clearance process. The PAR-Q+ was developed to use formalized probes to clarify problematic responses and to explore issues arising from currently diagnosed chronic disease or condition. The original PARmed-X tool is replaced with an interactive computer program (ePARmed-X+) to clear prospective PA participants for either unrestricted or supervised PA or to direct them to obtain medical clearance. Evidence-based validation was also provided for the direct role of highly qualified university-educated exercise professionals in the PA clearance process. The risks associated with exercise during pregnancy were also evaluated. The systematic review and consensus process, conforming to the AGREE Instrument, has provided a sound evidence base for enhanced effectiveness of the clearance process for PA participation of both asymptomatic populations and persons with chronic diseases or conditions.

Background: We established a program of research to improve the development, reporting and evaluation of practice guidelines. We assessed the construct validity of the items and user's manual in the beta version of the AGREE II., Methods: We designed guideline excerpts reflecting high-and low-quality guideline content for 21 of the 23 items in the tool. We designed two study packages so that one low-quality and one high-quality version of each item were randomly assigned to each package. We randomly assigned 30 participants to one of the two packages. Participants reviewed and rated the guideline content according to the instructions of the user's manual and completed a survey assessing the manual., Results: In all cases, content designed to be of high quality was rated higher than low-quality content; in 18 of 21 cases, the differences were significant (p < 0.05). The manual was rated by participants as appropriate, easy to use, and helpful in differentiating guidelines of varying quality, with all scores above the mid-point of the seven-point scale. Considerable feedback was offered on how the items and manual of the beta-AGREE II could be improved., Interpretation: The validity of the items was established and the user's manual was rated as highly useful by users. We used these results and those of our study presented in part 1 to modify the items and user's manual. We recommend AGREE II (available at www.agreetrust.org) as the revised standard for guideline development, reporting and evaluation.

Background: We undertook research to improve the AGREE instrument, a tool used to evaluate guidelines. We tested a new seven-point scale, evaluated the usefulness of the original items in the instrument, investigated evidence to support shorter, tailored versions of the tool, and identified areas for improvement., Method: We report on one component of a larger study that used a mixed design with four factors (user type, clinical topic, guideline and condition). For the analysis reported in this article, we asked participants to read a guideline and use the AGREE items to evaluate it based on a seven-point scale, to complete three outcome measures related to adoption of the guideline, and to provide feedback on the instrument's usefulness and how to improve it., Results: Guideline developers gave lower-quality ratings than did clinicians or policy-makers. Five of six domains were significant predictors of participants' outcome measures (p < 0.05). All domains and items were rated as useful by stakeholders (mean scores > 4.0) with no significant differences by user type (p > 0.05). Internal consistency ranged between 0.64 and 0.89. Inter-rater reliability was satisfactory. We received feedback on how to improve the instrument., Interpretation: Quality ratings of the AGREE domains were significant predictors of outcome measures associated with guideline adoption: guideline endorsements, overall intentions to use guidelines, and overall quality of guidelines. All AGREE items were assessed as useful in determining whether a participant would use a guideline. No clusters of items were found more useful by some users than others. The measurement properties of the seven-point scale were promising. These data contributed to the refinements and release of the AGREE II.

Background: Self-administered surveys are an essential methodological tool for health services and knowledge translation research, and engaging end-users of the research is critical. However, few documented accounts of the efforts invested in recruitment of multiple different stakeholders to one health services research study exist. Here, we highlight the challenges of recruiting key stakeholders (policy-makers, clinicians, guideline developers) to a Canadian Institutes of Health Research (CIHR) funded health services research (HSR) study aimed to develop an updated and refined version of a guideline appraisal tool, the AGREE., Methods: Using evidence-based methods of recruitment, our goal was to recruit 192 individuals: 80 international guideline developers, 80 Canadian clinicians and 32 Canadian policy/decision-makers. We calculated the participation rate and the recruitment efficiency., Results: We mailed 873 invitation letters. Of 838 approached, our participation rate was 29%(240) and recruitment efficiency, 19%(156). One policy-maker manager did not allow policy staff to participate in the study., Conclusions: Based on the results from this study, we suggest that future studies aiming to engage similar stakeholders in HSR over sample by at least 5 times to achieve their target sample size and allow for participant withdrawals. We need continued efforts to communicate the value of research between researchers and end-users of research (policy-makers, clinicians, and other researchers), integration of participatory research strategies, and promotion of the value of end-user involvement in research. Future research to understand methods of improving recruitment efficiency and engaging key stakeholders in HSR is warranted.

Background: Practice guidelines (PGs) are systematically developed statements intended to assist in patient, practitioner, and policy decisions. The AGREE II is the revised and updated standard tool for guideline development, reporting and evaluation. It is comprised of 23 items and a user's Manual. The AGREE II is ready for use., Objectives: To develop, execute, and evaluate the impact of two internet-based educational interventions designed to accelerate the capacity of stakeholders to use the AGREE II: a multimedia didactic tutorial with a virtual coach, and a higher intensity training program including both the didactic tutorial and an interactive practice exercise component., Methods: Participants (clinicians, developers, and policy makers) will be randomly assigned to one of three conditions. Condition one, didactic tutorial -- participants will go through the on-line AGREE II tutorial supported by a virtual coach and review of the AGREE II prior to appraising the test PG. Condition two, tutorial + practice -- following the multimedia didactic tutorial with a virtual coach, participants will review the on-line AGREE II independently and use it to appraise a practice PG. Upon entering their AGREE II score for the practice PG, participants will be given immediate feedback on how their score compares to expert norms. If their score falls outside a predefined range, the participant will receive a series of hints to guide the appraisal process. Participants will receive an overall summary of their performance appraising the PG compared to expert norms. Condition three, control arm -- participants will receive a PDF copy of the AGREE II for review and to appraise the test PG on-line. All participants will then rate one of ten test PGs with the AGREE II. The outcomes of interest are learners' performance, satisfaction, self-efficacy, mental effort, and time-on-task; comparisons will be made across each of the test groups., Discussion: Our research will test innovative educational interventions of various intensities and instructional design to promote the adoption of AGREE II and to identify those strategies that are most effective for training. The results will facilitate international capacity to apply the AGREE II accurately and with confidence and to enhance the overall guideline enterprise.