Search

Objectives: The evaluation of health benefits and harms of an intervention with GRADE Evidence to Decision (EtD) frameworks includes judgments if the effects are "trivial," "small," "moderate," or "large." Such judgments ideally require the a priori establishment of decision thresholds (DTs), whose empirical derivation for single outcomes has been previously described. In this article, we provide a methodological approach to estimate DTs for composite endpoints based on disutilities., Study Design and Setting: We generated an approach that involves the computation of pooled disutilities, in which the disutility of each outcome comprised in the composite endpoint is weighted by the respective relative frequency. We also applied a modeling approach based on probability distributions to account for uncertainty in the estimates. We present a practical example of the determination of DTs associated with the development of at least one adverse event following treatment with intranasal medications for rhinitis that we used in the Allergic Rhinitis and its Impact on Asthma guidelines., Results: We provide the methodological steps of a modeling-based approach to compute pooled disutilities and, as a result, DTs for composite endpoints. We have developed a webtool (https://compositedt.med.up.pt/) that allows for a simple implementation of the proposed approach. Applying our approach to a practical example, we concluded that rhinitis nasal medications, compared to placebo, were associated with a trivial harm from adverse events., Conclusion: We propose an approach for estimating DTs for composite endpoints, which may be particularly valuable whenever composite endpoints are used for clinical research, clinical practice, and decision-making., Competing Interests: Declaration of competing interest None., (Copyright © 2024 Elsevier Inc. All rights reserved.)

Introduction: GRADE and other Evidence to Decision (EtD) frameworks are widely used by guideline development groups (GDG) and other decision-makers. When GDGs judge the magnitude of desirable and undesirable health outcomes on EtDs, they typically categorize them as trivial, small, moderate or large. However, generic judgment or decision thresholds (DTs) that could guide the user about such estimates of effect size or serve as references for interpretation of findings are not yet available. The objective of this study was to empirically derive DTs for EtD judgments about the magnitude of dichotomously assessed health benefits and harms., Methods: We conducted a methodological randomised controlled trial to derive empirical DTs across conditions and health outcomes. We invited stakeholders including clinicians, epidemiologists, decision scientists, health research methodologists, experts in Health Technology Assessment (HTA), members of GDGs, patient representatives, and the public to participate in the trial. We employed randomly assigned case scenarios to elicit ranges of absolute risk differences (ARD) judged as small and moderate effects from study participants. We then used the collected data to derive empirical decision thresholds. We also investigated the validity of our DTs by measuring the agreement between judgments that were made by GDGs in the past and the judgments that our DTs approach would suggest if applied to the same guideline data., Results: A total of 445 stakeholders accessed the survey of which 409 were randomised and 288 rated at least one case scenario. Based on these participants, the study findings support our a priori hypothesis of a difference in the DTs for trivial, small, moderate, and large effects and are suggestive of a relation between raters' judgments and the joint measure of absolute effects and outcome values. The results permit the use and calculation of DTs for a variety of scenarios and we present three ways of how to use the results practically., Conclusions: In this trial we confirmed that empirically derived DTs discriminate between judgments on the EtDs. These DTs can be used for judgments about desirable and undesirable health effects in systematic reviews or to initiate and inform a discussion with a GDG. This ensures consistency in judgments across different guideline questions and promotes transparency in judgments., (Copyright © 2024. Published by Elsevier Inc.)

Objectives: The PANELVIEW questionnaire identifies the strengths and weaknesses of the process and methods used for developing health guidelines from the guideline development group's perspective. To expand its use, PANELVIEW ideally should be available in different languages. We aimed to cross-culturally adapt PANELVIEW into Spanish and assess its acceptability, validity, and reliability., Study Design and Setting: To translate and culturally adapt PANELVIEW to Spanish, we followed International Society for Pharmacoeconomics and Outcomes Research's Translation and Cultural Adaptation Good Practice Principles guidelines. The process consisted of 1) forward and back translation, 2) input from an expert panel, and 3) cognitive debriefing interviews. We assessed the content validity with experts in guideline development who rated instrument items for relevance to determine the item content validity index and scale content validity index (I-CVI and S-CVI). We tested the reliability with health guidelines panels from Spanish-speaking countries and measured internal consistency (Cronbach's alpha). We examined acceptability through the number of missing responses for each item., Results: The content comparison between the back-translation and the original version showed that most items (24/34) were conceptually equivalent but with grammatical differences. Through the cognitive interviews, we identified six items with wording issues, ten with clarity issues, and two with applicability issues. I-CVI ranged from 0.77 to 1.00, with two items needing revision. S-CVI was 0.92, showing excellent content validity. The PANELVIEW Spanish version demonstrated very good reliability (Cronbach's alpha coefficient of 0.96). Panel members responded to all items, showing good acceptability., Conclusion: The PANELVIEW Spanish version was conceptually equivalent to the original version and provided satisfactory evidence of acceptability, validity and reliability., Competing Interests: Declaration of competing interest W.W., Elie Akl and H.S. will retain the copyright of the PANELVIEW tool through McMaster University (Technology ID 20-031), and the tool may be licensed in the future. There are no competing interests for any other author., (Copyright © 2024 Elsevier Inc. All rights reserved.)

Background and Objective: Grading of Recommendations, Assessment, Development, and Evaluation (GRADE)-ADOLOPMENT is widely applied to efficiently use existing credible guidelines and contextualize them to a target setting. To highlight the experiences of a Latin American Guideline Development Group (GDG) applying GRADE-ADOLOPMENT to adapt the American Society of Hematology's clinical practice guideline on managing venous thromboembolisms (VTEs) in Latin America., Study Design and Setting: We employed a mixed-method postevaluation using self-administered surveys and semistructured interviews. We assessed the Latin American GDG (1) general satisfaction and confidence using the approach, (2) their ratings on the usefulness, appropriateness, and importance of GRADE-ADOLOPMENT and its tools to inform their judgements, and (3) any additional facilitators and barriers to refine the process., Results: Eleven of the 14 GDG members, including nine panelists and two methodologists, provided survey responses and eight participated in the interview. Respondents felt "mostly" or "completely" satisfied with the adapted guideline. Eight panelists who were surveyed agree that GRADE-ADOLOPMENT is useful in countries with limited resources. Although panelists expressed initial apprehensions in their understanding of the process, they demonstrated enhanced confidence in their capacity to apply GRADE after completing workshop training and by acquiring experience. Panelists reiterated the importance of considering evidence-to-decision (EtD) criteria (ie, resources, feasibility, and cost-effectiveness) when adapting recommendations. The GDG encountered challenges with collecting local and regional data, prioritizing recommendations while considering intraregional diversity, and the lengthy publication period, although the latter stemmed from procedures not related to GRADE-ADOLOPMENT., Conclusions: GRADE-ADOLOPMENT is an important tool to facilitate the adaptation and uptake of clinical practice guidelines in novel settings. The GDG felt satisfied with their overall experience using the GRADE-ADOLOPMENT approach. However, their experience could have been optimized if they had access to robust regional evidence, more recommendations to adapt from, and worked with more efficient guideline production timelines., Competing Interests: Declaration of competing interest The authors have no conflict of interest to declare., (Copyright © 2024. Published by Elsevier Inc.)

Background and Objective: The Grading of Recommendations, Assessment, Development and Evaluations (GRADE)-ADOLOPMENT methodology has been widely used to adopt, adapt, or de novo develop recommendations from existing or new guideline and evidence synthesis efforts. The objective of this guidance is to refine the operationalization for applying GRADE-ADOLOPMENT., Methods: Through iterative discussions, online meetings, and email communications, the GRADE-ADOLOPMENT project group drafted the updated guidance. We then conducted a review of handbooks of guideline-producing organizations, and a scoping review of published and planned adolopment guideline projects. The lead authors refined the existing approach based on the scoping review findings and feedback from members of the GRADE working group. We presented the revised approach to the group in November 2022 (approximately 115 people), in May 2023 (approximately 100 people), and twice in September 2023 (approximately 60 and 90 people) for approval., Results: This GRADE guidance shows how to effectively and efficiently contextualize recommendations using the GRADE-ADOLOPMENT approach by doing the following: (1) showcasing alternative pathways for starting an adolopment effort; (2) elaborating on the different essential steps of this approach, such as building on existing evidence-to-decision (EtDs), when available or developing new EtDs, if necessary; and (3) providing examples from adolopment case studies to facilitate the application of the approach. We demonstrate how to use contextual evidence to make judgments about EtD criteria, and highlight the importance of making the resulting EtDs available to facilitate adolopment efforts by others., Conclusion: This updated GRADE guidance further operationalizes the application of GRADE-ADOLOPMENT based on over 6 years of experience. It serves to support uptake and application by end users interested in contextualizing recommendations to a local setting or specific reality in a short period of time or with limited resources., Competing Interests: Declaration of competing interest The authors are members or contributors to the GRADE working group. Zachary Munn is supported by an NHMRC investigator grant APP 1195676. Alexander G. Mathioudakis was supported by the National Institute for Health and Care Research Manchester Biomedical Research Centre (NIHR Manchester BRC, NIHR203308) and by an NIHR Clinical Lectureship in Respiratory Medicine. The views expressed are those of the authors and not necessarily those of the NIHR or the Department of Health and Social Care., (Copyright © 2024. Published by Elsevier Inc.)

Background and Objective: To identify COVID-19 actionable statements (e.g., recommendations) focused on specific disadvantaged populations in the living map of COVID-19 recommendations (eCOVIDRecMap) and describe how health equity was assessed in the development of the formal recommendations., Methods: We employed the place of residence, race or ethnicity or culture, occupation, gender or sex, religion, education, socio-economic status, and social capital-Plus framework to identify statements focused on specific disadvantaged populations. We assessed health equity considerations in the evidence to decision frameworks (EtD) of formal recommendations for certainty of evidence and impact on health equity criteria according to the Grading of Recommendations, Assessment, Development, and Evaluations criteria., Results: We identified 16% (124/758) formal recommendations and 24% (186/819) good practice statements (GPS) that were focused on specific disadvantaged populations. Formal recommendations (40%, 50/124) and GPS (25%, 47/186) most frequently focused on children. Seventy-six percent (94/124) of the recommendations were accompanied with EtDs. Over half (55%, 52/94) of those considered indirectness of the evidence for disadvantaged populations. Considerations in impact on health equity criterion most frequently involved implementation of the recommendation for disadvantaged populations (17%, 16/94)., Conclusion: Equity issues were rarely explicitly considered in the development COVID-19 formal recommendations focused on specific disadvantaged populations. Guidance is needed to support the consideration of health equity in guideline development during health emergencies., Competing Interests: Declaration of Competing Interest Several authors (L.R., M.L., G.H.) were involved in the development of guidelines that were included in the study. However, they did not participate in the evaluation of health equity considerations in the study. Authors hold sole responsibility for the views expressed in the manuscript, which may not necessarily reflect the opinion or policy of the Pan American Health Organization, Canadian Rheumatology Association. The remaining authors report no conflicts of interest., (Copyright © 2023 Elsevier Inc. All rights reserved.)

Objectives: To highlight how using the Grading of Recommendations, Assessment, Development and Evaluations (GRADE) approach to understand the certainty in the evidence about the impact of climate change in health outcomes increases transparency. Also, how GRADE can enhance communication and decisions about adaptation and mitigation strategies., Study Design and Setting: We developed a narrative review based on an assessment of exiting systematic reviews addressing the effect of climate change on health outcomes and the impact of mitigation and adaptation strategies., Results: Adopting structured approaches such as GRADE to tackle the impact of climate change on health may help to (1) define the specific question to be addressed; (2) summarize the evidence in a structured way and assess uncertainty; (3) provide a systematic framework to move from evidence to action and to offer recommendations of different strength; (4) provide a systematic way to adapt recommendations to specific settings; and (5) provide a framework to assess the certainty of modeled evidence., Conclusion: In this article, we describe epidemiologic principles that could be used to move decision-making in climate change forward., (Copyright © 2023 Elsevier Inc. All rights reserved.)

Objectives: Grading of Recommendations Assessment, Development and Evaluation (GRADE) guidance to rate the certainty domain of imprecision is presently not fully operationalized for rating down by two levels and when different baseline risk or uncertainty in these risks are considered. In addition, there are scenarios in which lowering the certainty of evidence by three levels for imprecision is more appropriate than lowering it by two levels. In this article, we conceptualize and operationalize rating down for imprecision by one, two and three levels for imprecision using the contextualized GRADE approaches and making decisions., Methods: Through iterative discussions and refinement in online meetings and through email communication, we developed draft guidance to rating the certainty of evidence down by up to three levels based on examples. The lead authors revised the approach according to the feedback and the comments received during these meetings and developed GRADE guidance for how to apply it. We presented a summary of the results to all attendees of the GRADE Working Group meeting for feedback in October 2021 (approximately 80 people) where the approach was formally approved., Results: This guidance provides GRADE's novel approach for the considerations about rating down for imprecision by one, two and three levels based on serious, very serious and extremely serious concerns. The approach includes identifying or defining thresholds for health outcomes that correspond to trivial or none, small, moderate or large effects and using them to rate imprecision. It facilitates the use of evidence to decision frameworks and also provides guidance for how to address imprecision about implausible large effects and trivial or no effects using the concept of the 'review information size' and for varying baseline risks. The approach is illustrated using practical examples, an online calculator and graphical displays and can be applied to dichotomous and continuous outcomes., Conclusion: In this GRADE guidance article, we provide updated guidance for how to rate imprecision using the partially and fully contextualized GRADE approaches for making recommendations or decisions, considering alternate baseline risks and for both dichotomous and continuous outcomes., (Copyright © 2022 Elsevier Inc. All rights reserved.)

Objectives: Guideline panels must assess the magnitude of health benefits and harms to develop sensible recommendations. However, they rarely use explicit thresholds. In this paper we report on the piloting and the use thresholds for benefits and harms., Study Design and Setting: We piloted the use of thresholds in a Chilean COVID-19 living guideline. For each of the critical outcomes, we asked panelists to suggest values of the thresholds for large, moderate, small, or trivial or no effect. We collected this information through a survey and an on-line discussion., Results: Twelve panelists decided on thresholds for three critical outcomes (mortality, need for mechanical ventilation and serious adverse events). For all outcomes, an absolute risk reduction was considered larger with more than 50 events, moderate with less than 50 events, small with less than 25 events, and trivial with less than 10 events. Having these a priori thresholds in place significantly impacted on the development of recommendations., Conclusion: Explicit thresholds were a valuable addition to the judgment of the certainty in the evidence, to decide the direction and strength of the recommendation and to evaluate the need for update. We believe this is a line of research worth perusing., (Copyright © 2022 Elsevier Inc. All rights reserved.)

Objectives: To describe divergence between actionable statements issued by coronavirus disease 2019 (COVID-19) guideline developers cataloged on the "COVID-19 Recommendations and Gateway to Contextualization" platform., Study Design and Setting: We defined divergence as at least two comparable actionable statements with different explicit judgments of strength, direction, or subgroup consideration of the population or intervention. We applied a content analysis to compare guideline development methods for a sample of diverging statements and to evaluate factors associated with divergence., Results: Of the 138 guidelines evaluated, 85 (62%) contained at least one statement that diverged from another guideline. We identified 223 diverging statements in these 85 guidelines. We grouped statements into 66 clusters. Each cluster addressed the same population, intervention, and comparator group or just similar interventions. Clinical practice statements were more likely to diverge in an explicit judgment of strength or direction compared to public health statements. Statements were more likely to diverge in strength than direction. The date of publication, used evidence, interpretation of evidence, and contextualization considerations were associated with divergence., Conclusion: More than half of the assessed guidelines issued at least one diverging statement. This study helps in understanding the types of differences between guidelines issuing comparable statements and factors associated with their divergence., (Copyright © 2022. Published by Elsevier Inc.)

Background: Health guideline development requires sequential prioritization of the guideline topic, questions, and health outcomes. In this paper we report on new approaches for prioritizing questions and outcomes in guidelines., Methods: Ten guideline panels on venous thromboembolism rated potential guideline questions on a 9-point scale according to their overall importance and 6 criteria: common in practice, uncertainty in practice, variation in practice, new evidence available, cost consequences, not previously addressed. We randomized panelists to rate one potential question with and without the 6 criteria. Panelists rated importance of outcomes, defined with health outcome descriptors (HODs), using a 9-point scale, and health utility of outcomes on a visual analogue scale., Results: Of 469 potential questions identified, 72.5% were rated as important but not of high priority, and 25.4% as high priority. Each criterion was significantly associated with the overall importance rating. The overall importance rating means were 5.96 (SD 2.38) and 6.53 (SD 2.45) (P = 0.25) for those randomized to rate questions with and without the criteria, respectively. The mean importance rating for 121 outcomes was 6.01 (SD 1.25), with 35.5% rated as critical for decision-making. Panelists provided health utility ratings for 127 outcomes, with a minimum mean rating of 0.12 (SD 0.10) and maximum of 0.91 (SD 0.15)., Conclusion: Our structured process provided information to help explain perspectives of question importance, to facilitate panels' outcome prioritization, and to facilitate decision-making in guideline development., (Copyright © 2021. Published by Elsevier Inc.)

Objective: To propose a taxonomy and framework that identifies and presents actionable statements in guidelines., Study Design and Setting: We took an iterative approach reviewing case studies of guidelines produced by the World Health Organization and the American Society of Hematology to develop an initial conceptual framework. We then tested it using randomly selected recommendations from published guidelines addressing COVID-19 from different organizations, evaluated its results, and refined it before retesting. The urgency and availability of evidence for development of these recommendations varied. We consulted with experts in research methodology and guideline developers to improve the final framework., Results: The resulting taxonomy and framework distinguishes five types of actional statements: formal recommendations; research recommendations; good practice statements; implementation considerations, tools and tips; and informal recommendations. These statements should respond to a priori established criteria and require a clear structure and recognizable presentation in a guideline. Most importantly, this framework identifies informal recommendations that differ from formal recommendations by how they consider evidence and in their development process., Conclusion: The identification, standardization and explicit labelling of actionable statements according to the framework may support guideline developers to create actionable statements with clear intent, avoid informal recommendations and improve their understanding and implementation by users., (Copyright © 2021. Published by Elsevier Inc.)

Objective: Stakeholders involved in developing recommendations need to have a common understanding of health outcomes and the perspective of affected individuals. In this paper we report on the development and application of health outcome descriptors (HODs) to inform decision-making by panels developing guideline recommendations., Study Design and Setting: Ten American Society of Hematology guideline panels addressing the management of venous thromboembolism developed HODs, rated their importance and health utility, applied them to prioritize outcomes, and to balance potential benefits and harms to formulate recommendations., Results: It was feasible to involve 18 panelists in developing 127 HODs. There was high agreement (82%) across the ten panels about outcomes perceived as critical or important for decision-making. Panelists' utility ratings of the outcomes were strongly correlated with panelists' outcome importance ratings (Pearson's r=-0.88). HODs were incorporated into Grading of Recommendations Assessment, Development and Evaluation (GRADE) evidence-to-decision (EtD) frameworks to support a shared understanding of health outcomes in panel deliberations., Conclusion: HODs serve as a valuable tool to promote an explicit, common understanding of health outcomes during clinical guideline development and across different stakeholders. They are helpful across multiple steps of guideline development to facilitate panels' judgements, aiming to avoid variable implicit interpretations of health outcomes., (Copyright © 2021 Elsevier Inc. All rights reserved.)

Objectives: To describe how systematic review authors report and address categories of participants with potential missing outcome data of trial participants., Study Design and Setting: Methodological survey of systematic reviews reporting a group-level meta-analysis., Results: We included a random sample of 50 Cochrane and 50 non-Cochrane systematic reviews. Of these, 25 reported in their methods section a plan to consider at least one of the 10 categories of missing outcome data; 42 reported in their results, data for at least one category of missing data. The most reported category in the methods and results sections was "unexplained loss to follow-up" (n = 34 in methods section and n = 6 in the results section). Only 19 reported a method to handle missing data in their primary analyses, which was most often complete case analysis. Few reviews (n = 9) reported in the methods section conducting sensitivity analysis to judge risk of bias associated with missing outcome data at the level of the meta-analysis; and only five of them presented the results of these analyses in the results section., Conclusion: Most systematic reviews do not explicitly report sufficient information on categories of trial participants with potential missing outcome data or address missing data in their primary analyses., (Copyright © 2018 Elsevier Inc. All rights reserved.)

Objectives: Evidence-based clinical practice guidelines provide recommendations to assist clinicians in decision-making and to reduce the gap between best current research evidence and clinical practice. However, some argue that providing preappraised evidence summaries alone, rather than recommendations, is more appropriate. The objective of the study is to evaluate clinicians' preferences, and understanding of the evidence and intended course of action in response to evidence summaries with and without recommendations., Study Design Setting: We included practicing clinicians attending educational sessions across 10 countries. Clinicians were randomized to receive relevant clinical scenarios supported by research evidence of low or very low certainty and accompanied by either strong or weak recommendations developed with the GRADE system. Within each group, participants were further randomized to receive the recommendation plus the corresponding evidence summary or the evidence summary alone. We evaluated participants' preferences and understanding for the presentation strategy, as well as their intended course of action., Results: One hundred eighty-nine of 219 (86%) and 201 of 248 (81%) participants preferred having recommendations accompanying evidence summaries for both strong and weak recommendations, respectively. Across all scenarios, less than half of participants correctly interpreted information provided in the evidences summaries (e.g., estimates of effect, certainty in the research evidence). The presence of a recommendation resulted in a more appropriate intended course of action for two scenarios involving strong recommendations., Conclusion: Evidence summaries alone are not enough to impact clinicians' course of action. Clinicians clearly prefer having recommendations accompanying evidence summaries in the context of low or very low certainty of evidence (Trial registration NCT02006017)., (Copyright © 2018 Elsevier Inc. All rights reserved.)

Objective: To provide GRADE guidance for assessing risk of bias across an entire body of evidence consequent on missing data for systematic reviews of both binary and continuous outcomes., Study Design and Setting: Systematic survey of published methodological research, iterative discussions, testing in systematic reviews, and feedback from the GRADE Working Group., Results: Approaches begin with a primary meta-analysis using a complete case analysis followed by sensitivity meta-analyses imputing, in each study, data for those with missing data, and then pooling across studies. For binary outcomes, we suggest use of "plausible worst case" in which review authors assume that those with missing data in treatment arms have proportionally higher event rates than those followed successfully. For continuous outcomes, imputed mean values come from other studies within the systematic review and the standard deviation (SD) from the median SDs of the control arms of all studies., Conclusions: If the results of the primary meta-analysis are robust to the most extreme assumptions viewed as plausible, one does not rate down certainty in the evidence for risk of bias due to missing participant outcome data. If the results prove not robust to plausible assumptions, one would rate down certainty in the evidence for risk of bias., (Copyright © 2017 Elsevier Inc. All rights reserved.)

Background: Guideline developers can: (1) adopt existing recommendations from others; (2) adapt existing recommendations to their own context; or (3) create recommendations de novo. Monetary and nonmonetary resources, credibility, maximization of uptake, as well as logical arguments should guide the choice of the approach and processes., Objectives: To describe a potentially efficient model for guideline production based on adoption, adaptation, and/or de novo development of recommendations utilizing the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Evidence to Decision (EtD) frameworks., Study Design and Setting: We applied the model in a new national guideline program producing 22 practice guidelines. We searched for relevant evidence that informs the direction and strength of a recommendation. We then produced GRADE EtDs for guideline panels to develop recommendations., Results: We produced a total of 80 EtD frameworks in approximately 4 months and 146 EtDs in approximately 6 months in two waves. Use of the EtD frameworks allowed panel members understand judgments of others about the criteria that bear on guideline recommendations and then make their own judgments about those criteria in a systematic approach., Conclusion: The "GRADE-ADOLOPMENT" approach to guideline production combines adoption, adaptation, and, as needed, de novo development of recommendations. If developers of guidelines follow EtD criteria more widely and make their work publically available, this approach should prove even more useful., (Copyright © 2016 The Author(s). Published by Elsevier Inc. All rights reserved.)

Objectives: Explicit reporting of absolute measures is important to ensure treatment effects are correctly interpreted. We examined the extent to which authors report absolute effects for patient-important outcomes in abstracts of systematic review (SR)., Study Design and Setting: We searched OVID MEDLINE and Cochrane Database of Systematic Reviews to identify eligible SRs published in the year 2010. Citations were stratified into Cochrane and non-Cochrane reviews, with repeated random sampling in a 1:1 ratio. Paired reviewers screened articles and recorded abstract characteristics, including reporting of effect measures for the most patient-important outcomes of benefit and harm., Results: We included 96 Cochrane and 94 non-Cochrane reviews. About 117 (77.5%) relative measures were reported in abstracts for outcomes of benefit, whereas only 34 (22.5%) absolute measures were reported. Similarly, for outcomes of harm, 41 (87.2%) relative measures were provided in abstracts, compared with only 6 (12.8%) absolute measures. Eighteen (9.5%) abstracts reported both absolute and relative measures for outcomes of benefit, whereas only two (1.1%) abstracts reported both measures for outcomes of harm. Results were similar between Cochrane and non-Cochrane reviews., Conclusion: SR abstracts seldom report measures of absolute effect. Journal editors should insist that authors report both relative and absolute effects for patient-important outcomes., (Copyright © 2016 Elsevier Inc. All rights reserved.)

Objectives: The current format of summary of findings (SoFs) tables for presenting effect estimates and associated quality of evidence improve understanding and assist users finding key information in systematic reviews. Users of SoF tables have demanded alternative formats to express findings from systematic reviews., Study Design and Setting: We conducted a randomized controlled trial among systematic review users to compare the relative merits of a new format with the current formats of SoF tables regarding understanding, accessibility of information, satisfaction, and preference. Our primary goal was to show that the new format is not inferior to the current format., Results: Of 390 potentially eligible subjects, 290 were randomized. Of seven items testing understanding, three showed similar results, two showed small differences favoring the new format, and two (understanding risk difference and quality of the evidence associated with a treatment effect) showed large differences favoring the new format [63% (95% confidence interval {CI}: 55, 71) and 62% (95% CI: 52, 71) more correct answers, respectively]. Respondents rated information in the alternative format as more accessible overall and preferred the new format over the current format., Conclusions: While providing at least similar levels of understanding for some items and increased understanding for others, users prefer the new format of SoF tables., (Copyright © 2016 Elsevier Inc. All rights reserved.)

An increasing number of organizations worldwide are using new and improved standards for developing trustworthy clinical guidelines. One of such approaches, developed by the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) working group, offers systematic and transparent guidance in moving from evidence to recommendations. The GRADE strategy concentrates on four factors: the balance between benefits and harms, the certainty of the evidence, values and preferences, and resource considerations. However, it also considers issues around feasibility, equity, and acceptability of recommendations. GRADE distinguishes two types of recommendations: strong and weak. Strong recommendations reflect a clear preference for one alternative and should apply to all or almost all patients, obviating the need for a careful review of the evidence with each patient. Weak recommendations are appropriate when there is a close balance between desirable and undesirable consequences of alternative management strategies, uncertainty regarding the effects of the alternatives, uncertainty or variability in patients' values and preferences, or questionable cost-effectiveness. Weak recommendations usually require accessing the underlying evidence and a shared decision-making approach. Clinicians using GRADE recommendations should understand the meaning of the strength of the recommendation, be able to critically appraise the recommendation, and apply trustworthy recommendations according to their strength., (Copyright © 2016 Elsevier Inc. All rights reserved.)

Objectives: Expressing treatment effects in relative terms yields larger numbers than expressions in absolute terms, affecting the judgment of the clinicians and patients regarding the treatment options. It is uncertain how authors of systematic reviews (SRs) absolute effect estimates are reported in. We therefore undertook a systematic survey to identify and describe the reporting and methods for calculating absolute effect estimates in SRs., Study Design and Setting: Two reviewers independently screened title, abstract, and full text and extracted data from a sample of Cochrane and non-Cochrane SRs. We used regression analyses to examine the association between study characteristics and the reporting of absolute estimates for the most patient-important outcome., Results: We included 202 SRs (98 Cochrane and 104 non-Cochrane), most of which (92.1%) included standard meta-analyses including relative estimates of effect. Of the 202 SRs, 73 (36.1%) reported absolute effect estimates for the most patient-important outcome. SRs with statistically significant effects were more likely to report absolute estimates (odds ratio, 2.26; 95% confidence interval: 1.08, 4.74). The most commonly reported absolute estimates were: for each intervention, risk of adverse outcomes expressed as a percentage (41.1%); number needed to treat (26.0%); and risk for each intervention expressed as natural units or natural frequencies (24.7%). In 12.3% of the SRs that reported absolute effect estimates for both benefit and harm outcomes, harm outcomes were reported exclusively as absolute estimates. Exclusively reporting of beneficial outcomes as absolute estimates occurred in 6.8% of the SRs., Conclusions: Most SRs do not report absolute effects. Those that do often report them inadequately, thus requiring users of SRs to generate their own estimates of absolute effects. For any apparently effective or harmful intervention, SR authors should report both absolute and relative estimates to optimize the interpretation of their findings., (Copyright © 2016 Elsevier Inc. All rights reserved.)

Objectives: In 2007 the World Health Organization (WHO) adopted the GRADE system for development of public health guidelines. Previously we found that many strong recommendations issued by WHO are based on evidence for which there is only low or very low confidence in the estimates of effect (discordant recommendations). GRADE guidance indicates that such discordant recommendations are rarely appropriate but suggests five paradigmatic situations in which discordant recommendations may be warranted. We sought to provide insight into the many discordant recommendations in WHO guidelines., Study Design and Setting: We examined all guidelines that used the GRADE method and were approved by the WHO Guideline Review Committee between 2007 and 2012. Teams of reviewers independently abstracted data from eligible guidelines and classified recommendations either into one of the five paradigms for appropriately-formulated discordant recommendations or into three additional categories in which discordant recommendations were inconsistent with GRADE guidance: 1) the evidence warranted moderate or high confidence (a misclassification of evidence) rather than low or very low confidence; 2) good practice statements; or 3) uncertainty in the estimates of effect would best lead to a conditional (weak) recommendation., Results: The 33 eligible guidelines included 160 discordant recommendations, of which 98 (61.3%) addressed drug interventions and 132 (82.5%) provided some rationale (though not entirely explicit at times) for the strong recommendation. Of 160 discordant recommendations, 25 (15.6%) were judged consistent with one of the five paradigms for appropriate recommendations; 33 (21%) were based on evidence warranting moderate or high confidence in the estimates of effect; 29 (18%) were good practice statements; and 73 (46%) warranted a conditional, rather than a strong recommendation., Conclusion: WHO discordant recommendations are often inconsistent with GRADE guidance, possibly threatening the integrity of the process. Further training in GRADE methods for WHO guideline development group members may be necessary, along with further research on what motivates the formulation of such recommendations., (Copyright © 2016 Elsevier Inc. All rights reserved.)

Background: The World Health Organization (WHO) classifies a substantial proportion of their recommendations as strong despite low or very low confidence (certainty) in estimates of effect. Such discordant recommendations are often inconsistent with Grading of Recommendations Assessment, Development and Evaluation (GRADE) guidance., Objective: To gain the perspective of senior WHO methodology chairs regarding panels' use of GRADE, particularly regarding discordant recommendations., Data Sources: Senior active GRADE methodologists who had served on at least two WHO panels and were an author on at least one peer-reviewed published article on GRADE methodology., Methods: Five eligible methodologists participated in detailed semistructured interviews. Respondents answered questions regarding how they were viewed by other panelists and WHO leadership, and how they handled situations when panelists made discordant recommendations they felt were inappropriate. They also provided information on how the process can be improved. Interviews were recorded and transcribed, and inductive content analysis was used to derive codes, categories, and emergent themes., Results: Three themes emerged from the interviews of five methodologists: (1) The perceived role of methodologists in the process, (2) Contributors to discordant recommendations, and (3) Strategies for improvement. Salient findings included (1) a perceived tension between methodologists and WHO panels as a result of panel members' resistance to adhering to GRADE guidance; (2) both financial and nonfinancial conflicts of interest among panel members as an explanation for discordant recommendations; and (3) the need for greater clarity of, and support for, the role of methodologists as co-chairs of panels., Conclusions: These findings suggest that the role of the GRADE methodologist as a co-chair needs to be clarified by the WHO leadership. They further suggest the need for additional training for panelists, quality monitoring, and feedback to ensure optimal use of GRADE in guideline development at WHO., (Copyright © 2016 Elsevier Inc. All rights reserved.)

Objective: Many strong recommendations issued by the World Health Organization (WHO) are based on low- or very low-quality (low certainty) evidence (discordant recommendations). Many such discordant recommendations are inconsistent with the Grading of Recommendations Assessment, Development and Evaluation (GRADE) guidance. We sought to understand why WHO makes discordant recommendations inconsistent with GRADE guidance., Study Design and Setting: We interviewed panel members involved in guidelines approved by WHO (2007-2012) that included discordant recommendations. Interviews, recorded and transcribed, focused on use of GRADE including the reasoning underlying, and factors contributing to, discordant recommendations., Results: Four themes emerged: strengths of GRADE, challenges and barriers to GRADE, strategies to improve GRADE application, and explanations for discordant recommendations. Reasons for discordant recommendations included skepticism about the value of making conditional recommendations; political considerations; high certainty in benefits (sometimes warranted, sometimes not) despite assessing evidence as low certainty; and concerns that conditional recommendations will be ignored., Conclusion: WHO panelists make discordant recommendations inconsistent with GRADE guidance for reasons that include limitations in their understanding of GRADE. Ensuring optimal application of GRADE at WHO and elsewhere likely requires selecting panelists who have a commitment to GRADE principles, additional training of panelists, and formal processes to maximize adherence to GRADE principles., (Copyright © 2016 Elsevier Inc. All rights reserved.)

Objectives: The American College of Chest Physicians Antithrombotic Guidelines ninth iteration placed restrictions on panelists with recommendations on which they disclosed a primary conflict of interest (COI). We aimed to describe panelists' financial and intellectual COI and evaluate to what extent, beyond assessing financial COI, assessing intellectual COI affected COI management., Study Design and Setting: We classified financial and intellectual COI into primary (causes voting restriction) and secondary (no restrictions). We analyzed disclosures respectively with panelists and recommendations as units of analysis., Results: One hundred two panelists made 4,030 disclosures for 431 recommendations. The median number (and range) of panelists per recommendation who disclosed the various categories of COI was 0 (0-5) for primary financial COI, 0 (0-4) for secondary financial COI, 0 (0-7) for primary intellectual COI, and 1 (0-6) for secondary intellectual COI. Of the 431 recommendations, 63 (15%) had at least one panelist with a primary intellectual COI but no primary financial COI., Conclusion: COI had a relatively low prevalence and a skewed distribution, many panelists with none and some with many disclosures. A substantial number of disclosures should have resulted in restrictions based on intellectual COI in the absence of financial COI., (Copyright © 2014 Elsevier Inc. All rights reserved.)

Objectives: Expert guideline panelists are sometimes reluctant to offer weak/conditional/contingent recommendations. Grading of Recommendations Assessment, Development and Evaluation (GRADE) guidance warns against strong recommendations when confidence in effect estimates is low or very low, suggesting that such recommendations may seldom be justified. We aim to characterize the classification of strength of recommendations and confidence in estimates in World Health Organization (WHO) guidelines that used the GRADE approach and graded both strength and confidence (GRADEd)., Study Design and Setting: We reviewed all WHO guidelines (January 2007 to December 2012), identified those that were GRADEd, and, in these, examined the classifications of strong and weak and associated confidence in estimates (high, moderate, low, and very low)., Results: We identified 116 WHO guidelines in which 43 (37%) were GRADEd and had 456 recommendations, of which 289 (63.4%) were strong and 167 (36.6%) were conditional/weak. Of the 289 strong recommendations, 95 (33.0%) were based on evidence warranting low confidence in estimates and 65 (22.5%) on evidence warranting very low confidence in estimates (55.5% strong recommendations overall based on low or very low confidence in estimates)., Conclusion: Strong recommendations based on low or very low confidence estimates are very frequently made in WHO guidelines. Further study to determine the reasons for such high uncertainty recommendations is warranted., (Copyright © 2014 Elsevier Inc. All rights reserved.)