Search

Background The Cochrane Bias Methods Group recently developed the “Risk of Bias (ROB) in Non-randomized Studies of Interventions” (ROBINS-I) tool to assess ROB for non-randomized studies of interventions (NRSI). It is important to establish consistency in its application and interpretation across review teams. In addition, it is important to understand if specialized training and guidance will improve the reliability of the results of the assessments. Therefore, the objective of this cross-sectional study is to establish the inter-rater reliability (IRR), inter-consensus reliability (ICR), and concurrent validity of ROBINS-I. Furthermore, as this is a relatively new tool, it is important to understand the barriers to using this tool (e.g., time to conduct assessments and reach consensus—evaluator burden). Methods Reviewers from four participating centers will appraise the ROB of a sample of NRSI publications using the ROBINS-I tool in two stages. For IRR and ICR, two pairs of reviewers will assess the ROB for each NRSI publication. In the first stage, reviewers will assess the ROB without any formal guidance. In the second stage, reviewers will be provided customized training and guidance. At each stage, each pair of reviewers will resolve conflicts and arrive at a consensus. To calculate the IRR and ICR, we will use Gwet’s AC1 statistic. For concurrent validity, reviewers will appraise a sample of NRSI publications using both the New-castle Ottawa Scale (NOS) and ROBINS-I. We will analyze the concordance between the two tools for similar domains and for the overall judgments using Kendall’s tau coefficient. To measure the evaluator burden, we will assess the time taken to apply the ROBINS-I (without and with guidance), and the NOS. To assess the impact of customized training and guidance on the evaluator burden, we will use the generalized linear models. We will use Microsoft Excel and SAS 9.4 to manage and analyze study data, respectively. Discussion The quality of evidence from systematic reviews that include NRS depends partly on the study-level ROB assessments. The findings of this study will contribute to an improved understanding of the ROBINS-I tool and how best to use it.

Objective: The objective of the study was to clarify how the Grading of Recommendations Assessment, Development and Evaluation (GRADE) concept of certainty of evidence applies to certainty ratings of test accuracy. Study Design and Setting: After initial brainstorming with GRADE Working Group members, we iteratively refined and clarified the approaches for defining ranges when assessing the certainty of evidence for test accuracy within a systematic review, health technology assessment, or guideline. Results: Ranges can be defined both for single test accuracy and for comparative accuracy of multiple tests. For systematic reviews and health technology assessments, approaches for defining ranges include some that do not require value judgments regarding downstream health outcomes. Key challenges arise in the context of a guideline that requires ranges for sensitivity and specificity that are set considering possible effects on all critical outcomes. We illustrate possible approaches and provide an example from a systematic review of a direct comparison between two test strategies. Conclusions: This GRADE concept paper provides a framework for assessing, presenting, and making decisions based on the certainty of evidence for test accuracy. More empirical research is needed to support future GRADE guidance on how to best operationalize the candidate approaches.

The objective of this paper is to explain how to apply, interpret, and present the results of a new instrument to assess the risk of bias (RoB) in non-randomized studies (NRS) dealing with effects of environmental exposures on health outcomes. This instrument is modeled on the Risk Of Bias In Non-randomized Studies of Interventions (ROBINS-I) instrument. The RoB instrument for NRS of exposures assesses RoB along a standardized comparison to a randomized target experiment, instead of the study-design directed RoB approach. We provide specific guidance for the integral steps of developing a research question and target experiment, distinguishing issues of indirectness from RoB, making individual-study judgments, and performing and interpreting sensitivity analyses for RoB judgments across a body of evidence. Also, we present an approach for integrating the RoB assessments within the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) framework to assess the certainty of the evidence in the systematic review. Finally, we guide the reader through an overall assessment to support the rating of all domains that determine the certainty of a body of evidence using the GRADE approach. Keywords: Risk of bias, Environmental health, GRADE, Non-randomized studies, Study limitations, ROBINS

Background A new tool, “risk of bias (ROB) instrument for non-randomized studies of exposures (ROB-NRSE),” was recently developed. It is important to establish consistency in its application and interpretation across review teams. In addition, it is important to understand if specialized training and guidance will improve the reliability in the results of the assessments. Therefore, the objective of this cross-sectional study is to establish the inter-rater reliability (IRR), inter-consensus reliability (ICR), and concurrent validity of the new ROB-NRSE tool. Furthermore, as this is a relatively new tool, it is important to understand the barriers to using this tool (e.g., time to conduct assessments and reach consensus—evaluator burden). Methods Reviewers from four participating centers will apprise the ROB of a sample of NRSE publications using ROB-NRSE tool in two stages. For IRR and ICR, two pairs of reviewers will assess the ROB for each NRSE publication. In the first stage, reviewers will assess the ROB without any formal guidance. In the second stage, reviewers will be provided customized training and guidance. At each stage, each pair of reviewers will resolve conflicts and arrive at a consensus. To calculate the IRR and ICR, we will use Gwet’s AC1 statistic. For concurrent validity, reviewers will appraise a sample of NRSE publications using both the Newcastle-Ottawa Scale (NOS) and ROB-NRSE tool. We will analyze the concordance between the two tools for similar domains and for the overall judgments using Kendall’s tau coefficient. To measure evaluator burden, we will assess the time taken to apply ROB-NRSE tool (without and with guidance), and the NOS. To assess the impact of customized training and guidance on the evaluator burden, we will use the generalized linear models. We will use Microsoft Excel and SAS 9.4, to manage and analyze study data, respectively. Discussion The quality of evidence from systematic reviews that include NRSE depends partly on the study-level ROB assessments. The findings of this study will contribute to an improved understanding of ROB-NRSE and how best to use it.

Background Once a proposed topic has been identified for a systematic review and has undergone a question formulation stage, a protocol must be developed that specifies the scope and research questions in detail and outlines the methodology for conducting the systematic review. Rationale Framework modifications are often needed to accommodate increased complexity. We describe and give examples of adaptations and alternatives to traditional analytic frameworks. Discussion This article identifies and describes elements of frameworks and how they can be adapted to inform the protocol and conduct of systematic reviews of complex interventions. Modifications may be needed to adapt the population, intervention, comparators, and outcomes normally used in protocol development to successfully describe complex interventions; in some instances, alternative frameworks may be better suited. Possible approaches to analytic frameworks for complex interventions that illustrate causal and associative linkages are outlined, including time elements, which systematic reviews of complex interventions may need to address. The need for and specifics of the accommodations vary with details of a specific systematic review. This in turn helps determine whether traditional frameworks are sufficient, can be refined, or if alternate frameworks must be adopted.

Objectives\ud This article provides updated GRADE guidance about how authors of systematic reviews and health technology assessments (HTA) and guideline developers can assess the results and the certainty of evidence (also known as quality of the evidence or confidence in the estimates) of a body of evidence addressing test accuracy (TA).\ud Study Design and Setting\ud We present an overview of the GRADE approach and guidance for rating certainty in TA in clinical and public health and review the presentation of results of a body of evidence regarding tests. Part 1 of the two parts in this 21st guidance article about how to apply GRADE focuses on understanding study design issues in test accuracy, provide an overiew of the domains and describe risk of bias and indirectness specifically.\ud Results\ud Supplemented by practical examples, we describe how raters of the evidence using GRADE can evaluate study designs focusing on tests and how they apply the GRADE domains risk of bias and indirectness to a body of evidence of TA studies.\ud Conclusions\ud Rating the certainty of a body of evidence using GRADE in Cochrane and other reviews and World Health Organization and other guidelines dealing with in TA studies helped refining our approach. The resulting guidance will help applying GRADE successfully for questions and recommendations focusing on tests.

Objectives This article provides updated GRADE guidance about how authors of systematic reviews and health technology assessments (HTA) and guideline developers can rate the certainty of evidence (also known as quality of the evidence or confidence in the estimates) of a body of evidence addressing test accuracy (TA) on the domains imprecision, inconsistency, publication bias and other domains. It also provides guidance for how to present synthesized information in evidence profiles and summary of findings tables. Study Design and Setting We present guidance for rating certainty in TA in clinical and public health and review the presentation of results of a body of evidence regarding tests. Results Supplemented by practical examples, we describe how raters of the evidence can apply the GRADE domains inconsistency, imprecision, and publication bias to a body of evidence of TA studies. Conclusions Using GRADE in Cochrane and other reviews as well as World Health Organization and other guidelines helped refining the GRADE approach for rating the certainty of a body of evidence from TA studies. While several of the GRADE domains (e.g., imprecision and magnitude of the association) require further methodological research to help operationalize them, judgments need to be made on the basis of what is known so far.

Radical radiotherapy (RT) is a potentially curative treatment in non-small cell lung cancer (NSCLC) and is delivered in conventional 2-Gy fractions, hypofractionated and ablative stereotactic courses. No reliable, predictive biomarkers for the clinical events of local control, appearance of distant metastases and development of toxicity have been introduced in routine clinical practice. Such a test would enable the Radiotherapist to tailor the clinical management of individual patients, considering their pre-treatment characteristics, in order reduce the risk of recurrence or toxicity e.g., dose modification, accelerated fractionation, hypofractionation, or concurrent systemic therapy. The aim of this review was to map the published literature relating to investigations of the potential predictive value of patient or treatment characteristics in radical RT for NSCLC. These investigations should remain a research focus for disease control given the upward trends in lung cancer incidence, and for the avoidance of toxicity, given the survivorship afforded to the cohort of patients that do well with radical RT, or with the increasing range of systemic agents following metastatic relapse. The conclusion of the presented analysis is that there are no published, effective and validated predictive tools for estimation of risk of local/distant recurrence or toxicity after radical RT for NSCLC. The authors have identified an important space for future research in the field of lung cancer radiotherapy.

Objectives To revise 2010 guidance on grading the strength of evidence (SOE) of the effectiveness of drugs, devices, and other preventive and therapeutic interventions in systematic reviews produced by the Evidence-based Practice Center (EPC) program, established by the US Agency for Healthcare Research and Quality (AHRQ). Study Design and Setting A cross-EPC working group reviewed authoritative systems for grading SOE [primarily the approach from the Grading of Recommendations Assessment, Development and Evaluation (GRADE) working group] and conducted extensive discussions with GRADE and other experts. Results Updated guidance continues to be conceptually similar to GRADE. Reviewers are to evaluate SOE separately for each major treatment comparison for each major outcome. We added reporting bias as a required domain and retained study limitations (risk of bias), consistency, directness, and precision (and three optional domains). Additional guidance covers scoring consistency, precision, and reporting bias, grading bodies of evidence with randomized controlled trials and observational studies, evaluating single study bodies of evidence, using studies with high risk of bias, and presenting findings with greater clarity and transparency. SOE is graded high, moderate, low, or insufficient, reflecting reviewers' confidence in the findings for a specific treatment comparison and outcome. Conclusion No single approach for grading SOE suits all reviews, but a more consistent and transparent approach to reporting summary information will make reviews more useful to the broad range of audiences that AHRQ's work aims to reach. EPC working groups will consider ongoing challenges and modify guidance as needed, on issues such as combining trials and observational studies in bodies of evidence, weighting domains, and combining qualitative and quantitative syntheses.

Objectives The Agency for Healthcare Research and Quality (AHRQ) Effective Health Care Program conducts systematic reviews of health-care topics nominated by stakeholders. Topics undergo refinement to ensure relevant questions of appropriate scope and useful reviews. Input from key informants, experts, and a literature scan informs changes in the nominated topic. AHRQ convened a work group to assess approaches and develop recommendations for topic refinement. Study Design and Setting Work group members experienced in topic refinement generated a list of questions and guiding principles relevant to the refinement process. They discussed each issue and reached agreement on recommendations. Results Topics should address important health-care questions or dilemmas, consider stakeholder priorities and values, reflect the state of the science, and be consistent with systematic review research methods. Guiding principles of topic refinement are fidelity to the nomination, relevance, research feasibility, responsiveness to stakeholder inputs, reduced investigator bias, transparency, and suitable scope. Suggestions for stakeholder engagement, synthesis of input, and reporting are discussed. Refinement decisions require judgment and balancing guiding principles. Variability in topics precludes a prescriptive approach. Conclusion Accurate, rigorous, and useful systematic reviews require well-refined topics. These guiding principles and methodological recommendations may help investigators refine topics for reviews.

Introduction Rapid reviews (RR), using abbreviated systematic review (SR) methods, are becoming more popular among decision-makers. This World Health Organization commissioned study sought to summarize RR methods, identify differences, and highlight potential biases between RR and SR. Methods Review of RR methods (Key Question 1 [KQ1]), meta-epidemiologic studies comparing reliability/ validity of RR and SR methods (KQ2), and their potential associated biases (KQ3). We searched Medline, EMBASE, Cochrane Library, grey literature, and checked reference lists, used personal contacts, and crowdsourcing (e.g. email listservs). Selection and data extraction was conducted by one reviewer (KQ1) or two reviewers independently (KQ2-3). Results Across all KQs, we identified 42,743 citations through the literature searches. KQ1: RR methods from 29 organizations were reviewed. There was no consensus on which aspects of the SR process to abbreviate. KQ2: Studies comparing the conclusions of RR and SR (n = 9) found them to be generally similar. Where major differences were identified, it was attributed to the inclusion of evidence from different sources (e.g. searching different databases or including different study designs). KQ3: Potential biases introduced into the review process were well-identified although not necessarily supported by empirical evidence, and focused mainly on selective outcome reporting and publication biases. Conclusion RR approaches are context and organization specific. Existing comparative evidence has found similar conclusions derived from RR and SR, but there is a lack of evidence comparing the potential of bias in both evidence synthesis approaches. Further research and decision aids are needed to help decision makers and reviewers balance the benefits of providing timely evidence with the potential for biased findings.

Non-randomized studies of the effects of interventions are critical to many areas of health care evaluation, but their results may be biased. It is therefore important to understand and appraise their strengths and weaknesses. We developed ROBINS-I (“Risk Of Bias In Non-randomized Studies - of Interventions”), a new tool for evaluating risk of bias in estimates of the comparative effectiveness (harm or benefit) of interventions from studies that did not use randomization to allocate units (individuals or clusters of individuals) to comparison groups. The tool will be particularly useful to those undertaking systematic reviews that include non-randomized studies.

Objectives: The aim of this paper is to describe a conceptual framework for how to consider health equity in the Grading Recommendations Assessment and Development Evidence (GRADE) guideline development process. Study Design and Setting: Consensus-based guidance developed by the GRADE working group members and other methodologists. Results: We developed consensus-based guidance to help address health equity when rating the certainty of synthesized evidence (i.e., quality of evidence). When health inequity is determined to be a concern by stakeholders, we propose five methods for explicitly assessing health equity: (1) include health equity as an 'outcome; (2) consider patient-important outcomes relevant to health equity; (3) assess differences in the relative effect size of the treatment; (4) assess differences in baseline risk and the differing impacts on absolute effects; and (5) assess indirectness of evidence to disadvantaged populations and/or settings. Conclusion: The most important priority for research on health inequity and guidelines is to identify and document examples where health equity has been considered explicitly in guidelines. Although there is a weak scientific evidence base for assessing health equity, this should not discourage the explicit consideration of how guidelines and recommendations affect the most vulnerable members of society. (C) 2017 The Authors. Published by Elsevier Inc.

Impaired glucose tolerance and impaired fasting glucose, referred to as prediabetes, are predisease states that may progress to type 2 diabetes mellitus. The value of intervening in these states, especially with respect to cardiovascular outcomes, is unclear. The purpose of this study was to develop a broad synopsis of the available literature through an evidence map of systematic reviews about interventions in adults with prediabetes. The Cochrane Library Issue 4, 2011, Ovid MEDLINE (1946–January 4, 2012) and grey literature were searched. Studies were selected according to defined eligibility criteria. Fourteen relevant systematic reviews, of mostly high quality, were retrieved. The majority of reviews evaluated pharmacotherapeutic interventions or diet/exercise/lifestyle. A few reviews assessed complementary alternative medicine, behavioural strategies, standard prevention education or gastric bypass surgery. Very few reviews reported data on clinical cardiovascular endpoints. Most reviews reported either shorter-term surrogate markers of cardiovascular outcomes or progression to type 2 diabetes mellitus. Based on an assessment of authors’ overall conclusions, the value of pharmacotherapeutic interventions and diet/exercise/lifestyle ranged from beneficial to inconclusive, behavioural strategies were beneficial, and gastric bypass surgery was probably beneficial. The value of complementary alternative medicine was inconclusive. A considerable amount of evidence pertaining to interventions in prediabetes exists. Future research should synthesize the available systematic review evidence base in an overview of reviews. In addition, more primary research should be conducted to assess clinical cardiovascular endpoints.

Apply and compare two methods that identify signals for the need to update systematic reviews, using three Evidence-based Practice Center reports on omega-3 fatty acids as test cases.We applied the RAND method, which uses domain (subject matter) expert guidance, and a modified Ottawa method, which uses quantitative and qualitative signals. For both methods, we conducted focused electronic literature searches of recent studies using the key terms from the original reports. We assessed the agreement between the methods and qualitatively assessed the merits of each system.Agreement between the two methods was "substantial" or better (kappa0.62) in three of the four systematic reviews. Overall agreement between the methods was "substantial" (kappa=0.64, 95% confidence interval [CI] 0.45-0.83).The RAND and modified Ottawa methods appear to provide similar signals for the possible need to update systematic reviews in this pilot study. Future evaluation with a broader range of clinical topics and eventual comparisons between signals to update reports and the results of full evidence review updates will be needed. We propose a hybrid approach combining the best features of both methods, which should allow efficient review and assessment of the need to update.

Objective This article reports on literature surveillance methods to identify new evidence eligible for updating systematic reviews. Study Design and Setting Five surveillance search approaches are tested in the context of identifying studies that would signal major or invalidating new evidence for existing systematic reviews of health care interventions. Recall for each search approach was assessed as proportion of a composite yield of relevant studies across all search approaches that were identified by that approach. Screening burden was the number of studies that would need to be reviewed to identify the evidence that would necessitate updating. Results Searches were tested in a cohort of 77 systematic reviews. No one method yielded consistently high recall of relevant new evidence, so combinations of the strategies were examined. A search algorithm based on PubMed's related article search in combination with a subject searching using clinical queries was the most effective combination, retrieving all relevant new records in 68 cases. Screening burden was a median of 71 new records per review (inter-quartile range: 42–161). Conclusion Surveillance for emerging evidence that signals the need to update systematic reviews is feasible using a combination of subject searching and searching based on the PubMed's related article function.

The intrathecal drug-delivery system (IDDS) is one mode of infusing analgesic medications directly into the cerebrospinal fluid in close proximity to their site of action. This modality has been employed in patients with refractory pain either due to malignant or non-malignant causes for over 30 years. Unfortunately, and despite the number of years it has been in use, there is still a scarcity of rigorous evidence to guide its integration into clinical practice. Current best evidence is inconclusive as to the comparative effectiveness and harms of the IDDS relative to routine medical care of patients. There are far more systematic reviews than high-quality primary comparative studies of the IDDS vs. conventional pain treatment. Existing clinical practice recommendations are best viewed as expert opinion with competing interests. This article will review the existing literature for indications, contraindications, consensus statements, different technologies, and complications of the IDDS. Although approved analgesics for IDDS delivery are limited to morphine and ziconotide, many other analgesics, alone or in combination, are routinely used in this setting. This review will also focus on the pharmacology, clinical efficacy, and safety of intrathecal medications extensively used in clinical practice; including agents approved, unapproved, and under development.

Background Anecdotal evidence suggests a possible link between travel and venous thromboembolism (VTE). We systematically evaluated the evidence from observational studies. Methods We searched studies evaluating the risk of venous thrombosis in relation to traveling from MEDLINE and EMBASE up to March 2004, together with a hand search of reference lists from retrieved literature, and we contacted some of the experts. Observational studies estimating the risks of VTE and isolated calf vein thrombosis were eligible. Methodologic quality was assessed based on prior criteria, and meta-analysis was considered where applicable. Results A total of 194 English-language publications were initially identified. Sixteen studies were included: 9 casecontrol, 2 prospective controlled, and 5 other observational studies. They differed drastically in study designs, selection of controls where applicable, mode and duration of travel, and subtypes of VTE under consideration. Ten studies concluded that travel, mostly through air and of prolonged duration, is a risk factor for venous thrombosis and/or pulmonary embolism, and the risk increases for passengers with preexisting venous thrombosis risk factors. Outcomes examined ranged from asymptomatic isolated calf muscle vein thrombosis to severe fatal pulmonary embolism. Conclusions Current literature is controversial over any association between travel and VTE, and although the quality and power of these studies have been variable, studies of higher quality have shown a strong and significant association between prolonged air travel and VTE. No conclusions could be drawn about other modes of transportation. Since VTE is a disease of multifactorial causation, those with preexisting VTE risk factors are most vulnerable.

Background: Cardiovascular disease and type 2 diabetes are examples of chronic diseases that impose significant morbidity and mortality in the general population worldwide. Most chronic diseases are associated with underlying preventable risk factors, such as elevated blood pressure, high blood glucose or glucose intolerance, high lipid levels, physical inactivity, excessive sedentary behaviours, and overweight/obesity. The occurrence of intermediate outcomes during childhood increases the risk of disease in adulthood. Sugar-sweetened beverages are known to be significant sources of additional caloric intake, and given recent attention to their contribution in the development of chronic diseases, a systematic review is warranted. We will assess whether the consumption of sugar-sweetened beverages in children is associated with adverse health outcomes and what the potential moderating factors are. Methods/Design: Of interest are studies addressing sugar-sweetened beverage consumption, taking a broad perspective. Both direct consumption studies as well as those evaluating interventions that influence consumption (e.g. school policy, educational) will be relevant. Non-specific or multi-faceted behavioural, educational, or policy interventions may also be included subject to the level of evidence that exists for the other interventions/exposures. Comparisons of interest and endpoints of interest are pre-specified. We will include randomized controlled trials, controlled clinical trials, interrupted time series studies, controlled before-after studies, prospective and retrospective comparative cohort studies, case–control studies, and nested case–control designs. The MEDLINE®, Embase, The Cochrane Library, CINAHL, ERIC, and PsycINFO® databases and grey literature sources will be searched. The processes for selecting studies, abstracting data, and resolving conflicts are described. We will assess risk of bias using design-specific tools. To determine sets of confounding variables that should be adjusted for, we have developed causal directed acyclic graphs and will use those to inform our risk of bias assessments. Meta-analysis will be conducted where appropriate; parameters for exploring statistical heterogeneity and effect modifiers are pre-specified. The Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach will be used to determine the quality of evidence for outcomes.

peer-reviewed Context. Delirium often presents difficult management challenges in the context of goals of care in palliative care settings. Objectives. The aim was to formulate an analytical framework for further research on delirium in palliative care settings, prioritize the associated research questions, discuss the inherent methodological challenges associated with relevant studies, and outline the next steps in a program of delirium research.Methods. We combined multidisciplinary input from delirium researchers and knowledge users at an international delirium study planning meeting, relevant literature searches, focused input of epidemiologic expertise, and a meeting participant and coauthor survey to formulate a conceptual research framework and prioritize research questions.Results. Our proposed framework incorporates three main groups of research questions: the first was predominantly epidemiologic, such as delirium occurrence rates, risk factor evaluation, screening, and diagnosis; the second covers pragmatic management questions; and the third relates to the development of predictive models for delirium outcomes. Based on aggregated survey responses to each research question or domain, the combined modal ratings of "very'' or "extremely'' important confirmed their priority.Conclusion. Using an analytical framework to represent the full clinical care pathway of delirium in palliative care settings, we identified multiple knowledge gaps in relation to the occurrence rates, assessment, management, and outcome prediction of delirium in this population. The knowledge synthesis generated from adequately powered, multicenter studies to answer the framework's research questions will inform decision making and policy development regarding delirium detection and management and thus help to achieve better outcomes for patients in palliative care settings. (C) 2014 American Academy of Hospice and Palliative Medicine. Published by Elsevier Inc. All rights reserved. PUBLISHED peer-reviewed

Background Perioperative pain management has recently been revolutionized with the recognition of novel mechanisms and introduction of newer drugs. Many randomized trials have studied the use of the gabapentinoid anti-epileptic, pregabalin, in acute pain. Published systematic reviews suggest that using pregabalin for perioperative pain management may decrease analgesic requirements and pain scores, at the expense of troublesome side effects. A major limitation of the extant reviews is the lack of rigorous investigation of clinical characteristics that would maximize the benefit harms ratio in favor of surgical patients. We posit that effects of pregabalin for perioperative pain management vary by the type of surgical pain model and propose this systematic review protocol to update previous systematic reviews and investigate the heterogeneity in findings across subgroups of surgical pain models. Methods/Design Using a peer-reviewed search strategy, we will search key databases for clinical trials on perioperative pregabalin use in adults. The electronic searches will be supplemented by scanning the reference lists of included studies. No limits of language, country or year will be imposed. Outcomes will include pain; use of co-analgesia, particularly opioids; enhanced recovery; and drug-related harms. We will focus on the identification of surgical models and patient characteristics that have shown benefit and adverse effects from pregabalin. Two clinical experts will independently screen the studies for inclusion using eligibility criteria established a priori. Data extracted by the reviewers will then be verified. Publication bias will be assessed, as will risk of bias using the Cochrane Risk of Bias tool. Meta-analysis and meta-regression are planned if the studies are deemed statistically, methodologically and clinically homogenous. Evidence will be graded for its strength for a select number of outcomes. Discussion We will explore the findings of perioperative clinical trials studying the use of pregabalin for acute pain. We will comment on the implications of the findings and provide further direction for the appropriate use of pregabalin in acute pain. This protocol will attempt to bridge the growing gap between clinical experience and emerging evidence, and has the potential to aid future guideline development in the perioperative use of pregabalin. Trial registration PROSPERO registration number CRD42012002078

Background The objective of this systematic review was to examine the benefits, harms and pharmacokinetic interactions arising from the co-administration of commonly used dietary supplements with cardiovascular drugs. Many patients on cardiovascular drugs take dietary supplements for presumed benefits and may be at risk for adverse supplement-drug interactions. Methods The Allied and Complementary Medicine Database, the Cochrane Library, EMBASE, International Bibliographic Information on Dietary Supplements and MEDLINE were searched from the inception of the review to October 2011. Grey literature was also reviewed. Two reviewers independently screened records to identify studies comparing a supplement plus cardiovascular drug(s) with the drug(s) alone. Reviewers extracted data using standardized forms, assessed the study risk of bias, graded the strength of evidence and reported applicability. Results Evidence was obtained from 65 randomized clinical trials, 2 controlled clinical trials and 1 observational study. With only a few small studies available per supplement, evidence was insufficient for all predefined gradable clinical efficacy and harms outcomes, such as mortality and serious adverse events. One long-term pragmatic trial showed no benefit from co-administering vitamin E with aspirin on a composite cardiovascular outcome. Evidence for most intermediate outcomes was insufficient or of low strength, suggesting no effect. Incremental benefits were noted for triglyceridemia with omega-3 fatty acid added to statins; and there was an improvement in levels of high-density lipoprotein cholesterol with garlic supplementation when people also consumed nitrates Conclusions Evidence of low-strength indicates benefits of omega-3 fatty acids (plus statin, or calcium channel blockers and antiplatelets) and garlic (plus nitrates or warfarin) on triglycerides and HDL-C, respectively. Safety concerns, however, persist.

The evidence for testing thiopurine S-methyltransferase (TPMT) enzymatic activity or genotype before starting therapy with thiopurine-based drugs is unclear.To examine the sensitivity and specificity of TPMT genotyping for TPMT enzymatic activity, reducing harm from thiopurine by pretesting, and the association of thiopurine toxicity with TPMT status in adults and children with chronic inflammatory diseases.MEDLINE, EMBASE, the Cochrane Library, and Ovid HealthSTAR (from inception to December 2010) and BIOSIS and Genetics Abstracts (to May 2009).Two reviewers screened records and identified relevant studies in English.Data on patient characteristics, outcomes, and risk for bias were extracted by one reviewer and independently identified by another.54 observational studies and 1 randomized, controlled trial were included. Insufficient evidence addressed the effectiveness of pretesting. Genotyping sensitivity to identify patients with low and intermediate TPMT enzymatic activity ranged from 70.33% to 86.15% (lower-bound 95% CI, 54.52% to 70.88%; upper-bound CI, 78.50% to 96.33%). Sparse data precluded estimation of genotype sensitivity to identify patients with low to absent enzymatic activity. Genotyping specificity approached 100%. Compared with noncarriers, heterozygous and homozygous genotypes were both associated with leukopenia (odds ratios, 4.29 [CI, 2.67 to 6.89] and 20.84 [CI, 3.42 to 126.89], respectively). Compared with intermediate or normal activity, low TPMT enzymatic activity was significantly associated with myelotoxicity and leukopenia.Available evidence was not rigorous and was underpowered to detect a difference in outcomes.Insufficient evidence addresses the effectiveness of TPMT pretesting in patients with chronic inflammatory diseases. Estimates of the sensitivity of genotyping are imprecise. Evidence confirms the known associations of leukopenia or myelotoxicity with reduced TPMT activity or variant genotype.Agency for Healthcare Research and Quality.

Evidence-based practice requires translating research findings into clinical and policy decision making. Clinical practice guidelines (CPGs) serve this purpose by evaluating evidence and making recommendations about therapeutic and diagnostic interventions and clinical management strategies. Systematic reviews are considered the best available evidence and are often used in the development of CPGs [1,2]. Since guideline development involves an assessment of the overall quality of evidence and complex balancing of trade-offs between the important benefits and harms of any given intervention, arbitrariness, value judgements, and subjectivity ultimately come into play in the guideline development process and associated recommendations [3]. In order to minimize cognitive bias in interpreting evidence and make the inherently subjective process more transparent and consistent, CPGs have traditionally employed formal systems or frameworks to understand and grade the quality of the body of evidence and strength of recommendations [4,5]. One such framework is the grading quality of evidence and strength of recommendations (GRADE), which is commonly used by guideline panels in deriving health care recommendations. GRADE was developed to overcome some of the deficiencies of earlier efforts [6]. GRADE defines the quality of evidence as the collective level of confidence guideline developers have about the validity of estimates of benefits and harms for any given intervention, and the strength of guideline recommendation as the extent of collective confidence that adherence to the recommendation will do more good than harm [7]. It urges guideline developers to consider all important patient outcomes of benefit and harm, to systematically evaluate the quality of their estimates, and to assess the trade-offs between evidence of benefits and harms, the preferences and values placed by patients on outcomes, the opportunity cost associated with the recommendation, and the feasibility of recommendations given a clinical setting before formulating guideline recommendations. Details of the GRADE approach have been published elsewhere [8]. In a new Policy Forum published in this issue of PLoS Medicine, Kavanagh [9] questions the external consistency of the GRADE framework by comparing the Surviving Sepsis Campaign (SSC) guideline recommendations developed in 2004 and updated in 2008. Moreover, Kavanagh expresses his concerns on the processes of the GRADE development and its formal validation. Had we likened the GRADE approach to an instrument or a health profile built on discrete logic to capture evidence, we would have concurred with some of Kavanagh’s criticism of GRADE. However, we see GRADE as a framework uncovering implicit subjectivity and invoking a systematic, explicit, judicious, and transparent approach to interpreting, as opposed to ‘‘capturing’’ evidence. It reveals how values are assigned to judgments, but what values are assigned it does not dictate simply because it cannot dictate. Below we first present our concern about one aspect of the GRADE framework and then our perspective on the various criticisms of it.

Objective To develop two checklists for the quality of observational studies of incidence or risk factors of diseases. Study Design and Setting Initial development of the checklists was based on a systematic literature review. The checklists were refined after pilot trials of validity and reliability were conducted by seven experts, who tested the checklists on 10 articles. Results The checklist for studies of incidence or prevalence of chronic disease had six criteria for external validity and five for internal validity. The checklist for risk factor studies had six criteria for external validity, 13 criteria for internal validity, and two aspects of causality. A Microsoft Access database produced automated standardized reports about external and internal validities. Pilot testing demonstrated face and content validities and discrimination of reporting vs. methodological qualities. Interrater agreement was poor. The experts suggested future reliability testing of the checklists in systematic reviews with preplanned protocols, a priori consensus about research-specific quality criteria, and training of the reviewers. Conclusion We propose transparent and standardized quality assessment criteria of observational studies using the developed checklists. Future testing of the checklists in systematic reviews is necessary to develop reliable tools that can be used with confidence.

Background Botulinum toxin A is a commonly used biological medication in the field of facial plastic surgery. Currently, there are three distinct formulations of botulinum toxin A, each with their purported benefits and advantages. However, there is considerable confusion as to the relative efficacy and side-effects associated with each formulation. Therefore, the purpose of this paper is to systematically assess published studies and perform a meta-analysis to determine if there is a significant advantage of any of the individual formulations. Methods/design A systematic literature search was performed for all relevant English language randomized controlled trials using Embase, Cumulative Index to Nursing and Allied Health Literature (CINAHL), MEDLINE, World Health Organization (WHO) International Clinical Trials Registry Platform, European Union (EU) Clinical Trials Register, Cochrane Library databases of clinical trials, and ClinicalTrials.gov. Inclusion criteria included any randomized controlled trial (RCT) that assessed the use of botulinum toxin for cosmetic purposes. The included articles were also analyzed for bias using the Cochrane Collaboration’s tool for assessing the risk of bias in RCTs. Discussion The results of this review will provide clinicians with an unbiased, high level of evidence of the comparative efficacy of individual preparations of botulinum toxin A. Trial registration PROSPERO: CRD4201200337

Background Cardiovascular disease and type 2 diabetes are examples of chronic diseases that impose significant morbidity and mortality in the general population worldwide. Most chronic diseases are associated with underlying preventable risk factors, such as elevated blood pressure, high blood glucose or glucose intolerance, high lipid levels, physical inactivity, excessive sedentary behaviours, and overweight/obesity. The occurrence of intermediate outcomes during childhood increases the risk of disease in adulthood. Sugar-sweetened beverages are known to be significant sources of additional caloric intake, and given recent attention to their contribution in the development of chronic diseases, a systematic review is warranted. We will assess whether the consumption of sugar-sweetened beverages in children is associated with adverse health outcomes and what the potential moderating factors are. Methods/Design Of interest are studies addressing sugar-sweetened beverage consumption, taking a broad perspective. Both direct consumption studies as well as those evaluating interventions that influence consumption (e.g. school policy, educational) will be relevant. Non-specific or multi-faceted behavioural, educational, or policy interventions may also be included subject to the level of evidence that exists for the other interventions/exposures. Comparisons of interest and endpoints of interest are pre-specified. We will include randomized controlled trials, controlled clinical trials, interrupted time series studies, controlled before-after studies, prospective and retrospective comparative cohort studies, case–control studies, and nested case–control designs. The MEDLINE®, Embase, The Cochrane Library, CINAHL, ERIC, and PsycINFO® databases and grey literature sources will be searched. The processes for selecting studies, abstracting data, and resolving conflicts are described. We will assess risk of bias using design-specific tools. To determine sets of confounding variables that should be adjusted for, we have developed causal directed acyclic graphs and will use those to inform our risk of bias assessments. Meta-analysis will be conducted where appropriate; parameters for exploring statistical heterogeneity and effect modifiers are pre-specified. The Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach will be used to determine the quality of evidence for outcomes. Systematic review registration PROSPERO CRD42014009641.

Authors of Cochrane reviews are expected to update their reviews every 2 years. The updating process helps to ensure that reviews are current and include recent evidence. However, the updating process is time-consuming for authors, particularly when Cochrane methods evolve and authors are required to revisit some of the originally included studies. The Cochrane Collaboration's ‘Risk of bias’ tool is a mandatory component of Cochrane reviews, providing an assessment of the potential biases of included studies. The tool has been modified most recently in 2011, and the expectation is that new versions will continue to be produced and utilised in all Cochrane reviews. In this commentary we discuss, in the context of updating scenarios that are likely to be encountered, the potential options systematic review authors may have recourse to when the Cochrane Collaboration's ‘Risk of bias’ tool has been modified between the original review and its update. We recommend that authors who are updating reviews should revise their original assessments of included studies using the most recent version of the risk of bias tool. Despite the increased workload, use of the most recent version of the tool facilitates consistency of methods and reporting both across and within reviews, and ensures currency to the methodological rigour.

Background Systematic reviews (SRs) can become outdated as new evidence emerges over time. Organizations that produce SRs need a surveillance method to determine when reviews are likely to require updating. This report describes the development and initial results of a surveillance system to assess SRs produced by the Agency for Healthcare Research and Quality (AHRQ) Evidence-based Practice Center (EPC) Program. Methods Twenty-four SRs were assessed using existing methods that incorporate limited literature searches, expert opinion, and quantitative methods for the presence of signals triggering the need for updating. The system was designed to begin surveillance six months after the release of the original review, and thenceforth every six months for any review not classified as being a high priority for updating. The outcome of each round of surveillance was a classification of the SR as being low, medium or high priority for updating. Results Twenty-four SRs underwent surveillance at least once, and ten underwent surveillance a second time during the 18 months of the program. Two SRs were classified as high, five as medium, and 17 as low priority for updating. The time lapse between the searches conducted for the original reports and the updated searches (search time lapse - STL) ranged from 11 months to 62 months: The STL for the high priority reports were 29 months and 54 months; those for medium priority reports ranged from 19 to 62 months; and those for low priority reports ranged from 11 to 33 months. Neither the STL nor the number of new relevant articles was perfectly associated with a signal for updating. Challenges of implementing the surveillance system included determining what constituted the actual conclusions of an SR that required assessing; and sometimes poor response rates of experts. Conclusion In this system of regular surveillance of 24 systematic reviews on a variety of clinical interventions produced by a leading organization, about 70% of reviews were determined to have a low priority for updating. Evidence suggests that the time period for surveillance is yearly rather than the six months used in this project.

Systematic reviews are often advocated as the best source of evidence to guide clinical decisions and health care policy, yet we know little about the extent to which they require updating.To estimate the average time to changes in evidence that are sufficiently important to warrant updating systematic reviews.Survival analysis of 100 quantitative systematic reviews.Systematic reviews published from 1995 to 2005 and indexed in ACP Journal Club. Eligible reviews evaluated a specific drug or class of drug, device, or procedure and included only randomized or quasi-randomized, controlled trials.Quantitative signals for updating were changes in statistical significance or relative changes in effect magnitude of at least 50% involving 1 of the primary outcomes of the original systematic review or any mortality outcome. Qualitative signals included substantial differences in characterizations of effectiveness, new information about harm, and caveats about the previously reported findings that would affect clinical decision making.The cohort of 100 systematic reviews included a median of 13 studies and 2663 participants per review. A qualitative or quantitative signal for updating occurred for 57% of reviews (95% CI, 47% to 67%). Median duration of survival free of a signal for updating was 5.5 years (CI, 4.6 to 7.6 years). However, a signal occurred within 2 years for 23% of reviews and within 1 year for 15%. In 7%, a signal had already occurred at the time of publication. Only 4% of reviews had a signal within 1 year of the end of the reported search period; 11% had a signal within 2 years of the search. Shorter survival was associated with cardiovascular topics (hazard ratio, 2.70 [CI, 1.36 to 5.34]) and heterogeneity in the original review (hazard ratio, 2.15 [CI, 1.12 to 4.11]).Judgments of the need for updating were made without involving content experts.In a cohort of high-quality systematic reviews directly relevant to clinical practice, signals for updating occurred frequently and within a relatively short time.