Search

Rationale, Aims, and Objectives Methods Results Conclusion The previous studies demonstrated that the Grading of Recommendations Assessment, Development and Evaluation (GRADE) system, a leading method for evaluating the certainty (quality) of scientific evidence (CoE), cannot reliably differentiate between various levels of CoE when the objective is to accurately assess the magnitude of the treatment effect. An estimated effect size is a function of multiple factors, including the true underlying treatment effect, biases, and other nonlinear factors that affect the estimate in different directions. We postulate that non‐weighted, simple linear tallying can provide more accurate estimates of the probability of a true estimate of treatment effects as a function of CoE.We reasoned that stable treatment effect estimates over time indicate truthfulness. We compared odds ratios (ORs) from meta‐analyses (MAs) before and after updates, hypothesising that a ratio of odds ratios (ROR) equal to 1 will be more commonly observed in higher versus lower CoE. We used a subset of a previously analysed data set consisting of 82 Cochrane pairs of MAs in which CoE has not changed with the updated MA. If the linear model is valid, we would expect a decrease in the number of ROR = 1 cases as we move from high to moderate, low, and very low CoE.We found a linear relationship between the probability of a potentially ‘true’ estimate of treatment effects as a function of CoE (assuming a 10% ROR error margin) (R2 = 1; p = 0.001). The probability of potentially ‘true’ estimates decreases by 21% (95% CI: 18%–24%) for each drop in the rating of CoE. A linear relationship with a 5% ROR error margin was less clear, likely due to a smaller sample size. Still, higher CoE showed a significantly greater probability of ‘true’ effects (53%) compared to non‐high (i.e., moderate, low, or very low) CoE (25%); p = 0.032.This study confirmed linear relationship between CoE and the probability of potentially ‘true’ estimates. We found that the probability of potentially “true” estimates decreases by about 20% for each drop in CoE (from about 80% for high to 55% for moderate to 35% to low and 15% to very low CoE). [ABSTRACT FROM AUTHOR]

The Evaluation of Genomic Applications in Practice and Prevention Working Group was first convened in 2005 to develop and test evidence-based methods for the evaluation of genomic tests in transition from research to clinical and public health practice. Over the ensuing years, the Working Group has met 26 times, publishing eight recommendation statements, two methods papers, and one outcomes paper, as well as planning and serving as technical experts on numerous associated systematic reviews. Evaluation of Genomic Applications in Practice and Prevention methods have evolved to address implications of the proliferation of genome-wide association studies and are currently expanding to face challenges expected from clinical implementation of whole-genome sequencing tests. In this article, we review the work of the Evaluation of Genomic Applications in Practice and Prevention Working Group over the first 8 years of its existence with an emphasis on lessons learned throughout the process. It is hoped that in addition to the published methods of the Working Group, the lessons we have learned along the way will be informative to others who are producers and consumers of evidence-based guidelines in the field of genomic medicine.

Background: Current methods for developing clinical practice guidelines have several limitations: they are characterised by the "black box" operation—a process with defined inputs and outputs but an incomplete understanding of its internal workings; they have "the integration problem"—a lack of framework for explicitly integrating factors such as patient preferences and trade‐offs between benefits and harms; they generate one recommendation at a time that typically are not connected in a coherent analytical framework; and they apply to "average" patients, while clinicians and their patients seek advice tailored to individual circumstances. Methods: We propose augmenting the current guideline development method by converting evidence‐based pathways into fast‐and‐frugal decision trees (FFTs) and integrating them with generalised decision curve analysis to formulate clear, individualised management recommendations. Results: We illustrate the process by developing recommendations for the management of heparin‐induced thrombocytopenia (HIT). We converted evidence‐based pathways for HIT, developed by the American Society of Hematology, into an FFT. Here, we consider only thrombotic complications and major bleeding. We leveraged the predictive potential of FFTs to compare the effects of argatroban, bivalirudin, fondaparinux, and direct oral anticoagulants (DOACs) using generalised decision curve analysis. We found that DOACs were superior to other treatments if the FFT‐predicted probability of HIT exceeded 3%. Conclusions: The proposed analytical framework connects guidelines, pathways, FFTs, and decision analysis, offering risk‐tailored personalised recommendations and addressing current guideline development critiques. [ABSTRACT FROM AUTHOR]

Background: To realize the potential of precision medicine, predictive models should be integrated within the framework of decision analysis, such as the decision curve analysis (DCA). To date, its application has required individual patient data (IPD) that are often unavailable. Performing DCA using aggregate data without requiring IPD may advance the goals of precision medicine. Methods: We present a statistical framework demonstrating that DCA can be conducted by using only the mean and standard deviation (SD) from the raw probabilities of the predictive model. We tested our theoretical framework by performing extensive simulations and comparing the aggregate‐based DCA with IPD DCA. The latter was conducted using IPD from four predictive models that employed logistic regression, Cox or competing risk time‐to‐event modeling including (a) statins for primary prevention of cardiovascular disease (n = 4859), (b) hospice referral for terminally ill patients (n = 9104), (c) use of thromboprophylaxis for preventing venous thromboembolism in patients with cancer (n = 425) and (d) prevention of sinusoidal obstruction syndrome after hematopoietic cell transplantation (SCT) (n = 80). Results: Simulations assuming perfect calibration showed that regardless of which probability distributions informed the predictive models, the differences in DCA were negligible. Similarly, for the adequately powered models, the results of DCA based on the summary data were similar to IPD‐derived DCA. The inherent instability of the predictive models, based on the smaller sample sizes, resulted in a somewhat larger discrepancy between aggregate and IPD‐based DCA. Conclusions: DCA informed by adequately powered and well‐calibrated models using only summary statistical estimates (mean and SD) approximates well models using IPD. Use of aggregate data will facilitate broader integration of predictive with decision modeling toward the goals of individualized decision‐making. [ABSTRACT FROM AUTHOR]

Rationale: Decision curve analysis (DCA) helps integrate prediction models with treatment assessments to guide personalised therapeutic choices among multiple treatment options. However, the current versions of DCA do not explicitly model treatment effects in the analysis but implicitly or holistically assess therapeutic benefits and harms. In addition, the existing DCA cannot allow the comparison of multiple treatments using a standard metric. Aims and Objectives: To develop a generalised version of DCA (gDCA) by decomposing holistically assessed net benefits and harms into patient preferences versus empirical evidence (as obtained in the trials, meta‐analyses of clinical studies, etc.) to allow individualised comparison of single or multiple treatments using a common metric. Methods: We reformulated DCA by (1) decomposing holistic, implicit utilities into specific utilities related to treatment effects and patient's relative values (RV) about disease outcomes versus treatment harms, (2) explicitly modelling each treatment effect at the level of probabilities and/or utilities (outcomes) in a decision tree, and (3) avoiding scaling effects employed in the original DCA to enable comparison of treatment effects against the common metrics. We used data from a published network meta‐analysis of randomised trials to inform the use of statin treatment according to Framingham Risk Model. Results: We illustrate the analysis by modelling the effects of three statins in the primary prevention of cardiovascular disease. We performed simultaneous comparisons against standard metrics (RV) for all treatments. We examined for which RV values, a predictive model for guiding personalised treatment, outperformed the strategies of treating everyone or treating no one. We found that the magnitude of benefits (efficacy) seems more important than the simple ratio of efficacy/harms. Conclusion: We describe gDCA for evaluating single or multiple treatments to help tailor therapy toward individual risk characteristics. gDCA further helps integrate the principles of evidence‐based medicine with decision analysis. [ABSTRACT FROM AUTHOR]