Search

Objectives: To systematically survey Cochrane reviews' approaches to calculating, presenting, and interpreting pooled estimates of patient-reported outcome measures (PROMs)., Study Design and Setting: We retrospectively selected 200 Cochrane reviews that met the eligibility criteria. Two researchers independently extracted the pooled effect measures and approaches for pooling and interpreting the effect measures, reaching consensus through discussions., Results: When primary studies used the same PROM, Cochrane review authors most often used mean differences (MDs) (81.9%) for calculating the pooled effect measures; when primary studies used different PROMs, the review authors often applied standardized mean differences (SMDs) (54.3%). Although in most cases (80.1%) the review authors interpreted the importance of effect, they failed, in 48.5% of the pooled effect measures, to report criteria for categorizing the magnitude of effect. When authors interpreted the importance of the effect, for those with primary studies using the same PROM, they most often referred to the minimally important differences (MIDs) (75.0%); for those with primary studies using different PROMs, the approaches used varied., Conclusion: Cochrane review authors most often used MDs or SMDs for calculating and presenting the pooled effect measures of PROs but often failed to make explicit their criteria for categorizing the magnitude of effect., (Copyright © 2023 The Authors. Published by Elsevier Inc. All rights reserved.)

Objectives: Minimal important difference (MID), the smallest change or difference that patients perceive as important, aids interpretation of change in patient-reported outcome measure (PROM) scores. A credibility instrument that assesses the methodological rigor of an anchor-based MID includes one core item addressing the correlation between the PROM and the anchor. However, the majority of MID studies in the literature fail to report the correlation. To address this issue, we extended the anchor-based MID credibility instrument by adding an item addressing construct proximity as an alternative to the correlation item., Study Design and Setting: Informed by an MID methodological survey, we added an alternative item-a subjective assessment of similarity of the constructs (i.e., construct proximity) between PROM and anchor-to the correlation item and generated principles for the assessment. We sampled 101 MIDs and analyzed the assessments performed by each pair of raters. By calculating weighted Cohen's kappa, we assessed the reliability of the assessments., Results: Construct proximity assessment is based on the anticipated association between the anchor and PROM constructs: the closer the anticipated association, the higher the rating. Our detailed principles address the most frequently used anchors: transition ratings, measures of satisfaction, other PROMs, and clinical measures. The assessments showed acceptable agreement (weighted kappa 0.74, 95% CI 0.55-0.94) between raters., Conclusion: In the absence of a reported correlation coefficient, construct proximity assessment provides a useful alternative in the credibility assessment of anchor-based MID estimates., (Copyright © 2023 Elsevier Inc. All rights reserved.)

Background and Objectives: To evaluate reporting of minimal important difference (MID) estimates using anchor-based methods for patient-reported outcome measures (PROMs), and the association with reporting deficiencies on their credibility., Methods: Systematic survey of primary studies empirically estimating MIDs. We searched Medline, EMBASE, PsycINFO, and the Patient-Reported Outcome and Quality of Life Instruments Database until October 2018. We evaluated study reporting, focusing on participants' demographics, intervention(s), characteristics of PROMs and anchors, and MID estimation method(s). We assessed the impact of reporting issues on credibility of MID estimates., Results: In 585 studies reporting on 5,324 MID estimates for 526 distinct PROMs, authors frequently failed to adequately report key characteristics of PROMs and MIDs, including minimum and maximum values of PROM scale, measure of variability accompanying the MID estimate and number of participants included in the MID calculation. Across MID estimates (n = 5,324), the most serious reporting issues impacting credibility included infrequent reporting of the correlation between the anchor and PROM (66%), inadequate details to judge precision of MID point estimate (13%), and insufficient information about the threshold used to ascertain MIDs (16%)., Conclusion: Serious issues of incomplete reporting in the MID literature threaten the optimal use of MID estimates to inform the magnitude of effects of interventions on PROMs., (Copyright © 2022 Elsevier Inc. All rights reserved.)

Objectives: This study aims to describe the distribution of anchor-based minimal important change (MIC) estimates in standard deviation (SD) units and examine if the robustness of such estimates depends on the specific SD used or on the methodological credibility of the anchor-based estimates., Design and Setting: We included all anchor-based MIC estimates from studies published in MEDLINE and relevant literature databases upto October 2018. Each MIC was converted to SD units using baseline, endpoint, and change from baseline SDs. We performed a descriptive analysis of MICs in SD units and checked how the distribution would change if MICs with low methodological credibility were excluded from the analysis., Results: We included 1,009 MIC estimates from 182 studies. The medians and interquartile ranges of MICs in SD units were 0.43 (0.25 to 0.69), 0.42 (0.22 to 0.70), and 0.51 (0.28 to 0.78) for baseline, endpoint, and change SD units, respectively. Some MICs were extremely large or small. The distribution did not change significantly after excluding MICs estimated by less credible methods., Conclusions: The size of the universally applicable MIC in SD units could not be determined. Anchor-based MICs in SD units were widely distributed, with more than half in the range of 0.2 to 0.8., (Copyright © 2022. Published by Elsevier Inc.)

Objective: To systematically survey the literature addressing the reporting of studies estimating anchor-based minimal important differences (MIDs) and choice of optimal MIDs., Study Design and Setting: We searched Medline, Embase and PsycINFO from 1987 to March 2020. Teams of two reviewers independently identified eligible publications and extracted quotations addressing relevant issues for reporting and/or selecting anchor-based MIDs. Using a coding list, we assigned the same code to quotations capturing similar or related issues. For each code, we generated an 'item', i.e., a specific phrase or sentence capturing the underlying concept. When multiple concepts existed under a single code, the team created multiple items for that code. We clustered codes addressing a broader methodological issue into a 'category' and classified items as relevant for reporting, relevant for selecting an anchor-based MID, or both., Results: We identified 136 eligible publications that provided 6 categories (MID definition, anchors, patient-reported outcome measures, generalizability and statistics) and 24 codes. These codes contained 34 items related to reporting MID studies, of which 29 were also related to selecting MIDs., Conclusion: The systematic survey identified items related to reporting of anchor-based MID studies and selecting optimal MIDs. These provide a conceptual framework to inform the design of studies related to MIDs, and a basis for developing a reporting standard and a selection approach for MIDs., (Copyright © 2021 Elsevier Inc. All rights reserved.)

Objective: To provide practical principles and examples to help GRADE users make optimal choices regarding their ratings of certainty of evidence using a minimally or partially contextualized approach., Study Design and Setting: Based on the GRADE clarification of certainty of evidence in 2017, a project group within the GRADE Working Group conducted iterative discussions and presentations at GRADE Working Group meetings to refine this construct and produce practical guidance., Results: Systematic review and health technology assessment authors need to clarify what it is in which they are rating their certainty of evidence (i.e., the target of their certainty rating). The decision depends on the degree of contextualization (partially or minimally contextualized), thresholds (null, small, moderate or large effect threshold), and where the point estimate lies in relation to the chosen threshold(s). When the 95% confidence interval crosses multiple possible thresholds (i.e., including both large benefit and large harm), it is not worthwhile for authors to determine the target of certainty rating., Conclusion: GRADE provides practical principles to help systematic review and health technology assessment authors specify the target of their certainty of evidence rating., (Copyright © 2021 The Author(s). Published by Elsevier Inc. All rights reserved.)

Objectives: The objective of the study was to develop an inventory summarizing all anchor-based minimal important difference (MID) estimates for patient-reported outcome measures (PROMs) available in the medical literature., Study Design and Setting: We searched MEDLINE, EMBASE, CINAHL, PsycINFO, and the Patient-Reported Outcome and Quality of Life Instruments Database internal library (January 1989-October 2018). We included primary studies empirically calculating an anchor-based MID estimate for any PROM in adults and adolescents. Pairs of reviewers independently screened and selected studies, extracted data, and evaluated the credibility of the MIDs., Results: We identified 585 eligible studies, the majority conducted in Europe (n = 211) and North America (n = 179), reporting 5,324 MID estimates for 526 distinct PROMs. Investigators conducted their studies in the context of patients receiving surgical (n = 105, 18%), pharmacological (n = 85, 15%), rehabilitation (n = 65, 11%), or a combination of interventions (n = 194, 33%). Of all MID estimates, 59% (n = 3,131) used a global rating of change anchor. Major credibility limitations included weak correlation (n = 1,246, 23%) or no information regarding the correlation (n = 3,498, 66%) between the PROM and anchor and imprecision in the MID estimate (n = 2,513, 47%)., Conclusion: A large number of MIDs for assisting in the interpretation of PROMs exist. The MID inventory will facilitate the use of MID estimates to inform the interpretation of the magnitude of treatment effects in clinical research and guideline development., (Copyright © 2020 Elsevier Inc. All rights reserved.)

Objective: The objective of the study was to systematically survey the methodological literature and collect suggested criteria for assessing the credibility of effect modification and associated rationales., Study Design and Setting: We searched MEDLINE, Embase, and WorldCat up to March 2018 for publications providing guidance for assessing the credibility of effect modification identified in randomized trials or meta-analyses. Teams of two investigators independently identified eligible publications and extracted credibility criteria and authors' rationale, reaching consensus through discussion. We created a taxonomy of criteria that we iteratively refined during data abstraction., Results: We identified 150 eligible publications that provided 36 criteria and associated rationales. Frequent criteria included significant test for interaction (n = 54), a priori hypothesis (n = 49), providing a causal explanation (n = 47), accounting for multiplicity (n = 45), testing a small number of effect modifiers (n = 38), and prespecification of analytic details (n = 39). For some criteria, we found more than one rationale; some criteria were connected through a common rationale. For some criteria, experts disagreed regarding their suitability (e.g., added value of stratified randomization; trustworthiness of biologic rationales)., Conclusion: Methodologists have expended substantial intellectual energy providing criteria for critical appraisal of apparent effect modification. Our survey highlights popular criteria, expert agreement and disagreement, and where more work is needed, including testing criteria in practice., (Copyright © 2019. Published by Elsevier Inc.)

Objective: To assess analytic approaches randomized controlled trial (RCT) authors use to address missing participant data (MPD) for patient-important continuous outcomes., Study Design and Setting: We conducted a systematic survey of RCTs published in 2014 in the core clinical journals that reported at least one patient-important outcome analyzed as a continuous variable., Results: Among 200 studies, 187 (93.5%) trials explicitly reported whether MPD occurred. In the 163 (81.5%) trials that reported the occurrence of MPD, the median and interquartile ranges of the percentage of participants with MPD were 11.4% (2.5%-22.6%).Among the 147 trials in which authors made clear their analytical approach to MPD, the approaches chosen included available data only (109, 67%); mixed-effect models (10, 6.1%); multiple imputation (9, 4.5%); and last observation carried forward (9, 4.5). Of the 163 studies reporting MPD, 16 (9.8%) conducted sensitivity analyses examining the impact of the MPD and (18, 11.1%) discussed the risk of bias associated with MPD., Conclusion: RCTs reporting continuous outcomes typically have over 10% of participant data missing. Most RCTs failed to use optimal analytic methods, and very few conducted sensitivity analyses addressing the possible impact of MPD or commented on how MPD might influence risk of bias., (Copyright © 2017 Elsevier Inc. All rights reserved.)