Search

David Ruben Teindl Laursen,1–4 Asger Sand Paludan-Müller,2 Asbjørn Hróbjartsson1,3,4 1Centre for Evidence-Based Medicine Odense (CEBMO), Odense University Hospital, Odense, Denmark; 2Nordic Cochrane Centre, Rigshospitalet, Copenhagen, Denmark; 3Department of Clinical Research, University of Southern Denmark, Odense, Denmark; 4Odense Patient data Explorative Network (OPEN), Odense University Hospital, Odense, Denmark Background: Run-in periods are occasionally used in randomized clinical trials to exclude patients after inclusion, but before randomization. In theory, run-in periods increase the probability of detecting a potential treatment effect, at the cost of possibly affecting external and internal validity. Adequate reporting of exclusions during the run-in period is a prerequisite for judging the risk of compromised validity. Our study aims were to assess the proportion of randomized clinical trials with run-in periods, to characterize such trials and the types of run-in periods and to assess their reporting. Materials and methods: This was an observational study of 470 PubMed-indexed randomized controlled trial publications from 2014. We compared trials with and without run-in periods, described the types of run-in periods and evaluated the completeness of their reporting by noting whether publications stated the number of excluded patients, reasons for exclusion and baseline characteristics of the excluded patients. Results: Twenty-five trials reported a run-in period (5%). These were larger than other trials (median number of randomized patients 217 vs 90, P=0.01) and more commonly industry trials (11% vs 3%, P

Routinely collected health data (RCD) including electronic health records, disease registries, health administrative data and wearables data are not specifically collected for research purposes. Analysis of these data poses unique methodological challenges that must be addressed when conducting research, particularly as availability and use increase. This scoping review aimed to identify methodological challenges in research using RCD from existing literature (registered protocol: https://doi.org/10.17605/OSF.IO/EBM4D). We searched 6 electronic databases, including medical, health economics, nursing and psychology research databases, between Jan 2015 and Jan 2023, combining multiple “RCD” and “research” search terms (e.g., epidemiologic, informatics, pharmaceutical research). After screening abstracts and full-texts, we doubly extracted methodological themes, categorizing them into different study stages. We screened more than 23,000 records and included 430 papers. Bias and confounding were the most common methodological issues identified, discussed in relation to both study design and data analysis. Data quality, including data accuracy, validation, completeness, timeliness and cleaning, also posed substantial challenges, particularly during data processing stage. Record linkage and conducting analyses using distributed health networks also pose unique methodological challenges. Heterogeneity, incorporating social determinants of health and statistical models that address methodological challenges are also described in the literature. External validity and reporting are important considerations for RCD research. Our review identified several methodological challenges facing researchers using RCD. These issues should be addressed to ensure methodologically sound research. These findings will inform the development of a standardized protocol template and accompanying educational platform aimed at enhancing methodological quality and transparency when conducting research using RCD.

We investigated to which degree commercial funding is associated with estimated intervention effects in randomized trials. We included meta-epidemiological studies with published data on the association between commercial funding and results or conclusions of randomized trials. We searched five databases and other sources. We selected one result per meta-epidemiological study, preferably unadjusted ratio of odds ratios (ROR), for example, odds ratio(commercial funding)/odds ratio(noncommercial funding). We pooled RORs in random-effects meta-analyses (ROR <1 indicated exaggerated intervention effects in commercially funded trials), subgrouped (preplanned) by study aim: commercial funding per se versus risk of commercial funder influence. We included eight meta-epidemiological studies (264 meta-analyses, 2725 trials). The summary ROR was 0.95 (95% confidence interval 0.85-1.06). Subgroup analysis revealed a difference (p = 0.02) between studies of commercial funding per se, ROR 1.06 (0.95-1.17) and studies of risk of commercial funder influence, ROR 0.88 (0.79-0.97). In conclusion, we found no statistically significant association between commercial funding and estimated intervention effects when combining studies of commercial funding per se and studies of risk of commercial funder influence. A preplanned subgroup analysis indicated that trials with high risk of commercial funder influence exaggerated intervention effects by 12% (21%-3%), on average. Our results differ from previous theoretical considerations and findings from methodological studies and therefore call for confirmation. We suggest it is prudent to interpret results from commercially funded trials with caution, especially when there is a risk that the funder had direct influence on trial design, conduct, analysis, or reporting.

Abstract Background Orthopaedic surgeons disagree considerably when classifying fractures of the proximal humerus. However, the clinical implications of low observer agreement remain unclear. The purpose of the study was to compare the agreement on Neer classification with the agreement on treatment recommendations. Methods We conducted a multi-centre observer-study. Five experienced shoulder surgeons independently assessed a consecutive series of 193 radiographs at two occasions three months apart. All pairs of radiographs were classified according to Neer. Subsequently, the observers were asked to recommend one of three treatment modalities for each case: non-operative treatment, locking plate osteosynthesis, or hemiarthroplasty. Results At both classification rounds mean kappa-values for inter-observer agreement on treatment recommendations (0.48 and 0.52) were significantly higher than the agreement on Neer classification (0.33 and 0.36) (p Conclusions We found a significantly higher agreement on treatment recommendations compared to agreement on fracture classification. The low observer agreement on the Neer classification reported in several observer studies may have less clinical importance than previously assumed. However, inter-observer agreement did not exceed moderate levels.

Abstract Background It can be challenging for patients and clinicians to properly interpret a change in the clinical condition after a treatment has been given. It is not known to which extent spontaneous improvement, effect of placebo and effect of active interventions contribute to the observed change from baseline, and we aimed at quantifying these contributions. Methods Systematic review and meta-analysis, based on a Cochrane review of the effect of placebo interventions for all clinical conditions. We selected all trials that had randomised the patients to three arms: no treatment, placebo and active intervention, and that had used an outcome that was measured on a continuous scale or on a ranking scale. Clinical conditions that had been studied in less than three trials were excluded. Results We analysed 37 trials (2900 patients) that covered 8 clinical conditions. The active interventions were psychological in 17 trials, physical in 15 trials, and pharmacological in 5 trials. Overall, across all conditions and interventions, there was a statistically significant change from baseline in all three arms. The standardized mean difference (SMD) for change from baseline was -0.24 (95% confidence interval -0.36 to -0.12) for no treatment, -0.44 (-0.61 to -0.28) for placebo, and -1.01 (-1.16 to -0.86) for active treatment. Thus, on average, the relative contributions of spontaneous improvement and of placebo to that of the active interventions were 24% and 20%, respectively, but with some uncertainty, as indicated by the confidence intervals for the three SMDs. The conditions that had the most pronounced spontaneous improvement were nausea (45%), smoking (40%), depression (35%), phobia (34%) and acute pain (25%). Conclusion Spontaneous improvement and effect of placebo contributed importantly to the observed treatment effect in actively treated patients, but the relative importance of these factors differed according to clinical condition and intervention.

Randomised trials are often funded by commercial companies and methodological studies support a widely held suspicion that commercial funding may influence trial results and conclusions. However, these studies often have a risk of confounding and reporting bias. The risk of confounding is markedly reduced in meta-epidemiological studies that compare fairly similar trials within meta-analyses, and risk of reporting bias is reduced with access to unpublished data. Therefore, we initiated the COMmercial Funding In Trials (COMFIT) study aimed at investigating the impact of commercial funding on estimated intervention effects in randomised clinical trials based on a consortium of researchers who agreed to share meta-epidemiological study datasets with information on meta-analyses and trials included in meta-epidemiological studies. Here, we describe the COMFIT study, its database, and descriptive results. We included meta-epidemiological studies with published or unpublished data on trial funding source and results or conclusions. We searched five bibliographic databases and other sources. We invited authors of eligible meta-epidemiological studies to join the COMFIT consortium and to share data. The final construction of the COMFIT database involves checking data quality, identifying trial references, harmonising variable categories, and removing non-informative meta-analyses as well as correlated meta-analyses and trial results. We included data from 17 meta-epidemiological studies, covering 728 meta-analyses and 6841 trials. Seven studies (405 meta-analyses, 3272 trials) had not published analyses on the impact of commercial funding, but shared unpublished data on funding source. On this basis, we initiated the construction of a combined database. Once completed, the database will enable comprehensive analyses of the impact of commercial funding on trial results and conclusions with increased statistical power and a markedly reduced risk of confounding and reporting bias. [This article was written on behalf of the COMFIT Consortium.]

Randomized clinical trials underpin evidence-based clinical practice, but flaws in their conduct may lead to biased estimates of intervention effects and hence invalid treatment recommendations. The main approach to the empirical study of bias is to collate a number of meta-analyses and, within each, compare the results of trials with and without a methodological characteristic such as blinding of participants and health professionals. Estimated within-meta-analysis differences are combined across meta-analyses, leading to an estimate of mean bias. Such "meta-epidemiological" studies are published in increasing numbers and have the potential to inform trial design, assessment of risk of bias, and reporting guidelines. However, their interpretation is complicated by issues of confounding, imprecision, and applicability. We developed a guide for interpreting meta-epidemiological studies, illustrated using MetaBLIND, a large study on the impact of blinding. Applying generally accepted principles of research methodology to meta-epidemiology, we framed 10 questions covering the main issues to consider when interpreting results of such studies, including risk of systematic error, risk of random error, issues related to heterogeneity, and theoretical plausibility. We suggest that readers of a meta-epidemiological study reflect comprehensively on the research question posed in the study, whether an experimental intervention was unequivocally identified for all included trials, the risk of misclassification of the trial characteristic, and the risk of confounding, i.e the adequacy of any adjustment for the likely confounders. We hope that our guide to interpretation of results of meta-epidemiological studies is helpful for readers of such studies.

In 2020 we published MetaBLIND, a large meta-epidemiological study on the impact of masking on effect estimates in randomized clinical trials (RCTs). While masking is an established methodological practice in RCTs it is not clear to what extent results of non-masked RCTs are biased. Surprisingly, we found no evidence of an impact of masking on effect estimates, on average. Michiel Tack commented on the MetaBLIND study, and here we respond. The issues he raised were examples of standard themes when interpreting meta-epidemiological studies, which we have discussed at some length elsewhere, and did not warrant change of our conclusion. We maintain that the MetaBLIND results do not provide a sufficient basis for recommending abandoning masking as a methodological safeguard. [ABSTRACT FROM AUTHOR]