Search

Standard meta-analysis methods are vulnerable to bias from incomplete reporting of results (both publication and outcome reporting bias) and poor study quality. Several alternative methods have been proposed as being less vulnerable to such biases. To evaluate these claims independently we simulated study results under a broad range of conditions first with no bias, then introducing simulated publication bias, outcome reporting bias, and bias from poor study quality. We then implemented common and the proposed bias robust meta-analysis methods and compared the mean bias and mean squared error (MSE) for four estimates of effect and the coverage probability of seven confidence intervals. We found that no methods perform well in the presence of any substantial bias. A regression based extension to Egger's test gave an estimate of effect with lower mean bias than standard methods in the presence of publication bias or poor study quality, but had a substantially worse MSE except in very specific conditions. Coverage of all 95% confidence intervals was very poor with increasing numbers of studies in biased conditions, often falling below 50%. The Knapp-Hartung interval performed closest to nominal coverage with fewer than 10 studies in most conditions, and the Henmi-Copas interval generally performed best with more than 10 studies. There was no evidence that a multiplicative term for heterogeneity improved coverage. Multiple forms of bias remain problematic for all meta-analysis methods, with very poor performance under conceivable conditions.

A "P" value, or the magnitude or direction of results can influence decisions about whether, when, and how research findings are disseminated. Regardless of whether an entire study or a particular study result is unavailable because investigators considered the results to be unfavorable, bias in a meta-analysis may occur when available results differ systematically from missing results. In this article, we summarize the empirical evidence for various reporting biases that lead to study results being unavailable for inclusion in systematic reviews, with a focus on health research. These biases include publication bias and selective nonreporting bias. We describe processes that systematic reviewers can use to minimize the risk of bias due to missing results in meta-analyses of health research, such as comprehensive searches and prospective approaches to meta-analysis. We also outline methods that have been designed for assessing risk of bias due to missing results in meta-analyses of health research, including using tools to assess selective nonreporting of results, ascertaining qualitative signals that suggest not all studies were identified, and generating funnel plots to identify small-study effects, one cause of which is reporting bias.

Immortal time may arise in survival analyses when individuals are assigned to treatment strategies based on post-eligibility information or selected based on post-assignment eligibility criteria. Selection based on eligibility criteria applied after treatment assignment results in immortal time when the analysis starts the follow-up at assignment. Misclassification of assignment to treatment strategies based on treatment received after eligibility results in immortal time when the treatment strategies are not distinguishable at the start of follow-up. Target trial emulation prevents the introduction of immortal time by explicitly specifying eligibility and assignment to the treatment strategies, and by synchronizing them at the start of follow-up. We summarize analytic approaches that prevent immortal time when longitudinal data are available to emulate the target trial from the time of treatment assignment. The term "immortal time bias" suggests that the source of the bias is the immortal time, but it is selection or misclassification that generates the immortal time, leading to bias. [ABSTRACT FROM AUTHOR]

Hospitalized patients with COVID-19 are at high risk for thrombosis and systemic inflammation. Anticoagulation can improve outcomes, but trials have varied in the doses used. This systematic review and prospective meta-analysis includes and analyzes randomized trials registered by September 2024. The analysis compares therapeutic-, prophylactic-, and intermediate-dose anticoagulation with heparins on 28-day mortality, major bleeding, and other clinical outcomes. Background: Reported results of clinical trials assessing higher-dose anticoagulation in patients hospitalized for COVID-19 have been inconsistent. Purpose: To estimate the association of higher- versus lower-dose anticoagulation with clinical outcomes. Data Sources: Randomized trials were identified from the World Health Organization's International Clinical Trials Registry Platform and ClinicalTrials.gov with no restriction by trial status or language. Study Selection: Eligible randomized trials assigned patients hospitalized for COVID-19 to higher- versus lower-dose anticoagulation. Data Extraction: 20 eligible trials provided data in a prospectively agreed format. Two further studies were included based on published data. The primary outcome was all-cause mortality 28 days after randomization. Secondary outcomes were progression to invasive mechanical ventilation or death, thromboembolic events, and major bleeding. Data Synthesis: Therapeutic- compared with prophylactic-dose anticoagulation with heparins reduced 28-day mortality (OR, 0.77 [95% CI, 0.64 to 0.93]; I 2  = 29%; 11 trials, 6297 patients, of whom 5456 required low or no oxygen at randomization). The ORs for 28-day mortality were 1.21 (CI, 0.93 to 1.58; I 2  = 0%) for therapeutic-dose compared with intermediate-dose anticoagulation (6 trials, 1803 patients, 843 receiving noninvasive ventilation at randomization) and 0.95 (CI, 0.76 to 1.19; I 2  = 0%; 10 trials, 3897 patients, 2935 receiving no or low oxygen at randomization) for intermediate- versus prophylactic-dose anticoagulation. Treatment effects appeared broadly consistent across predefined patient subgroups, although some analyses were limited in power. Higher- compared with lower-dose anticoagulation was associated with fewer thromboembolic events, but a greater risk for major bleeding. Conclusion: Therapeutic-dose compared with prophylactic-dose anticoagulation reduced 28-day mortality. Mortality was similar for intermediate-dose compared with prophylactic-dose anticoagulation and higher for therapeutic-dose compared with intermediate-dose anticoagulation, although this comparison was not estimated precisely. Primary Funding Source: No direct funding. (PROSPERO: CRD42020213461) [ABSTRACT FROM AUTHOR]

Despite a major increase in the range and number of software offerings now available to help researchers produce evidence syntheses, there is currently no generic tool for producing figures to display and explore the risk-of-bias assessments that routinely take place as part of systematic review. However, tools such as the R programming environment and Shiny (an R package for building interactive web apps) have made it straightforward to produce new tools to help in producing evidence syntheses. We present a new tool, "robvis" (Risk-Of-Bias VISualization), available as an R package and web app, which facilitates rapid production of publication-quality risk-of-bias assessment figures. We present a timeline of the tool's development and its key functionality.

Meta-analyses based on systematic literature reviews are commonly used to obtain a quantitative summary of the available evidence on a given topic. However, the reliability of any meta-analysis is constrained by that of its constituent studies. One major limitation is the possibility of small-study effects, when estimates from smaller and larger studies differ systematically. Small-study effects may result from reporting biases (ie, publication bias), from inadequacies of the included studies that are related to study size, or from reasons unrelated to bias. We propose two estimators based on the median and mode to increase the reliability of findings in a meta-analysis by mitigating the influence of small-study effects. By re-examining data from published meta-analyses and by conducting a simulation study, we show that these estimators offer robustness to a range of plausible bias mechanisms, without making explicit modelling assumptions. They are also robust to outlying studies without explicitly removing such studies from the analysis. When meta-analyses are suspected to be at risk of bias because of small-study effects, we recommend reporting the mean, median and modal pooled estimates.

Randomized clinical trials underpin evidence-based clinical practice, but flaws in their conduct may lead to biased estimates of intervention effects and hence invalid treatment recommendations. The main approach to the empirical study of bias is to collate a number of meta-analyses and, within each, compare the results of trials with and without a methodological characteristic such as blinding of participants and health professionals. Estimated within-meta-analysis differences are combined across meta-analyses, leading to an estimate of mean bias. Such "meta-epidemiological" studies are published in increasing numbers and have the potential to inform trial design, assessment of risk of bias, and reporting guidelines. However, their interpretation is complicated by issues of confounding, imprecision, and applicability. We developed a guide for interpreting meta-epidemiological studies, illustrated using MetaBLIND, a large study on the impact of blinding. Applying generally accepted principles of research methodology to meta-epidemiology, we framed 10 questions covering the main issues to consider when interpreting results of such studies, including risk of systematic error, risk of random error, issues related to heterogeneity, and theoretical plausibility. We suggest that readers of a meta-epidemiological study reflect comprehensively on the research question posed in the study, whether an experimental intervention was unequivocally identified for all included trials, the risk of misclassification of the trial characteristic, and the risk of confounding, i.e the adequacy of any adjustment for the likely confounders. We hope that our guide to interpretation of results of meta-epidemiological studies is helpful for readers of such studies.

Studies combined in a meta-analysis often have differences in their design and conduct that can lead to heterogeneous results. A random-effects model accounts for these differences in the underlying study effects, which includes a heterogeneity variance parameter. The DerSimonian-Laird method is often used to estimate the heterogeneity variance, but simulation studies have found the method can be biased and other methods are available. This paper compares the properties of nine different heterogeneity variance estimators using simulated meta-analysis data. Simulated scenarios include studies of equal size and of moderate and large differences in size. Results confirm that the DerSimonian-Laird estimator is negatively biased in scenarios with small studies and in scenarios with a rare binary outcome. Results also show the Paule-Mandel method has considerable positive bias in meta-analyses with large differences in study size. We recommend the method of restricted maximum likelihood (REML) to estimate the heterogeneity variance over other methods. However, considering that meta-analyses of health studies typically contain few studies, the heterogeneity variance estimate should not be used as a reliable gauge for the extent of heterogeneity in a meta-analysis. The estimated summary effect of the meta-analysis and its confidence interval derived from the Hartung-Knapp-Sidik-Jonkman method are more robust to changes in the heterogeneity variance estimate and show minimal deviation from the nominal coverage of 95% under most of our simulated scenarios.

Meta-analyses are an important tool within systematic reviews to estimate the overall effect size and its confidence interval for an outcome of interest. If heterogeneity between the results of the relevant studies is anticipated, then a random-effects model is often preferred for analysis. In this model, a prediction interval for the true effect in a new study also provides additional useful information. However, the DerSimonian and Laird method--frequently used as the default method for meta-analyses with random effects--has been long challenged due to its unfavorable statistical properties. Several alternative methods have been proposed that may have better statistical properties in specific scenarios. In this paper, we aim to provide a comprehensive overview of available methods for calculating point estimates, confidence intervals, and prediction intervals for the overall effect size under the random-effects model. We indicate whether some methods are preferable than others by considering the results of comparative simulation and real-life data studies.

Network meta‐analysis (NMA) combines evidence from multiple trials to compare the effectiveness of a set of interventions. In many areas of research, interventions are often complex, made up of multiple components or features. This makes it difficult to define a common set of interventions on which to perform the analysis. One approach to this problem is component network meta‐analysis (CNMA) which uses a meta‐regression framework to define each intervention as a subset of components whose individual effects combine additively. In this article, we are motivated by a systematic review of complex interventions to prevent obesity in children. Due to considerable heterogeneity across the trials, these interventions cannot be expressed as a subset of components but instead are coded against a framework of characteristic features. To analyse these data, we develop a bespoke CNMA‐inspired model that allows us to identify the most important features of interventions. We define a meta‐regression model with covariates on three levels: intervention, study, and follow‐up time, as well as flexible interaction terms. By specifying different regression structures for trials with and without a control arm, we relax the assumption from previous CNMA models that a control arm is the absence of intervention components. Furthermore, we derive a correlation structure that accounts for trials with multiple intervention arms and multiple follow‐up times. Although, our model was developed for the specifics of the obesity data set, it has wider applicability to any set of complex interventions that can be coded according to a set of shared features. [ABSTRACT FROM AUTHOR]

Background and aims: Chronic non‐cancer pain (CNCP) is one of the most common causes of disability globally. Opioid prescribing to treat CNCP remains widespread, despite limited evidence of long‐term clinical benefit and evidence of harm such as problematic pharmaceutical opioid use (POU) and overdose. The study aimed to measure the prevalence of POU in CNCP patients treated with opioid analgesics. Method: A comprehensive systematic literature review and meta‐analysis was undertaken using MEDLINE, Embase and PsycINFO databases from inception to 27 January 2021. We included studies from all settings with participants aged ≥ 12 with non‐cancer pain of ≥ 3 months duration, treated with opioid analgesics. We excluded case–control studies, as they cannot be used to generate prevalence estimates. POU was defined using four categories: dependence and opioid use disorder (D&OUD), signs and symptoms of D&OUD (S&S), aberrant behaviour (AB) and at risk of D&OUD. We used a random‐effects multi‐level meta‐analytical model. We evaluated inconsistency using the I2 statistic and explored heterogeneity using subgroup analyses and meta‐regressions. Results: A total of 148 studies were included with > 4.3 million participants; 1% of studies were classified as high risk of bias. The pooled prevalence was 9.3% [95% confidence interval (CI) = 5.7–14.8%; I2 = 99.9%] for D&OUD, 29.6% (95% CI = 22.1–38.3%, I2 = 99.3%) for S&S and 22% (95% CI = 17.4–27.3%, I2 = 99.8%) for AB. The prevalence of those at risk of D&OUD was 12.4% (95% CI = 4.3–30.7%, I2 = 99.6%). Prevalence was affected by study setting, study design and diagnostic tool. Due to the high heterogeneity, the findings should be interpreted with caution. Conclusions: Problematic pharmaceutical opioid use appears to be common in chronic pain patients treated with opioid analgesics, with nearly one in 10 experiencing dependence and opioid use disorder, one in three showing signs and symptoms of dependence and opioid use disorder and one in five showing aberrant behaviour. [ABSTRACT FROM AUTHOR]

Quantitative evidence synthesis methods aim to combine data from multiple medical trials to infer relative effects of different interventions. A challenge arises when trials report continuous outcomes on different measurement scales. To include all evidence in one coherent analysis, we require methods to "map" the outcomes onto a single scale. This is particularly challenging when trials report aggregate rather than individual data. We are motivated by a meta‐analysis of interventions to prevent obesity in children. Trials report aggregate measurements of body mass index (BMI) either expressed as raw values or standardized for age and sex. We develop three methods for mapping between aggregate BMI data using known or estimated relationships between measurements on different scales at the individual level. The first is an analytical method based on the mathematical definitions of z‐scores and percentiles. The other two approaches involve sampling individual participant data on which to perform the conversions. One method is a straightforward sampling routine, while the other involves optimization with respect to the reported outcomes. In contrast to the analytical approach, these methods also have wider applicability for mapping between any pair of measurement scales with known or estimable individual‐level relationships. We verify and contrast our methods using simulation studies and trials from our data set which report outcomes on multiple scales. We find that all methods recreate mean values with reasonable accuracy, but for standard deviations, optimization outperforms the other methods. However, the optimization method is more likely to underestimate standard deviations and is vulnerable to non‐convergence. [ABSTRACT FROM AUTHOR]

Non-randomised studies of the effects of interventions are critical to many areas of healthcare evaluation, but their results may be biased. It is therefore important to understand and appraise their strengths and weaknesses. We developed ROBINS-I (“Risk Of Bias In Non-randomised Studies - of Interventions”), a new tool for evaluating risk of bias in estimates of the comparative effectiveness (harm or benefit) of interventions from studies that did not use randomisation to allocate units (individuals or clusters of individuals) to comparison groups. The tool will be particularly useful to those undertaking systematic reviews that include non-randomised studies.

Systematic reviews often encounter primary studies that report multiple effect sizes based on data from the same participants. These have the potential to introduce statistical dependency into the meta-analytic data set. In this paper, we provide a tutorial on dealing with effect size multiplicity within studies in the context of meta-analyses of intervention and association studies, recommending a three-step approach. The first step is to define the research question and consider the extent to which it mainly reflects interest in mean effect sizes (which we term a "convergent" approach) or an interest in exploring heterogeneity (which we term a "divergent" approach). A second step is to identify the types of multiplicities that appear in the initial database of effect sizes relevant to the research question, and we propose a categorization scheme to differentiate them. The third step is to select a strategy for dealing with each type of multiplicity. The researcher can choose between a "reductionist" meta-analytic approach, which is characterized by inclusion of a single effect size per study, and an "integrative" approach, characterized by inclusion of multiple effect sizes per study. We present an overview of available analysis strategies for dealing with effect size multiplicity within studies and provide recommendations intended to help researchers decide which strategy might be preferable in particular situations. Last, we offer caveats and cautions about addressing the challenges multiplicity poses for systematic reviews and meta-analyses.

ObjectiveTo assess the impact of inadequate water and sanitation on diarrhoeal disease in low- and middle-income settings.MethodsThe search strategy used Cochrane Library, MEDLINE & PubMed, Global Health, Embase and BIOSIS supplemented by screening of reference lists from previously published systematic reviews, to identify studies reporting on interventions examining the effect of drinking water and sanitation improvements in low- and middle-income settings published between 1970 and May 2013. Studies including randomised controlled trials, quasi-randomised trials with control group, observational studies using matching techniques and observational studies with a control group where the intervention was well defined were eligible. Risk of bias was assessed using a modified Ottawa-Newcastle scale. Study results were combined using meta-analysis and meta-regression to derive overall and intervention-specific risk estimates.ResultsOf 6819 records identified for drinking water, 61 studies met the inclusion criteria, and of 12,515 records identified for sanitation, 11 studies were included. Overall, improvements in drinking water and sanitation were associated with decreased risks of diarrhoea. Specific improvements, such as the use of water filters, provision of high-quality piped water and sewer connections, were associated with greater reductions in diarrhoea compared with other interventions.ConclusionsThe results show that inadequate water and sanitation are associated with considerable risks of diarrhoeal disease and that there are notable differences in illness reduction according to the type of improved water and sanitation implemented.

ObjectiveTo assess the impact of inadequate water and sanitation on diarrhoeal disease in low- and middle-income settings.MethodsThe search strategy used Cochrane Library, MEDLINE & PubMed, Global Health, Embase and BIOSIS supplemented by screening of reference lists from previously published systematic reviews, to identify studies reporting on interventions examining the effect of drinking water and sanitation improvements in low- and middle-income settings published between 1970 and May 2013. Studies including randomised controlled trials, quasi-randomised trials with control group, observational studies using matching techniques and observational studies with a control group where the intervention was well defined were eligible. Risk of bias was assessed using a modified Ottawa-Newcastle scale. Study results were combined using meta-analysis and meta-regression to derive overall and intervention-specific risk estimates.ResultsOf 6819 records identified for drinking water, 61 studies met the inclusion criteria, and of 12,515 records identified for sanitation, 11 studies were included. Overall, improvements in drinking water and sanitation were associated with decreased risks of diarrhoea. Specific improvements, such as the use of water filters, provision of high-quality piped water and sewer connections, were associated with greater reductions in diarrhoea compared with other interventions.ConclusionsThe results show that inadequate water and sanitation are associated with considerable risks of diarrhoeal disease and that there are notable differences in illness reduction according to the type of improved water and sanitation implemented.