Search

A considerable proportion of quantitative research remains unpublished once completed. Little research has documented non-dissemination and dissemination bias in qualitative research. This study aimed to generate evidence on the extent of non-dissemination in qualitative research. We followed a cohort of qualitative studies presented as conference abstracts to ascertain their subsequent publication status. We searched for subsequent full publication in MEDLINE, in the Cumulative Index to Nursing & Allied Health Literature and in Google Scholar. We matched abstracts to subsequent publications according to authors, method of data collection and phenomenon of interest. Fisher's exact test was calculated to examine associations between study characteristics and publication. Factors potentially associated with time to publication were evaluated with Cox regression analysis. For 91 of 270 included abstracts (33.70%; 95% CI 28.09%-39.68%), no full publication was identified. Factors that were found to be associated with subsequent full publication were oral presentation (OR 4.62; 95% CI 2.43-8.94) and university affiliation (OR 1.96; 95% CI 1.05-3.66). Compared to oral presentations, studies presented as posters took longer time to reach full publication (hazard ratio 0.35, 95% CI 0.21-0.58). This study shows that it was not possible to retrieve a full publication for over one-third of abstracts. Our findings suggest that where this non-dissemination is systematic, it may lead to distortions of the qualitative evidence-base for decision-making through dissemination bias. Our findings are congruent with those of other studies. Further research might investigate non-dissemination of qualitative studies in other disciplines to consolidate our findings.

Using the Hartung-Knapp method and 95% prediction intervals (PIs) in random-effects meta-analyses is recommended by experts but rarely applied. Therefore, we aimed to reevaluate statistically significant meta-analyses using the Hartung-Knapp method and 95% PIs. In this methodological study, three databases were searched from January 2010 to July 2019. We included systematic reviews reporting a statistically significant meta-analysis of at least four randomized controlled trials in advanced cancer patients using either a fixed-effect or random-effects model. We investigated the impact of switching from fixed-effect to random-effects meta-analysis and of using the recommended Hartung-Knapp method in random-effects meta-analyses. Furthermore, we calculated 95% PIs for all included meta-analyses. We identified 6234 hits, of which 261 statistically significant meta-analyses were included. Our recalculations of these 261 meta-analyses produced statistically significant results in 132 of 138 fixed-effect and 114 of 123 random-effects meta-analyses. When switching to a random-effects model, 19 of 132 fixed-effect meta-analyses (14.4%) were no longer statistically significant. Using the Hartung-Knapp method in random-effects meta-analyses resulted in 34 of 114 nonsignificant meta-analyses (29.8%). In the full sample (N = 261), the "null effect" was included by the 95% PI in 195 (74.7%) and the "opposite effect" (e.g., hazard ratio 0.5, opposite effect 2) in 98 meta-analyses (37.5%). Using the Hartung-Knapp method and PIs substantially influenced the interpretation of many published, statistically significant meta-analyses. We strongly encourage researchers to check if using the Hartung-Knapp method and reporting 95% PIs is appropriate in random-effects meta-analyses.

The Peto odds ratio is a well-known effect measure in meta-analysis of binary outcomes. For pairwise comparisons, the Peto odds ratio estimator can be severely biased in the situation of unbalanced sample sizes in the two treatment groups or large treatment effects. In this publication, we evaluate Peto odds ratio estimators in the setting of multi-arm studies and in network meta-analysis using illustrative examples. We observe that Peto odds ratio estimators in a multi-arm study are inconsistent if the observed event probabilities are different or the sample sizes of treatment groups are unbalanced. The same problem emerges in network meta-analysis including only two-arm studies and translates to indirect comparisons of pairwise meta-analyses. We conclude that the Peto odds ratio should not be used as effect measure in network meta-analysis or indirect comparisons of pairwise meta-analyses.

In the era of the "reproducibility crisis" and the "P-value controversy" new ways of presentation and interpretation of the results of a meta-analysis are desirable. One suggestion that has been made for single studies almost six decades ago and taken up now and then is the P-value function. For a given outcome, this function assigns a P-value to each possible hypothetical value, given the data. Moreover, the P-value function simultaneously provides two-sided confidence intervals for all possible alpha levels. An application to meta-analysis, while suggested early, has not been widely established. We introduce the drapery plot that presents the P-value function for all individual studies and pooled estimates in a meta-analysis as curves and the prediction range for a single future study. We also present a scaled variant with the test statistic on the y-axis. Both plots visualize the full information of a pairwise meta-analysis. We see a drapery plot as a complementary figure to a forest plot. It may be even an alternative in meta-analyses with many studies where forest plots tend to become very large and complex.

Standard generic inverse variance methods for the combination of single proportions are based on transformed proportions using the logit, arcsine, and Freeman-Tukey double arcsine transformations. Generalized linear mixed models are another more elaborate approach. Irrespective of the approach, meta-analysis results are typically back-transformed to the original scale in order to ease interpretation. Whereas the back-transformation of meta-analysis results is straightforward for most transformations, this is not the case for the Freeman-Tukey double arcsine transformation, albeit possible. In this case study with five studies, we demonstrate how seriously misleading the back-transformation of the Freeman-Tukey double arcsine transformation can be. We conclude that this transformation should only be used with special caution for the meta-analysis of single proportions due to potential problems with the back-transformation. Generalized linear mixed models seem to be a promising alternative.

The LFK index has been promoted as an improved method to detect bias in meta‐analysis. Putatively, its performance does not depend on the number of studies in the meta‐analysis. We conducted a simulation study, comparing the LFK index test to three standard tests for funnel plot asymmetry in settings with smaller or larger group sample sizes. In general, false positive rates of the LFK index test markedly depended on the number and size of studies as well as the between‐study heterogeneity with values between 0% and almost 30%. Egger's test adhered well to the pre‐specified significance level of 5% under homogeneity, but was too liberal (smaller groups) or conservative (larger groups) under heterogeneity. The rank test was too conservative for most simulation scenarios. The Thompson–Sharp test was too conservative under homogeneity, but adhered well to the significance level in case of heterogeneity. The true positive rate of the LFK index test was only larger compared with classic tests if the false positive rate was inflated. The power of classic tests was similar or larger than the LFK index test if the false positive rate of the LFK index test was used as significance level for the classic tests. Under ideal conditions, the false positive rate of the LFK index test markedly and unpredictably depends on the number and sample size of studies as well as the extent of between‐study heterogeneity. The LFK index test in its current implementation should not be used to assess funnel plot asymmetry in meta‐analysis. [ABSTRACT FROM AUTHOR]

In systematic reviews, meta-analyses are routinely applied to summarize the results of the relevant studies for a specific research question. If one can assume that in all studies the same true effect is estimated, the application of a meta-analysis with common effect (commonly referred to as fixed-effect meta-analysis) is adequate. If between-study heterogeneity is expected to be present, the method of choice is a meta-analysis with random effects. The widely used DerSimonian and Laird method for meta-analyses with random effects has been criticized due to its unfavorable statistical properties, especially in the case of very few studies. A working group of the Cochrane Collaboration recommended the use of the Knapp-Hartung method for meta-analyses with random effects. However, as heterogeneity cannot be reliably estimated if only very few studies are available, the Knapp-Hartung method, while correctly accounting for the corresponding uncertainty, has very low power. Our aim is to summarize possible methods to perform meaningful evidence syntheses in the situation with only very few (ie, 2-4) studies. Some general recommendations are provided on which method should be used when. Our recommendations are based on the existing literature on methods for meta-analysis with very few studies and consensus of the authors. The recommendations are illustrated by 2 examples coming from dossier assessments of the Institute for Quality and Efficiency in Health Care.

Network meta-analysis has evolved into a core method for evidence synthesis in health care. In network meta-analysis, 3 or more treatments for a given medical condition are compared, based on a number of clinical studies, usually randomized controlled trials. Often, many different endpoints are investigated, related to different aspects of the patient's outcome, such as efficacy, safety, acceptability, or costs of a treatment. Different outcomes may lead to different rankings of the treatments. We use the existing theory of partially ordered sets and show how the relations between the treatments in a network meta-analysis can be illustrated by Hasse diagrams, that is, directed graphs showing the partial order relations, and by structured scatter plots and biplots.

Systematic reviewers conducting pairwise meta-analyses sometimes encounter multi-arm studies. To include these studies, and to avoid a unit-of-analysis error, often two or more arms are combined or the control arm is split. In this tutorial, we present 5 different approaches that can be used. Particularly, we present a novel approach (method 4) that to the best of our knowledge has not been presented before. We demonstrate their application on 3 selected data sets, discuss their scope of application and their advantages and limitations, and give recommendations.

Meta-analysis of a survival endpoint is typically based on the pooling of hazard ratios (HRs). If competing risks occur, the HRs may lose translation into changes of survival probability. The cumulative incidence functions (CIFs), the expected proportion of cause-specific events over time, re-connect the cause-specific hazards (CSHs) to the probability of each event type. We use CIF ratios to measure treatment effect on each event type. To retrieve information on aggregated, typically poorly reported, competing risks data, we assume constant CSHs. Next, we develop methods to pool CIF ratios across studies. The procedure computes pooled HRs alongside and checks the influence of follow-up time on the analysis. We apply the method to a medical example, showing that follow-up duration is relevant both for pooled cause-specific HRs and CIF ratios. Moreover, if all-cause hazard and follow-up time are large enough, CIF ratios may reveal additional information about the effect of treatment on the cumulative probability of each event type. Finally, to improve the usefulness of such analysis, better reporting of competing risks data is needed. Copyright © 2015 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

In systematic reviews based on network meta-analysis, the network structure should be visualized. Network plots often have been drawn by hand using generic graphical software. A typical way of drawing networks, also implemented in statistical software for network meta-analysis, is a circular representation, often with many crossing lines. We use methods from graph theory in order to generate network plots in an automated way. We give a number of requirements for graph drawing and present an algorithm that fits prespecified ideal distances between the nodes representing the treatments. The method was implemented in the function netgraph of the R package netmeta and applied to a number of networks from the literature. We show that graph representations with a small number of crossing lines are often preferable to circular representations. Copyright © 2015 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]