Effect of methodological quality measures in spinal surgery research: a metaepidemiological study

Background context Methodological quality measures of trials in meta-analyses have been shown to influence the pooled effect sizes in several medical fields. However, for spinal surgery, influence of quality measures has not been assessed. Purpose The purpose of this study was to analyze the influence of quality measures in studies on effectiveness in spinal surgery. Study design A metaepidemiological study was performed on meta-analyses within spinal surgery. Methods A systematic search was performed in MEDLINE, Cochrane Database, and EMBASE in August 2009. The effect sizes, defined as risk of positive clinical outcome, of trials included in the meta-analyses were assessed. The differences in effect sizes were calculated as risk differences (RDs). Relation of the RDs to potential quality measures such as sponsoring, randomization, allocation concealment, blinding, and study size was assessed with metaregression adjusted for multiple testing. Results Seven reviews consisting of 118 studies were included. Data provided by the systematic reviews alone were insufficient to analyze the effect of quality measures. Metaregression analysis of 76 of the individual trials reporting clinical outcome, though, showed that sample size, strict randomization, and outcome blinding were significant quality measures influencing study effect. Risk difference of effect from validly randomized studies was higher compared with not validly randomized and comparative observational trials (5.4%; 95% confidence interval [CI], 1.2–9.6; p=.044). Studies with adequate observer blinding showed a 7.2% lower RD (95% CI, 0.8–13.7; p=.049). For each increase of 100 patients, the RD decreased 3.6% (95% CI, 0.5–6.8; p=.098). Conclusions Contrary to basic methodological assertions, formal and strict randomization appeared to produce a significantly higher RD in spinal surgery research. Sufficient sample size and observer blinding, on the other hand, led to a lower RD as expected. These findings imply that effect of quality measures assessed in metaepidemiological studies should not be too easily translated to research in spinal surgery.