Search

A systematic review identifies, appraises and synthesises all the empirical evidence from studies that meet prespecified eligibility criteria to answer a specific research question. As part of the appraisal, researchers use explicit methods to assess risk of bias in the results’ from included studies that contribute to the review’s findings, to improve our confidence in the review’s conclusions. Randomised controlled trials included in Cochrane Reviews have used a specific risk of bias tool to assess these included studies since 2008. In 2019, a new version of this tool, Risk of Bias 2 (RoB 2), was launched to improve its usability and to reflect current understanding of how the causes of bias can influence study results. Cochrane implemented RoB 2 in a phased approach, with users of the tool informing guidance development. This paper highlights learning for all systematic reviewers (Cochrane and non-Cochrane) from the phased implementation, highlighting differences between the original version of the tool and RoB 2, consideration of reporting systematic review protocols or full review reports that have used RoB 2, and some tips shared by authors during the pilot phase of the implementation.

Results from clinical trials can be deemed trustworthy only if they are properly conducted and their methods are fully reported. The SPIRIT and CONSORT checklists, which have improved clinical trial design, conduct and reporting, are being updated to reflect recent advances and improve the assessment of healthcare interventions.

Excessive anterior pelvic tilt is suspected of causing femoroacetabular impingement, low back pain, and sacroiliac joint pain. Non-surgical treatment may decrease symptoms and is seen as an alternative to invasive and complicated surgery. However, the effect of non-surgical modalities in adults is unclear. The aim of this review was to investigate patient- and observer-reported outcomes of non-surgical intervention in reducing clinical symptoms and/or potential anterior pelvic tilt in symptomatic and non-symptomatic adults with excessive anterior pelvic tilt, and to evaluate the certainty of evidence.MEDLINE, EMBASE, Web of Science and Cochrane (CENTRAL) databases were searched up to March 2019 for eligible studies. Two reviewers assessed risk of bias independently, using the Cochrane Risk of Bias tool for randomized trials and the ROBINS-I tool for non-randomized studies. Data were synthesized qualitatively. The GRADE approach was used to assess the overall certainty of evidence.Of 2013 citations, two randomized controlled trials (RCTs) (n= 72) and two non-RCTs (n= 23) were included. One RCT reported a small reduction (< 2°) in anterior pelvic tilt in non-symptomatic men. The two non-RCTs reported a statistically significant reduction in anterior pelvic tilt, pain, and disability in symptomatic populations. The present review was based on heterogeneous study populations, interventions, and very low quality of evidence.No overall evidence for the effect of non-surgical treatment in reducing excessive anterior pelvic tilt and potentially related symptoms was found. High-quality studies targeting non-surgical treatment as an evidence-based alternative to surgical interventions for conditions related to excessive anterior pelvic tilt are warranted.Cite this article: EFORT Open Rev2020;5:37-45. DOI: 10.1302/2058-5241.5.190017

The ESPGHAN 2012 coeliac disease (CD) diagnostic guidelines aimed to guide physicians in accurately diagnosing CD and permit omission of duodenal biopsies in selected cases. Here, an updated and expanded evidence-based guideline is presented. Literature databases and other sources of information were searched for studies that could inform on 10 formulated questions on symptoms, serology, human leukocyte antigen genetics, and histopathology. Eligible articles were assessed using QUADAS2. GRADE provided a basis for statements and recommendations. Various symptoms are suggested for case finding, with limited contribution to diagnostic accuracy. If CD is suspected, measurement of total serum IgA and IgA-antibodies against transglutaminase 2 (TGA-IgA) is superior to other combinations. We recommend against deamidated gliadin peptide antibodies (DGP-IgG/IgA) for initial testing. Only if total IgA is low/undetectable, an IgG-based test is indicated. Patients with positive results should be referred to a paediatric gastroenterologist/specialist. If TGA-IgA is =10 times the upper limit of normal (10× ULN) and the family agrees, the no-biopsy diagnosis may be applied, provided endomysial antibodies (EMA-IgA) will test positive in a second blood sample. Human leukocyte antigen DQ2-/DQ8 determination and symptoms are not obligatory criteria. In children with positive TGA-IgA <10× ULN at least 4 biopsies from the distal duodenum and at least 1 from the bulb should be taken. Discordant results between TGA-IgA and histopathology may require re-evaluation of biopsies. Patients with no/mild histological changes (Marsh 0/I) but confirmed autoimmunity (TGA-IgA/EMA-IgA+) should be followed closely. CD diagnosis can be accurately established with or without duodenal biopsies if given recommendations are followed.

IMPORTANCE When analyzing results of randomized clinical trials, the treatment with the greatest specific effect compared with its placebo control is considered to be the most effective one. Although systematic variations of improvements in placebo control groups would have important implications for the interpretation of placebo-controlled trials, the knowledge base on the subject is weak. OBJECTIVE To investigate whether different types of placebo treatments are associated with different responses using the studies of migraine prophylaxis for this analysis. DESIGN, SETTING, AND PARTICIPANTS We searched relevant sources through February 2012 and contacted the authors to identify randomized clinical trials on the prophylaxis of migraine with an observation period of at least 8 weeks after randomization that compared an experimental treatment with a placebo control group. We calculated pooled random-effects estimates according to the type of placebo for the proportions of treatment response. We performed meta-regression analyses to identify sources of heterogeneity. In a network meta-analysis, direct and indirect comparisons within and across trials were combined. Additional analyses were performed for continuous outcomes. EXPOSURE Active migraine treatment and the placebo control conditions. MAIN OUTCOMES AND MEASURES Proportion of treatment responders, defined as having an attack frequency reduction of at least 50%. Other available outcomes in order of preference included a reduction of 50% or greater in migraine days, the number of headache days, or headache score or a significant improvement as assessed by the patients or their physicians. RESULTS Of the 102 eligible trials, 23 could not be included in the meta-analyses owing to insufficient data. Sham acupuncture (proportion of responders, 0.38 [95% CI, 0.30-0.47]) and sham surgery (0.58 [0.37-0.77]) were associated with a more pronounced reduction of migraine frequency than oral pharmacological placebos (0.22 [0.17-0.28]) and were the only significant predictors of response in placebo groups in multivariable analyses (P = .005 and P = .001, respectively). Network meta-analysis confirmed that more patients reported response in sham acupuncture groups than in oral pharmacological placebo groups (odds ratio, 1.88 [95% CI, 1.30-2.72]). Corresponding analyses for continuous outcomes showed similar findings. CONCLUSIONS AND RELEVANCE Sham acupuncture and sham surgery are associated with higher responder ratios than oral pharmacological placebos. Clinicians who treat patients with migraine should be aware that a relevant part of the overall effect they observe in practice might be due to nonspecific effects and that the size of such effects might differ between treatment modalities.

Background:Clinical trials are commonly done without blinded outcome assessors despite the risk of bias. We wanted to evaluate the effect of nonblinded outcome assessment on estimated effects in randomized clinical trials with outcomes that involved subjective measurement scales.Methods:We conducted a systematic review of randomized clinical trials with both blinded and nonblinded assessment of the same measurement scale outcome. We searched PubMed, EMBASE, PsycINFO, CINAHL, Cochrane Central Register of Controlled Trials, HighWire Press and Google Scholar for relevant studies. Two investigators agreed on the inclusion of trials and the outcome scale. For each trial, we calculated the difference in effect size (i.e., standardized mean difference between nonblinded and blinded assessments). A difference in effect size of less than 0 suggested that nonblinded assessors generated more optimistic estimates of effect. We pooled the differences in effect size using inverse variance random-effects meta-analysis and used metaregression to identify potential reasons for variation.Results:We included 24 trials in our review. The main meta-analysis included 16 trials (involving 2854 patients) with subjective outcomes. The estimated treatment effect was more beneficial when based on nonblinded assessors (pooled difference in effect size −0.23 [95% confidence interval (CI) −0.40 to −0.06]). In relative terms, nonblinded assessors exaggerated the pooled effect size by 68% (95% CI 14% to 230%). Heterogeneity was moderate (I2= 46%, p= 0.02) and unexplained by metaregression.Interpretation:We provide empirical evidence for observer bias in randomized clinical trials with subjective measurement scale outcomes. A failure to blind assessors of outcomes in such trials results in a high risk of substantial bias.

Abstract: Objective:To analyse the role of the control group in the methodology of clinical placebo effect evaluating trials.Setting:Department of Medical Philosophy and Clinical Theory, University of Copenhagen.Methods:A theoretical methodological analysis.Results:At least with present trial designs, it is impossible, with certainty, to exclude a potential placebo effect from the so-called control group. The placebo effect, in other words, is uncontrollable. Conclusion:I suggest that the notions of control group and placebo treatment group are replaced by the notions of “placebo effect maximising group” and “placebo effect minimising group”, thus stressing the fact that what such trials measure is a relative effect, and an underestimation of the absolute placebo effect. The illusion of absolute placebo effect evaluation must be acknowledged when articles referring to empirical trials on the placebo effect are interpreted.

Abstract. Twenty-four orthopaedic surgeons classified 42 pairs of radiographs according to the Neer system for proximal humeral fractures. Mean kappa value for inter-observer agreement was 0.27 (95% CI 0.26-0.28) with no clinically significant difference between orthopaedic residents (n=9), fellows (n=6) and specialists (n=9). Mean kappa for agreement of displacement versus non-displacement was 0.41 (95% CI 0.39-0.43) overall, and 0.50 (95% CI 0.45-0.56) within the specialist group. The agreement found in our study is unsatisfactory from a clinical perspective. Résumé. 24 chirurgiens orthopédistes ont classé 42 paires de radiographies selon la classification de Neer pour les fractures humérales proximales. La valeur moyenne du kappa pour l'accord inter-observateur était 0.27 (95% CI 0.26-0.28) sans différence significative entre résidents (n=9), chirurgiens juniors (n=6) et chirurgiens spécialistes (n=9). La valeur moyenne du kappa pour l'accord sur le critère déplacement ou non-déplacement était globalement 0.41 (95% CI 0.39-0.43), et, dans le groupe spécialiste 0.50 (95% CI 0.45-0.56). Dans une perspective clinique l'accord trouvé dans notre étude est peu satisfaisant.

ObjectivesTo study the impact of blinding on estimated treatment effects, and their variation between trials; differentiating between blinding of patients, healthcare providers, and observers; detection bias and performance bias; and types of outcome (the MetaBLIND study).DesignMeta-epidemiological study.Data sourceCochrane Database of Systematic Reviews (2013-14).Eligibility criteria for selecting studiesMeta-analyses with both blinded and non-blinded trials on any topic.Review methodsBlinding status was retrieved from trial publications and authors, and results retrieved automatically from the Cochrane Database of Systematic Reviews. Bayesian hierarchical models estimated the average ratio of odds ratios (ROR), and estimated the increases in heterogeneity between trials, for non-blinded trials (or of unclear status) versus blinded trials. Secondary analyses adjusted for adequacy of concealment of allocation, attrition, and trial size, and explored the association between outcome subjectivity (high, moderate, low) and average bias. An ROR lower than 1 indicated exaggerated effect estimates in trials without blinding.ResultsThe study included 142 meta-analyses (1153 trials). The ROR for lack of blinding of patients was 0.91 (95% credible interval 0.61 to 1.34) in 18 meta-analyses with patient reported outcomes, and 0.98 (0.69 to 1.39) in 14 meta-analyses with outcomes reported by blinded observers. The ROR for lack of blinding of healthcare providers was 1.01 (0.84 to 1.19) in 29 meta-analyses with healthcare provider decision outcomes (eg, readmissions), and 0.97 (0.64 to 1.45) in 13 meta-analyses with outcomes reported by blinded patients or observers. The ROR for lack of blinding of observers was 1.01 (0.86 to 1.18) in 46 meta-analyses with subjective observer reported outcomes, with no clear impact of degree of subjectivity. Information was insufficient to determine whether lack of blinding was associated with increased heterogeneity between trials. The ROR for trials not reported as double blind versus those that were double blind was 1.02 (0.90 to 1.13) in 74 meta-analyses.ConclusionNo evidence was found for an average difference in estimated treatment effect between trials with and without blinded patients, healthcare providers, or outcome assessors. These results could reflect that blinding is less important than often believed or meta-epidemiological study limitations, such as residual confounding or imprecision. At this stage, replication of this study is suggested and blinding should remain a methodological safeguard in trials.

ObjectiveTo characterise and analyse the experiences of trial researchers of if and how conflicts of interest had unduly influenced clinical trials they had worked on, what management strategies they had used to minimise any potential influence, and their experiences and views on conflicts of interest more generally.DesignQualitative interview study.ParticipantsTrial researchers who had participated in at least 10 clinical trials with methodological or statistical expertise. Researchers differed by geographical location, educational background, and experience with different types of funders. Interviewees were identified by searches on Web of Science and snowball sampling. 52 trial researchers were approached by email; 20 agreed to be interviewed.SettingInterviews conducted by telephone, recorded, transcribed verbatim, imported to NVivo 12, and analysed by systematic text condensation. Semistructured interviews focused on financial and non-financial conflicts of interest.ResultsThe interviewees had participated in a median of 37.5 trials and were mainly male physicians who had experience with commercial and non-commercial trial funders. Two predefined themes (influence of conflicts of interest and management strategies) and two additional themes (definition and reporting of conflicts of interest) emerged. Examples of perceived influence of conflicts of interest were: choice of inferior comparator, manipulation of the randomisation process, prematurely stopping the trials, fabrication of data, blocking access to data, and spin (eg, overly favourable interpretation of the results). Examples of strategies to manage conflicts of interest were: disclosure procedures, exclusion of the funder from design and analysis, independent committees, contracts ensuring complete access to the data, and no restriction by the funder on analysis and reporting. Interviewees used different definitions or thresholds for what they considered to be conflicts of interest, and they described different criteria for when to report them. Some interviewees considered non-commercial financial conflicts of interest (eg, funding of trials by governmental health agencies with a political agenda) to be equally or more important than commercial financial conflicts of interest (eg, funding by drug and device companies), but more challenging to report and manage.ConclusionThis study described how trial researchers perceive conflicts of interest unduly influencing clinical trials they had worked on, and the management strategies they used to prevent these influences. The results indicated considerable variability in researchers’ understanding of what conflicts of interest are and when they should be reported.

ObjectiveTo investigate the association between conflicts of interest and favourable recommendations in clinical guidelines, advisory committee reports, opinion pieces, and narrative reviews.DesignSystematic review.Eligibility criteriaStudies that compared the association between conflicts of interest and favourable recommendations of drugs or devices (eg, recommending a drug) in clinical guidelines, advisory committee reports, opinion pieces (eg, editorials), or narrative reviews.Data sourcesPubMed, Embase, Cochrane Methodology Register (from inception to February 2020), reference lists, Web of Science, and grey literature.Data extraction and analysisTwo authors independently extracted data and assessed the methodological quality of the studies. Pooled relative risks and 95% confidence intervals were estimated using random effects models (relative risk >1 indicates that documents with conflicts of interest more often had favourable recommendations than documents with no conflicts of interest). Financial and non-financial conflicts of interest were analysed separately, and the four types of documents were analysed separately (preplanned) and combined (post hoc).Results21 studies that analysed 106 clinical guidelines, 1809 advisory committee reports, 340 opinion pieces, and 497 narrative reviews were included. Unpublished data were received for 11 studies (eight full datasets and three summary datasets). 15 studies showed risk of confounding because the compared documents could differ in factors other than conflicts of interest (eg, different drugs used for different populations). The relative risk for associations between financial conflicts of interest and favourable recommendations for clinical guidelines was 1.26 (95% confidence interval 0.93 to 1.69; four studies of 86 clinical guidelines), for advisory committee reports was 1.20 (0.99 to 1.45; four studies of 629 advisory committee reports), for opinion pieces was 2.62 (0.91 to 7.55; four studies of 284 opinion pieces), and for narrative reviews was 1.20 (0.97 to 1.49; four studies of 457 narrative reviews). An analysis of all four types of documents combined supported these findings (1.26, 1.09 to 1.44). In one study that investigated specialty interests, the association between including radiologists as authors of guidelines and recommending routine breast cancer was: relative risk 2.10, 95% confidence interval 0.92 to 4.77; 12 clinical guidelines).ConclusionsWe interpret our findings to indicate that financial conflicts of interest are associated with favourable recommendations of drugs and devices in clinical guidelines, advisory committee reports, opinion pieces, and narrative reviews. Limitations of this review were risk of confounding in the included studies and the statistical imprecision of individual analyses of each document type. It is not certain whether non-financial conflicts of interest influence recommendations.Systematic review registrationCochrane Methodology Review Protocol MR000040.

ObjectiveTo assess the effects of coding and coding conventions on summaries and tabulations of adverse events data on suicidality within clinical study reports.DesignSystematic electronic search for adverse events of suicidality in tables, narratives, and listings of adverse events in individual patients within clinical study reports. Where possible, for each event we extracted the original term reported by the investigator, the term as coded by the medical coding dictionary, medical coding dictionary used, and the patient’s trial identification number. Using the patient’s trial identification number, we attempted to reconcile data on the same event between the different formats for presenting data on adverse events within the clinical study report.Setting9 randomised placebo controlled trials of duloxetine for major depressive disorder submitted to the European Medicines Agency for marketing approval.Data sourcesClinical study reports obtained from the EMA in 2011.ResultsSix trials used the medical coding dictionary COSTART (Coding Symbols for a Thesaurus of Adverse Reaction Terms) and three used MedDRA (Medical Dictionary for Regulatory Activities). Suicides were clearly identifiable in all formats of adverse event data in clinical study reports. Suicide attempts presented in tables included both definitive and provisional diagnoses. Suicidal ideation and preparatory behaviour were obscured in some tables owing to the lack of specificity of the medical coding dictionary, especially COSTART. Furthermore, we found one event of suicidal ideation described in narrative text that was absent from tables and adverse event listings of individual patients. The reason for this is unclear, but may be due to the coding conventions used.ConclusionData on adverse events in tables in clinical study reports may not accurately represent the underlying patient data because of the medical dictionaries and coding conventions used. In clinical study reports, the listings of adverse events for individual patients and narratives of adverse events can provide additional information, including original investigator reported adverse event terms, which can enable a more accurate estimate of harms.

ObjectiveTo determine, using research on duloxetine for major depressive disorder as an example, if there are inconsistencies between protocols, clinical study reports, and main publicly available sources (journal articles and trial registries), and within clinical study reports themselves, with respect to benefits and major harms.DesignData on primary efficacy analysis and major harms extracted from each data source and compared.SettingNine randomised placebo controlled trials of duloxetine (total 2878 patients) submitted to the European Medicines Agency (EMA) for marketing approval for major depressive disorder.Data sourcesClinical study reports, including protocols as appendices (total 13 729 pages), were obtained from the EMA in May 2011. Journal articles were identified through relevant literature databases and contacting the manufacturer, Eli Lilly. Clinicaltrials.gov and the manufacturer’s online clinical trial registry were searched for trial results.ResultsClinical study reports fully described the primary efficacy analysis and major harms (deaths (including suicides), suicide attempts, serious adverse events, and discontinuations because of adverse events). There were minor inconsistencies in the population in the primary efficacy analysis between the protocol and clinical study report and within the clinical study report for one trial. Furthermore, we found contradictory information within the reports for seven serious adverse events and eight adverse events that led to discontinuation but with no apparent bias. In each trial, a median of 406 (range 177-645) and 166 (100-241) treatment emergent adverse events (adverse events that emerged or worsened after study drug was started) in the randomised phase were not reported in journal articles and Lilly trial registry reports, respectively. We also found publication bias in relation to beneficial effects.ConclusionClinical study reports contained extensive data on major harms that were unavailable in journal articles and in trial registry reports. There were inconsistencies between protocols and clinical study reports and within clinical study reports. Clinical study reports should be used as the data source for systematic reviews of drugs, but they should first be checked against protocols and within themselves for accuracy and consistency.

High quality protocols facilitate proper conduct, reporting, and external review of clinical trials. However, the completeness of trial protocols is often inadequate. To help improve the content and quality of protocols, an international group of stakeholders developed the SPIRIT 2013 Statement (Standard Protocol Items: Recommendations for Interventional Trials). The SPIRIT Statement provides guidance in the form of a checklist of recommended items to include in a clinical trial protocol.This SPIRIT 2013 Explanation and Elaboration paper provides important information to promote full understanding of the checklist recommendations. For each checklist item, we provide a rationale and detailed description; a model example from an actual protocol; and relevant references supporting its importance. We strongly recommend that this explanatory paper be used in conjunction with the SPIRIT Statement. A website of resources is also available (www.spirit-statement.org).The SPIRIT 2013 Explanation and Elaboration paper, together with the Statement, should help with the drafting of trial protocols. Complete documentation of key trial elements can facilitate transparency and protocol review for the benefit of all stakeholders.

OBJECTIVE: To evaluate the impact of non-blinded outcome assessment on estimated treatment effects in randomised clinical trials with binary outcomes. Design Systematic review of trials with both blinded and non-blinded assessment of the same binary outcome. For each trial we calculated the ratio of the odds ratios—the odds ratio from non-blinded assessments relative to the corresponding odds ratio from blinded assessments. A ratio of odds ratios <1 indicated that non-blinded assessors generated more optimistic effect estimates than blinded assessors. We pooled the individual ratios of odds ratios with inverse variance random effects meta-analysis and explored reasons for variation in ratios of odds ratios with meta-regression. We also analysed rates of agreement between blinded and non-blinded assessors and calculated the number of patients needed to be reclassified to neutralise any bias. DATA SOURCES: PubMed, Embase, PsycINFO, CINAHL, Cochrane Central Register of Controlled Trials, HighWire Press, and Google Scholar. Eligibility criteria for selecting studies Randomised clinical trials with blinded and non-blinded assessment of the same binary outcome. RESULTS: We included 21 trials in the main analysis (with 4391 patients); eight trials provided individual patient data. Outcomes in most trials were subjective—for example, qualitative assessment of the patient’s function. The ratio of the odds ratios ranged from 0.02 to 14.4. The pooled ratio of odds ratios was 0.64 (95% confidence interval 0.43 to 0.96), indicating an average exaggeration of the non-blinded odds ratio by 36%. We found no significant association between low ratios of odds ratios and scores for outcome subjectivity (P=0.27); non-blinded assessor’s overall involvement in the trial (P=0.60); or outcome vulnerability to non-blinded patients (P=0.52). Blinded and non-blinded assessors agreed in a median of 78% of assessments (interquartile range 64-90%) in the 12 trials with available data. The exaggeration of treatment effects associated with non-blinded assessors was induced by the misclassification of a median of 3% of the assessed patients per trial (1-7%). CONCLUSIONS: On average, non-blinded assessors of subjective binary outcomes generated substantially biased effect estimates in randomised clinical trials, exaggerating odds ratios by 36%. This bias was compatible with a high rate of agreement between blinded and non-blinded outcome assessors and driven by the misclassification of few patients.

OBJECTIVE: To study the inter-observer variation related to extraction of continuous and numerical rating scale data from trial reports for use in meta-analyses. Design Observer agreement study. DATA SOURCES: A random sample of 10 Cochrane reviews that presented a result as a standardised mean difference (SMD), the protocols for the reviews and the trial reports (n=45) were retrieved. DATA EXTRACTION: Five experienced methodologists and five PhD students independently extracted data from the trial reports for calculation of the first SMD result in each review. The observers did not have access to the reviews but to the protocols, where the relevant outcome was highlighted. The agreement was analysed at both trial and meta-analysis level, pairing the observers in all possible ways (45 pairs, yielding 2025 pairs of trials and 450 pairs of meta-analyses). Agreement was defined as SMDs that differed less than 0.1 in their point estimates or confidence intervals. RESULTS: The agreement was 53% at trial level and 31% at meta-analysis level. Including all pairs, the median disagreement was SMD=0.22 (interquartile range 0.07-0.61). The experts agreed somewhat more than the PhD students at trial level (61% v 46%), but not at meta-analysis level. Important reasons for disagreement were differences in selection of time points, scales, control groups, and type of calculations; whether to include a trial in the meta-analysis; and data extraction errors made by the observers. In 14 out of the 100 SMDs calculated at the meta-analysis level, individual observers reached different conclusions than the originally published review. CONCLUSIONS: Disagreements were common and often larger than the effect of commonly used treatments. Meta-analyses using SMDs are prone to observer variation and should be interpreted with caution. The reliability of meta-analyses might be improved by having more detailed review protocols, more than one observer, and statistical expertise.

OBJECTIVE:s To study the analgesic effect of acupuncture and placebo acupuncture and to explore whether the type of the placebo acupuncture is associated with the estimated effect of acupuncture. Design Systematic review and meta-analysis of three armed randomised clinical trials. DATA SOURCES: Cochrane Library, Medline, Embase, Biological Abstracts, and PsycLIT. DATA EXTRACTION: and analysis Standardised mean differences from each trial were used to estimate the effect of acupuncture and placebo acupuncture. The different types of placebo acupuncture were ranked from 1 to 5 according to assessment of the possibility of a physiological effect, and this ranking was meta-regressed with the effect of acupuncture. DATA SYNTHESIS: Thirteen trials (3025 patients) involving a variety of pain conditions were eligible. The allocation of patients was adequately concealed in eight trials. The clinicians managing the acupuncture and placebo acupuncture treatments were not blinded in any of the trials. One clearly outlying trial (70 patients) was excluded. A small difference was found between acupuncture and placebo acupuncture: standardised mean difference –0.17 (95% confidence interval –0.26 to –0.08), corresponding to 4 mm (2 mm to 6 mm) on a 100 mm visual analogue scale. No statistically significant heterogeneity was present (P=0.10, I2=36%). A moderate difference was found between placebo acupuncture and no acupuncture: standardised mean difference –0.42 (–0.60 to –0.23). However, considerable heterogeneity (P<0.001, I2=66%) was also found, as large trials reported both small and large effects of placebo. No association was detected between the type of placebo acupuncture and the effect of acupuncture (P=0.60). CONCLUSIONS: A small analgesic effect of acupuncture was found, which seems to lack clinical relevance and cannot be clearly distinguished from bias. Whether needling at acupuncture points, or at any site, reduces pain independently of the psychological impact of the treatment ritual is unclear.

OBJECTIVE: To evaluate how often sample size calculations and methods of statistical analysis are pre-specified or changed in randomised trials. Design Retrospective cohort study. DATA SOURCE: Protocols and journal publications of published randomised parallel group trials initially approved in 1994-5 by the scientific-ethics committees for Copenhagen and Frederiksberg, Denmark (n=70). MAIN OUTCOME MEASURE: Proportion of protocols and publications that did not provide key information about sample size calculations and statistical methods; proportion of trials with discrepancies between information presented in the protocol and the publication. RESULTS: Only 11/62 trials described existing sample size calculations fully and consistently in both the protocol and the publication. The method of handling protocol deviations was described in 37 protocols and 43 publications. The method of handling missing data was described in 16 protocols and 49 publications. 39/49 protocols and 42/43 publications reported the statistical test used to analyse primary outcome measures. Unacknowledged discrepancies between protocols and publications were found for sample size calculations (18/34 trials), methods of handling protocol deviations (19/43) and missing data (39/49), primary outcome analyses (25/42), subgroup analyses (25/25), and adjusted analyses (23/28). Interim analyses were described in 13 protocols but mentioned in only five corresponding publications. CONCLUSION: When reported in publications, sample size calculations and statistical methods were often explicitly discrepant with the protocol or not pre-specified. Such amendments were rarely acknowledged in the trial publication. The reliability of trial reports cannot be assessed without having access to the full protocols.

OBJECTIVE:s To compare how allocation concealment is described in publications of randomised clinical trials and corresponding protocols, and to estimate how often trial publications with unclear allocation concealment have adequate concealment according to the protocol. Design Cohort study of 102 sets of trial protocols and corresponding publications. Setting Protocols of randomised trials approved by the scientific and ethical committees for Copenhagen and Frederiksberg, 1994 and 1995. MAIN OUTCOME MEASURES: Frequency of adequate, unclear, and inadequate allocation concealment and sequence generation in trial publications compared with protocols, and the proportion of protocols where methods were reported to be adequate but descriptions were unclear in the trial publications. RESULTS: 96 of the 102 trials had unclear allocation concealment according to the trial publication. According to the protocols, 15 of these 96 trials had adequate allocation concealment (16%, 95% confidence interval 9% to 24%), 80 had unclear concealment (83%, 74% to 90%), and one had inadequate concealment. When retrospectively defined loose criteria for concealment were applied, 83 of the 102 trial publications had unclear concealment. According to their protocol, 33 of these 83 trials had adequate allocation concealment (40%, 29% to 51%), 49 had unclear concealment (59%, 48% to 70%), and one had inadequate concealment. CONCLUSIONS: Most randomised clinical trials have unclear allocation concealment on the basis of the trial publication alone. Most of these trials also have unclear allocation concealment according to their protocol.