Search

Background There is broad recognition of the importance of evidence in informing clinical decisions. When information from all studies included in a systematic review (“review”) does not contribute to a meta-analysis, decision-makers can be frustrated. Our objectives were to use the field of eyes and vision as a case study and examine the extent to which authors of Cochrane reviews conducted meta-analyses for their review’s pre-specified main outcome domain and the reasons that some otherwise eligible studies were not incorporated into meta-analyses. Methods We examined all completed systematic reviews published by Cochrane Eyes and Vision, as of August 11, 2017. We extracted information about each review’s outcomes and, using an algorithm, categorized one outcome as its “main” outcome. We calculated the percentage of included studies incorporated into meta-analyses for any outcome and for the main outcome. We examined reasons for non-inclusion of studies into the meta-analysis for the main outcome. Results We identified 175 completed reviews, of which 125 reviews included two or more studies. Across these 125 reviews, the median proportions of studies incorporated into at least one meta-analysis for any outcome and for the main outcome were 74% (interquartile range [IQR] 0–100%) and 28% (IQR 0–71%), respectively. Fifty-one reviews (41%) could not conduct a meta-analysis for the main outcome, mostly because fewer than two included studies measured the outcome (21/51 reviews) or the specific measurements for the outcome were inconsistent (16/51 reviews). Conclusions Outcome choice during systematic reviews can lead to few eligible studies included in meta-analyses. Core outcome sets and improved reporting of outcomes can help solve some of these problems.

Background In an effort to identify previously unrecognized aspects of editorial decision-making, we explored the words and phrases that one group of editors used during their meetings. Methods We performed an observational study of discussions at manuscript meetings at JAMA, a major US general medical journal. One of us (KD) attended 12 editorial meetings in 2003 as a visitor and took notes recording phrases from discussion surrounding 102 manuscripts. In addition, editors attending the meetings completed a form for each manuscript considered, listing the reasons they were inclined to proceed to the next step in publication and reasons they were not (DR attended 4/12 meetings). We entered the spoken and written phrases into NVivo 2.0. We then developed a schema for classifying the editors' phrases, using an iterative approach. Results Our classification schema has three main themes: science, journalism, and writing. We considered 2,463 phrases, of which 87 related mainly to the manuscript topic and were not classified (total 2,376 classified). Phrases related to science predominated (1,274 or 54%). The editors, most of whom were physicians, also placed major weight on goals important to JAMA's mission (journalism goals) such as importance to medicine, strategic emphasis for the journal, interest to the readership, and results (729 or 31% of phrases). About 16% (n = 373) of the phrases used related to writing issues, such as clarity and responses to the referees' comments. Conclusion Classification of editorial discourse provides insight into editorial decision making and concepts that need exploration in future studies.

Background The inclusion of randomized controlled trials (RCTs) reported in conference abstracts in systematic reviews is controversial, partly because study design information and risk of bias is often not fully reported in the abstract. The Association for Research in Vision and Ophthalmology (ARVO) requires trial registration of abstracts submitted for their annual conference as of 2007. Our goal was to assess the feasibility of obtaining study design information critical to systematic reviews, but not typically included in conference abstracts, from the trial registration record. Methods We reviewed all conference abstracts presented at the ARVO meetings from 2007 through 2009, and identified 496 RCTs; 154 had a single matching registration record in ClinicalTrials.gov. Two individuals independently extracted information from the abstract and the ClinicalTrials.gov record, including study design, sample size, inclusion criteria, masking, interventions, outcomes, funder, and investigator name and contact information. Discrepancies were resolved by consensus. We assessed the frequencies of reporting variables appearing in the abstract and the trial register and assessed agreement of information reported in both sources. Results We found a substantial amount of study design information in the ClinicalTrials.gov record that was unavailable in the corresponding conference abstract, including eligibility criteria associated with gender (83%; 128/154); masking or blinding of study participants (53%, 82/154), persons administering treatment (30%, 46/154), and persons measuring the outcomes (40%, 61/154)); and number of study centers (58%; 90/154). Only 34% (52/154) of abstracts explicitly described a primary outcome, but a primary outcome was included in the “Primary Outcome” field in the ClinicalTrials.gov record for 82% (126/154) of studies. One or more study interventions were reported in each abstract, but agreed exactly with those reported in ClinicalTrials.gov only slightly more than half the time (88/154, 56%). We found no contact information for study investigators in the abstract, but this information was available in less than one quarter of ClinicalTrial.gov records (17%; 26/154). Conclusion RCT design information not reported in conference abstracts is often available in the corresponding ClinicalTrials.gov registration record. Sometimes there is conflicting information reported in the two sources and further contact with the trial investigators may still be required.