Search

Background Research waste is a costly problem for scientific research, with poor design and conduct of the research being key elements which contribute to wastage. Interventions to address poor design and conduct may save time and money. The objective of the study was to map the interventions that have been evaluated for improving the design or conduct of scientific research to identify any gaps in the evidence.Methods We undertook a systematic scoping review. We searched MEDLINE, EMBASE, EconLit, ERIC, Social Policy and Practice, HMIC, ProQuest Dissertations and Theses Global and MetaArXiv from 1stJanuary 2012 to 13thJune 2022. Evaluated interventions that aimed to improve the design or conduct of scientific research by targeting researchers or research teams were included. Screening was completed by two reviewers and data charting by a single reviewer with another reviewer checking.Results A total of 81 evaluated interventions were included. Most of the interventions targeted research conduct, primarily focussed on registration, publishing, and reporting. Most included studies used observational evaluation methods. Categorising the interventions by the behaviour change wheel framework we found that most studies utilised restriction, coercion, and persuasion and fewer used enablement, training, or incentivisation to achieve their aims.Conclusions More evaluations of interventions aimed at how researchers design their research are needed, these should be developed appropriately and evaluated for effectiveness using experimental methods.

Objective.Establishing a research agenda on standardizing pain measurement in clinical trials in rheumatic and musculoskeletal diseases (RMD).Methods.Discussion during a meeting at the Outcome Measures in Rheumatology (OMERACT) 2018, prepared by a systematic review of existing core outcome sets and a patient online survey.Results.Several key questions were debated: Is pain a symptom or a disease? Are pain core (sub)domains consistent across RMD? How to account for pain mechanistic descriptors (e.g., central sensitization) in pain measurement?Conclusion.Characterizing and assessing the spectrum of pain experience across RMD in a standardized fashion is the objective of the OMERACT Pain Working Group.

ObjectiveTo evaluate the ability of The BMJeditors to predict the number of times submitted research manuscripts will be cited.DesignCohort study.SettingManuscripts submitted to The BMJ, reviewed, and subsequently scheduled for discussion at a prepublication meeting between 27 August 2015 and 29 December 2016.Participants10 BMJresearch team editors.Main outcome measuresReviewed manuscripts were rated independently by attending editors for citation potential in the year of first publication plus the next year: no citations, below average (<10 citations), average (10-17 citations), or high (>17 citations). Predicted citations were subsequently compared with actual citations extracted from Web of Science (WOS).ResultsOf 534 manuscripts reviewed, 505 were published as full length articles (219 in The BMJ)by end of 2019 and indexed in WOS, 22 were unpublished, and one abstract was withdrawn. Among the 505 manuscripts, the median (IQR [range]) number of citations in the year of publication plus the following year was 9 (4-17 [0-150]); 277 (55%) manuscripts were cited <10 times, 105 (21%) were cited 10-17 times, and 123 (24%) cited >17 times. Manuscripts accepted by The BMJwere cited more highly (median 12 (IQR 7-24) citations) than those rejected (median 7 (3-12) citations). For all 10 editors, predicted ratings tended to increase in line with actual citations, but with considerable variation within categories; nine failed to identify the correct citation category for >50% (range 31%-52%) of manuscripts, and κ ranged between 0.01 to 0.19 for agreement between predicted and actual categories. Editors more often rated papers that achieved high actual citation counts as having low citation potential than the reverse. Collectively, the mean percentage of editors predicting the correct citation category was 43%, and for 160 (32%) manuscripts at least 50% of editors predicted the right category.ConclusionsEditors weren’t good at estimating the citation potential of manuscripts individually or as a group; there is no wisdom of the crowd when it comes to BMJeditors.

Objective To assess the uptake of the rheumatoid arthritis core outcome set using a new assessment method of calculating uptake from data in clinical trial registry entries.Design Review of randomised trials.Setting ClinicalTrials.gov.Subjects 273 randomised trials of drug interventions for the treatment of rheumatoid arthritis and registered in ClinicalTrials.gov between 2002 and 2016. Full publications were identified for completed studies from information in the trial registry or from an internet search using Google and the citation database Web of Science.Main outcome measure The percentage of trials reporting or planning to measure the rheumatoid arthritis core outcome set calculated from the information presented in the trial registry and compared with the percentage reporting the rheumatoid arthritis core outcome set in the resulting trial publications.Results The full rheumatoid arthritis core outcome set was reported in 81% (116/143) of trials identified on the registry as completed (or terminated) for which results were found in either the published literature or the registry. For trials identified on the registry as completed (or terminated), using information only available in the registry gives an estimate for uptake of 77% (145/189).Conclusions The uptake of the rheumatoid arthritis core outcome set in clinical trials has continued to increase over time. Using the information on outcomes listed for completed or terminated studies in a trial registry provides a reasonable estimate of the uptake of a core outcome set and is a more efficient and up-to-date approach than examining the outcomes in published trial reports. The method proposed may provide an efficient approach for an up-to-date assessment of the uptake of the 300 core outcome sets already published.

ObjectiveTo determine the extent and nature of selective non-reporting of harm outcomes in clinical studies that were eligible for inclusion in a cohort of systematic reviews.DesignCohort study of systematic reviews from two databases.SettingOutcome reporting bias in trials for harm outcomes (ORBIT II) in systematic reviews from the Cochrane Library and a separate cohort of systematic reviews of adverse events.Participants92 systematic reviews of randomised controlled trials and non-randomised studies published in the Cochrane Library between issue 9, 2012 and issue 2, 2013 (Cochrane cohort) and 230 systematic reviews published between 1 January 2007 and 31 December 2011 in other publications, synthesising data on harm outcomes (adverse event cohort).MethodsA 13 point classification system for missing outcome data on harm was developed and applied to the studies.Results86% (79/92) of reviews in the Cochrane cohort did not include full data from the main harm outcome of interest of each review for all of the eligible studies included within that review; 76% (173/230) for the adverse event cohort. Overall, the single primary harm outcome was inadequately reported in 76% (705/931) of the studies included in the 92 reviews from the Cochrane cohort and not reported in 47% (4159/8837) of the 230 reviews in the adverse event cohort. In a sample of primary studies not reporting on the single primary harm outcome in the review, scrutiny of the study publication revealed that outcome reporting bias was suspected in nearly two thirds (63%, 248/393).ConclusionsThe number of reviews suspected of outcome reporting bias as a result of missing or partially reported harm related outcomes from at least one eligible study is high. The declaration of important harms and the quality of the reporting of harm outcomes must be improved in both primary studies and systematic reviews.

OBJECTIVE: To examine the prevalence of outcome reporting bias—the selection for publication of a subset of the original recorded outcome variables on the basis of the results—and its impact on Cochrane reviews. Design A nine point classification system for missing outcome data in randomised trials was developed and applied to the trials assessed in a large, unselected cohort of Cochrane systematic reviews. Researchers who conducted the trials were contacted and the reason sought for the non-reporting of data. A sensitivity analysis was undertaken to assess the impact of outcome reporting bias on reviews that included a single meta-analysis of the review primary outcome. RESULTS: More than half (157/283 (55%)) the reviews did not include full data for the review primary outcome of interest from all eligible trials. The median amount of review outcome data missing for any reason was 10%, whereas 50% or more of the potential data were missing in 70 (25%) reviews. It was clear from the publications for 155 (6%) of the 2486 assessable trials that the researchers had measured and analysed the review primary outcome but did not report or only partially reported the results. For reports that did not mention the review primary outcome, our classification regarding the presence of outcome reporting bias was shown to have a sensitivity of 88% (95% CI 65% to 100%) and specificity of 80% (95% CI 69% to 90%) on the basis of responses from 62 trialists. A third of Cochrane reviews (96/283 (34%)) contained at least one trial with high suspicion of outcome reporting bias for the review primary outcome. In a sensitivity analysis undertaken for 81 reviews with a single meta-analysis of the primary outcome of interest, the treatment effect estimate was reduced by 20% or more in 19 (23%). Of the 42 meta-analyses with a statistically significant result only, eight (19%) became non-significant after adjustment for outcome reporting bias and 11 (26%) would have overestimated the treatment effect by 20% or more. CONCLUSIONS: Outcome reporting bias is an under-recognised problem that affects the conclusions in a substantial proportion of Cochrane reviews. Individuals conducting systematic reviews need to address explicitly the issue of missing outcome data for their review to be considered a reliable source of evidence. Extra care is required during data extraction, reviewers should identify when a trial reports that an outcome was measured but no results were reported or events observed, and contact with trialists should be encouraged.