Search

INTRODUCTION: Children, adolescents, and young adults (AYAs) with cancer are a special population who are subjected to a number of unique challenges, stressors, and barriers to high-quality psychological care. In a recent systematic review of measurement properties, we found that 5 of 18 identified patient-reported outcome measures (PROMs) had sufficient psychometric properties to justify their use. A next step is to analyze the reliability of these scale scores in a reliability-generalization meta-analysis. METHODS: We conducted a systematic review of three databases for all studies reporting reliability data for previously identified PROMs. Included studies were further required to include patients with cancer, or survivors of cancer, ages 2–39. We next synthesized alpha and test-retest coefficients using best statistical practices, according to prespecified subgroups, where possible. We considered a threshold of 0.7 to represent sufficient evidence of reliability. RESULTS: Seventy-one studies were included. Overall, reliability coefficients for scale and subscale scores exceeded 0.7. Subgroup analyses were limited by incomplete reporting and a lack of sufficient studies for each subgroup; however, where conducted, these subgroup analyses showed significant differences in the reliability of self-reports versus proxy reports and original versus adapted versions of PROMs. DISCUSSION: We recommend better reporting of reliability data in future studies of children and AYAs with cancer. We discourage relying on historical reliability data in different samples and the reporting of only ranges of reliability coefficients for subscales. Our study suggests that significant differences in the reliability of PROMs may be associated with the PROM respondent and the version of the PROM, thus highlighting the need for further investigation.

Introduction: Children, adolescents, and young adults with cancer are a vulnerable population in whom improvements in psycho-oncology care would be valuable. We previously published a scoping review exploring what is known about positive psychological constructs in this population. A key finding was that evidence-based patient-reported outcome measures (PROMs) were needed to standardize psycho-oncology research. Methods: We undertook a systematic review of measurement properties focused on PROMs in the pediatric and adolescent/young adults (AYAs) with cancer populations. We searched for PROMs that measured at least 1 of 15 previously identified, key positive constructs. This systematic review followed COSMIN guidelines, considered the gold standard manual for systematic reviews of measurement properties. Results: We identified 20 (22 if proxy reports are counted separately) unique PROMs that met inclusion criteria. Of those, only five were recommended for use given the current evidence. The remainder may be used if no alternative exists. Only eight PROMs had any evidence of content validity, considered the most important psychometric property by COSMIN. Many studies were at risk of bias owing to incomplete or suboptimal methods. Discussion: Five PROMs are considered reliable for the pediatric and AYA with cancer populations. A number of PROMs require additional research to ensure their items and instructions are comprehensive, relevant, and comprehensible to children and AYAs. Given the overall lack of research, this review may be considered a starting point for the future refinement of a core set of PROMs to measure positive psychological constructs.

Objectives A cross-sectional analysis of orthopedic trauma randomized controlled trial (RCT) abstracts to assess the frequency and manifestations of spin, the misrepresentation or distortion of research findings, in orthopedic trauma clinical trials. Methods The top 5 orthopedic trauma journals were searched from January 1, 2012, to December 31, 2017. RCTs with nonsignificant endpoints (p > .05) were analyzed for spin in the abstract. The primary endpoint of our investigation was the frequency and type of spin. The secondary endpoint was to assess whether funding source was associated with the presence of spin. Due to the low reporting of funding sources no statistics were able to be computed for this outcome. Results Our PubMed search yielded 517 articles. Primary screening excluded 303 articles, and full text evaluation excluded an additional 161. Overall, 53 articles were included. Spin was identified in 35 of the 53 (66.0%) abstracts analyzed. Evidence of spin was found in 21 (39.6%) abstract results sections and 22 (41.5%) abstract conclusion sections. Of the 21 RCTs reporting a clinical trial registry, 3 (14.3%) had evidence of selective reporting bias. Conclusions Orthopedic trauma RCTs from highly ranked journals with nonsignificant endpoints published from 2012 to 2017 frequently have spin in their abstracts. Abstracts with evidence of spin may influence a reader's perception of new drugs or procedures. In orthopedic trauma, the implications of spin may affect the treatment of patients with orthopedic trauma, so efforts to mitigate spin in RCT abstracts must be prioritized.

OBJECTIVE We investigated the randomized controlled trials (RCTs) related to acute respiratory distress syndrome (ARDS) to assess the presentation and frequency of misrepresented research findings, also known as spin. METHODS We searched PubMed (MEDLINE) for studies published from January 1, 2011 to December 31, 2019. We included randomized controlled trials with an ARDS intervention and a nonsignificant primary endpoint. Trial screening and data extraction was performed on all studies independently and in duplicate. The primary endpoint was to investigate the frequency and manifestation of spin in RCT abstracts. Our secondary endpoint was to investigate associations between funding source and spin. RESULTS Our PubMed search returned 766 articles with 37 meeting inclusion criteria. Spin was present in 14 (14/37, 37.8%; 95% CI 22.5%-55.2%) abstracts. The most common manifestations of spin were claiming benefit based on a significant secondary endpoint (6/14, 42.9%), followed by the use of 'trend' statements, such as 'trend toward significance' (2/14, 14.3%; 95% CI 1.8%-42.8%). The most common spin in abstract conclusions was in the form of claiming benefit due to a significant secondary endpoint (3/4, 75%; 95% CI 19.4%-99.4%). Our secondary endpoint did not identify a significant difference in the prevalence of spin in publicly funded (5/19, 26.3%; 95% CI 9.1%-51.2%) compared to privately funded (4/12, 33.3%; 95% CI 9.9%-65.1%) studies (p>.05). CONCLUSIONS RCTs of ARDS interventions with nonsignificant primary endpoints often included spin in the abstract. Spin in the abstract may influence clinician appraisal and interpretation of diagnostic or treatment modalities.

Aim: To evaluate the abstracts and articles of randomized controlled trials (RCTs) in pain literature and general anesthesia for the evidence of spin. Materials & methods: Following a priori protocol, we used a PubMed search for RCTs from a 2-year period. Data extracted using the Boutron spin definition for criteria. Results: A total of 163 articles were identified as RCTs with clear primary and secondary end points. A total of 33 trials were identified with nonsignificant primary end points. An 18/33 were found to have spin (54.5%). The spin was identified in the results, title and conclusion 12/18, 3/18 and 3/18, respectively. Conclusion: Spin was found in 54% of pain RCTs. With sensitive clinical concerns, such as chronic pain, any misrepresentation of validity could prove to have significant clinical consequences.

Conflicts of interest (COIs) in healthcare are increasingly discussed in the literature, yet these relationships continue to influence healthcare. Research has consistently shown that financial COIs shape prescribing practices, medical education and guideline recommendations. In 2009, the Institute of Medicine (IOM, now the National Academy of Medicine) published Conflicts of Interest in Medical Research, Practice, and Education—one of the most comprehensive reviews of empirical research on COIs in medicine. Ten years after publication of theIOM’s report, we review the current state of COIs within medicine. We also provide specific recommendations for enhancing scientific integrity in medical research, practice, education and editorial practices.

© 2020, The Royal Society of Medicine. In recent years there has been increasing awareness of problems that have undermined trust in medical research. This review outlines some of the most important issues including research culture, reporting biases, and statistical and methodological issues. It examines measures that have been instituted to address these problems and explores the success and limitations of these measures. The paper concludes by proposing three achievable actions which could be implemented to deliver significantly improved transparency and mitigation of bias. These measures are as follows: (1) mandatory registration of interests by those involved in research; (2) that journals support the ‘registered reports’ publication format; and (3) that comprehensive study documentation for all publicly funded research be made available on a World Health Organization research repository. We suggest that achieving such measures requires a broad-based campaign which mobilises public opinion. We invite readers to feedback on the proposed actions and to join us in calling for their implementation.

Context The misrepresentation and distortion of research findings, known as “spin,” has been shown to affect clinical decision making. Spin has been found in randomized controlled trials (RCTs) published in various fields of medicine. Objective To evaluate the abstracts of RCTs found in the cardiology literature for spin. Methods The authors searched PubMed using a specific string of keywords to identify previously published articles documenting RCTs of cardiovascular treatments in humans. To be included, a cardiology trial had to randomize humans to an intervention, statistically compare 2 or more groups, and have a nonsignificant primary endpoint. Records were excluded if they did not meet these criteria. Data extraction was double-blinded and done using a pilot-tested Google Form. Items extracted from each trial included the title, journal, funding source, comparator arm, primary endpoint, statistical analysis of the primary endpoint, secondary endpoints, statistical analysis of secondary endpoints, and trial registration number (if reported). The 2 authors who screened records for inclusion were then asked whether spin was present in the abstract of the randomized trial. Spin in the title, abstract results, abstract conclusions, and selection of reported endpoints were considered. Results Of the 651 PubMed citations retrieved by our search string, 194 RCTs with a clearly defined primary endpoint were identified. Of these 194 RCTs, 66 trials contained nonsignificant primary endpoints and were evaluated for spin. Of these trials, spin was identified in 18 of the 66 abstracts (27.3%). Conclusions Spin was present in our sample of cardiology RCTs. Spin may influence clinical decision making by creating false impressions of the true validity of a drug or intervention.

Aims Perhaps the most important step when designing and conducting randomized controlled trials (RCTs) in addiction is to put methodological safeguards in place to minimize the likelihood for bias to affect trial outcomes. In this study, we applied the revised Cochrane risk of bias tool (ROB 2) to RCTs of drug, alcohol or tobacco interventions. Methods We searched for trials published in 15 addiction medicine journals over a 7-year period. Our primary endpoint is the risk of bias of included studies. We conducted a sensitivity analysis of publicly funded trials. Results Overall, included RCTs were most often at high risk of bias per our judgments (244/487, 50.1%). However, significant proportions of included RCTs were at low risk of bias (123/487, 25.3%) or some concerns for bias (120/497, 24.6%). RCTs with behavioral modification interventions (19/44, 43.2%) and alcohol interventions (80/150, 53.3%) had the highest proportion of high-risk judgments. In a sensitivity analysis of publicly funded RCTs), 195/386 (50.5%) were at high risk of bias. Conclusions Approximately half of included drug, alcohol or tobacco RCTs in our sample were judged to be at high risk of bias with the most common reason being a lack of proper blinding or proper description of blinding. Key action items to reduce bias in future addiction RCTs include adequate randomization, blinding and inclusion of a trial registry number and protocol.

Our study examines the association between the favorability of percutaneous coronary intervention (PCI) and/or coronary artery bypass surgery (CABG) and the presence of conflicts of interest (COIs) among authors.We used the "Citing Articles" tool on the New England Journal of Medicine website to identify editorials on the use of PCI/CABG for stable ischemic heart disease. Authors were rated as "supportive," "neutral," or "critical" of these interventions based on the content of their editorials. COIs for each author were identified using past publications found on Scopus, PubMed, or a general internet search.A total of 606 articles were identified, and data were extracted from 56 of them. Among the 149 authors, 64 (43.0%) had a COI. Of these 64 authors, 19 (29.7%) disclosed their COI, while 45 (70.3%) did not. Overall, among authors with a COI, there was no association between disclosed and undisclosed COIs and the authors' view of PCI/CABG [χEditors should adopt policies to counteract the unbalancing effects that COIs have on medical opinions and evidence.

Study objective We aim to investigate spin in emergency medicine abstracts, using a sample of randomized controlled trials from high-impact-factor journals with statistically nonsignificant primary endpoints. Methods This study investigated spin in abstracts of emergency medicine randomized controlled trials from emergency medicine literature, with studies from 2013 to 2017 from the top 5 emergency medicine journals and general medical journals. Investigators screened records for inclusion and extracted data for spin. We considered evidence of spin if trial authors focused on statistically significant results, interpreted statistically nonsignificant results as equivalent or noninferior, used favorable rhetoric in the interpretation of nonsignificant results, or claimed benefit of an intervention despite statistically nonsignificant results. Results Of 772 abstracts screened, 114 randomized controlled trials reported statistically nonsignificant primary endpoints. Spin was found in 50 of 114 abstracts (44.3%). Industry-funded trials were more likely to have evidence of spin in the abstract (unadjusted odds ratio 3.4; 95% confidence interval 1.1 to 11.9). In the abstracts' results, evidence of spin was most often due to authors' emphasizing a statistically significant subgroup analysis (n=9). In the abstracts' conclusions, spin was most often due to authors' claiming they accomplished an objective that was not a prespecified endpoint (n=14). Conclusion Spin was prevalent in the selected randomized controlled trial, emergency medicine abstracts. Authors most commonly incorporated spin into their reports by focusing on statistically significant results for secondary outcomes or subgroup analyses when the primary outcome was statistically nonsignificant. Spin was more common in studies that had some component of industry funding.

A multidisciplinary international cohort of 72 expert statisticians and researchers recently proposed lowering the p value threshold from 0.05 to 0.005 to mitigate distortion of trial results and decrease bias. We hereby explored how a change to the p value threshold may alter the statistical significance of primary endpoints in gastroenterology (GE) randomized control trials (RCTs). We analyzed RCTs published in the 20 highest ranked GE and medicine journals. For each trial, we extracted the p values for the corresponding primary endpoints. We retrieved 233 RCTs, of which 159 were included in the final analysis yielding 202 primary endpoints. Of these endpoints, 60% had a p value less than 0.05 and when a threshold of less than 0.005 was applied, approximately 50% retained significance. We endorse a lower p value threshold as an actionable, provisional measure for improving statistical inference in GE RCTs until more long-term solutions become available.

INTRODUCTION: Given the changing landscape of colorectal cancer, systematic reviews are likely to play a key role in advancing the understanding of prevention, diagnosis, and treatment. METHODS: We conducted a cross-sectional investigation of the risk of bias and reporting quality of systematic reviews referenced by colon and rectal cancer National Comprehensive Cancer Network (NCCN) guidelines. We used two widely accepted tools: Risk of Bias in Systematic reviews (ROBIS) and Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA). RESULTS: Using ROBIS, only 3 (4.8%) systematic reviews were judged with low risk of bias, 35 (55.6%) systematic reviews were judged with unclear risk of bias, and 25 (39.7%) systematic reviews were judged with high risk of bias. Across all systematic reviews, the individual bias domains at the highest risk of bias were domains 1 (protocol and eligibility criteria) and 2 (methods to identify and select studies). Across all studies, the median adherence to PRISMA was 74.1% (IQR 69.2–80.0%), corresponding to approximately 20 of 27 items. CONCLUSIONS: Systematic reviews cited in NCCN guidelines for colon and rectal cancer are frequently at unclear or high risk of bias and do not report key systematic review items that are important for the critical appraisal of results. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (10.1007/s11606-020-05639-y) contains supplementary material, which is available to authorized users.

Recent proposals to change the p value threshold from 0.05 to 0.005 or to retire statistical significance altogether have garnered much criticism and debate.1 2 As of the writing of our manuscript, the proposal to eliminate statistical significance testing, backed by over 800 signatories, achieved record-breaking status on Altmetrics, with an attention score exceeding 13 000 derived from 19 000 Twitter comments and 35 news stories. We appreciate the renewed enthusiasm for tackling important issues related to the analysis, reporting and interpretation of scientific research results. Our perspective, however, focuses on the current use and reporting of statistical significance and where we should go from here. 1. We begin by saying that p values themselves are not flawed. Rather, the use, misuse or abuse of p values in ways antithetical to rigorous scientific pursuits is the flaw. If p values are a hammer, scientists are the hammer wielders. One would not discard the hammer if the wielder, when using the hammer, repeatedly missed the nail. Similarly, one would not discard the hammer if the wielder used the hammer in a way not suited to the hammer’s purpose, such as in an attempt to drive a screw. Rather, one would expect that the fault lies with the hammer-wielder and recommend ways to refine the hammer’s use. Thus, a focus on …

Introduction A recent proposal suggests changing the threshold for statistical significance from a P value of .05 to .005 to minimize bias and increase reproducibility of future studies. P values less than .05 but greater than .005 would be reclassified as “suggestive”, whereas P values less than .005 would be considered significant. The present study explores how lowering the P value threshold would affect the interpretation of previously published orthopaedic trauma randomized controlled trials (RCTs) and whether outcomes from these trials would maintain statistical significance under the proposed P value threshold. Methods All RCTs published between January 01, 2016 and January 31, 2018 in the Journal of Orthopaedic Trauma, Injury, and Archives of Orthopaedic and Trauma Surgery were screened by at least 2 authors. Data from included trials were extracted in blinded and duplicate fashion. All P values for primary endpoints were included from each study. Results We identified 124 primary endpoints from 48 trials: 39.5% (49/124) of endpoints had a P value less than .05 and 60.5% (75/124) had a P value greater than .05. Overall, 51.0% (25/49) of statistically significant primary endpoints were less than .005, while 49.0% (24/49) would be reclassified as suggestive. Conclusion Based on our results, adopting a lower threshold of significance would heavily alter the significance of orthopaedic trauma RCTs and should be further evaluated and cautiously considered when viewing the effect such a proposal on orthopaedic practice.

The challenges facing the evidence-based medicine (EBM) movement are well documented.1 2 Yet, the problems facing early-career and mid-career researchers (EMCRs) working in the ecosystem of EBM have not been articulated. The coming together of a cohort of EMCRs from across the globe enabled this articulation.3 The 2019 EBMLive conference (see box 1) provided a space for EMCRs to discuss problems, exchange ideas and create a list of potential solutions. This article outlines four key problems faced by EMCRs and their potential solutions (see box 2). Box 1 ### The EBMLive Conference The EBMLive Conference (www.ebmlive.org) is a joint partnership between the Centre for Evidence-Based Medicine and the BMJ , designed to develop, disseminate, and implement better evidence for better healthcare . Since inception, EBMLive has worked tirelessly to include the voice of students and early-career researchers. Building on previous work, the inaugural Doug Altman Scholarship3 and Building Capacity Bursaries were launched in 2019 to fund the travel and attendance of early-career researchers from across the globe to attend the conference. Box 2 ### Problems facing EMCR and their potential solutions In the lead up to the EBMLive conference, Doug Altman Scholars submitted personal and general problems they have faced as early-career researchers. The responses were synthesised and shared with the Scholars to generate further discussion. During EBMLive, the problems and ideas for potential solutions were discussed and presented during dedicated sessions for early- and mid-career researchers. The key list of problems facing early-career and mid-career researchers and their potential solutions are as follows: 1. Tokenistic training of evidence-based medicine 2. Emphasis of quantity over quality 3. Lack …

ImportanceClear and comprehensive descriptions of clinical trial interventions are necessary to translate new results into clinical practice. The TIDieR checklist was developed to be a minimum set of key items considered essential to high-quality reporting of clinical trial interventions.ObjectiveTo determine the quality of reporting of recent Food and Drug Administration (FDA)-approved oncology interventions.DesignCross-sectional investigation.Setting/Participants/InterventionRecent, FDA-approved haematology/oncology anticancer interventions.Main outcome measureQuality of reporting.ResultsAcross all included trials (n=96), a median of 8–9 (out of 12) TIDieR items were reported. Seven TIDieR items had >90% adherence, including individual-level and study-level modifications of drugs and dosing schedules. Three items were less often reported: intervention provider, including training and expertise (7/192, 3.6%); trial institution infrastructure (0/192, 0.0%); and how intervention compliance was assessed (59/192, 30.7%). Publication of a protocol improved intervention reporting (pConclusionsIn this analysis of clinical trials of recent, FDA-approved anticancer interventions, we found good adherence to the TIDieR checklist. These studies were homogeneous in their structure and included information; some TIDieR items were always or never/rarely reported. Clinical trial effect sizes may not translate to real-world practice for a number of reasons. Thus, to aid the translation of trial effect sizes to real-world practice, we recommend authors adhere to the TIDieR checklist and describe the infrastructure of trial centres and describe who provided the intervention, along with their expertise.

AIM The fragility index is calculated by changing one outcome event to a nonevent within a trial until the associated P value exceeds 0.05. In this study, we assessed the robustness, risk of bias (RoB), and power of randomized controlled trials that underlie recommendations set forth by the American College of Gastroenterology (ACG) on managing dyspepsia and Helicobacter pylori infections. METHODS All citations referenced in the guidelines were screened for inclusion criteria. The fragility indexes for eligible trials were then calculated. The likelihood and sources of bias in the included trials were evaluated by the Cochrane 'RoB' Tool 2.0. RESULTS The median fragility index for the 52 trials was three events. Five studies (9.6%) resulted in a fragility index of 0 when statistical analysis was applied. For the 52 trials, 12 (23.1%) were at a low RoB, 15 (28.8%) had some concerns, and 25 (48.1%) were at a high RoB. High RoB was most commonly due to bias of selection in the reported result (15.5%). CONCLUSION A median of three events was needed to nullify statistical significance in 52 trials that underpin guideline recommendations on the management of dyspepsia and H. pylori infections. In addition, concerns for RoB were found for these trials.

HIV/AIDS is associated with significant morbidity, mortality, and financial burden. For these reasons, robust clinical evidence is critical. We aim to investigate the fragility index, fragility quotient, and risk of bias of clinical trial endpoints in HIV medicine. The fragility index represents the minimum amount of trial endpoint “nonevents” changed to “events” in one trial arm required to nullify statistical significance. The fragility quotient contextualized the fragility index by dividing the index by the total trial sample size. We selected eligible trials from the Department of Health and Human Services guideline for the use of antiretroviral agents in HIV-1-infected adults and adolescents. We calculated the fragility index and fragility quotient for all included trials. The Cochrane “risk of bias” Tool 2.0 was used to evaluate the likelihood and sources of bias in the included trials. Thirty-nine RCTs were included for our analysis of fragility. Thirty-six were included for our analysis of the risk of bias. The median fragility index was 5. Three RCTs were at high risk of bias, all due to the selection of the endpoint or statistical test. Twenty had some concerns for risk of bias. The analyzed HIV medicine RCT endpoints were fragile, overall. This indicates that a median of 5 patients across all included studies would nullify the statistical significance of the endpoints. Furthermore, we found evidence that concerns for bias are present at a high rate. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (10.1007/s11606-019-04928-5) contains supplementary material, which is available to authorized users.

371 Background: The number of female veterans is increasing by more than 18,000 per year, and 700 veterans enrolled in VA healthcare are diagnosed with breast cancer each year. The American Cancer Society recommends screening MRI in women with 20-25% or greater lifetime risk of breast cancer, and primary care physicians and oncologists order breast MRIs regularly. The Michael E. DeBakey Veteran’s Affairs Medical Center (MEDVAMC) performs mammograms and ultrasounds on site, but breast MRI is not currently available. Care in the Community arranges for veterans to obtain these studies at outside facilities. This is a complex process, requiring coordination among multiple departments at the MEDVAMC, outside MRI facilities, and patients. Methods: We created a comprehensive process map for the current breast MRI ordering process to determine areas for improvement. We reviewed all breast MRI orders from 1/2019 to 5/2021 at the MEDVAMC and determined the cancellation rate and the rate of benign (BIRADS 1-2) vs non benign (BIRADS 0, 3-5) screening outcomes. We created a Pareto analysis of cancellation reasons. Our process measures were percentage of placed breast MRI orders that were cancelled and delay from MRI order to MRI performed. Our outcome measures were percentage of order cancellations due to incorrect or incomplete orders and delay from MRI order placement to results uploaded. Results: Of the 434 orders that were placed for breast MRIs, 64% were cancelled. Only 117 of the 167 patients that had MRIs ordered ultimately received one. Of the 104 patients without known malignancy who competed an MRI, 45% had non-benign findings requiring follow-up. Our pareto analysis showed that the top cancellation reason was incorrect orders, usually contrast or side verbiage. In June 2021, the CITC order set for MRI was changed from a free text box to discrete, clickable options for side and contrast. In December 2021, we held educational interventions on how to avoid delay or cancellation. Our p-chart of percentage of cancellations due to incorrect ordering shows an extremely promising decrease, with the last seven consecutive subgroups of 15 patients below the mean. The order cancellation percentage is also decreasing. The average delay pre-intervention from MRI order placement to results uploaded was 67 days. In January 2022, we implemented a breast MRI nurse navigator to help acquire results and expedite scheduling. Post-intervention, the average delay has decreased to 38.5 days. Using XmR and S charts, we have also shown an increase in precision. Conclusions: Providing discrete, clickable options within the MRI order set reduced the number of cancellations due to incorrect verbiage. Early data suggests that the educational intervention improved the cancellation rate. Instituting a nurse navigator decreased both the delay and variability for the time from order placement to results received, which will reduce the delay to follow-up of non-benign findings.

Objective This study assesses the quality and completeness of systematic reviews (SRs) included by the National Comprehensive Cancer Network (NCCN) cancer screening clinical practice guidelines (CPGs). Methods We evaluated SRs according to PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) and AMSTAR-2 (A Measurement Tool to Assess systematic Reviews). Results Seven NCCN CPGs were included with 109 SRs. The mean PRISMA percent completeness of included SRs was 71 % (range 0.1–1.0). The mean AMSTAR-2 percent completeness was 56 % (range 0.05−0.99). Of the 70 SRs assessed via AMSTAR-2, 42 (60 %) received a “critically low” rating, 11 (15.7 %) received “low” ratings, and 17 (24.3 %) received “moderate”. None of the SRs received a “high” rating. Conclusion Lack of adherence to AMSTAR-2 and PRISMA reporting standards among the SRs included is prevalent. We suggest improved reporting of SR inclusion criteria and evaluation to bolster the reporting quality of SRs underpinning CPG recommendations.

Objective: To assess the quality of reporting in diagnostic accuracy studies (DAS) referenced by the Quality Improvement Guidelines for Diagnostic Arteriography and their adherence to the Standards for Reporting of Diagnostic Accuracy (STARD) statement. Materials and Methods: Citations specific to the Society of Interventional Radiology’s Quality Improvement Guidelines for Diagnostic Arteriography were collected. Using the 34-item STARD checklist, two authors in a duplicate and blinded fashion documented the number of items reported per diagnostic accuracy study. Authors met, and any discrepancies were resolved in a resolution meeting. Results: Of the 26 diagnostic accuracy studies included, the mean number of STARD items reported was 17.8 (SD ± 3.1). The median adherence was 18 (IQR, 17-19) items. Ten articles were published prior to 2003, the original date of STARD publication, and 16 articles were published after 2003. The mean number of reported items for the articles published prior to STARD 2003, and after STARD 2003 was 17.4 (SD ± 2.4) and 18.1 (SD ± 3.5), respectively. There were 14 STARD items that demonstrated an adherence of < 25%, and 13 an adherence > 75%. Conclusion: The dichotomous distribution of adherence to the STARD statement by DAS investigated demonstrates that areas of deficient reporting may be present and require attention to ensure complete and transparent reporting in the future.

Introduction: Children and adolescents/young adults (AYAs) with cancer are a vulnerable population susceptible to numerous late effects, such as fatigue and depression, which may diminish their long-term psychological, physical, spiritual, and emotional health. A well-rounded understanding of how positive psychological constructs affect the quality of care and treatment outcomes is therefore warranted. Methods: We conducted a scoping review of 15 positive psychological constructs in children and AYAs with cancer. The primary research questions were (1) what is known about positive psychological constructs in children and AYAs with cancer; (2) what value is ascribed to these constructs by patients? Results: Two hundred seventy-six articles were included after database search and screening. These studies were mostly observational or qualitative and conducted in North America. Constructs were often poorly defined, and measurement tools used to gather data were wide ranging. Numerous factors were correlated with increased or decreased expression of certain constructs, but overall themes were difficult to identify. Similarly, patients often spoke of what increased or decreased expression of a construct, with less emphasis on what they implicitly value. Discussion: This scoping review found ample evidence for what increases or decreases expression of positive psychological constructs, but this evidence was observational and often conflicting. In the future, we recommend the development of a core set of psychological outcomes, with definitions and corresponding measurement tools. We further recommend an emphasis on randomized trials to more rigorously study how expression of constructs can be improved and what effect this has on the quality of life.

Objective: Reproducibility of systemic reviews (SRs) can be hindered by the presence of citation bias. Citation bias may occur when authors of SRs conduct hand-searches of included study reference lists to identify additional studies. Such a practice may lead to exaggerated SR summary effects. The purpose of this paper is to examine the prevalence of hand-searching reference lists in otolaryngology SRs.Methods: The authors searched for systematic reviews published in eight clinical otolaryngology journals using the Cochrane Library and PubMed, with the date parameter of January 1, 2008, to December 31, 2017. Two independent authors worked separately to extract data from each SR for the following elements: whether reference lists were hand-searched, other kinds of supplemental searching, PRISMA adherence, and funding source. Following extraction, the investigators met to review discrepancies and achieve consensus.Results: A total of 539 systemic reviews, 502 from clinical journals and 37 from the Cochrane library, were identified. Of those SRs, 72.4% (390/539) hand-searched reference lists, including 97.3% (36/37) of Cochrane reviews. For 228 (58.5%) of the SRs that hand-searched reference lists, no other supplemental search (e.g., search of trial registries) was conducted.Conclusions: These findings indicate that hand-searching reference lists is a common practice in otolaryngology SRs. Moreover, a majority of studies at risk of citation bias did not attempt to mitigate the bias by conducting additional supplemental searches. The implication is that summary effects in otolaryngology systematic reviews may be biased toward statistically significant findings.

Introduction. Children and adolescents/young adults (AYAs) with cancer are a vulnerable population susceptible to numerous late effects, such as fatigue and depression, that may diminish their long-term psychosocial, physical, spiritual, and emotional health. A well-rounded understanding of how positive psychosocial constructs affect the quality of care and treatment outcomes is therefore warranted.Methods. We conducted a scoping review of 15 positive psychosocial constructs in children and AYAs with cancer. The primary research questions were 1) what is known about positive psychosocial constructs in children and AYAs with cancer; 2) what value is ascribed to these constructs by patients?. Results. 276 articles were included after database search and screening. These studies were mostly observational or qualitative and conducted in North America. Constructs were often poorly defined, and measurement tools used to gather data were wide-ranging. Numerous factors were correlated with increased or decreased expression of certain constructs, but overall themes were difficult to identify. Similarly, patients often spoke of what increased or decreased expression of a construct, with less emphasis on what they implicitly value.Discussion. This scoping review found ample evidence for what increases or decreases expression of positive psychosocial constructs, but this evidence was observational and often conflicting. In the future, we recommend the development of a core set of psychosocial outcomes, with definitions and corresponding measurement tools. We further recommend an emphasis on randomized trials to more rigorously study how expression of constructs can be improved and what effect this has on quality of life.

IMPORTANCE: Selective outcome reporting bias in oncology drug advertisements may encourage misconceptions about a drug’s efficacy profile. OBJECTIVE: We sought to determine the rates of selective outcome reporting in published cancer clinical trials and in television and print advertisements for anticancer medications. We also quantified the number of advertisements that did not include or cite any studies with mature overall survival (OS) data (i.e., data with all required patient events for final analysis). DESIGN/SETTING/PARTICIPANTS: We conducted a cross-sectional investigation of advertisements uploaded to the AdPharm Database (repository of pharmaceutical advertisements); the clinical trials supporting the ads; and the trial registrations associated with the trials. Data were extracted by two investigators who were blinded to each other’s data. MAIN OUTCOME MEASURES: The first co-primary objective was to investigate selective outcome reporting between trial registrations and published trials. The second co-primary objective was to investigate selective outcome reporting between the same published trials and drug advertisements. RESULTS: We included 74 advertisements and 48 clinical trials. Print ads were the most common (n = 66), and most print advertisements were targeted to health care providers (n = 55, 83.3%). Overall, 41/48 (85.4%) trials were registered prior to study enrollment, and 41/48 (85.4%) did not deviate from the registered primary endpoints. Across all advertisements (n = 74), statistically significant endpoints were more often reported (unadjusted risk ratio [uRR] 1.26; 95% confidence interval [CI] (1.14–1.40)) and 22/55 (40.0%) advertisements cited trials with immature overall survival data (i.e., data without the required number of events for final analysis). CONCLUSIONS: In our sample, statistically significant endpoints were more commonly reported than nonsignificant endpoints. Immature endpoints (those analyzed before the required number of accrued patient events) were often reported. By reporting only significant endpoints and those that are immature, advertisers may encourage misconceptions about a drug’s efficacy profile.

e18641 Background: There are two million female veterans across the United States, a number increasing by more than 18,000 per year. 1 in 8 women will be diagnosed with breast cancer in their lifetime, and studies have shown that servicewomen are 40% more likely to get breast cancer than civilians. The American Cancer Society (ACS) recommends screening MRI in women with 20-25% or greater lifetime risk of breast cancer, and both primary care physicians and oncologists are now ordering these more frequently. The Michael E. DeBakey Veteran’s Affairs Medical Center (MEDVAMC) performs screening mammograms and ultrasounds, but breast MRI is not currently available at the facility. Care in the Community arranges for veterans to obtain these studies at outside facilities. This is a complex process, requiring coordination among multiple departments, the outside MRI facility, and the patient. There are a significant number of MRI order cancellations per month, which can lead to delays in diagnosis or treatment. Methods: We created a comprehensive process map for the current breast MRI ordering process to determine areas for improvement. We looked at all breast MRI orders starting in 12/2020 at the MEDVAMC and determined the rate of cancellation and the rate of benign (BIRADS 1-2) vs non benign (BIRADS 0, 3-5) screening outcomes. We created a Pareto analysis to determine the most common cancellation reasons and p-charts of the percent of cancelled MRI orders per month and a run chart of the percent due to incorrect verbiage. Results: Of the 243 orders placed for MRI for 124 patients from 12/2020 to 1/2022, 64.2% were cancelled. Of the 57% of patients (71/124) with complete MRIs, 35.2% had non-benign findings requiring follow up, excluding known malignancies. Our Pareto analysis showed that most cancellations were due to incorrect verbiage in the order. An intervention in 6/2021 changed the MRI order from a free text box to clickable options, followed by an educational intervention in 12/2021. Cancellations due to incorrect verbiage decreased overall after June, however it has not yet reached significance. It is too early to determine if the educational intervention caused a significant change, however the cancellation rate has decreased. Conclusions: Providing discrete, clickable options within the MRI order has reduced the number of cancellations due to incorrect verbiage, though we have not yet reached significance. Early data suggests that the educational intervention has improved the cancellation rate. Fewer cancellations will lead to more timely studies, which will in turn lead to faster follow up of non-benign findings. Further directions include using a nurse navigator to reduce confusion and delays, further simplifying the ordering process, checking patient phone numbers in clinic, and sending patients home with information on the importance of screening MRI to reduce veteran cancellation.

OBJECTIVE Spin, the misrepresentation and distortion of research findings, has been shown to affect clinical decision making. Spin has been found in randomized controlled trials (RCTs) published in various fields of medicine, but no study has tested for the presence of spin in otolaryngology RCTs. The purpose of this study is to evaluate the abstracts of RCTs found in the otolaryngology literature for spin. METHODS In this cross-sectional analysis, we analyzed the abstracts of RCTs for spin using a pilot-tested form. Double data extraction was performed by two blinded authors, and discrepancies were resolved using mutual discussion. RESULTS Out of the 534 PubMed citations retrieved by our search string, 162 parallel-group RCTs with clearly defined primary and secondary endpoints were identified. Further analysis identified 47 trials with nonsignificant primary outcomes, which were then evaluated for spin. Spin was identified in 33 of the 47 (70%) abstracts. Spin was found in the results sections of 25 (53%) of the included abstracts and was found in the conclusion section of 27 (57%) of the abstracts. Spin was not present in the titles of any of the included studies. CONCLUSION Spin was common in our sample of otolaryngology RCTs. Spin may potentially create false impressions about the true validity of a drug or intervention. Further research needs to test for potential clinical implications of spin in the otolaryngology literature. LEVEL OF EVIDENCE NA. Laryngoscope, 2018.

IntroductionComplete reporting of systematic reviews of interventions is essential to the interpretation of research findings and the reproducibility of research results. The Template for Intervention Description and Replication (TIDieR) checklist—and the version specific to systematic reviews (TIDieR-SR)—was created to provide authors and researchers an evidence-based guide for reporting trial and systematic review interventions. In this study, we apply TIDieR-SR to Cochrane systematic reviews of surgical interventions.MethodsWe searched the Cochrane Database for relevant systematic reviews. Two investigators applied inclusion/exclusion criteria to all titles/abstracts and full texts. These same investigators extracted all data in duplicate while masked to the other’s data. The primary outcome was adherence to TIDieR-SR items.ResultsTwo hundred and thirty-eight systematic reviews were included. Overall, included SRs adhered to a median of 6 (IQR 5–7) out of eight TIDieR-SR items. The item with the lowest adherence was item 7 (share intervention materials, 1/238 (0.4%).DiscussionOur results are encouraging, but the generalisability of our findings is compromised by the inclusion of only Cochrane systematic reviews. Future reporting of intervention materials is likely to improve the application of effective surgical interventions in the clinical practice.

Introduction Noninferiority trials can show that new treatments with slightly less efficacy are safer, cheaper, or easier to administer. However, the conclusions of noninferiority trials depend on robust methodology. Methods We conducted a 6 year cross-sectional investigation of the methodological quality of oncology noninferiority trials published in the top 10 oncology journals. Four key quality criteria were investigated. Results Nonefficacy benefits of the new treatment were stated in 88/110 (80.0 %) trials. Justification for the noninferiority margin was provided in 79/110 (71.8 %) trials. Authors most often used previous data as justification for the chosen margin (n = 42). In 15 noninferiority trials the percent preserved effect could be calculated and the median effect preserved was 56.8 %. Conclusions The oncology noninferiority trials included in our study had key methodological shortcomings, counterbalanced by a clear delineation of expected nonefficacy benefits of the new treatment.

Regulatory agencies around the world have been using flexible requirements for approval of new drugs, especially for cancer drugs. The US Food and Drug Administration (FDA) is mostly the first agency to approve new drugs worldwide, mainly due to the faster terms of the accelerated pathway and breakthrough therapy designation. Surrogate endpoints and preliminary data (e.g. single-arm and phase 2 studies) are used for these new approvals, however larger effect sizes are expected. We aim to compare FDA Accelerated vs Regular Pathway approvals and Breakthrough therapy designations (BTD) for lung cancer treatments between 2006 and 2018 regarding study design, sample size, outcome measures and effect size. We assessed the FDA database to collect data from studies that formed the basis of approvals of new drugs or indications for lung cancer spanning from 2006 to 2018. We found that accelerated pathway approvals are based on significantly more single-arm studies with small sample sizes and surrogate primary endpoints. However, effect size was not different between the pathways. A large proportion of studies used to support regular pathway approvals also showed these characteristics that are related to low quality and uncertain evidence. Compared to other approvals, BTD were more frequently based on single-arm studies. There was no significant difference in use of surrogate endpoints or sample size. 44% of BTD were based on studies demonstrating large effect sizes, proportionally more than approvals not receiving this designation. In conclusion, based on the indicators of evidence quality we extracted, criteria's for granting accelerated approval and breakthrough therapy designation seen not clear. Faster approvals are in the majority full of uncertainties which should be viewed with caution and the patient have to be communicated to allow shared decision making. Post-marketing validation is essential.

The Psychopharmacologic Drug Advisory Committee (PDAC) is one of 33 advisory committees of the Food and Drug Administration (FDA). During committee meetings, an open public hearing takes place where speakers provide testimonies about the drug in question and are asked, not required, to disclose any conflicts of interests (COIs) before speaking. These speakers may present with COIs which include, but are not limited to, reimbursement for travel and lodging by the pharmaceutical company to attend the meeting; previous or current payments for consulting from the pharmaceutical company and compensation as a paid investigator in previously conducted clinical trials for the drug under review. Our study aimed to investigate the characteristics and COIs of public speakers at PDAC meetings of the FDA. We evaluated 145 public speakers at FDA committee meetings over a 10-year period. We found a total of 52 public speakers disclosed a COI with travel and lodging being the most prominent. Among these speakers, 82.4% provided a positive testimony regarding the psychiatric drug in question. Speakers who had the condition in question were not more likely to provide a positive statement than those who did not. Our results showed that disclosing a COI was associated with increased odds of public speakers providing a favourable testimony for the recommendation of psychiatric drugs. The implications of these findings are concerning since COIs have the potential to skew public speaker’s testimonies and persuade committee members to recommend a drug through emotionally charged tactics.

ObjectivesEvaluate the completeness of reporting of addiction randomised controlled trials (RCTs) using the Consolidated Standards of Reporting Trials (CONSORT) statement.SettingNot applicable.ParticipantsRCTs identified using a PubMed search of 15 addiction journals and a 5-year cross-section.Outcome measuresCompleteness of reporting.ResultsOur analysis of 394 addiction RCTs found that the mean number of CONSORT items reported was 19.2 (SD 5.2), out of a possible 31. Twelve items were reported in 75% of RCTs. Journal endorsement of CONSORT was found to improve the number of CONSORT items reported.ConclusionsPoor reporting quality may prohibit readers from critically appraising the methodological quality of addiction trials. We recommend journal endorsement of CONSORT since our study and those previous have shown that CONSORT endorsement improves the quality of reporting.

Importance Reproducible research practices are essential to biomedical research because these practices promote trustworthy evidence. In systematic reviews and meta-analyses, reproducible research practices ensure that summary effects used to guide patient care are stable and trustworthy. Objective To evaluate the reproducibility in theory of meta-analyses in oncology systematic reviews cited by the 49 National Comprehensive Cancer Network (NCCN) guidelines for the treatment of cancer by site and evaluate whether Cochrane reviews or systematic reviews that report adherence to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines use more reproducible research practices. Design, Setting, and Participants A cross-sectional investigation of all systematic reviews with at least 1 meta-analysis and at least 1 included randomized clinical trial (RCT) that are cited by NCCN guidelines for treatment of cancer by site. We scanned the reference list of all NCCN guidelines (n = 49) for potential systematic reviews and meta-analyses. All retrieved studies were screened, and data were extracted, independently and in duplicate. The analysis was carried out between May 6, 2018, and January 28, 2019. Main Outcomes and Measures The frequency of reproducible research practices, defined as (1) effect estimate and measure of precision (eg, hazard ratio with 95% confidence interval); (2) clear list of studies included for each analysis; and (3) for subgroup and sensitivity analyses, it must be clear which studies were included in each group or level. Results We identified 1124 potential systematic reviews, and 154 meta-analyses comprising 3696 meta-analytic effect size estimates were included. Only 2375 of the 3696 meta-analytic estimates (64.3%), including subgroup and sensitivity analyses, were reproducible in theory. Forest plots appear to improve the reproducibility of meta-analyses. All meta-analytic estimates were reproducible in theory in 100 systematic reviews (64.9%), and in 15 systematic reviews (9.7%), no meta-analytic estimates could potentially be reproduced. Data were said to be imputed in 29 meta-analyses, but none specified which data. Only 1 meta-analysis included a link to an online data set. Conclusions and Relevance More reproducible research practices are needed in oncology meta-analyses, as suggested by those that are cited by the NCCN. Reporting meta-analyses in forest plots and requirements for full data sharing are recommended.

We have identified ‘spin’ in abstracts of randomised controlled trials (RCTs) with nonsignificant primary endpoints in psychiatry and psychology journals. This is a cross-sectional review of clinical trials with nonsignificant primary endpoints published in psychiatry and psychology journals from January 2012 to December 2017. The main outcome was the frequency and manifestation of spin in the abstracts. We define spin as the ‘use of specific reporting strategies, from whatever motive, to highlight that the experimental treatment is beneficial, despite a statistically nonsignificant difference for the primary outcome, or to distract the reader from statistically nonsignificant results’. We have also assessed the relationship between industry funding and spin. Of the 486 RCTs examined, 116 were included in our analysis of spin. Spin was identified in 56% (n=65) of those included. Spin was found in 2 (2%) titles, 24 (21%) abstract results sections and 57 (49.1%) abstract conclusion sections. Evidence of spin was simultaneously identified in both results and conclusions sections in 15% of RCTs (n=17). Twelve articles reported industry funding (10%). Industry funding was not associated with increased odds of spin in the abstract (unadjusted OR: 1.0; 95% CI: 0.3 to 3.2). We found no relationship between industry funding and spin in abstracts. These findings raise concerns about the effects spin may have on clinicians. Further steps could be taken to address spin, including inviting reviewers to comment on the presence of spin and updating Consolidated Standards of Reporting Trials guidelines to contain language discouraging spin.

Background: Statistical analysis of systematic reviews allows the results of previous studies to be combined and synthesized to assess the overall health effect of the intervention in question. Systematic reviews can also be used to guide the creation of clinical practice guidelines and are considered to have a high level of evidence. Thus, it is important that their methodological quality is of the highest standard. Publication bias presents 2 problems: (1) studies with significant results may be overrepresented in systematic reviews and meta-analyses (“false positives”) and (2) studies without significant results may not be included in systematic reviews and meta-analyses (“false negatives”) because each study, on its own, was underpowered, meaning that some treatment options that may have clinical benefit will not be adopted. Methods: We performed a study to evaluate the techniques used by authors to report and evaluate publication bias in the top 10 orthopaedic journals as well as 3 orthopaedic-related Cochrane groups. Two authors independently screened the titles and abstracts to identify systematic reviews and meta-analyses. We assessed publication bias in the systematic reviews that did not assess publication bias themselves. Results: Our final sample included 694 systematic reviews or meta-analyses that met our inclusion criteria. Our review included 502 studies (72%) that focused on clinical outcomes, with the majority of the remaining studies focused on predictive and prognostic accuracy (20%) or diagnostic accuracy (5%). Publication bias was discussed in 295 (42.5%) of the included studies and was assessed in 135 (19.5%). Of the studies that assessed publication bias, 31.9% demonstrated evidence of publication bias. Only 43% and 22% of studies that involved use of the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines discussed and assessed publication bias, respectively. Conclusions: Publication bias is infrequently discussed and assessed in the high-impact orthopaedic literature. Furthermore, nearly one-third of the studies that assessed for publication bias demonstrated evidence of publication bias. In addition to these shortcomings, fewer than half of these studies involved use of the PRISMA guidelines and yet only one-fourth of the studies assessed for publication bias. Clinical Relevance: By understanding the degree to which publication bias is discussed and presented in high-impact orthopaedic literature, changes can be made by journals and researchers alike to improve the overall quality of research produced and reported.