Search

Inter-rater reliability (IRR) is one of the commonly used tools for assessing the quality of ratings from multiple raters. However, applicant selection procedures based on ratings from multiple raters usually result in a binary outcome; the applicant is either selected or not. This final outcome is not considered in IRR, which instead focuses on the ratings of the individual subjects or objects. We outline the connection between the ratings' measurement model (used for IRR) and a binary classification framework. We develop a simple way of approximating the probability of correctly selecting the best applicants which allows us to compute error probabilities of the selection procedure (i.e., false positive and false negative rate) or their lower bounds. We draw connections between the IRR and the binary classification metrics, showing that binary classification metrics depend solely on the IRR coefficient and proportion of selected applicants. We assess the performance of the approximation in a simulation study and apply it in an example comparing the reliability of multiple grant peer review selection procedures. We also discuss other possible uses of the explored connections in other contexts, such as educational testing, psychological assessment, and health-related measurement, and implement the computations in the R package IRR2FPR., (© 2024 The Authors. British Journal of Mathematical and Statistical Psychology published by John Wiley & Sons Ltd on behalf of British Psychological Society.)

Peer review is an essential step in scientific progress and clinical improvement, providing opportunity for research to be critically evaluated and improved by one's colleagues. Pharmacists from all job settings are called to serve as peer reviewers in the ever-growing publication landscape of the profession. Despite challenges to engagement such as time and compensation, peer review provides considerable professional development for both authors and reviewers alike. This article will serve as a practical guide for peer reviewers, discussing best practices as well as the handling of different situations that may arise during the process., Competing Interests: Disclosure Jordan R. Covvey is an Associate Editor and the other authors are members of the Editorial Advisory Board of the Journal of the American Pharmacists Association (JAPhA). The authors declare no other relevant conflicts of interest or financial relationships., (Copyright © 2024 American Pharmacists Association®. All rights reserved.)

Background: Reviewers rarely comment on the same aspects of a manuscript, making it difficult to properly assess manuscripts' quality and the quality of the peer review process. The goal of this pilot study was to evaluate structured peer review implementation by: 1) exploring whether and how reviewers answered structured peer review questions, 2) analysing reviewer agreement, 3) comparing that agreement to agreement before implementation of structured peer review, and 4) further enhancing the piloted set of structured peer review questions., Methods: Structured peer review consisting of nine questions was piloted in August 2022 in 220 Elsevier journals. We randomly selected 10% of these journals across all fields and IF quartiles and included manuscripts that received two review reports in the first 2 months of the pilot, leaving us with 107 manuscripts belonging to 23 journals. Eight questions had open-ended fields, while the ninth question (on language editing) had only a yes/no option. The reviews could also leave Comments-to-Author and Comments-to-Editor . Answers were independently analysed by two raters, using qualitative methods., Results: Almost all the reviewers ( n = 196, 92%) provided answers to all questions even though these questions were not mandatory in the system. The longest answer (Md 27 words, IQR 11 to 68) was for reporting methods with sufficient details for replicability or reproducibility. The reviewers had the highest (partial) agreement (of 72%) for assessing the flow and structure of the manuscript, and the lowest (of 53%) for assessing whether interpretation of the results was supported by data, and for assessing whether the statistical analyses were appropriate and reported in sufficient detail (52%). Two thirds of the reviewers ( n = 145, 68%) filled out the Comments-to-Author section, of which 105 (49%) resembled traditional peer review reports. These reports contained a Md of 4 (IQR 3 to 5) topics covered by the structured questions. Absolute agreement regarding final recommendations (exact match of recommendation choice) was 41%, which was higher than what those journals had in the period from 2019 to 2021 (31% agreement, P = 0.0275)., Conclusions: Our preliminary results indicate that reviewers successfully adapted to the new review format, and that they covered more topics than in their traditional reports. Individual question analysis indicated the greatest disagreement regarding the interpretation of the results and the conducting and the reporting of statistical analyses. While structured peer review did lead to improvement in reviewer final recommendation agreements, this was not a randomized trial, and further studies should be performed to corroborate this. Further research is also needed to determine whether structured peer review leads to greater knowledge transfer or better improvement of manuscripts., Competing Interests: Mario Malički is Co-Editor-in-Chief of Research Integrity and Peer Review journal. Bahar Mehmani is an employee of Elsevier, the publisher of the journals studied in this article and the owner of the submission system used for piloting the structured peer review, and for collecting the data and reviewer responses that were analysed in this study. Elsevier is also the owner of the Scopus database which was used to select journals from different impact factor quartiles and subject areas., (© 2024 Malički and Mehmani.)

Objective: This paper aims to provide an introductory resource for beginner peer reviewers in psychiatry and the broader biomedical science field. It will provide a concise overview of the peer review process, alongside some reviewing tips and tricks., Conclusion: The peer review process is a fundamental aspect of biomedical science publishing. The model of peer review offered varies between journals and usually relies on a pool of volunteers with differing levels of expertise and scope. The aim of peer review is to collaboratively leverage reviewers' collective knowledge with the objective of increasing the quality and merit of published works. The limitations, methodology and need for transparency in the peer review process are often poorly understood. Although imperfect, the peer review process provides some degree of scientific rigour by emphasising the need for an ethical, comprehensive and systematic approach to reviewing articles. Contributions from junior reviewers can add significant value to manuscripts., Competing Interests: DisclosureThe author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Purpose: With expansion of academic cancer center networks across geographically-dispersed sites, ensuring high-quality delivery of care across all network affiliates is essential. We report on the characteristics and efficacy of a radiation oncology peer-review quality assurance (QA) system implemented across a large-scale multinational cancer network., Methods and Materials: Since 2014, weekly case-based peer-review QA meetings have been standard for network radiation oncologists with radiation oncology faculty at a major academic center. This radiotherapy (RT) QA program involves pre-treatment peer-review of cases by disease site, with disease-site subspecialized main campus faculty members. This virtual QA platform involves direct review of the proposed RT plan as well as supporting data, including relevant pathology and imaging studies for each patient. Network RT plans were scored as being concordant or nonconcordant based on national guidelines, institutional recommendations, and/or expert judgment when considering individual patient-specific factors for a given case. Data from January 1, 2014, through December 31, 2019, were aggregated for analysis., Results: Between 2014 and 2019, across 8 network centers, a total of 16,601 RT plans underwent peer-review. The network-based peer-review case volume increased over the study period, from 958 cases in 2014 to 4,487 in 2019. A combined global nonconcordance rate of 4.5% was noted, with the highest nonconcordance rates among head-and-neck cases (11.0%). For centers that joined the network during the study period, we observed a significant decrease in the nonconcordance rate over time (3.1% average annual decrease in nonconcordance, P = 0.01); among centers that joined the network prior to the study period, nonconcordance rates remained stable over time., Conclusions: Through a standardized QA platform, network-based multinational peer-review of RT plans can be achieved. Improved concordance rates among newly added network affiliates over time are noted, suggesting a positive impact of network membership on the quality of delivered cancer care., Competing Interests: Disclosures The authors declare no relevant financial disclosures or conflicts of interest. Data sharing not available at this time., (Copyright © 2024 American Society for Radiation Oncology. Published by Elsevier Inc. All rights reserved.)

Over the past 15 years, the radiology community has made great progress moving from a system of score-based peer review to one of peer learning. Much has been learned along the way. In peer learning, cases in which learning opportunities are identified are reviewed solely for the purpose of fostering learning and improvement. This article defines peer learning and peer review and emphasizes the difference; looks back at the 20-year history of score-based peer review and transition to peer learning; outlines the problems with score-based peer review and the key elements of peer learning; discusses the current state of peer learning; and outlines future challenges and opportunities., (Copyright © 2024 Elsevier Inc. All rights reserved.)

Aim: This scoping review examined development strategies for preparing reviewers to critically appraise the content of manuscripts submitted to peer-reviewed journals., Background: The journal peer review process is the crux of building the science of nursing education to inform teaching and learning., Method: Using the Joanna Briggs Institute procedure for scoping reviews, five databases were searched for articles published in English in peer-reviewed health sciences journals between 2012 and 2022 that included strategies for developing journal peer reviewers., Results: Of the 44 articles included in the review, a majority were commentaries (52%) published by medicine (61%), followed by nursing (9%) and multidisciplinary journals (9%). Reviewer development strategies aligned with three themes: pedagogical approaches, resources, and personal practices., Conclusion: Although multiple disciplines addressed peer reviewer development, a comprehensive and effective approach was not reported in the reviewed literature. The findings can inform a multilevel reviewer development program led by academic nurse educators., Competing Interests: The authors have declared no conflict of interest., (Copyright © 2023 National League for Nursing.)

Objectives: Ensuring high-quality radiotherapy requires peer-reviewing target volumes. The Royal College of Radiologists recommends peer review specifically for individual target volumes in cases of gynaecological cancers. This study presents the outcomes of implementing an on-demand peer review system for gynaecological cancers within our institute., Methods: The peer review process was planned for gynaecological cancer cases intended for curative radiotherapy. After junior clinical oncologists (COs) completed the segmentation, two senior COs specializing in gynaecological cancers conducted the peer review. All peer review outcomes were recorded prospectively. The audit process compliance, the proportion of patients requiring major and minor modifications in target volumes, the direction of changes, and the factors influencing these changes were reported., Results: A total of 230 patients were eligible, and out of these, 204 (88.3%) patients underwent at least one peer review. Among the patients, 108 required major modifications in their target volumes. P-charts revealed a stabilization in the need for major modifications at the end of three months, indicating that 38.2% and 28% of patients still required major modifications for the nodal and primary CTV, respectively. Multivariable analysis demonstrated that major modifications were associated with the use of extended field radiotherapy and radical radiation in non-cervical primary cases., Conclusions: An on-demand peer review system was feasible and resulted in clinically meaningful, major modifications in the target volumes for 53% of patients., Advances in Knowledge: Gynaecological cancers require ongoing peer review to ensure quality of care in radiotherapy. A flexible on-demand system not only ensures that patient treatment start is not delayed but also has an important educational role for junior trainees., (© The Author(s) 2024. Published by Oxford University Press on behalf of the British Institute of Radiology. All rights reserved. For permissions, please email: journals.permissions@oup.com.)

The process of peer review has been the gold standard for evaluating medical science, but significant pressures from the recent COVID-19 pandemic, new methods of communication, larger amounts of research, and an evolving publication landscape have placed significant pressures on this system. A task force convened by the American College of Cardiology identified the 5 most significant controversies associated with the current peer-review process: the effect of preprints, reviewer blinding, reviewer selection, reviewer incentivization, and publication of peer reviewer comments. Although specific solutions to these issues will vary, regardless of how scientific communication evolves, peer review must remain an essential process for ensuring scientific integrity, timely dissemination of information, and better patient care. In medicine, the peer-review process is crucial because harm can occur if poor-quality data or incorrect conclusions are published. With the dramatic increase in scientific publications and new methods of communication, high-quality peer review is more important now than ever., Competing Interests: Funding Support and Author Disclosures Dr Kusumoto is chair of the Peer Review Task Force; and is chair of the ACC Scientific Publications Committee. Dr Bittl is a member of the ACC Scientific Publications Committee. Dr Creager is a member of the ACC Scientific Publications Committee. Dr Dauerman is associate editor of the Journal of the American College of Cardiology; has received research funding from Medtronic and Boston Scientific; and is a consultant for Medtronic and Boston Scientific. Dr Lala is deputy editor of the Journal of Cardiac Failure; and has received speaker honoraria from Zoll Medical and Abbott. Dr McDermott is deputy editor of JAMA; has received research funding from Regeneron and Helixmith; and has received other research support from Helixmith, ArtAssist, Hershey, Mars, Chromadex, and ResearveAge. Ms Varieur Turco is divisional senior director of publishing for the ACC. Dr Taqueti is immediate past chair of the ACC Scientific Publications Committee; and is chair of the ACC Digital Transformation Committee. Dr Fuster is editor-in-chief of the Journal of the American College of Cardiology., (Copyright © 2023 American College of Cardiology Foundation. Published by Elsevier Inc. All rights reserved.)

There is growing recognition that animal methods bias, a preference for animal-based methods where they are not necessary or where nonanimal-based methods may already be suitable, can impact the likelihood or timeliness of a manuscript being accepted for publication. Following April 2022 workshop about animal methods bias in scientific publishing, a coalition of scientists and advocates formed a Coalition to Illuminate and Address Animal Methods Bias (COLAAB). The COLAAB has developed this guide to be used by authors who use nonanimal methods to avoid and respond to animal methods bias from manuscript reviewers. It contains information that researchers may use during 1) study design, including how to find and select appropriate nonanimal methods and preregister a research plan, 2) manuscript preparation and submission, including tips for discussing methods and choosing journals and reviewers that may be more receptive to nonanimal methods, and 3) the peer review process, providing suggested language and literature to aid authors in responding to biased reviews. The author's guide for addressing animal methods bias in publishing is a living resource also available online at animalmethodsbias.org, which aims to help ensure fair dissemination of research that uses nonanimal methods and prevent unnecessary experiments on animals., (© 2023 The Authors. Advanced Science published by Wiley-VCH GmbH.)

Recognizing that very few potential reviewers and authors receive formal training on peer review, we provide guidance on peer reviewing manuscripts and on being responsive to reviewer comments. Peer review provides benefits to all parties involved. Serving as a peer reviewer gives perspective on the editorial process, fosters relationships with journal editors, gives insights into novel research, and provides a means of demonstrating topical expertise. When responding to peer reviewers, authors have the opportunity to strengthen the manuscript, sharpen the message, and address areas of potential misunderstanding. First, we provide guidance on how to peer review a manuscript. Reviewers should consider the importance of the manuscript, its rigor, and clarity of presentation. Reviewer comments should be as specific as possible. They should also be constructive and respectful in tone. Reviews typically include a list of major comments focused on methodology and interpretation and may also include a list of minor comments that pinpoint specific areas of clarification. Opinions expressed as comments to the editor are confidential. Second, we provide guidance on being responsive to reviewer comments. Authors are encouraged to approach reviewer comments as a collaboration and to view this exercise as an opportunity to strengthen their work. Response comments should be presented respectfully and systematically. The author's goal is to signal that they have engaged directly and thoughtfully with each comment. In general, when an author has questions regarding reviewer comments or how to respond, they are invited to contact the editor to review., (Copyright © 2023 Academy of Consultation-Liaison Psychiatry. Published by Elsevier Inc. All rights reserved.)

Purpose: Quality assurance (QA) is critical to the success of radiation therapy (RT) for patients with cancer and affects clinical outcomes. We report longitudinal findings of a prospective peer review evaluation system implemented at a major academic health system as part of RT QA during a 10-year period., Methods and Materials: All cases treated within our department undergo prospective multidisciplinary peer review and are assigned a grade (A, B, or C). "A" cases require no changes, "B" cases require minor modification, and "C" cases require major modification before treatment planning. The z-ratio test for the significance of the difference between the 5-year baseline (2012-2016) and follow-up (2017-2021) period was used to compare grades between the 2 periods. A 2-tailed P value <.05 was considered significant., Results: Of the 20,069 cases, 15,659 (78%) were curative and were analyzed. The fraction of A cases decreased from 74.8% (baseline) to 64.5% (follow-up), whereas B cases increased from 19.4% to 35.4% and C cases decreased from 5.8% to 0.1%. Of the 9 treatment locations, the main hospital site had a higher percentage of A grades relative to community locations in the baseline (78.6% vs 67.8%; P < .002) and follow-up (66.9% vs 62.3%; P < .002) periods. There was a decrease in the percentage of A cases from the baseline to the follow-up period regardless of plan type (complex vs intermediate vs simple). There was a decrease in the percentage of A cases among specialists from baseline to follow-up (78.2% to 67.7%; P < .002) and among generalists from baseline to follow-up (69.7% to 61.7%; P < .002)., Conclusions: Our 10-year experience in contour peer review identified increased opportunities in improving treatment plan quality over time. The drop in A scores and rise in B scores suggests increased scrutiny and findings-based improvements over time, whereas the drop in C scores indicates amelioration of "major failures" addressed in the startup years. Peer review rounds upstream of treatment planning provide valuable RT QA and should be considered by other departments to enhance the quality and consistency of RT., (Copyright © 2023 American Society for Radiation Oncology. Published by Elsevier Inc. All rights reserved.)

Background: Endoscopic retrograde cholangiopancreatography (ERCP) is a commonly performed procedure worldwide. The aim of this study was to examine cases of mortality after ERCP to identify clinical incidents that are potentially preventable, to improve patient safety., Methods: The Australian and New Zealand Audit of Surgical Mortality provides an independent and externally peer-reviewed audit of surgical mortality pertaining to potentially avoidable issues. A retrospective review of prospectively collected data within this database was performed for the 8-year audit period from 1 January 2009 to 31 December 2016. Clinical incidents were identified by assessors through first- or second-line review, and thematically coded into periprocedural stages. These themes were then qualitatively analysed., Results: There were 58 potentially avoidable deaths following ERCP, with 85 clinical incidents. Preprocedural incidents were most common (n = 37), followed by postprocedural (n = 32) and then intraprocedural (n = 8). Communication issues occurred across the periprocedural period (n = 8). Preprocedural incidents included delay to procedure, inadequate resuscitative management, decision to perform procedure and inadequate assessment. Intraprocedural incidents comprised technical factors and inadequate support. Postprocedural incidents involved inappropriate treatment, delay in definitive surgical treatment or in recognizing complications, inappropriate second-line intervention and inadequate assessment. Communication incidents comprised inadequate documentation, failure to escalate care and poor inter-clinician communication., Conclusion: Causes of mortality following ERCP are wide-ranging, and reviewing clinical incidents associated with potentially avoidable mortality can serve to inform and educate practitioners. In collating a subset of cases in which procedure-related mortality was deemed avoidable, a series of cautionary tales about ERCP is presented that may provide cues to practitioners on improving patient safety and inform future surgical practice., (© 2023 The Authors. ANZ Journal of Surgery published by John Wiley & Sons Australia, Ltd on behalf of Royal Australasian College of Surgeons.)

Background and Purpose: Peer evaluations are often utilized to allow student pharmacists practice in giving and receiving feedback. In a small class setting, these can easily be completed and feedback distributed quickly. However, in the larger class setting, reviewing and disseminating peer feedback can be quite cumbersome, especially if using paper format. The purpose of this educational activity was to create a process for peer evaluations that allows for efficient collection and dissemination of peer feedback of presentations of student pharmacists and describe the student experience with this new format., Educational Activity and Setting: In Research Topics in Pharmacy II, an electronic peer-evaluation tool was created using electronic examination software to collect and distribute this peer review in a timely fashion during and after each class session. At the completion of this course, a survey was distributed to collect student pharmacists' perception of this electronic peer-review process., Findings: A total of 63 of 91 students (69%) completed the survey. The majority of the students (98.4%) "strongly agreed" or "agreed" the peer-evaluation items made it easy to provide feedback to their peers and 79% preferred this electronic method of feedback vs. paper format. Overall, 93.6% of student pharmacists felt they were more engaged during the presentations as a result of providing electronic feedback., Summary: Maximizing our resources by creating an electronic peer evaluation with our current examination software, allowed for an efficient means of obtaining and disseminating peer review that was timely and well-received by students., (Copyright © 2021 Elsevier Inc. All rights reserved.)

Objective: To perform the first systematic review of all available gender-affirming surgery (GAS) publications across all procedures to assess both outcomes reported in the literature and the methods used for outcome assessment., Summary of Background Data: Rapidly increasing clinical volumes of gender-affirming surgeries have stimulated a growing need for high-quality clinical research. Although some procedures have been performed for decades, each individual procedure has limited data, necessitating synthesis of the entire literature to understand current knowledge and guide future research., Methods: A systematic review was performed following Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines to identify all outcomes measures in GAS cohorts, including PCOs, complications, and functional outcomes. Outcome data were pooled to assess currently reported complication, satisfaction, and other outcome rates., Results: Overall, 15,186 references were identified, 4162 papers advanced to abstract review, and 1826 underwent full-text review. After review, there were 406 GAS cohort publications. Of non-genitoplasty titles, 35 were mastectomy, 6 mammoplasty, 21 facial feminization, and 31 voice/cartilage. Although 59.1% of non-genitoplasty papers addressed PCOs in some form, only 4.3% used instruments partially-validated in transgender patients. Overall, data were reported heterogeneously and were biased towards high-volume centers., Conclusions: This study represents the most comprehensive review of GAS literature. By aggregating all previously utilized measurement instruments, this study offers a foundation for discussions about current methodologic limitations and what dimensions must be included in assessing surgical success. We have assembled a comprehensive list of outcome instruments; this offers an ideal starting basis for emerging discussions between patients and providers about deficiencies which new, better instruments and metrics must address. The lack of consistent use of the same outcome measures and validated GAS-specific instruments represent the 2 primary barriers to high-quality research where improvement efforts should be focused., Competing Interests: The authors declare no conflict of interests., (Copyright © 2021 Wolters Kluwer Health, Inc. All rights reserved.)

Background: The COVID-19 pandemic exposed racism as a public health crisis embedded in structural processes. Editors of surgical research journals pledged their commitment to improve structure and process through increasing diversity in the peer review and editorial process; however, little benchmarking data are available., Methods: A survey of editorial board members from high impact surgical research journals captured self-identified demographics. Analysis of manuscript submissions from 2016 to 2020 compared acceptance for diversity, equity, and inclusion (DEI)-focused manuscripts to overall rates., Results: 25.6% of respondents were female, 2.9% Black, and 3.3% Hispanic. There was variation in the diversity among journals and in the proportion of DEI submissions they attract, but no clear correlation between DEI acceptance rates and board diversity., Conclusions: Diversity among board members reflects underrepresentation of minorities seen among surgeons nationally. Recruitment and retention of younger individuals, representing more diverse backgrounds, may be a strategy for change. DEI publication rates may benefit from calls for increasing DEI scholarship more so than changes to the peer review process., (Copyright © 2021 Elsevier Inc. All rights reserved.)

Purpose: Recent calls to improve transparency in peer review have prompted examination of many aspects of the peer-review process. Peer-review systems often allow confidential comments to editors that could reduce transparency to authors, yet this option has escaped scrutiny. Our study explores 1) how reviewers use the confidential comments section and 2) alignment between comments to the editor and comments to authors with respect to content and tone., Methods: Our dataset included 358 reviews of 168 manuscripts submitted between January 1, 2019 and August 24, 2020 to a health professions education journal with a single blind review process. We first identified reviews containing comments to the editor. Then, for the reviews with comments, we used procedures consistent with conventional and directed qualitative content analysis to develop a coding scheme and code comments for content, tone, and section of the manuscript. For reviews in which the reviewer recommended "reject," we coded for alignment between reviewers' comments to the editor and to authors. We report descriptive statistics., Results: 49% of reviews contained comments to the editor (n = 176). Most of these comments summarized the reviewers' impression of the article (85%), which included explicit reference to their recommended decision (44%) and suitability for the journal (10%). The majority of comments addressed argument quality (56%) or research design/methods/data (51%). The tone of comments tended to be critical (40%) or constructive (34%). For the 86 reviews recommending "reject," the majority of comments to the editor contained content that also appeared in comments to the authors (80%); additional content tended to be irrelevant to the manuscript. Tone frequently aligned (91%)., Conclusion: Findings indicate variability in how reviewers use the confidential comments to editor section in online peer-review systems, though generally the way they use them suggests integrity and transparency to authors., Competing Interests: Three authors have editorial roles in Perspectives on Medical Education: Erik Driessen (Editor-in-Chief), Lauren Maggio (Deputy Editor-in-Chief), and Anthony Artino (Associate Editor). Driessen and Maggio receive an honorarium for their editorial roles. This does not alter our adherence to PLOS ONE policies on sharing data and materials.

Competing Interests: Competing interests: We have read and understood The BMJ policy on declaration of interests and have the following interests to declare: All authors declare no conflict of interest. The Secretariat of the Independent Panel for Pandemic Preparedness and Response is independent.

Research suggests that scientists based at prestigious institutions receive more credit for their work than scientists based at less prestigious institutions, as do scientists working in certain countries. We examined the extent to which country- and institution-related status signals drive such differences in scientific recognition. In a preregistered survey experiment, we asked 4,147 scientists from six disciplines (astronomy, cardiology, materials science, political science, psychology and public health) to rate abstracts that varied on two factors: (i) author country (high status vs lower status in science); (ii) author institution (high status vs lower status university). We found only weak evidence of country- or institution-related status bias, and mixed regression models with discipline as random-effect parameter indicated that any plausible bias not detected by our study must be small in size., Competing Interests: MN, CB, EB, MP, JA No competing interests declared, (© 2021, Nielsen et al.)

Retractions of COVID-19 literature in both preprints and the peer-reviewed literature serve as a reminder that there are still challenging issues underlying the integrity of the biomedical literature. The risks to academia become larger when such retractions take place in high-ranking biomedical journals. In some cases, retractions result from unreliable or nonexistent data, an issue that could easily be avoided by having open data policies, but there have also been retractions due to oversight in peer review and editorial verification. As COVID-19 continues to affect academics and societies around the world, failures in peer review might also constitute a public health risk. The effectiveness by which COVID-19 literature is corrected, including through retractions, depends on the stringency of measures in place to detect errors and to correct erroneous literature. It also relies on the stringent implementation of open data policies.

This Perspective presents comments intended for junior researchers by Carol A. Lange, Editor-in-Chief, Endocrinology, and Stephen R. Hammes, former Editor-in-Chief, Molecular Endocrinology, and former co-Editor-in-Chief, Endocrinology., Principal Points: 1. Know when you are ready and identify your target audience.2. Select an appropriate journal.3. Craft your title and abstract to capture your key words and deliver your message.4. Tell a clear and impactful story.5. Review, polish, and perfect your manuscript., (Published by Oxford University Press on behalf of the Endocrine Society 2021.)

Background: While teaching evaluation systems are common in academia, very little information is available regarding formal coaching and peer review of teaching performance in surgery. This article is a report on the development and implementation of a peer review of operative teaching program., Methods: Our process was designed using a multistep sequential model which included developing a peer review of teaching instrument that was piloted to study the efficacy and utility of the tool., Results: Thirty-nine peer reviews of teaching were conducted. Among the most frequent challenges that faculty identified were allowing residents to struggle/give autonomy, judging when to take over the case, communicating effectively, being patient, balancing education and patient safety, and giving feedback., Conclusions: Our peer review of teaching program is systematic, feasible, and can be adopted by other surgery departments. Faculty's identified strengths and challenges have been incorporated into our faculty development curricula., (Copyright © 2020 Elsevier Inc. All rights reserved.)

Objective:  To estimate the human resources required for a retrospective quality review of different percentages of all routine diagnostic procedures in the Department of Radiology at Bern University Hospital, Switzerland., Materials and Methods:  Three board-certified radiologists retrospectively evaluated the quality of the radiological reports of a total of 150 examinations (5 different examination types: abdominal CT, chest CT, mammography, conventional X-ray images and abdominal MRI). Each report was assigned a RADPEER score of 1 to 3 (score 1: concur with previous interpretation; score 2: discrepancy in interpretation/not ordinarily expected to be made; score 3: discrepancy in interpretation/should be made most of the time). The time (in seconds, s) required for each review was documented and compared. A sensitivity analysis was conducted to calculate the total workload for reviewing different percentages of the total annual reporting volume of the clinic., Results:  Among the total of 450 reviews analyzed, 91.1 % (410/450) were assigned a score of 1 and 8.9 % (40/450) were assigned scores of 2 or 3. The average time (in seconds) required for a peer review was 60.4 s (min. 5 s, max. 245 s). The reviewer with the greatest clinical experience needed significantly less time for reviewing the reports than the two reviewers with less clinical expertise (p < 0.05). Average review times were longer for discrepant ratings with a score of 2 or 3 (p < 0.05). The total time requirement calculated for reviewing all 5 types of examination for one year would be more than 1200 working hours., Conclusion:  A retrospective peer review of reports of radiological examinations using the RADPEER system requires considerable human resources. However, to improve quality, it seems feasible to peer review at least a portion of the total yearly reporting volume., Key Points:   · A systematic retrospective assessment of the content of radiological reports using the RADPEER system involves high personnel costs.. · The retrospective assessment of all reports of a clinic or practice seems unrealistic due to the lack of highly specialized personnel.. · At least part of all reports should be reviewed with the aim of improving the quality of reports.., Citation Format: · Maurer MH, Brönnimann M, Schroeder C et al. Time Requirement and Feasibility of a Systematic Quality Peer Review of Reporting in Radiology. Fortschr Röntgenstr 2021; 193: 160 - 167., Competing Interests: The authors declare that they have no conflict of interest., (Thieme. All rights reserved.)

Objectives: A measurement tool to assess systematic reviews (SRs) 2 (AMSTAR 2) allows for deriving the overall confidence in an SR. We investigated how authors derived the overall confidence rating and whether different schemes lead to different results., Study Design and Setting: We compared three different schemes (original 7-item scheme, a self-developed 5-item scheme, and the AMSTAR Web site) to derive the overall confidence in AMSTAR 2 using two distinct samples of SRs. Multiple bibliographic databases were searched for articles to analyze how AMSTAR 2 was applied by others., Results: In both samples (n = 60 and n = 58), the Friedman test revealed a significant difference between the schemes (P < 0.001). The Web site scheme was the least strict one, whereas between the 5-item and 7-item scheme, no differences were found in post hoc analyses. We included 53 publications applying AMSTAR 2 identified in our literature search. Only 37 of them (70%) used the original 7-item scheme. Less than half of them (18 of 37) reported how they derived the overall rating., Conclusion: Authors should clearly report how they have derived the overall rating when applying AMSTAR 2. Reporting should allow for reproducing the overall ratings for editors, peer reviewers, and readers., (Copyright © 2020 Elsevier Inc. All rights reserved.)

Competing Interests: Competing interests: The author reports no conflicts of interest.

Objective: Based on a national survey of program directors we developed a letter of recommendation (LOR) scoring rubric (SR) to assess LORs submitted to a pediatric residency program. The objective was to use the SR to analyze: the consistency of LOR ratings across raters and LOR components that contributed to impression of the LOR and candidate., Methods: We graded 30 LORs submitted to a pediatric residency program that were evenly distributed based on final rank by our program. The SR contained 3 sections (letter features, phrases, and applicant abilities) and 2 questions about the quality of the LOR (LORQ) and impression of the candidate (IC) after reading the LOR on a 5-point Likert scale. Inter-rater reliability was calculated with intraclass correlation coefficients (ICC(2,1)). Pearson (r) correlations and stepwise multivariate linear regression modeling predicted LORQ and IC. Mean scores of phrases, features, and applicant abilities were analyzed with ANOVA and Bonferroni correction., Results: Phrases (ICC(2,1) = 0.82, p<0.001)) and features (ICC(2,1) = 0.60, p<0.001)) were rated consistently, while applicant abilities were not (ICC(2,1) = 0.28, p<0.001)). For features, LORQ (R2 = 0.75, p<0.001) and IC (R2 = 0.58, p<0.001) were best predicated by: writing about candidates' abilities, strength of recommendation, and depth of interaction with the applicant. For abilities, LORQ (R2 = 0.47, p<0.001) and IC (R2 = 0.51, p<0.001) were best predicted by: clinical reasoning, leadership, and communication skills (0.2). There were significant differences for phrases and features (p<0.05)., Conclusions: The SR was consistent across raters and correlates with impression of LORQ and IC. This rubric has potential as a faculty development tool for writing LORS., Competing Interests: The authors have declared that no competing interests exist.

The coronavirus disease 2019 pandemic has affected business on numerous fronts in unprecedented and abrupt ways. From site closures and local "stay-at-home orders" to travel advisories and restrictions, the day-to-day practice of toxicologic pathology has been impacted dramatically and rapidly. A critical function of Toxicologic Pathologists is performing pathology peer review for nonclinical studies. Traditionally, corroborating the findings of histological assessment could be achieved through shipment of histopathological slides to the peer review pathologist, or by the peer review pathologist traveling to the location of the slides (eg, the test facility). Since early 2020, many pathologists have been unable to perform the latter due to local, regional, national, test facility, company, and/or personal restrictions. The disruption for some has been minimal, while others are working from home for the first time. We recommend that contingency plans for all peer review procedures and personnel should be in-place to accommodate sudden and unexpected workflow transitions. Now, more than ever, approaching peer reviews with enhanced adaptability will help ensure success.

Purpose: While peer review is critical for quality and safety in radiotherapy, there are neither formal guidelines nor format examples for brachytherapy (BT) peer review. We report on a gynecologic BT peer-review method implemented at a high-volume academic center., Methods and Materials: We analyzed discussions at bimonthly gynecologic BT peer-review rounds between July and December 2018. Rounds consisted of 2-5 attending physicians with gynecologic BT expertise, 1-2 BT physicists, and trainees. Peer-review targets included clinical case review, contours, implant technique, dose/fractionation, and target/organ-at-risk (OAR) dosimetry. The projected/final target and OAR dosimetry were analyzed., Results: 55 separate implants from 44 patients were reviewed. Implants were mostly reviewed after the first BT fraction (n = 16, 29%) or at another time point during BT (n = 20, 36%). One (2%) implant was presented prospectively. The applicator type and BT technique were reviewed for all implants. Dose/fractionation was evaluated for 46 implants (84%); contours were discussed for 21 (38%). Target and OAR dosimetry were reviewed for 54 (98%) and 28 implants (51%), respectively. Six cases (11%) underwent minor changes to the applicator type to improve target and/or OAR dosimetry. One case (2%) had a major change recommended to the dose/fractionation., Conclusions: Gynecologic BT peer review may enhance BT quality by allowing for implant optimization and formal review of challenging cases, ultimately improving medical decision-making and team communication. Peer review should be implemented in centers offering gynecologic BT., (Published by Elsevier Inc.)

Nursing Peer Review is a foundational and essential element of professional nursing practice. It is a systematic methodology to improve nurse and patient outcomes. The process can be labor-intensive and cumbersome in managing data from diverse data sources, especially if the process is manual. Directors of Professional Practice in a health care system partnered with an external vendor to create an interactive software platform where technology was leveraged to streamline the review process including review of aggregate data and trend analyses and generate reports using an electronic database. This resulted in a 75% reduction in the number of steps and subsequently the time required to complete the review process from initial screening to referral and closure. The generation of actionable data facilitated active engagement of clinical nurses in addressing identified clinical issues using process improvement and evidence-based practice methods. A critical feature of the software platform is that it provides actionable data that can be used to improve patient safety and fosters accountability for clinical nurses to promote self-regulation of nursing practice.

The aim of this systematic review is to provide a narrative synthesis of the peer-reviewed literature regarding the role of psychodynamics in community psychology. The authors screened 301 records on the topic, found in major citation databases (Scopus and Web of Science) without time or language restrictions. Ten articles addressing the review question were identified, showing the contributions of interpersonal psychoanalysis, Adlerian psychology, the Tavistock psychodynamic model, and Lacanian psychoanalysis. Several points of synergy between community psychology and psychodynamics were outlined mainly concerning empowerment theory, preventative and ecological perspective, power, and social order. Besides, the view on the community life, the role of emotion, and the conceptualization of the unconscious domain are discussed. Implications for community interventions are highlighted, regarding clients' demands, the role of community practitioners, and the use of transference/countertransference in consultative work. Limitations and future directions are also considered., (© 2020 Wiley Periodicals LLC.)

Purpose: To present the longitudinal results of a prospective peer review evaluation system (PES) before treatment planning., Methods and Materials: All cases undergoing radiation therapy (RT) at high-volume academic institutions were graded in daily prospective multidisciplinary contouring rounds (CRs). The clinical suitability for RT, prescription, contours, and written directives were peer reviewed, compared with departmental care pathways, and recorded in a prospective database. Grades were assigned as follows: A (score 4.0) = no deficiencies; B (3.0) = minor modifications of the planning target volume, organs at risk, written directives, or a prescription/care pathway mismatch; and C (2.0) = incomplete target volume or organ-at-risk contours, unsuitable use or inappropriate planned administration of RT, significant contour modifications, prescription changes, or laterality modifications. Information was pooled to determine pretreatment planning work performance by assigning a grade point average (GPA) for each physician as well as compositely., Results: A total of 11,843 treatment plans from 7854 patients were reviewed using the PES from September 2013 to May 2018. Twenty-seven point nine percent of cases (n = 3303) required modifications before treatment planning commenced. The overall breakdown of grades was 72.1% As, 21.7% Bs, and 6.2% Cs. The median physician CR GPA was 3.60 (average 3.7) with a range of 3.0 to 3.9. Seventy-five percent of physicians demonstrated improvement of their CR GPA since inception of the program, and all physicians demonstrated a drop in the percentage of cases that were assigned a grade of C., Conclusions: The PES can transparently quantify clinical performance in a single metric. The PES was impactful, with 75% of physicians demonstrating improvement in their CR GPA over time. In contrast to traditional chart rounds, this peer review was meaningful when done before planning commenced, a trend that was observed throughout the study period. Twenty-seven point nine percent of all cases required modification before starting treatment planning, and 6.2% of cases required significant remediation., (Copyright © 2019 American Society for Radiation Oncology. Published by Elsevier Inc. All rights reserved.)

Objective: To develop a tool to assess the quality of peer-review reports in biomedical research., Methods: We conducted an online survey intended for biomedical editors and authors. The survey aimed to (1) determine if participants endorse the proposed definition of peer-review report quality; (2) identify the most important items to include in the final version of the tool and (3) identify any missing items. Participants rated on a 5-point scale whether an item should be included in the tool and they were also invited to comment on the importance and wording of each item. Principal component analysis was performed to examine items redundancy and a general inductive approach was used for qualitative data analysis., Results: A total of 446 biomedical editors and authors participated in the survey. Participants were mainly male (65.9%), middle-aged (mean=50.3, SD=13) and with PhD degrees (56.4%). The majority of participants (84%) agreed on the definition of peer-review report quality we proposed. The 20 initial items included in the survey questionnaire were generally highly rated with a mean score ranging from 3.38 (SD=1.13) to 4.60 (SD=0.69) (scale 1-5). Participants suggested 13 items that were not included in the initial list of items. A steering committee composed of five members with different expertise discussed the selection of items to include in the final version of the tool. The final checklist includes 14 items encompassed in five domains (Importance of the study, Robustness of the study methods, Interpretation and discussion of the study results, Reporting and transparency of the manuscript, Characteristics of peer reviewer's comments)., Conclusion: Assessment of Review reports with a Checklist Available to eDItors and Authors tool could be used regularly by editors to evaluate the reviewers' work, and also as an outcome when evaluating interventions to improve the peer-review process., Competing Interests: Competing interests: None declared., (© Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY. Published by BMJ.)