Search

Introduction: Despite the importance of peer review to publications, there is no generally accepted approach for editorial evaluation of a peer review’s value to a journal editor’s decision-making. The graduate medical education editors of the Western Journal of Emergency Medicine Special Issue in Educational Research & Practice (Special Issue) developed and studied the holistic editor’s scoring rubric (HESR) with the objective of assessing the quality of a review and an emphasis on the degree to which it informs a holistic appreciation for the submission under consideration.Methods: Using peer-review guidelines from several journals, the Special Issue’s editors formulated the rubric as descriptions of peer reviews of varying degree of quality from the ideal to the unacceptable. Once a review was assessed by each editor using the rubric, the score was submitted to a third party for blinding purposes. We compared the performance of the new rubric to a previously used semantic differential scale instrument. Kane’s validity framework guided the evaluation of the new scoring rubric around three basic assumptions: improved distribution of scores; relative consistency rather than absolute inter-rater reliability across editors; and statistical evidence that editors valued peer reviews that contributed most to their decision-making.Results: Ninety peer reviews were the subject of this study, all were assessed by two editors. Compared to the highly skewed distribution of the prior rating scale, the distribution of the new scoring rubric was bell shaped and demonstrated full use of the rubric scale. Absolute agreement between editors was low to moderate, while relative consistency between editor’s rubric ratings was high. Finally, we showed that recommendations of higher rated peer reviews were more likely to concur with the editor’s formal decision.Conclusion: Early evidence regarding the HESR supports the use of this instrument in determining the quality of peer reviews as well as its relative importance in informing editorial decision-making.

Introduction: Historically, there have been no systematic programs for teaching peer review, leaving trainees to learn by trial and error. Recently, a number of publications have advocated for programs where experienced reviewers mentor trainees to more efficiently acquire this knowledge.Objective: Our goal was to develop an introductory learning experience that intentionally fosters peer-review skills.Methods: The Council of Residency Directors in Emergency Medicine (CORD) offered education fellowship directors the opportunity to mentor their fellows by reviewing submitted manuscript(s) supplemented by educational material provided by their journal. Reviews were collaboratively created. The decision letter that was sent to manuscript authors was also sent to the mentees; it included all reviewers’ and editor’s comments, as feedback. In 2022, fellows received a post-experience survey regarding prior experiences and their perspectives of the mentored peer-review experience. Results: From 2020–2022, participation grew from 14 to 30 education fellowships, providing 76 manuscript peer reviews. The 2022 survey-response rate of 87% (20/23) revealed that fellows were inexperienced in education scholarship prior to participation: 30% had authored an education paper, and 10% had performed peer review of an education manuscript. Overall, participants were enthusiastic about the program and anxious to participate the following year. In addition, participants identified a number of benefits of the mentored experience including improved understanding of the scholarship process; informing fellows’ scholarly pursuits; improved conceptualization of concepts learned elsewhere in training; and learning through exposure to scholarship. Conclusion: This program’s early findings suggest that collaboration between academic societies and interested graduate medical education faculty has the potential to formalize the process of learning peer review, benefitting all involved stakeholders.

Introduction: In 2015, with a stated goal of disseminating best teaching practices and developing a community of educational scholars, the Council of Emergency Medicine Directors (CORD) and th eClerkship Directors of Emergency Medicine (CDEM) created an annual Special Issue in Educational Research and Practice (Special Issue) in cooperation with the Western Journal of Emergency Medicine. The intention of this study was to analyze the impact of this effort to date.Methods: Bibliometric data was gathered on all four special issues, 2015-2019, from the Web of Scienceand then verified with the eScholarship website. Authorship, academic affiliation, date published, articletype, and format were tabulated for descriptive analysis. Using metrics from Google Scholar, alternative scholarly impact metrics (altmetrics), and the eScholarship website, the authors identified top articles and grouped them into themes.Results: Of the 136 articles included in the first four years of the Special Issue, 126 represented peer reviewed publications with an overall acceptance rate of 25.0% (126/505). Authors from this cohort represented 103 of the 182 (56.6%) Accreditation Council for Graduate Medical Education (ACGME) programs in existence at the time of the inaugural issue. Multi-institutional studies represented 34.9% (44/126) of the peer-reviewed publications. Traditional and alternative publication metrics are reported to assess the impact of articles from the Special Issues.Conclusion: The Special Issue is a proven outlet to share best practices, innovations, and research related to education. Additionally, the infrastructure of this process promotes the development of individual faculty and a community of teaching scholars.

Introduction: The emergency medicine clerkship director serves an important role in the education of medical students. The authors sought to update the demographic and academic profile of the emergency medicine clerkship director. Methods: We developed and implemented a comprehensive questionnaire, and used it to survey all emergency medicine clerkship directors at United States allopathic medical schools accredited by the Liaison Committee on Medical Education. We analyzed and interpreted data using descriptive statistics.Results: One hundred seven of 133 (80.4%) emergency medicine clerkship directors completed the survey. Clerkship Director’s mean age was 39.7 years (SD-7.2), they were more commonly male 68.2%, of Caucasian racial backgrounds and at the instructor or assistant professor (71.3%) level. The mean number of years of experience as clerkship director was 5.5 (SD-4.5). The mean amount of protected time for clerkship administration reported by respondents was 7.3 hours weekly (SD-5.1), with the majority (53.8%) reporting 6 or more hours of protected time per week. However, 32.7% of emergency medicine clerkship directors reported not having any protected time for clerkship administration. Most clerkship directors (91.6%) held additional teaching responsibilities beyond their clerkship and many were involved in educational research (49.5%). The majority (79.8%), reported being somewhat or very satisfied with their job as clerkship director. Conclusion: Most clerkship directors were junior faculty at the instructor or assistant professor rank and were involved with a variety of educational endeavors beyond the clerkship. [West J Emerg Med. 2014;15(4):498–403.]

Objective: To use 360-degree evaluations within an Observed Structured Clinical Examination (OSCE) to assess medical student comfort level and communication skills with intimate partner violence (IPV) patients.Methods: We assessed a cohort of fourth year medical students’ performance using an IPV standardized patient (SP) encounter in an OSCE. Blinded pre- and post-tests determined the students’ knowledge and comfort level with core IPV assessment. Students, SPs and investigators completed a 360-degree evaluation that focused on each student’s communication and competency skills. We computed frequencies, means and correlations.Results: Forty-one students participated in the SP exercise during three separate evaluation periods. Results noted insignificant increase in students’ comfort level pre-test (2.7) and post-test (2.9). Although 88% of students screened for IPV and 98% asked about the injury, only 39% asked about verbal abuse, 17% asked if the patient had a safety plan, and 13% communicated to the patient that IPV is illegal. Using Likert scoring on the competency and overall evaluation (1, very poor and 5, very good), the mean score for each evaluator was 4.1 (competency) and 3.7 (overall). The correlations between trainee comfort level and the specific competencies of patient care, communication skill and professionalism were positive and significant (p

Introduction: We describe our adaptation of the team strategies and tools to enhance performance and patient safety (TeamSTEPPS) for use as an educational intervention for medical and nursing students. We hypothesized that participation in a team training program using the modified TeamSTEPPS program would positively affect knowledge and attitudes toward teamwork skills and their ability to recognize the presence and quality of team skills., Methods: Two hundred thirteen students participated in a 4-hour team training program that included a lecture followed by small group team training exercises. Nurse/physician pairs facilitated student team activities. Knowledge and attitudes were assessed before and after the educational intervention. Recognition of team skills was assessed using videos., Results: Statistically significant differences were found with participant knowledge (P<0.001) and attitude (P=0.004). Students were able to identify the presence and quality of team skills in the video vignettes. Overwhelmingly, the students recognized team skills much better in the success video than in the opportunity video (P<0.001), and they rated the success video much higher than the opportunity video (P<0.001) with regards to the quality of team skills., Conclusions: We successfully adapted TeamSTEPPS for use with our medical and nursing students, educated facilitators, and implemented the course. Our students improved their knowledge of vital team and communication skills, attitudes toward working as teams, and were able to identify effective team skills.