Search

The increasing use of technology in health care and health professions education is an invitation to examine how digital sources of evidence are used in making assessment claims. In this paper, we describe how four sets of terms-primary and secondary data; structured and unstructured data; development and use; and deterministic and generative-can aid in examining how data from digital sources are used in evaluating what learners know and can do. Drawing on multiple examples, this paper shows how the four sets of terms can help both developers and users of technology-based assessment systems., Competing Interests: Dr. Krumm receives research support from the Society for Improving Medical Professional Learning (SIMPL). Dr. George is the Executive Director of the Society for Improving Medical Professional Learning (SIMPL). This position is unpaid. Dr. Barone is a former employee of NBME. Dr. Smirnova received compensation from the Kern Institute for the Transformation of Medical Education, Medical College of Wisconsin, for her time helping coordinate the Next Era in Assessment meeting in Milwaukee, Wisconsin., (Copyright: © 2024 The Author(s).)

Background: Precision health is a burgeoning scientific discipline that aims to incorporate individual variability in biological, behavioral, and social factors to develop personalized health solutions. To date, emergency medicine has not deeply engaged in the precision health movement. However, rapid advances in health technology, data science, and medical informatics offer new opportunities for emergency medicine to realize the promises of precision health., Methods: In this article, we conceptualize precision emergency medicine as an emerging paradigm and identify key drivers of its implementation into current and future clinical practice. We acknowledge important obstacles to the specialty-wide adoption of precision emergency medicine and offer solutions that conceive a successful path forward., Results: Precision emergency medicine is defined as the use of information and technology to deliver acute care effectively, efficiently, and authentically to individual patients and their communities. Key drivers and opportunities include leveraging human data, capitalizing on technology and digital tools, providing deliberate access to care, advancing population health, and reimagining provider education and roles. Overcoming challenges in equity, privacy, and cost is essential for success. We close with a call to action to proactively incorporate precision health into the clinical practice of emergency medicine, the training of future emergency physicians, and the research agenda of the specialty., Conclusions: Precision emergency medicine leverages new technology and data-driven artificial intelligence to advance diagnostic testing, individualize patient care plans and therapeutics, and strategically refine the convergence of the health system and the community., (© 2024 The Author(s). Academic Emergency Medicine published by Wiley Periodicals LLC on behalf of Society for Academic Emergency Medicine.)

Purpose: Written assessment comments are needed to archive feedback and inform decisions. Regrettably, comments are often impoverished, leaving performance-relevant information undocumented. Research has focused on content and supervisor's ability and motivation to write it but has not sufficiently examined how well the undocumented information lends itself to being written as comments. Because missing information threatens the validity of assessment processes, this study examined the performance information that resists being written., Method: Two sequential data collection methods and multiple elicitation techniques were used to triangulate unwritten assessment comments. Between November 2022 and January 2023, physicians in Canada were recruited by email and social media to describe experiences with wanting to convey assessment information but feeling unable to express it in writing. Fifty supervisors shared examples via survey. From January to May 2023, a subset of 13 participants were then interviewed to further explain what information resisted being written and why it seemed impossible to express in writing and to write comments in response to a video prompt or for their own "unwritable" example. Constructivist grounded theory guided data collection and analysis., Results: Not all performance-relevant information was equally writable. Information resisted being written as assessment comments when it would require an essay to be expressed in writing, belonged in a conversation and not in writing, or was potentially irrelevant and unverifiable. In particular, disclosing sensitive information discussed in a feedback conversation required extensive recoding to protect the learner and supervisor-learner relationship., Conclusions: When documenting performance information as written comments is viewed as an act of disclosure, it becomes clear why supervisors may feel compelled to leave some comments unwritten. Although supervisors can be supported in writing better assessment comments, their failure to write invites a reexamination of expectations for documenting feedback and performance information as written comments on assessment forms., (Copyright © 2024 the Association of American Medical Colleges.)

Objective: Preliminary surgery positions are associated with a negative stigma, and this stigma may persist for those residents who later go on to obtain a categorical position. However, it is currently unknown if general surgery residents who complete a preliminary year perform differently than their categorical peers throughout training. To examine these potential differences, we compared operative performance and autonomy across all 5 years of training for those who completed a nondesignated preliminary training year with those who did not., Design: Faculty ratings of categorical general surgery residents were collected from the Society for Improving Medical and Professional Learning (SIMPL) application. Residents were categorized based on the completion of a nondesignated preliminary year ("Previous Prelim" [PP]) or not ("Traditional Categorical" [TC]). Operative performance and autonomy ratings were tracked over 5 training years and analyzed using generalized mixed effects models. Performance and autonomy outcomes were dichotomized: "not competent" or "competent" and "no meaningful autonomy" or "meaningful autonomy", respectively. Fixed effects included cohort group, academic month, case complexity, resident level (junior [PGYs 1 and 2] vs senior [PGYs 3, 4, and 5] residents), and an interaction term between cohort group and resident level, while random effects included resident, faculty, program, and procedure., Setting: Operative performance and autonomy ratings were collected from the SIMPL application between 2015 and 2023., Participants: A total of 89 general surgery residency programs, which encompassed 1,108 categorical general surgery residents., Results: A total of 42,416 evaluations were analyzed. Of these, 809 evaluations were from PP residents. Faculty ratings of PP vs TC residents revealed no significant differences in adjusted probabilities of achieving a "competent" rating as a junior (5.6%, 95% Confidence Interval [CI] 2.7%-11.1% vs 3.0%, 95% CI 2.4%-3.7%, p = 0.28) or senior (23.7%, 95% CI 12.2%-41% vs 34.1%, 95% CI 29.1%-39.5%, p = 0.59) resident. Similarly, no significant differences were identified in adjusted probabilities of achieving a "meaningful autonomy" rating between PP and TC junior (9.0%, 95% CI 5.4%-14.6% vs 8.1%, 95% CI 6.9%-9.5%, p = 0.97) or senior (42.3%, 95% CI 28.7%-57.1% vs 49.6%, 95% CI 44.9%-54.2%, p = 0.76) residents., Conclusions: PP and TC residents may have similar operative performance and autonomy ratings throughout their residency training. Although further study of more senior residents is warranted, these initial findings could help combat the negative stigma associated with a preliminary year of training., (Copyright © 2024 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.)

Competency-based medical education (CBME) has produced large collections of data, which can provide valuable information about trainees and medical education systems. Many organizations continue to struggle with accessing, collecting, governing, analyzing, and visualizing their clinical and/or educational data. This hinders data sharing efforts within and across organizations, which are foundational in supporting system-wide improvements. Challenges to data sharing within medical education include variability in legislation, existing data policies, heterogeneity of data, inadequate data infrastructure, and various intended purposes or uses. In this eye opener, the authors describe four case studies to illustrate some of the aforementioned challenges and characterize the complexity of data sharing within medical education along two dimensions: organizational (single vs. multiple) and data type (clinical and/or educational). With the goal of better supporting data sharing initiatives, the authors introduce an action-oriented blueprint that includes a three-stage process (i.e., preparation, execution, and iteration) to highlight crucial aspects of data sharing. This evidence-informed model incorporates current best practices and aims to support data sharing initiatives within their own organizations and across multiple organizations. Finally, organizations can use this model to conceptually guide and track their progression throughout the data sharing process., Competing Interests: Dr. George is the Executive Director of the Society for Improving Medical Professional Learning (SIMPL). This position is unpaid., (Copyright: © 2024 The Author(s).)

Objectives: Emergency physicians without specialized Emergency Medical Services (EMS) training are often required to provide online medical oversight. One common ethical question faced by these physicians is the assessment for decision-making capacity in a patient who does not accept EMS transport to the hospital. We sought expert consensus for a standardized set of guiding questions and recommendations to ensure a rigorous and feasible capacity assessment., Methods: A modified Delphi method approach was used to achieve group consensus among expert individuals. Nineteen physician experts were recruited from across the country, representing populations totaling over 22 million and a variety of urban, suburban, and rural practice environments. Experts completed a Round 1 survey that included 19 questions surrounding best practices for capacity evaluation among patients refusing transport. The threshold for consensus was predefined as 80% agreement. Participants gathered virtually meeting where the results from the first round were shared with the group. Discussion generated new items and refined the language of existing items. Following the virtual meeting, a Round 2 survey was conducted, and voted on by the panel for the items that did not meet consensus in Round 1., Results: After the first round, 15 of 19 items reached consensus. Three of the items that met consensus were universally noted to require language modification for clarification. A large portion of the discussion involved the proper method of integrating patient concerns around ambulance transport (e.g., cost of transport, financial concerns, social barriers) into the capacity assessment and whether alternate care options should be discussed. After the second round of voting, one additional item was reversed to meet consensus, resulting in a total of 16 items., Conclusions: A consensus expert panel was able to agree upon 16 standardized steps to guide best practices and assist emergency physicians in real-time evaluation of patients that refuse EMS transport.

Objectives: Women remain underrepresented in the emergency medicine (EM) workforce, academic EM, and institutional leadership. In order to support women physicians in EM, we must explore factors that contribute to attrition and workplace satisfaction. For example, tensions between workplace and familial roles are important to consider as women navigate careers in EM. The logistics and stressors of workplace lactation pose a particular challenge during an already stressful time for a new mother returning to work in a busy emergency department (ED), but limited empirical data exist regarding this experience. We aimed to explore the stressors associated with workplace lactation spaces in order to better inform the creation of lactation spaces for individuals working in EDs., Methods: Our team used an exploratory qualitative design to investigate lactation-specific stressors and understand their relationship to individuals' needs when lactating in EM workplace environments. A total of 40 individuals were interviewed, highlighting post-pregnancy return-to-work (RTW) experiences of medical students, residents, advanced practice professionals, nurses, fellows, and faculty. Interviews were coded and analyzed using thematic analysis., Results: We identified both tangible and intangible characteristics of lactation spaces that contribute to stress for lactating individuals. Additionally, we discovered that participants frequently noted a desire to work simultaneously while pumping in order to feel they were self-actualizing in their dual roles of parent and clinician. Among tangible items, access to a computer within lactation space was a key driver of ability to fulfill dual roles. Among intangible characteristics, we identified three distinct, yet interrelated, subthemes, including the need for lactation spaces to be respectful of individuals' time, privacy, and general health and well-being., Conclusions: This study suggests that meeting basic lactation needs with thoughtfully designed lactation spaces can empower individuals in their roles both as a lactating parent and a clinician. EM leadership can evaluate existing lactation spaces to ensure they meet the tangible and intangible needs of lactating physicians, trainees, advanced practice professionals, and nurses., Competing Interests: The authors declare no conflicts of interest., (© 2024 The Author(s). Journal of the American College of Emergency Physicians Open published by Wiley Periodicals LLC on behalf of American College of Emergency Physicians.)

Introduction: Optimizing the performance of emergency department (ED) teams impacts patient care, but the utility of current, team-based performance assessment tools to comprehensively measure this impact is underexplored. In this study we aimed to 1) evaluate ED team performance using current team-based assessment tools during an interprofessional in situ simulation and 2) identify characteristics of effective ED teams., Methods: This mixed-methods study employed case study methodology based on a constructivist paradigm. Sixty-three eligible nurses, technicians, pharmacists, and postgraduate year 2-4 emergency medicine residents at a tertiary academic ED participated in a 10-minute in situ simulation of a critically ill patient. Participants self-rated performance using the Team Performance Observation Tool (TPOT) 2.0 and completed a brief demographic form. Two raters independently reviewed simulation videos and rated performance using the TPOT 2.0, Team Emergency Assessment Measure (TEAM), and Ottawa Crisis Resource Management Global Rating Scale (Ottawa GRS). Following simulations, we conducted semi-structured interviews and focus groups with in situ participants. Transcripts were analyzed using thematic analysis., Results: Eighteen team-based simulations took place between January-April 2021. Raters' scores were on the upper end of the tools for the TPOT 2.0 (R1 4.90, SD 0.17; R2 4.53, SD 0.27, IRR [inter-rater reliability] 0.47), TEAM (R1 3.89, SD 0.19; R2 3.58, SD 0.39, IRR 0.73), and Ottawa GRS (R1 6.6, SD 0.56; R2 6.2, SD 0.54, IRR 0.68). We identified six themes from our interview data: team member entrustment; interdependent energy; leadership tone; optimal communication; strategic staffing; and simulation empowering team performance., Conclusion: Current team performance assessment tools insufficiently discriminate among high performing teams in the ED. Emergency department-specific assessments that capture features of entrustability, interdependent energy, and leadership tone may offer a more comprehensive way to assess an individual's contribution to a team's performance., Competing Interests: Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. This research was generously supported by grant funding from the Emergency Medicine Foundation/Council of Residency Directors in Emergency Medicine Education Scholarship Starter Grant. There are no other conflicts of interest or sources of funding to declare.

Phenomenon : Central to competency-based medical education is the need for a seamless developmental continuum of training and practice. Trainees currently experience significant discontinuity in the transition from undergraduate (UME) to graduate medical education (GME). The learner handover is intended to smooth this transition, but little is known about how well this is working from the GME perspective. In an attempt to gather preliminary evidence, this study explores U.S. program directors (PDs) perspective of the learner handover from UME to GME. Approach : Using exploratory qualitative methodology, we conducted semi-structured interviews with 12 Emergency Medicine PDs within the U.S. from October to November, 2020. We asked participants to describe their current perception of the learner handover from UME to GME. Then we performed thematic analysis using an inductive approach. Findings : We identified two main themes: The inconspicuous learner handover and barrier to creating a successful UME to GME learner handover. PDs described the current state of the learner handover as "nonexistent," yet acknowledged that information is transmitted from UME to GME. Participants also highlighted key challenges preventing a successful learner handover from UME to GME. These included: conflicting expectations, issues of trust and transparency, and a dearth of assessment data to actually hand over. Insights : PDs highlight the inconspicuous nature of learner handovers, suggesting that assessment information is not shared in the way it should be in the transition from UME to GME. Challenges with the learner handover demonstrate a lack of trust, transparency, and explicit communication between UME and GME. Our findings can inform how national organizations establish a unified approach to transmitting growth-oriented assessment data and formalize transparent learner handovers from UME to GME.

Abstract: Medical training programs and health care systems collect ever-increasing amounts of educational and clinical data. These data are collected with the primary purpose of supporting either trainee learning or patient care. Well-established principles guide the secondary use of these data for program evaluation and quality improvement initiatives. More recently, however, these clinical and educational data are also increasingly being used to train artificial intelligence (AI) models. The implications of this relatively unique secondary use of data have not been well explored. These models can support the development of sophisticated AI products that can be commercialized. While these products have the potential to support and improve the educational system, there are challenges related to validity, patient and learner consent, and biased or discriminatory outputs. The authors consider the implications of developing AI models and products using educational and clinical data from learners, discuss the uses of these products within medical education, and outline considerations that should guide the appropriate use of data for this purpose. These issues are further explored by examining how they have been navigated in an educational collaborative., (Copyright © 2023 the Association of American Medical Colleges.)

Purpose: The era of precision education is increasingly leveraging electronic health record (EHR) data to assess residents' clinical performance. But precision in what the EHR-based resident performance metrics are truly assessing is not fully understood. For instance, there is limited understanding of how EHR-based measures account for the influence of the team on an individual's performance-or conversely how an individual contributes to team performances. This study aims to elaborate on how the theoretical understandings of supportive and collaborative interdependence are captured in residents' EHR-based metrics., Method: Using a mixed methods study design, the authors conducted a secondary analysis of 5 existing quantitative and qualitative datasets used in previous EHR studies to investigate how aspects of interdependence shape the ways that team-based care is provided to patients., Results: Quantitative analyses of 16 EHR-based metrics found variability in faculty and resident performance (both between and within resident). Qualitative analyses revealed that faculty lack awareness of their own EHR-based performance metrics, which limits their ability to act interdependently with residents in an evidence-informed fashion. The lens of interdependence elucidates how resident practice patterns develop across residency training, shifting from supportive to collaborative interdependence over time. Joint displays merging the quantitative and qualitative analyses showed that residents are aware of variability in faculty's practice patterns and that viewing resident EHR-based measures without accounting for the interdependence of residents with faculty is problematic, particularly within the framework of precision education., Conclusions: To prepare for this new paradigm of precision education, educators need to develop and evaluate theoretically robust models that measure interdependence in EHR-based metrics, affording more nuanced interpretation of such metrics when assessing residents throughout training., (Copyright © 2024 the Association of American Medical Colleges.)

Purpose: Entrustable professional activities (EPAs) are a widely used framework for curriculum and assessment, yet the variability in emergency medicine (EM) training programs mandates the development of EPAs that meet the needs of the specialty as a whole. This requires eliciting and incorporating the perspectives of multiple stakeholders (i.e., faculty, residents, and patients) in the development of EPAs. Without a shared understanding of what a resident must be able to do upon graduation, we run the risk of advancing ill-prepared residents that may provide inconsistent care., Methods: In an effort to address these challenges, beginning in February 2020, the authors assembled an advisory board of 25 EM faculty to draft and reach consensus on a final list of EPAs that can be used across all training programs within the specialty of EM. Using modified Delphi methodology, the authors came to consensus on an initial list of 22 EPAs. The authors presented these EPAs to faculty supervisors, residents, and patients for refinement. The authors collated and analyzed feedback from focus groups of residents and patients using thematic analysis. The EPAs were subsequently refined based on this feedback., Results: Stakeholders in EM residency training endorsed a final revised list of 22 EPAs. Stakeholder focus groups highlighted two main thematic considerations that helped shape the finalized list of EM EPAs: attention to the meaningful nuances of EPA language and contextualizing the EPAs and viewing them developmentally., Conclusions: To foreground all key stakeholders within the EPA process for EM, the authors chose within the development process to draft; come to consensus; and refine EPAs for EM in collaboration with relevant faculty, patient, and resident stakeholders. Each stakeholder group contributed meaningfully to the content and intended implementation of the EPAs. This process may serve as a model for others in developing stakeholder-responsive EPAs., Competing Interests: The authors declare no conflicts of interest., (© 2024 Society for Academic Emergency Medicine.)

Abstract: In 1999, the National Labor Relations Board determined that residents function as employees, thereby allowing them to freely unionize. From 2020 to 2023, house staff (i.e., resident physicians and fellows) unions have significantly increased, and 8 physician training centers, representing nearly 4,000 house staff, have unionized since March 2021. While unions provide residents with an important tool in effecting change in their workplace, their introduction into the educational milieu has the potential to alter the program director (PD)-resident relationship. In this article, the authors use the educational alliance framework to detail 3 factors required to support a quality educational relationship between a resident and their PD. They also elaborate on how the introduction of unions may impact the PD-resident relationship and explore the potential unintended consequences of unionization as it pertains to this relationship. The authors then use 2 social psychology theories, naïve realism and motivated reasoning, to describe common framing dynamics that lead to conflict during collective bargaining processes. They conclude by offering strategies that PDs may use to mitigate tensions that arise in contract negotiations, even without a direct seat at the table. Ultimately, PDs should anticipate continued growth of resident unions and prepare themselves and their programs for the tensions that may arise from this action. The PD role as a neutral third party ought to be preserved, which is possible if all parties set reasonable expectations for the changes in the PD's role and responsibilities under a union. PDs should understand the 3 core aspects of the educational alliance and the importance of establishing credibility with their residents early on to build a strong foundation., (Copyright © 2024 the Association of American Medical Colleges.)

Graduate medical education (GME) and Clinical Competency Committees (CCC) have been evolving to monitor trainee progression using competency-based medical education principles and outcomes, though evidence suggests CCCs fall short of this goal. Challenges include that evaluation data are often incomplete, insufficient, poorly aligned with performance, conflicting or of unknown quality, and CCCs struggle to organize, analyze, visualize, and integrate data elements across sources, collection methods, contexts, and time-periods, which makes advancement decisions difficult. Learning analytics have significant potential to improve competence committee decision making, yet their use is not yet commonplace. Learning analytics (LA) is the interpretation of multiple data sources gathered on trainees to assess academic progress, predict future performance, and identify potential issues to be addressed with feedback and individualized learning plans. What distinguishes LA from other educational approaches is systematic data collection and advanced digital interpretation and visualization to inform educational systems. These data are necessary to: 1) fully understand educational contexts and guide improvements; 2) advance proficiency among stakeholders to make ethical and accurate summative decisions; and 3) clearly communicate methods, findings, and actionable recommendations for a range of educational stakeholders. The ACGME released the third edition CCC Guidebook for Programs in 2020 and the 2021 Milestones 2.0 supplement of the Journal of Graduate Medical Education (JGME Supplement) presented important papers that describe evaluation and implementation features of effective CCCs. Principles of LA underpin national GME outcomes data and training across specialties; however, little guidance currently exists on how GME programs can use LA to improve the CCC process. Here we outline recommendations for implementing learning analytics for supporting decision making on trainee progress in two areas: 1) Data Quality and Decision Making, and 2) Educator Development.

Over the past decade, medical education has shifted from a time-based approach to a competency-based approach for surgical training. This transition presents many new systemic challenges. The Society for Improving Medical Professional Learning (SIMPL) was created to respond to these challenges through coordinated collaboration across an international network of medical educators. The primary goal of the SIMPL network was to implement a workplace-based assessment and feedback platform. To date, SIMPL has developed, implemented, and sustained a platform that represents the earliest and largest effort to support workplace-based assessment at scale. The SIMPL model for collaborative improvement demonstrates a potential approach to addressing other complex systemic problems in medical education., Competing Interests: J.B.Z., S.S.S.-S., and B.C.G. have leadership positions, contracts, and grants with SIMPL. The remaining authors declare that they have nothing to disclose., (Copyright © 2023 The Author(s). Published by Wolters Kluwer Health, Inc.)

Background: Nationally, underrepresented minorities represent a significant proportion of the first-generation student population. These students also tend to report lower levels of belongingness compared to their peers, which may impact their wellness and be an important factor in their academic success. This study aimed to explore whether status as a first-generation student was associated with belongingness amongst medical students., Methods: In 2019, a previously validated 16-item survey was used to examine potential disparities in belongingness amongst groups of first-generation medical students. Differences between groups were assessed using a Mann-Whitney U-test for each individual item and three composite groupings of items regarding social belonging, academic belonging, and institutional support., Results: First-generation to college and first-generation to graduate school students reported lower belongingness across most individual items as well as in all three composite groups., Conclusions: Given that peer relationships and institutional support play an important role in medical student belonging, these findings represent an opportunity to address the specific needs of individuals from underrepresented groups in medicine. Doing so can support the academic and professional success of first-generation students and help close the diversity gap in medicine., Competing Interests: No competing interests were disclosed., (Copyright: © 2023 Delgado AC et al.)

Introduction: Individual assessments disregard team contributions, while team assessments disregard an individual's contributions. Interdependence has been put forth as a conceptual bridge between our educational traditions of assessing individual performance and our imminent challenge of assessing team-based performance without losing sight of the individual. The purpose of this study was to develop a more refined conceptualisation of interdependence to inform the creation of measures that can assess the interdependence of residents within health care teams., Methods: Following a constructivist grounded theory approach, we conducted 49 semi-structured interviews with various members of health care teams (e.g. physicians, nurses, pharmacists, social workers and patients) across two different clinical specialties-Emergency Medicine and Paediatrics-at two separate sites. Data collection and analysis occurred iteratively. Constant comparative inductive analysis was used, and coding consisted of three stages: initial, focused and theoretical., Results: We asked participants to reflect upon interdependence and describe how it exists in their clinical setting. All participants acknowledged the existence of interdependence, but they did not view it as part of a linear spectrum where interdependence becomes independence. Our analysis refined the conceptualisation of interdependence to include two types: supportive and collaborative. Supportive interdependence occurs within health care teams when one member demonstrates insufficient expertise to perform within their scope of practice. Collaborative interdependence, on the other hand, was not triggered by lack of experience/expertise within an individual's scope of practice, but rather recognition that patient care requires contributions from other team members., Conclusion: In order to assess a team's collective performance without losing sight of the individual, we need to capture interdependent performances and characterise the nature of such interdependence. Moving away from a linear trajectory where independence is seen as the end goal can also help support efforts to measure an individual's competence as an interdependent member of a health care team., (© 2023 Association for the Study of Medical Education and John Wiley & Sons Ltd.)

Purpose: Despite the recognized importance of collaborative communication among physicians, conflict at transitions of care remains a pervasive issue. Recent work has underscored how poor communication can undermine patient safety and organizational efficiency, yet little is known about how interphysician conflict (I-PC) impacts the physicians forced to navigate these tensions. The goal of this study was to explore the social processes and interpersonal interactions surrounding I-PC and their impact, using conversations regarding admission between internal medicine (IM) and emergency medicine (EM) as a lens to explore I-PC in clinical practice., Method: The authors used constructivist grounded theory to explore the interpersonal and social dynamics of I-PC. They used purposive sampling to recruit participants, including EM resident and attending physicians and IM attending physicians. The authors conducted hour-long, semistructured interviews between June and October 2020 using the Zoom video conferencing platform. Interviews were coded in 3 phases: initial line-by-line coding, focused coding, and recording. Constant comparative analysis was used to refine emerging codes, and the interview guide was iteratively updated., Results: The authors interviewed 18 residents and attending physicians about how engaging in I-PC led to both personal and professional harm. Specifically, physicians described how I-PC resulted in emotional distress, demoralization, diminished sense of professional attributes, and job dissatisfaction. Participants also described how emotional residue attached to past I-PC events primed the workplace for future conflict., Conclusions: I-PC may represent a serious yet underrecognized source of harm, not only to patient safety but also to physician well-being. Participants described both the personal and professional consequences of I-PC, which align with the core tenets of burnout. Burnout is a well-established threat to the physician workforce, but unlike many other contributors to burnout, I-PC may be modifiable through improved education that equips physicians with the skills to navigate I-PC throughout their careers.

Background: Safe and effective physician-to-physician patient handoffs are integral to patient safety. Unfortunately, poor handoffs continue to be a major cause of medical errors. Developing a better understanding of challenges faced by health care providers is critical to address this continued patient safety threat. This study addresses the gap in the literature exploring broad, cross-specialty trainee perspectives around handoffs and provides a set of trainee-informed recommendations for both training programs and institutions., Methods: Using a constructivist paradigm, the authors conducted a concurrent/embedded mixed method study to investigate trainees' experiences with patient handoffs across Stanford University Hospital, a large academic medical center. The authors designed and administered a survey instrument including Likert-style and open-ended questions to solicit information about trainee experiences from multiple specialties. The authors performed a thematic analysis of open-ended responses., Results: 687/1138 (60.4%) of residents and fellows responded to the survey, representing 46 training programs and over 30 specialties. There was wide variability in handoff content and process, most notably code status not being consistently mentioned a third of the time for patients who were not full code. Supervision and feedback about handoffs were inconsistently provided. Trainees identified multiple health-systems level issues that complicated handoffs and suggested solutions to these threats. Our thematic analysis identified five important aspects of handoffs: (1) handoff elements, (2) health-systems-level factors, (3) impact of the handoff, (4) agency (duty), and (5) blame and shame., Conclusions: Health systems, interpersonal, and intrapersonal issues affect handoff communication. The authors propose an expanded theoretical framework for effective patient handoffs and provide a set of trainee-informed recommendations for training programs and sponsoring institutions. Cultural and health-systems issues must be prioritized and addressed, as an undercurrent of blame and shame permeates the clinical environment., (© 2023. The Author(s).)

The use of Artificial Intelligence (AI) in medical education has the potential to facilitate complicated tasks and improve efficiency. For example, AI could help automate assessment of written responses, or provide feedback on medical image interpretations with excellent reliability. While applications of AI in learning, instruction, and assessment are growing, further exploration is still required. There exist few conceptual or methodological guides for medical educators wishing to evaluate or engage in AI research. In this guide, we aim to: 1) describe practical considerations involved in reading and conducting studies in medical education using AI, 2) define basic terminology and 3) identify which medical education problems and data are ideally-suited for using AI.

Competency-based medical education (CBME) is an outcomes-based approach to education and assessment that focuses on what competencies trainees need to learn in order to provide effective patient care. Despite this goal of providing quality patient care, trainees rarely receive measures of their clinical performance. This is problematic because defining a trainee's learning progression requires measuring their clinical performance. Traditional clinical performance measures (CPMs) are often met with skepticism from trainees given their poor individual-level attribution. Resident-sensitive quality measures (RSQMs) are attributable to individuals, but lack the expeditiousness needed to deliver timely feedback and can be difficult to automate at scale across programs. In this eye opener, the authors present a conceptual framework for a new type of measure - TRainee Attributable & Automatable Care Evaluations in Real-time (TRACERs) - attuned to both automation and trainee attribution as the next evolutionary step in linking education to patient care. TRACERs have five defining characteristics: meaningful (for patient care and trainees), attributable (sufficiently to the trainee of interest), automatable (minimal human input once fully implemented), scalable (across electronic health records [EHRs] and training environments), and real-time (amenable to formative educational feedback loops). Ideally, TRACERs optimize all five characteristics to the greatest degree possible. TRACERs are uniquely focused on measures of clinical performance that are captured in the EHR, whether routinely collected or generated using sophisticated analytics, and are intended to complement (not replace) other sources of assessment data. TRACERs have the potential to contribute to a national system of high-density, trainee-attributable, patient-centered outcome measures., Competing Interests: The authors have no competing interests to declare., (Copyright: © 2023 The Author(s).)

Purpose: The electronic health record (EHR) is frequently identified as a source of assessment data regarding residents' clinical performance. To better understand how to harness EHR data for education purposes, the authors developed and authenticated a prototype resident report card. This report card used EHR data exclusively and was authenticated with various stakeholders to understand individuals' reactions to and interpretations of EHR data when presented in this way., Methods: Using principles derived from participatory action research and participatory evaluation, this study brought together residents, faculty, a program director, and medical education researchers ( n  = 19) to develop and authenticate a prototype report card for residents. From February to September 2019, participants were invited to take part in a semistructured interview that explored their reactions to the prototype and provided insights about how they interpreted the EHR data., Results: Our results highlighted three themes: data representation, data value, and data literacy. Participants varied in terms of the best way to present the various EHR metrics and felt pertinent contextual information should be included. All participants agreed that the EHR data presented were valuable, but most had concerns about using it for assessment. Finally, participants had difficulties interpreting the data, suggesting that these data could be presented more intuitively and that residents and faculty may require additional training to fully appreciate these EHR data., Conclusions: This work demonstrated how EHR data could be used to assess residents' clinical performance, but it also identified areas that warrant further consideration, especially pertaining to data representation and subsequent interpretation. Providing residents and faculty with EHR data in a resident report card was viewed as most valuable when used to guide feedback and coaching conversations., Competing Interests: The authors declare no potential conflict of interest., (© 2023 Society for Academic Emergency Medicine.)

Purpose: Traditional quality metrics do not adequately represent the clinical work done by residents and, thus, cannot be used to link residency training to health care quality. This study aimed to determine whether electronic health record (EHR) data can be used to meaningfully assess residents' clinical performance in pediatric emergency medicine using resident-sensitive quality measures (RSQMs)., Method: EHR data for asthma and bronchiolitis RSQMs from Cincinnati Children's Hospital Medical Center, a quaternary children's hospital, between July 1, 2017, and June 30, 2019, were analyzed by ranking residents based on composite scores calculated using raw, unadjusted, and case-mix adjusted latent score models, with lower percentiles indicating a lower quality of care and performance. Reliability and associations between the scores produced by the 3 scoring models were compared. Resident and patient characteristics associated with performance in the highest and lowest tertiles and changes in residents' rank after case-mix adjustments were also identified., Results: 274 residents and 1,891 individual encounters of bronchiolitis patients aged 0-1 as well as 270 residents and 1,752 individual encounters of asthmatic patients aged 2-21 were included in the analysis. The minimum reliability requirement to create a composite score was met for asthma data (α = 0.77), but not bronchiolitis (α = 0.17). The asthma composite scores showed high correlations ( r = 0.90-0.99) between raw, latent, and adjusted composite scores. After case-mix adjustments, residents' absolute percentile rank shifted on average 10 percentiles. Residents who dropped by 10 or more percentiles were likely to be more junior, saw fewer patients, cared for less acute and younger patients, or had patients with a longer emergency department stay., Conclusions: For some clinical areas, it is possible to use EHR data, adjusted for patient complexity, to meaningfully assess residents' clinical performance and identify opportunities for quality improvement.

Objectives: Here we report the clinical performance of COVID-19 curbside screening with triage to a drive-through care pathway versus main emergency department (ED) care for ambulatory COVID-19 testing during a pandemic. Patients were evaluated from cars to prevent the demand for testing from spreading COVID-19 within the hospital., Methods: We examined the effectiveness of curbside screening to identify patients who would be tested during evaluation, patient flow from screening to care team evaluation and testing, and safety of drive-through care as 7-day ED revisits and 14-day hospital admissions. We also compared main ED efficiency versus drive-through care using ED length of stay (EDLOS). Standardized mean differences (SMD) >0.20 identify statistical significance., Results: Of 5931 ED patients seen, 2788 (47.0%) were walk-in patients. Of these patients, 1111 (39.8%) screened positive for potential COVID symptoms, of whom 708 (63.7%) were triaged to drive-through care (with 96.3% tested), and 403 (36.3%) triaged to the main ED (with 90.5% tested). The 1677 (60.2%) patients who screened negative were seen in the main ED, with 440 (26.2%) tested. Curbside screening sensitivity and specificity for predicting who ultimately received testing were 70.3% and 94.5%. Compared to the main ED, drive-through patients had fewer 7-day ED revisits (3.8% vs 12.5%, SMD = 0.321), fewer 14-day hospital readmissions (4.5% vs 15.6%, SMD = 0.37), and shorter EDLOS (0.56 vs 5.12 hours, SMD = 1.48)., Conclusion: Curbside screening had high sensitivity, permitting early respiratory isolation precautions for most patients tested. Low ED revisit, hospital readmissions, and EDLOS suggest drive-through care, with appropriate screening, is safe and efficient for future respiratory illness pandemics., Competing Interests: The authors have no conflicts of interest to disclose., (© 2022 The Authors. JACEP Open published by Wiley Periodicals LLC on behalf of American College of Emergency Physicians.)

Introduction: The strengths and weaknesses of virtual and in-person formats within continuing professional development (CPD) are incompletely understood. This study sought to explore attendees' perspectives across multiple specialties regarding benefits and limitations of conference formats and strategies for successful virtual and hybrid (i.e., in-person conferences with a virtual option) conferences., Methods: From December 2020 to January 2021, semistructured interviews were conducted with participants who attended both virtual and in-person CPD conferences. Purposive sampling was utilized to ensure diverse representation of gender, years in practice, location, academic rank, specialty, and practice type. Multiple specialties were intentionally sought to better understand the broader experience among physicians in general, rather than among a specific specialty. Using modified grounded theory approach with a constructivist-interpretivist paradigm, two investigators independently analyzed all interview transcripts. Discrepancies were resolved by in-depth discussion and negotiated consensus., Results: Twenty-six individuals across 16 different specialties were interviewed. We identified three overarching concepts: motivations to attend conferences, benefits and limitations of different conference formats, and strategies to optimize virtual and hybrid conferences. Specific motivators included both professional and personal factors. Benefits of in person included networking/community, immersion, and wellness, while the major limitation was integration with personal life. Benefits of virtual were flexibility, accessibility, and incorporation of technology, while limitations included technical challenges, distractions, limitations for tactile learning, and communication/connection. Benefits of hybrid included more options for access, while limitations included challenges with synchrony of formats and dilution of experiences. Strategies to improve virtual/hybrid conferences included optimizing technology/production, facilitating networking and engagement, and deliberate selection of content., Conclusions: This study identified several benefits and limitations of each medium as well as strategies to optimize virtual and hybrid CPD conferences. This may help inform future CPD conference planning for both attendees and conference planners alike., Competing Interests: The authors declare no potential conflict of interest., (© 2022 Society for Academic Emergency Medicine.)

Background: COVID-19 screening protocols rapidly evolved as a result of changing Centers for Disease Control and Prevention (CDC) and California Department of Public Health (CDPH) recommendations. These protocols led to operational improvements at one large academic medical centre using change management methods explained in Kotter's 8-stage change model., Methods: We reviewed all iterations of clinical process maps for identifying, isolating and assessing COVID-19 infections in paediatric and adult populations within one emergency department (ED) from 28 February 2020 to 5 April 2020. We incorporated CDC and CDPH criteria for the various roles of healthcare workers in ED patient assessment., Results: Using Kotter's 8-stage change model, we outlined the chronological evolution of basic screening criteria, as well as how these were reviewed, modified and implemented during the onset and through the time of greatest uncertainty of COVID-19 in the USA. Our results demonstrate a successful creation, and subsequent execution, of rapidly changing protocols across a large workforce., Conclusion: We effectively applied a business change management framework to the hospital management response during a pandemic; we share these experiences and challenges to inform and guide future operational decision making during times of rapid change., Competing Interests: Competing interests: None declared., (© Author(s) (or their employer(s)) 2022. No commercial re-use. See rights and permissions. Published by BMJ.)

Introduction: As the field of medical education evolves, there is a need to increase the quality of education scholarship and develop a cadre of research scholars; however, clinician educators (CEs) considering this career transition have limited formal training in education research methodology to heed this call. Therefore, a program that provides more advanced training in education scholarship for CEs without the financial and resource barriers of fellowships and masters programs is needed., Methods: The SAEM Advanced Research Methodology Evaluation and Design in Medical Education (ARMED MedEd) program is a longitudinal program for the beyond-beginner CE, seeking advanced training in education research. The program was created using a comprehensive needs assessment and included longitudinal training; small-group projects; dedicated project mentors; and integrated diversity, equity, and inclusion initiatives. Program participants applied for a grant upon program completion., Results: Twenty-one participants completed the course with 100% completing the baseline survey and 67% (14/21) completing the end-of-program survey. Participants reported improved perception of knowledge across all of the topics with a medium to large effect size, ranging from 0.40 to 0.62. When asked about impact on their network of potential collaborators, participants reported a median of 7 (interquartile range [IQR] 5-8) out of 9. When asked about the impact on their community of practice, participants reported a median of 7 (IQR 5-7) out of 9. When asked about the impact on their professional identity, participants reported a median of 7 (IQR 4-9) out of 9. Participants also reported an increase in both the quantity (mean of 2 ± 1 new mentors) and the quality (median score 7 [IQR 5-8] out of 9) of new research mentorship as a result of the program. Open-ended feedback was generally positive, with 100% reporting they would advise others to take this program., Conclusions: The SAEM ARMED MedEd program represents a proof of concept for an advanced education research program seeking to fill the research training gap for the beyond-beginner Clinician educators., Competing Interests: The authors declare no potential conflict of interest., (© 2022 Society for Academic Emergency Medicine.)

Objectives: The goal of this study was to develop and evaluate a novel curriculum and assessment tool for Core Entrustable Professional Activity (EPA) 10 competencies and entrustment scoring in a cohort of medical students in their emergency medicine (EM) clerkship using a framework of individualized, ad hoc, formative assessment. Core EPA 10 is an observable workplace-based activity for graduating medical students to recognize a patient requiring urgent or emergent care and initiate evaluation and management., Methods: This is a prospective, pretest-posttest study of medical students during their EM clerkship. Using the Thomas and Kern framework, we created a curriculum of simulation cases about chest pain/cardiac arrest and respiratory distress, which included novel assessment checklists, and instructional videos about recognizing and managing emergencies. Students were individually pretested on EPA 10 competencies using the simulation cases. Two raters scored students using standardized checklists. Students then watched instructional videos, underwent a posttest with the simulation cases, and were scored again by the two raters using the checklists. Differences between pretest and posttest scores were analyzed using paired t-tests and Wilcoxon signed-rank tests., Results: Seventy-three out of 85 (86%) students completed the curriculum. Mean scores from pretest to final posttest in the chest pain/cardiac arrest and respiratory distress cases significantly improved from 14.8/19 (SD 1.91), to 17.1/19 (SD = 1.00), t(68) = 10.56, p  < 0.001, and 8.5/13 (SD 1.79), to 11.1/13(SD 0.89), t(67) = 11.15, p  < 0.001, respectively. The kappa coefficients were 0.909 ( n  = 2698, p  < 0.001) and 0.931 ( n  = 1872, p  < 0.001). Median modified Chen entrustment scores improved from 1b (i.e., "Watch me do this") to 2b (i.e., "I'll watch you") for the chest pain/cardiac arrest case ( p  < 0.001) and 1b/2a (i.e., "Watch me do this"/ "Let's do this together") to 3a (i.e. "You go ahead, and I'll double-check all of your findings") for the respiratory distress case ( p  < 0.001)., Conclusion: A new directed curriculum of standardized simulation cases and asynchronous instructional videos improved medical student performance in EPA 10 competencies and entrustment scores. This study provides a curricular framework to support formative individualized assessments for EPA 10., Competing Interests: DTM reports grant money to the University of Colorado to conduct research conceived and written by DTM from the Society of Academic Emergency Medicine, Emergency Medicine Foundation. WG, HCW, KN, SSS, and MAG report no conflicts of interest., (© 2022 Society for Academic Emergency Medicine.)

Purpose: Feedback continues to present a challenge for competency-based medical education. Clear, consistent, and credible feedback is vital to supporting one's ongoing development, yet it can be difficult to gather clinical performance data about residents. This study sought to determine whether providing residents with electronic health record (EHR)-based report cards, as well as an opportunity to discuss these data with faculty trained using the R2C2 model, can help residents understand and interpret their clinical performance metrics., Method: Using action research methodology, the author team collected EHR data from July 2017 to February 2020, for all residents (n = 21) in one 5-year Emergency Medicine program and created personalized report cards for each resident. During October 6-17, 2020, 8 out of 17 eligible residents agreed to have their feedback conversations recorded and participate in a subsequent interview with a nonphysician member of the research team. Data were analyzed using thematic analysis, and the authors used inductive analysis to identify themes in the data., Results: In analyzing both the feedback conversations as well as the individual interviews with faculty and residents, the authors identified 2 main themes: (1) Reactions and responses to receiving personalized EHR data and (2) The value of EHR data for assessment and feedback purposes. All participants believed that EHR data metrics are useful for prompting self-reflection, and many pointed to their utility in providing suggestions for actionable changes in their clinical practice. For faculty, having a tool through which underperforming residents can be shown "objective" data about their clinical performance helps underscore the need for improvement, particularly when residents are resistant., Conclusions: The EHR is a valuable source of educational data, and this study demonstrates one of the many thoughtful ways it can be used for assessment and feedback purposes., (Copyright © 2022 by the Association of American Medical Colleges.)

Purpose: Despite the implementation of professionalism curricula and standardised communication tools, inter-physician conflict persists. In particular, the interface between emergency medicine (EM) and internal medicine (IM) has long been recognised as a source of conflict. The social nuances of this conflict remain underexplored, limiting educators' ability to comprehensively address these issues in the clinical learning environment. Thus, the authors explored EM and IM physicians' experiences with negotiating hospital admissions to better understand the social dynamics that contribute to inter-physician conflict and provide foundational guidance for communication best practices., Methods: Using a constructivist grounded theory (CGT) approach, the authors conducted 18 semi-structured interviews between June and October 2020 with EM and IM physicians involved in conversations regarding admissions (CRAs). They asked participants to describe the social exchanges that influenced these conversations and to reflect on their experiences with inter-physician conflict. Data collection and analysis occurred iteratively. The relationships between the codes were discussed by the research team with the goal of developing conceptual connections between the emergent themes., Results: Participants described how their approaches to CRAs were shaped by their specialty identity, and how allegiance to members of their group contributed to interpersonal conflict. This conflict was further promoted by a mutual sense of disempowerment within the organisation, misaligned expectations, and a desire to promote their group's prerogatives. Conflict was mitigated when patient care experiences fostered cross-specialty team formation and collaboration that dissolved traditional group boundaries., Conclusions: Conflict between EM and IM physicians during CRAs was primed by participants' specialty identities, their power struggles within the broader organisation, and their sense of duty to their own specialty. However, formation of collaborative inter-specialty physician teams and expansion of identity to include colleagues from other specialties can mitigate inter-physician conflict., (© 2021 Association for the Study of Medical Education and John Wiley & Sons Ltd.)

Background: The emergency department (ED) help desk is an undergraduate-run service learning program that screens ED patients for social needs, connects them to community resources, and follows-up to promote connections with resources. Students accepted to the program participate in a didactic course on the fundamentals of social emergency medicine as well as available community resources. Students also receive training around interviewing patients and use of screening software. Students commit to at least three quarters of service, during which they attend weekly team meetings., Methods: This qualitative study explores the impact of this service learning experience for students. Current and former students were identified by the director of the program. Purposive and snowball sampling was used to select a sample of participants that participated in a semistructured interview. Our codebook was developed inductively using thematic analysis. Themes were presented and discussed with the entire research team for further analysis and refinement. Data collection and analysis used a constant comparative approach, and data collection ceased when saturation was achieved., Results: Study participants consisted of current and former ED help desk student volunteers ( n  = 21). All participants believed that the ED help desk service learning experience prepared them for future careers by providing an experience that filled a gap in their education. We identified four main themes: (1) participants' perceived impact on patients, (2) learning from patients' experiences and differences, (3) appreciating patients' vulnerability and collaboratively addressing patients' needs, and (4) learning to navigate patients' social needs within the broader health care system., Conclusions: Our ED help desk service learning program offers a unique experience for students to learn about patients' social needs, participate in meaningfully interactions with patients, and empower themselves and patients to work together as coproducers of patients' care., Competing Interests: The authors have no potential conflicts to disclose., (© 2022 by the Society for Academic Emergency Medicine.)

Context: Coming face to face with a trainee who needs to be failed is a stern test for many supervisors. In response, supervisors have been encouraged to report evidence of failure through numerous assessment redesigns. And yet, there are lingering signs that some remain reluctant to engage in assessment processes that could alter a trainee's progression in the programme. Failure is highly consequential for all involved and, although rare, requires explicit study. Recent work identified a phase of disbelief that preceded identification of underperformance. What remains unknown is how supervisors come to recognise that a trainee needs to be failed., Methods: Following constructivist grounded theory methodology, 42 physicians and surgeons in British Columbia, Canada shared their experiences supervising trainees who profoundly underperformed, required extensive remediation or were dismissed from the programme. We identified recurring themes using an iterative, constant comparative process., Results: The shift from disbelieving underperformance to recognising failure involves three patterns: accumulation of significant incidents, discovery of an egregious error after negligible deficits or illumination of an overlooked deficit when pointed out by someone else. Recognising failure was accompanied by anger, certainty and a sense of duty to prevent harm., Conclusion: Coming to the point of recognising that a trainee needs to fail is akin to the psychological process of a tipping point where people first realise that noise is signal and cross a threshold where the pattern is no longer an anomaly. The co-occurrence of anger raises the possibility for emotions to be a driver of, and not only a barrier to, recognising failure. This warrants caution because tipping points, and anger, can impede detection of improvement. Our findings point towards possibilities for supporting earlier identification of underperformance and overcoming reluctance to report failure along with countermeasures to compensate for difficulties in detecting improvement once failure has been verified., (© 2021 Association for the Study of Medical Education and John Wiley & Sons Ltd.)

Introduction Digitalizing workplace-based assessments (WBA) holds the potential for facilitating feedback and performance review, wherein we can easily record, store, and analyze data in real time. When digitizing assessment systems, however, it is unclear what is gained and lost in the message as a result of the change in medium. This study evaluates the quality of comments generated in paper vs. electronic media and the influence of an assessor's seniority. Methods Using a realist evaluation framework, a retrospective database review was conducted with paper-based and electronic medium comments. A sample of assessments was examined to determine any influence of the medium on the word count and the Quality of Assessment for Learning (QuAL) score. A correlation analysis evaluated the relationship between word count and QuAL score. Separate univariate analyses of variance (ANOVAs) were used to examine the influence of the assessor's seniority and medium on word count, QuAL score, and WBA scores. Results The analysis included a total of 1,825 records. The average word count for the electronic comments (M=16) was significantly higher than the paper version (M=12; p=0.01). Longer comments positively correlated with QuAL score (r=0.2). Paper-based comments received lower QuAL scores (0.41) compared to electronic (0.51; p<0.01). Years in practice was negatively correlated with QuAL score (r=-0.08; p<0.001) as was word count (r=-0.2; p<0.001). Conclusion Digitization of WBAs increased the length of comments and did not appear to jeopardize the quality of WBAs; these results indicate higher-quality assessment data. True digital transformation may be possible by harnessing trainee data repositories and repurposing them to analyze for faculty-relevant metrics., Competing Interests: The authors have declared financial relationships, which are detailed in the next section., (Copyright © 2022, Chan et al.)

Aim: We aimed to better understand variation in opioid prescribing practices by investigating physician factors at one academic suburban Emergency Department (ED)., Methods: We retrospectively reviewed the electronic medical records of all patients given opioid prescriptions in the Stanford Health Care ED from 2009 to 2018. We described the variation in opioid prescriptions over time from 2009 to 2018, then dove deeper into a single year (July 1, 2017 to July 1, 2018). We described the number and type of opioid prescriptions at discharge and variation in attending physician opioid prescribing patterns using independent t-tests and a Fischer's exact test., Results: From 2009 to 2018, 657,037 patient visits occurred; 92,612 (14.1%) opioid prescriptions were written. Opioid prescriptions increased from 2009, peaked in 2015, then decreased. Individual providers wrote opioid prescriptions for 1 to 17% of their discharged patients. There was no significant difference in opioid prescribing based on provider gender (p = 0.456), fellow or attending status (p = 0.390), residency completed at Stanford Hospital (p = 0.593), residency completed within California (p = 0.493), or residency completed after 2010 (p = 0.589). Of the 371 providers who wrote opioid prescriptions from 2009 through 2018, 120 wrote prescriptions for patients who had already received at least three opioid prescriptions in the same year from the same department., Conclusion: This study could inform policymakers by describing patterns of variation in opioid prescribing over time and between providers. Although we did see significant differences in prescribing patterns from one provider to the next, those were not explained by the factors we examined. Further studies could investigate factors such as provider experience with pain and addiction, bias regarding particular pathologies, and concern around patient satisfaction scores., Competing Interests: Declaration of Competing Interest None of the authors report any conflicts of interest., (Copyright © 2021 Elsevier Inc. All rights reserved.)

Introduction: Distal radius fractures are common orthopedic injuries managed in emergency departments. Simulation-based mastery learning is widely recognized to improve provider competence for bedside procedures but has not been studied to teach fracture management. This study evaluated the effectiveness of a simulation-based mastery learning curriculum to teach distal radius fracture reduction to novice orthopedic surgery and emergency medicine residents., Methods: We created a novel mastery learning checklist using the Mastery Angoff method of standard setting, paired with a new simulation model designed for this project, to teach orthopedic surgery and emergency medicine interns (N = 22) at the study site. Orthopedic surgery and emergency medicine faculty members participated in checklist development, curriculum design, and implementation. Training included just-in-time asynchronous education with a readiness assessment test, in-classroom expert demonstration, and deliberate practice with feedback. Residents completed a pretest/posttest skills examination and a presurvey/postsurvey assessing procedural confidence., Results: Standard setting resulted in a 41-item checklist with minimum passing score of 37/41 items. All participants met or surpassed the minimum passing score on postexamination. Postsurvey confidence levels were significantly higher than presurvey in all aspects of the distal radius fracture procedure (P < 0.05)., Conclusions: This study demonstrated that a simulation-based mastery learning curriculum improved skills and confidence performing distal radius fracture reductions for orthopedic surgery and emergency medicine interns. Future planned studies include curriculum testing across additional institutions, examination of clinical impact, and application of mastery learning for other orthopedic procedures., Competing Interests: W.D. was the recipient of a grant from the Stanford Teaching and Mentoring Academy to support this project, which will be paid to the Department of Emergency Medicine. The other authors declare no conflict of interest., (Copyright © 2020 Society for Simulation in Healthcare.)

Background: The 13 core entrustable professional activities (EPAs) are key competency-based learning outcomes in the transition from undergraduate to graduate medical education in the United States. Five of these EPAs (EPA2: prioritizing differentials, EPA3: recommending and interpreting tests, EPA4: entering orders and prescriptions, EPA5: documenting clinical encounters, and EPA10: recognizing urgent and emergent conditions) are uniquely suited for web-based assessment., Objective: In this pilot study, we created cases on a web-based simulation platform for the diagnostic assessment of these EPAs and examined the feasibility and acceptability of the platform., Methods: Four simulation cases underwent 3 rounds of consensus panels and pilot testing. Incoming emergency medicine interns (N=15) completed all cases. A maximum of 4 "look for" statements, which encompassed specific EPAs, were generated for each participant: (1) performing harmful or missing actions, (2) narrowing differential or wrong final diagnosis, (3) errors in documentation, and (4) lack of recognition and stabilization of urgent diagnoses. Finally, we interviewed a sample of interns (n=5) and residency leadership (n=5) and analyzed the responses using thematic analysis., Results: All participants had at least one missing critical action, and 40% (6/15) of the participants performed at least one harmful action across all 4 cases. The final diagnosis was not included in the differential diagnosis in more than half of the assessments (8/15, 54%). Other errors included selecting incorrect documentation passages (6/15, 40%) and indiscriminately applying oxygen (9/15, 60%). The interview themes included psychological safety of the interface, ability to assess learning, and fidelity of cases. The most valuable feature cited was the ability to place orders in a realistic electronic medical record interface., Conclusions: This study demonstrates the feasibility and acceptability of a web-based platform for diagnostic assessment of specific EPAs. The approach rapidly identifies potential areas of concern for incoming interns using an asynchronous format, provides feedback in a manner appreciated by residency leadership, and informs individualized learning plans., (©Cynthia R Peng, Kimberly A Schertzer, Holly A Caretta-Weyer, Stefanie S Sebok-Syer, William Lu, Charissa Tansomboon, Michael A Gisondi. Originally published in JMIR Medical Education (https://mededu.jmir.org), 17.11.2021.)

Introduction: Individual assessment disregards the team aspect of clinical work. Team assessment collapses the individual into the group. Neither is sufficient for medical education, where measures need to attend to the individual while also accounting for interactions with others. Valid and reliable measures of interdependence are critical within medical education given the collaborative manner in which patient care is provided. Medical education currently lacks a consistent approach to measuring the performance between individuals working together as part of larger healthcare team. This review's objective was to identify existing approaches to measuring this interdependence., Methods: Following Arksey & O'Malley's methodology, we conducted a scoping review in 2018 and updated it to 2020. A search strategy involving five databases located >12 000 citations. At least two reviewers independently screened titles and abstracts, screened full texts (n = 161) and performed data extraction on twenty-seven included articles. Interviews were also conducted with key informants to check if any literature was missing and assess that our interpretations made sense., Results: Eighteen of the twenty-seven articles were empirical; nine conceptual with an empirical illustration. Eighteen were quantitative; nine used mixed methods. The articles spanned five disciplines and various application contexts, from online learning to sports performance. Only two of the included articles were from the field of Medical Education. The articles conceptualised interdependence of a group, using theoretical constructs such as collaboration synergy; of a network, using constructs such as degree centrality; and of a dyad, using constructs such as synchrony. Both descriptive (eg social network analysis) and inferential (eg multi-level modelling) approaches were described., Conclusion: Efforts to measure interdependence are scarce and scattered across disciplines. Multiple theoretical concepts and inconsistent terminology may be limiting programmatic work. This review motivates the need for further study of measurement techniques, particularly those combining multiple approaches, to capture interdependence in medical education., (© 2021 John Wiley & Sons Ltd and The Association for the Study of Medical Education.)

Study Objective: To explore the social and environmental conditions in emergency departments that contribute to perceived barriers and supports for workplace lactation among individuals working in emergency medicine., Methods: Constructivist grounded theory was used by our team to understand the social processes and behaviors associated with workplace lactation for health care professionals working in EDs. A total of 24 interviews of individuals in EDs with recent return-to-work experience after childbirth were performed. The interviews yielded 36 unique experiences (from 21 faculty, 12 trainees, and 3 nurses) because some participants had more than 1 child, in which case all lactation experiences were discussed. Interview transcriptions were coded and analyzed iteratively for the development of themes, per constructivist grounded theory., Results: Using constant comparative inductive methods, we describe 3 pervasive themes as they relate to workplace lactation that emerged from the analysis of interview data: (1) emergency medicine culture, (2) workplace lactation policies, and (3) supports for workplace lactation., Conclusion: Although formalized workplace lactation policies and other identifiable supports for workplace lactation aid individuals desiring to lactate after returning to work in EDs, many individuals still experience cultural barriers to their desired lactation habits. Policies and individual support systems may continue to fall short of meeting the needs of lactating individuals in emergency medicine unless broader cultural change occurs. Our work offers initial recommendations for shifting the landscape of lactation practices in emergency medicine., (Copyright © 2021 American College of Emergency Physicians. Published by Elsevier Inc. All rights reserved.)

Background: The number of residency applications submitted by medical students rises annually, resulting in increased work and costs for residency programs and applicants, particularly in emergency medicine. We propose a solution to this problem: an optional, two-stage Match with a "summer match" stage, in which applicants can submit a limited number of applications early. This would be conducted similarly to the early decision process for college admissions. The study objectives were to explore stakeholder opinions on the feasibility of a summer match and to identify the ideal logistic parameters to operationalize this proposal., Methods: We used exploratory qualitative methodology following a constructivist paradigm to develop an understanding of the potential impact of a summer match. We interviewed 34 key stakeholders in the U.S. residency application process identified through purposive sampling including educational administrators (program directors, designated institutional officials, medical school deans) and trainees (students, residents). We coded and thematically analyzed interview data in two stages using an inductive approach., Results: We identified six themes from the participant interviews that broadly reflected issues of the residency application process, value, and equity. These themes included disrupting the status quo, logistic concerns, match strategy, differential benefits, unintended consequences, and return on investment. Most study participants supported the summer match concept, with medical students and residents most in favor. We developed a theoretical summer match protocol based on these findings., Conclusions: A summer match may reduce the burdens of increasing residency applications and associated costs. Pilot testing is necessary to confirm this hypothesis and determine the impact of the proposed summer match protocol. Unintended consequences must be considered carefully during implementation., Competing Interests: The authors have no potential conflicts to disclose., (© 2021 by the Society for Academic Emergency Medicine.)

Entrustable professional activities (EPAs) describe activities that qualified professionals must be able to perform to deliver safe and effective care to patients. The entrustable aspect of EPAs can be used to assess learners through documentation of entrustment decisions, while the professional activity aspect can be used to map curricula. When used as an assessment framework, the entrustment decisions reflect supervisory judgments that combine trainees' relational autonomy and patient safety considerations. Thus, the design of EPAs incorporates the supervisor, trainee, and patient in a way that uniquely offers a link between educational outcomes and patient outcomes. However, achieving a patient-centered approach to education amidst both curricular and assessment obligations, educational and patient outcomes, and a supervisor-trainee-patient triad is not simple nor guaranteed. As medical educators continue to advance EPAs as part of their approach to competency-based medical education, the authors share a critical discussion of how patients are currently positioned in EPAs. In this article, the authors examine EPAs and discuss how their development, content, and implementation can result in emphasizing the trainee and/or supervisor while unintentionally distancing or hiding the patient. They consider creative possibilities for how EPAs might better integrate the patient as finding ways to better foreground the patient in EPAs holds promise for aligning educational outcomes and patient outcomes., (Copyright © 2021 by the Association of American Medical Colleges.)

Background: The use of simulators in medical education is critical for developing procedural competence prior to treating patients. Current training of emergency physicians to perform distal radius fracture reduction is inconsistent and inadequate., Approach: We developed a 3D printed distal radius fracture simulation training model that is easy to assemble and relatively inexpensive. We present step-by-step instructions to reproduce the model., Evaluation: The model was found to have high fidelity for training by both instructors and participants in a simulation-based mastery learning course., Reflection: We successfully designed a low cost, easy to reproduce, high fidelity model for use in a simulation-based mastery learning course to teach distal radius fracture reduction.

Background: Mastery learning has gained popularity for training residents in procedural skills due to its demonstrated superiority over traditional methods. However, no studies have compared the efficacy of traditional versus mastery learning methods in residency point-of-care ultrasound education. We hypothesized that mastery learning would improve residents' skills in performing the extended focused assessment with sonography in trauma (eFAST)., Methods: All first-year emergency medicine (EM) resident physicians at a single university hospital underwent a crossover randomized controlled trial to receive mastery-learning eFAST training either at the beginning of the academic year or 6 months into intern year. Participants were taught using a checklist validated by a panel of experts using Mastery Angoff methods and were given feedback on missed tasks until each trainee completed the eFAST with a minimum passing standard (MPS). Our primary outcome was technical proficiency between the two groups for eFAST examinations performed in the emergency department during the academic year., Results: Sixteen interns were enrolled; eight were randomized to each group. The group that received mastery training at the beginning of the year had mean clinical eFAST proficiency scores above the MPS in the first two quarters of the academic year, while the control group did not. Once the control group underwent eFAST mastery training at the midpoint of the year, both groups had mean proficiency scores above the MPS for the remainder of the year., Conclusion: Simulation-based mastery learning is an effective method of teaching the eFAST examination. This training during intern orientation conferred early proficiency in clinical performance of eFAST among EM residents. This difference in proficiency was no longer present after the control group received mastery learning education halfway through the academic year., Competing Interests: The authors have no potential conflicts to disclose., (© 2021 by the Society for Academic Emergency Medicine.)

Background: Increasing access to high-quality emergency and prehospital care is an important priority in low- and middle-income countries (LMICs). However, ensuring that emergency medical technicians (EMTs) maintain their clinical knowledge and proficiency with procedural skills is challenging, as continuing education requirements are still being introduced, and clinical instructional efforts need strengthening. We describe the development and implementation of an innovative asynchronous learning tool for EMTs in the form of a Web-based trivia game., Methods: Over 500 case-based multiple-choice questions (covering 10 essential prehospital content areas) were created by experts in prehospital education, piloted with EMT educators from LMICs, and delivered to EMTs through a Web-based quiz game platform over a 12-week period. We enrolled 252 participants from nine countries., Results: Thirty-two participants (12.7%) completed the entire 12-week game. Participants who completed the game were administered a survey with a 100% response rate. Ninety-three percent of participants used their mobile phone to access the game. Overall, participants reported that the interface was easy to use (93.8% agreed or strongly agreed), the game improved their knowledge (100% agreed or strongly agreed), and they felt better prepared for their jobs (100% agreed or strongly agreed). The primary motivators for participation were improving patient care (37.5%) and being recognized on the game's leaderboard (31.3%). All participants reported that they would engage in the game again (43.8% agreed and 56.3% strongly agreed) and would recommend the game to their colleagues (34.4% agreed and 65.6% strongly agreed)., Conclusions: In conclusion, a quiz game targeting EMT learners from LMICs was viewed as accessible and effective by participants. Future efforts should focus on increasing retention and trialing languages in addition to English., Competing Interests: The authors have no potential conflicts to disclose., (© 2021 by the Society for Academic Emergency Medicine.)

Problem: Given the complex interaction among patients, individual providers, health care teams, and the clinical environment, patient safety events with serious consequences are most likely to occur in intensive care units, operating rooms, and emergency departments (EDs). With low-frequency, high-risk events such as pediatric resuscitations, health care teams working in EDs may not have the clinical opportunity to identify deficiencies, review and reinforce knowledge and skills, and problem solve in authentic clinical conditions. Without creating opportunities to safely practice, hospitals run the risk of having health care teams and environments that are not prepared to provide optimal patient care., Approach: Researchers employed a case series design and used a train-the-trainer model for in situ simulation. They trained health care professionals (instructors) in 3 general, nonacademic EDs in the San Francisco Bay area of California to perform pediatric resuscitation in situ simulations in 2018-2019. In situ simulations occur in the clinical work environment with simulation participants (teams) who are health care professionals taking care of actual patients., Outcomes: Teams made up of physicians, nurses, and ED technicians were evaluated for clinical performance, teamwork, and communication during in situ simulations conducted by instructors at baseline, 6 months, and 12 months. Debriefing after the simulations identified multiple latent safety threats (i.e., unidentified potential safety hazards) that were previously unknown. Each ED's pediatric readiness-its ability to provide emergency care for children-was evaluated at baseline and 12 months., Next Steps: The authors will continue to monitor and examine the impact and sustainability of the pediatric in situ simulation program on pediatric readiness scores and its possible translation to other high-risk clinical settings, as well as explore the relationship between in situ simulations and patient outcomes., (Copyright © 2020 by the Association of American Medical Colleges.)

Context: Inadequate documentation of observed trainee incompetence persists despite research-informed solutions targeting this failure to fail phenomenon. Documentation could be impeded if assessment language is misaligned with how supervisors conceptualise incompetence. Because frameworks tend to itemise competence as well as being vague about incompetence, assessment design may be improved by better understanding and describing of how supervisors experience being confronted with a potentially incompetent trainee., Methods: Following constructivist grounded theory methodology, analysis using a constant comparison approach was iterative and informed data collection. We interviewed 22 physicians about their experiences supervising trainees who demonstrate incompetence; we quickly found that they bristled at the term 'incompetence,' so we began to use 'underperformance' in its place., Results: Physicians began with a belief and an expectation: all trainees should be capable of learning and progressing by applying what they learn to subsequent clinical experiences. Underperformance was therefore unexpected and evoked disbelief in supervisors, who sought alternate explanations for the surprising evidence. Supervisors conceptualised underperformance as: an inability to engage with learning due to illness, a life event or learning disorders, so that progression was stalled, or an unwillingness to engage with learning due to lack of interest, insight or humility., Conclusion: Physicians conceptualise underperformance as problematic progression due to insufficient engagement with learning that is unresponsive to intensified supervision. Although failure to fail tends to be framed as a reluctance to document underperformance, the prior phase of disbelief prevents confident documentation of performance and delays identification of underperformance. The findings offer further insight and possible new solutions to address under-documentation of underperformance., (© 2020 John Wiley & Sons Ltd and The Association for the Study of Medical Education.)

Data science is an inter-disciplinary field that uses computer-based algorithms and methods to gain insights from large and often complex datasets. Data science, which includes Artificial Intelligence techniques such as Machine Learning (ML), has been credited with the promise to transform Health Professions Education (HPE) by offering approaches to handle big (and often messy) data. To examine this promise, we conducted a critical review to explore: (1) published applications of data science and ML in HPE literature and (2) the potential role of data science and ML in shifting theoretical and epistemological perspectives in HPE research and practice. Existing data science studies in HPE are often not informed by theory, but rather oriented towards developing applications for specific problems, uses, and contexts. The most common areas currently being studied are procedural (e.g., computer-based tutoring or adaptive systems and assessment of technical skills). We found that epistemic beliefs informing the use of data science and ML in HPE poses a challenge for existing views on what constitutes objective knowledge and the role of human subjectivity for instruction and assessment. As a result, criticisms have emerged that the integration of data science in the field of HPE is in danger of becoming technically driven and narrowly focused in its approach to teaching, learning and assessment. Our findings suggest that researchers tend to formalize around the epistemological stance driven largely by traditions of a research paradigm. Future data science studies in HPE need to involve both education scientists and data scientists to ensure mutual advancements in the development of educational theory and practical applications. This may be one of the most important tasks in the integration of data science and ML in HPE research in the years to come.

Objective: In the era of competency-based medical education (CBME), the collection of more and more trainee data is being mandated by accrediting bodies such as the Accreditation Council for Graduate Medical Education and the Royal College of Physicians and Surgeons of Canada. However, few efforts have been made to synthesize the literature around the current issues surrounding workplace-based assessment (WBA) data. This scoping review seeks to synthesize the landscape of literature on the topic of data collection and utilization for trainees' WBAs in emergency medicine (EM)., Methods: The authors conducted a scoping review in the style of Arksey and O'Malley, seeking to synthesize and map literature on collecting, aggregating, and reporting WBA data. The authors extracted, mapped, and synthesized literature that describes, supports, and substantiates effective data collection and utilization in the context of the CBME movement within EM., Results: Our literature search retrieved 189 potentially relevant references (after removing duplicates) that were screened to 29 abstracts and papers relevant to collecting, aggregating, and reporting WBAs. Our analysis shows that there is an increasing temporal trend toward contributions in these topics, with the majority of the papers (16/29) being published in the past 3 years alone., Conclusion: There is increasing interest in the areas around data collection and utilization in the age of CBME. The field, however, is only beginning to emerge, leaving more work that can and should be done in this area., (© 2020 Society for Academic Emergency Medicine.)