Search

Introduction: Debriefing after simulation facilitates reflective thinking and learning. Eye-tracking augmented debriefing (ETAD) may provide advantages over traditional debriefing (TD) by leveraging video replay with first-person perspective. This multisite randomized controlled trial compared the impact of ETAD with TD (without eye-tracking and without video) after simulation on 4 outcomes: (1) resident metacognitive awareness (the primary outcome), (2) cognitive load (CL) of residents and debriefers, (3) alignment of resident self-assessment and debriefer assessment scores, and (4) resident and debriefer perceptions of the debriefing experience., Method: Fifty-four emergency medicine residents from 2 institutions were randomized to the experimental (ETAD) or the control (TD) arm. Residents completed 2 simulation stations followed by debriefing. Before station 1 and after station 2, residents completed a Metacognition Awareness Inventory (MAI). After each station, debriefers and residents rated their CL and completed an assessment of performance. After the stations, residents were interviewed and debriefers participated in a focus group., Results: There were no statistically significant differences in mean MAI change, resident CL, or assessment alignment between residents and debriefers. Debriefer CL was lower in the experimental arm. Interviews identified 4 themes: (1) reflections related to debriefing approach, (2) eye-tracking as a metacognitive sensitizer, (3) translation of metacognition to practice, and (4) ETAD as a strategy to manage CL. Residents reported that eye tracking improved the specificity of feedback. Debriefers relied less on notes, leveraged video timestamps, appreciated the structure of the eye-tracking video, and found the video useful when debriefing poor performers., Conclusions: There were no significant quantitative differences in MAI or resident CL scores; qualitative findings suggest that residents appreciated the benefits of the eye-tracking video review. Debriefers expended less CL and reported less perceived mental effort with the new technology. Future research should leverage longitudinal experimental designs to further understand the impact of eye-tracking facilitated debriefing., Competing Interests: The authors declare no conflict of interest., (Copyright © 2024 Society for Simulation in Healthcare.)

Purpose: Inflammatory bowel disease (IBD) has been considered a relative contraindication to radiation therapy (RT) because of the potential greater risk of RT-induced toxicities. This study aimed to assess acute toxicity outcomes in patients with IBD treated with abdominal/pelvic RT., Methods and Materials: After institutional review board approval, patients with IBD who received RT to the abdomen/pelvis were identified from an institutional research repository, and their electronic medical records were reviewed. The IBD cohort was matched 1:1 with controls according to all of the following: RT, gender, disease site, age, and year of RT. Acute toxicity was defined as toxicity occurring within 3 months of RT. Primary outcomes were assessed via univariable logistic regression models and the predicted probability of acute toxicity and acute gastrointestinal (GI) toxicity were plotted for the most significant covariates. IBD and control cohorts were compared on demographic and toxicity variables using χ 2 /Fisher exact tests and Kruskal-Wallis tests where appropriate., Results: We identified 62 patients with a median age of 64 years (IQR, 54-70 years) who received RT from 2006 to 2022. Patients were treated with intensity modulated RT (38; 61.3%), 3-dimensional conformal RT (12; 19.4%), and stereotactic body RT/brachytherapy (12; 19.4%). After RT, 28 (45.2%) and 23 (37.1%) patients experienced grade ≥2 acute (any) and acute GI toxicity, respectively. Higher overall RT dose and RT dose to small bowel were found to be significantly associated with increased risk of grade ≥2 acute toxicities (OR, 1.041 per unit Gy; 95% CI, 1.005-1.084; P = .034 and OR, 1.046; 95% CI, 1.018-1.082; P = .003, respectively). Between IBD and control cohorts, there were no significant differences in grade ≥2 acute (any) and acute GI toxicities (P = .710 and P = .704, respectively)., Conclusions: In patients with IBD treated with abdominal/pelvic RT for malignancy, RT was effective and well-tolerated. RT treatment planning should carefully consider the location(s) of IBD inflammation and dose to bowel structures, in particular, dose to the small bowel., (Copyright © 2024 Elsevier Inc. All rights reserved.)

Background: As competency-based medical education (CBME) curricula are introduced in residency programs across Canada, systematic evaluation efforts are needed to ensure fidelity of implementation. This study evaluated early outcomes of CBME implementation in one Canadian Physical Medicine and Rehabilitation program that was an early adopter of CBME, with an aim to inform continuous quality improvement initiatives and CBME implementation nationwide., Methods: Using Rapid Evaluation methodology, informed by the CBME Core Components Framework, the intended outcomes of CBME were compared to actual outcomes., Results: Results suggested that a culture of feedback and coaching already existed in this program prior to CBME implementation, yet faculty felt that CBME added a framework to support feedback. The small program size was valuable in fostering strong relationships and individualized learning. However, participants expressed concerns about CBME fostering a reductionist approach to the development of competence. Challenges existed with direct observation, clear expectations for off-service training experiences, and tracking trainee progress. There was trepidation surrounding national curricular change, yet the institution-wide approach to CBME implementation created shared experiences and a community of practice., Conclusions: Program evaluation can help understand gaps between planned versus enacted implementation of CBME, and foster adaptations to improve the fidelity of implementation., Competing Interests: The authors have no conflicts of interest to declare., (© 2024 Trier, Askari, Hanmore, Thompson, McGuire, Braund, Koch Hall, McEwen, Dalgarno, Dagnone; licensee Synergies Partners.)

Introduction: As an early adopter of competency-based medical education (CBME) our postgraduate institution was uniquely positioned to analyze implementation experience data across programs, while keeping institutional factors constant. We described participants' experiences related to CBME implementation across programs derived from early program evaluation efforts within our setting., Methods: This evaluation focused on eight residency programs at a medium-sized academic institution in Canada. Participants ( n  = 175) included program leaders, faculty, and residents. The study consisted of 3 phases: (1) describing intended implementation; (2) documenting enacted implementation; and (3) comparing intended with enacted implementation to inform adaptations. Each program's findings were summarized in technical reports which were then analyzed thematically. Cross program data were organized by themes., Results: Six themes were identified. All groups emphasized the need for ongoing refinement of CBME resulting from shared tensions such as increased assessment burden. However, there were some disparate CBME-related experiences between programs such as the experience with entrustable professional activities, the interpretation of retrospective entrustment anchors, and quality of feedback., Conclusion: We detected several cross-program successes and important challenges related to CBME. Our experience can inform other programs engaging in implementation and evaluation of CBME.

Postgraduate medical education is an essential societal enterprise that prepares highly skilled physicians for the health workforce. In recent years, PGME systems have been criticized worldwide for problems with variable graduate abilities, concerns about patient safety, and issues with teaching and assessment methods. In response, competency based medical education approaches, with an emphasis on graduate outcomes, have been proposed as the direction for 21st century health profession education. However, there are few published models of large-scale implementation of these approaches. We describe the rationale and design for a national, time-variable competency-based multi-specialty system for postgraduate medical education called Competence by Design. Fourteen innovations were bundled to create this new system, using the Van Melle Core Components of competency based medical education as the basis for the transformation. The successful execution of this transformational training system shows competency based medical education can be implemented at scale. The lessons learned in the early implementation of Competence by Design can inform competency based medical education innovation efforts across professions worldwide., Competing Interests: JRF, JK, LSS, FB, VN, ST, JF, KH, RST were employees of the Royal College.JS, AA, AO, AKH, LC, SD, DR, WJC, RBC, TD, BT, LF, WG, ND, BW, RS, RA, DD, AB, MG, and EVM performed contract work for the Royal College., (Copyright: © 2024 The Author(s).)

Objective: Approximately five years ago, the Royal College emergency medicine programs in Canada implemented a competency-based paradigm and introduced the use of Entrustable Professional Activities (EPAs) for assessment of units of professional activity to assess trainees. Many competency-based medical education (CBME) based curricula, involve assessing for entrustment through observations of EPAs. While EPAs are frequently assessed in clinical settings, simulation is also used. This study aimed to characterize the use of simulation for EPA assessment., Methods: A study interview guide was jointly developed by all study authors and followed best practices for survey development. A national interview was conducted with program directors or assistant program directors across all the Royal College emergency medicine programs across Canada. Interviews were conducted over Microsoft Teams, interviews were recorded and transcribed, using Microsoft Teams transcribing service. Sample transcripts were analyzed for theme development. Themes were then reviewed by co-authors to ensure they were representative of the participants' views., Results: A 64.7% response rate was achieved. Simulation has been widely adopted by EM training programs. All interviewees demonstrated support for the use of simulation for EPA assessment for many reasons, however, PDs acknowledged limitations and thematic analysis revealed certain themes and tensions for using simulation for EPA assessment. Thematic analysis revealed six major themes: widespread support for the use of simulation for EPA assessment, concerns regarding the potential for EPA assessment to become a "tick- box" exercise, logistical barriers limiting the use of simulation for EPA assessment, varied perceptions about the authenticity of using simulation for EPA assessment, the potential for simulation for EPA assessment to compromise learner psychological safety, and suggestions for the optimization of use of simulation for EPA assessment., Conclusions: Our findings offer insight for other programs and specialties on how simulation for EPA assessment can best be utilized. Programs should use these findings when considering using simulation for EPA assessment., (© 2024. The Author(s), under exclusive licence to Canadian Association of Emergency Physicians (CAEP)/ Association Canadienne de Médecine d'Urgence (ACMU).)

Traditional approaches to assessment in health professions education systems, which have generally focused on the summative function of assessment through the development and episodic use of individual high-stakes examinations, may no longer be appropriate in an era of competency based medical education. Contemporary assessment programs should not only ensure collection of high-quality performance data to support robust decision-making on learners' achievement and competence development but also facilitate the provision of meaningful feedback to learners to support reflective practice and performance improvement. Programmatic assessment is a specific approach to designing assessment systems through the intentional selection and combination of a variety of assessment methods and activities embedded within an educational framework to simultaneously optimize the decision-making and learning function of assessment. It is a core component of competency based medical education and is aligned with the goals of promoting assessment for learning and coaching learners to achieve predefined levels of competence. In Canada, postgraduate specialist medical education has undergone a transformative change to a competency based model centred around entrustable professional activities (EPAs). In this paper, we describe and reflect on the large scale, national implementation of a program of assessment model designed to guide learning and ensure that robust data is collected to support defensible decisions about EPA achievement and progress through training. Reflecting on the design and implications of this assessment system may help others who want to incorporate a competency based approach in their own country., Competing Interests: The authors have no competing interests to declare., (Copyright: © 2024 The Author(s).)

Competence committees (CCs) are a recent innovation to improve assessment decision-making in health professions education. CCs enable a group of trained, dedicated educators to review a portfolio of observations about a learner's progress toward competence and make systematic assessment decisions. CCs are aligned with competency based medical education (CBME) and programmatic assessment. While there is an emerging literature on CCs, little has been published on their system-wide implementation. National-scale implementation of CCs is complex, owing to the culture change that underlies this shift in assessment paradigm and the logistics and skills needed to enable it. We present the Royal College of Physicians and Surgeons of Canada's experience implementing a national CC model, the challenges the Royal College faced, and some strategies to address them. With large scale CC implementation, managing the tension between standardization and flexibility is a fundamental issue that needs to be anticipated and addressed, with careful consideration of individual program needs, resources, and engagement of invested groups. If implementation is to take place in a wide variety of contexts, an approach that uses multiple engagement and communication strategies to allow for local adaptations is needed. Large-scale implementation of CCs, like any transformative initiative, does not occur at a single point but is an evolutionary process requiring both upfront resources and ongoing support. As such, it is important to consider embedding a plan for program evaluation at the outset. We hope these shared lessons will be of value to other educators who are considering a large-scale CBME CC implementation., Competing Interests: Some individual authors received funding from the Royal College either as staff (LS, JK, JRF) or as consultants (AO, DD, RA, AKH, WJC)., (Copyright: © 2024 The Author(s).)

Program evaluation is an essential, but often neglected, activity in any transformational educational change. Competence by Design was a large-scale change initiative to implement a competency-based time-variable educational system in Canadian postgraduate medical education. A program evaluation strategy was an integral part of the build and implementation plan for CBD from the beginning, providing insights into implementation progress, challenges, unexpected outcomes, and impact. The Competence by Design program evaluation strategy was built upon a logic model and three pillars of evaluation: readiness to implement, fidelity and integrity of implementation, and outcomes of implementation. The program evaluation strategy harvested from both internally driven studies and those performed by partners and invested others. A dashboard for the program evaluation strategy was created to transparently display a real-time view of Competence by Design implementation and facilitate continuous adaptation and improvement. The findings of the program evaluation for Competence by Design drove changes to all aspects of the Competence by Design implementation, aided engagement of partners, supported change management, and deepened our understanding of the journey required for transformational educational change in a complex national postgraduate medical education system. The program evaluation strategy for Competence by Design provides a framework for program evaluation for any large-scale change in health professions education., Competing Interests: The authors have no competing interests to declare., (Copyright: © 2024 The Author(s).)

Coaching is an increasingly popular means to provide individualized, learner-centered, developmental guidance to trainees in competency based medical education (CBME) curricula. Aligned with CBME's core components, coaching can assist in leveraging the full potential of this educational approach. With its focus on growth and improvement, coaching helps trainees develop clinical acumen and self-regulated learning skills. Developing a shared mental model for coaching in the medical education context is crucial to facilitate integration and subsequent evaluation of success. This paper describes the Royal College of Physicians and Surgeons of Canada's coaching model, one that is theory based, evidence informed, principle driven and iteratively and developed by a multidisciplinary team. The coaching model was specifically designed, fit for purpose to the postgraduate medical education (PGME) context and implemented as part of Competence by Design (CBD), a new competency based PGME program. This coaching model differentiates two coaching roles, which reflect different contexts in which postgraduate trainees learn and develop skills. Both roles are supported by the RX-OCR process: developing R elationship/ R apport, setting e X pectations, O bserving, a C oaching conversation, and R ecording/ R eflecting. The CBD Coaching Model and its associated RX-OCR faculty development tool support the implementation of coaching in CBME. Coaching in the moment and coaching over time offer important mechanisms by which CBD brings value to trainees. For sustained change to occur and for learners and coaches to experience the model's intended benefits, ongoing professional development efforts are needed. Early post implementation reflections and lessons learned are provided., Competing Interests: Some individual authors received funding from the RC either as staff (FB) or consultants (WC, AH, AO, DR). No other competing interests., (Copyright: © 2024 The Author(s).)

Abstract: Although just-in-time training (JIT) is increasingly used in simulation-based health professions education, its impact on learning, performance, and patient outcomes remains uncertain. The aim of this study was to determine whether JIT simulation training leads to improved learning and performance outcomes. We included randomized or nonrandomized interventional studies assessing the impact of JIT simulation training (training conducted in temporal or spatial proximity to performance) on learning outcomes among health professionals (trainees or practitioners). Of 4077 citations screened, 28 studies were eligible for inclusion. Just-in-time training simulation training has been evaluated for a variety of medical, resuscitation, and surgical procedures. Most JIT simulation training occurred immediately before procedures and lasted between 5 and 30 minutes. Despite the very low certainty of evidence, this systematic review suggests JIT simulation training can improve learning and performance outcomes, in particular time to complete skills. There remains limited data on better patient outcomes and collateral educational effects., Competing Interests: The authors declare no conflict of interest., (Copyright © 2023 Society for Simulation in Healthcare.)

Background: Simulation has become a staple in the training of healthcare professionals with accumulating evidence on its effectiveness. However, guidelines for optimal methods of simulation training do not currently exist., Methods: Systematic reviews of the literature on 16 identified key questions were conducted and expert panel consensus recommendations determined using the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) methodology., Objective: These evidence-based guidelines from the Society for Simulation in Healthcare intend to support healthcare professionals in decisions on the most effective methods for simulation training in healthcare., Results: Twenty recommendations on 16 questions were determined using GRADE. Four expert recommendations were also provided., Conclusions: The first evidence-based guidelines for simulation training are provided to guide instructors and learners on the most effective use of simulation in healthcare., Competing Interests: All conflicts of interest and disclosures were assessed as not having influenced the construction of these guidelines. D.S. disclosed research support from Intuitive Surgical and Beckon Dickinson and consultant honoraria from J&J and Applied Medical. A.W.C. receives honoraria from SSH and is consultant for The Debriefing Academy. J.T.P. disclosed royalties from Oxford University Press and Springer Nature as coeditor for simulation and surgical education books, and research support from Avita Medical. The other authors declare no other conflict of interest., (Copyright © 2023 Society for Simulation in Healthcare.)

Residents and faculty have described a burden of assessment related to the implementation of competency-based medical education (CBME), which may undermine its benefits. Although this concerning signal has been identified, little has been done to identify adaptations to address this problem. Grounded in an analysis of an early Canadian pan-institutional CBME adopter's experience, this article describes postgraduate programs' adaptations related to the challenges of assessment in CBME. From June 2019-September 2022, 8 residency programs underwent a standardized Rapid Evaluation guided by the Core Components Framework (CCF). Sixty interviews and 18 focus groups were held with invested partners. Transcripts were analyzed abductively using CCF, and ideal implementation was compared with enacted implementation. These findings were then shared back with program leaders, adaptations were subsequently developed, and technical reports were generated for each program. Researchers reviewed the technical reports to identify themes related to the burden of assessment with a subsequent focus on identifying adaptations across programs. Three themes were identified: (1) disparate mental models of assessment processes in CBME, (2) challenges in workplace-based assessment processes, and (3) challenges in performance review and decision making. Theme 1 included entrustment interpretation and lack of shared mindset for performance standards. Adaptations included revising entrustment scales, faculty development, and formalizing resident membership. Theme 2 involved direct observation, timeliness of assessment completion, and feedback quality. Adaptations included alternative assessment strategies beyond entrustable professional activity forms and proactive assessment planning. Theme 3 related to resident data monitoring and competence committee decision making. Adaptations included adding resident representatives to the competence committee and assessment platform enhancements. These adaptations represent responses to the concerning signal of significant burden of assessment within CBME being experienced broadly. The authors hope other programs may learn from their institution's experience and navigate the CBME-related assessment burden their invested partners may be facing., (Copyright © 2023 by the Association of American Medical Colleges.)

Objectives: Simulation-based technical skills training is now ubiquitous in medicine, particularly for high acuity, low occurrence (HALO) procedures. Mastery learning and deliberate practice (ML + DP) are potentially valuable educational methods, however, they are resource intensive. We sought to compare the effect of deliberate practice and mastery learning versus self-guided practice on skill performance of the rare, life-saving procedure, a bougie-assisted cricothyroidotomy (BAC)., Methods: We conducted a multi-center, randomized study at five North American emergency medicine (EM) residency programs. We randomly assigned 176 EM residents to either the ML + DP or self-guided practice groups. Three blinded airway experts independently evaluated BAC skill performance by video review before (pre-test), after (post-test) and 6-12 months (retention) after the training session. The primary outcome was post-test skill performance using a global rating score (GRS). Secondary outcomes included performance time and skill performance at the retention test., Results: Immediately following training, GRS scores were significantly higher as mean performance improved from pre-test, (22, 95% CI = 21-23) to post-test (27, 95% CI = 26-28), (p < 0.001) for all participants. However, there was no difference between the groups on GRS scores (p = 0.2) at the post-test or at the retention test (p = 0.2). At the retention test, participants in the ML + DP group had faster performance times (66 s, 95% CI = 57-74) compared to the self-guided group (77 s, 95% CI = 67-86), (p < 0.01)., Conclusions: There was no significant difference in skill performance between groups. Residents who received deliberate practice and mastery learning demonstrated an improvement in skill performance time., (© 2023. The Author(s), under exclusive licence to Canadian Association of Emergency Physicians (CAEP)/ Association Canadienne de Médecine d'Urgence (ACMU).)

Objective: The purpose of this study was to identify the learning needs of emergency physicians returning to Emergency Medicine (EM) practice after clinical leaves of less than 2 years, summarize existing return to practice programs, and propose recommendations regarding ideal educational and support structures for these physicians both during their practice gaps and upon return to EM., Methods: A multiple-phased study was conducted to establish recommendations regarding ideal educational and support structures for emergency physicians returning from practice gaps of less than 2 years. The overall design involved an initial environmental scan of existing and exemplar programs and regulatory body positions, followed by interviews with EM Department Heads from across Canada, and then subsequent content analysis and recommendation derivation by EM medical education expert group consensus. These summary recommendations were further revised by consensus at the 2022 CAEP conference academic symposium to derive a final set of consensus recommendations., Conclusions: We have developed a set of recommendations regarding ideal educational and support structures for physicians experiencing gaps in practice of less than 2 years. This set of recommendations was informed by review of existing and exemplar programs, policies and experience of regulatory bodies, interviews with EM Department Heads across Canada, and a subsequent process of consensus at the 2022 CAEP conference academic symposium. It is hoped that this set of recommendations will inform discussions and potential strategies employed by departments to facilitate the smooth and effective return to EM practice for individuals experiencing gaps., (© 2023. The Author(s), under exclusive licence to Canadian Association of Emergency Physicians (CAEP)/ Association Canadienne de Médecine d'Urgence (ACMU).)

Background: Simulation-based assessment can complement workplace-based assessment of rare or difficult to assess Entrustable Professional Activities (EPAs). We aimed to compare the use of simulation-based assessment for resuscitation-focused EPAs in three postgraduate medical training programs and describe faculty perceptions of simulation-based assessment., Methods: EPA assessment scores and setting (simulation or workplace) were extracted from 2017-2020 for internal medicine, emergency medicine, and surgical foundations residents at the transition to discipline and foundations of discipline stages. A questionnaire was distributed to clinical competency committee members., Results: Eleven percent of EPA assessments were simulation-based. The proportion of simulation-based assessment did not differ between programs but differed between transition (38%) and foundations (4%) stages within surgical foundations only. Entrustment scores differed between settings in emergency medicine at the transition level only (simulation: 4.82 ± 0.60 workplace: 3.74 ± 0.93). 70% of committee members (n=20) completed the questionnaire. Of those that use simulation-based assessment, 45% interpret them differently than workplace-based assessments. 73% and 100% trust simulation for high-stakes and low-stakes assessment, respectively., Conclusions: The proportion of simulation-based assessment for resuscitation focused EPAs did not differ between three postgraduate medical training programs. Interpretation of simulation-based assessment data between committee members was inconsistent. All respondents trust simulation-based assessment for low-stakes, and the majority for high-stakes assessment. These findings have practical implications for the integration simulation into programs of assessment., Competing Interests: There are no conflicts of interest declared by the authors, financial or otherwise., (© 2023 Seed, Gauthier, Zevin, Hall, Chaplin; licensee Synergies Partners.)

Objectives: To evaluate the impact and feasibility of multisource feedback compared with traditional feedback for trauma team captains (TTCs)., Design: A mixed-methods, non-randomised prospective study., Setting: A level one trauma centre in Ontario, Canada., Participants: Postgraduate medical residents in emergency medicine and general surgery participating as TTCs. Selection was based on a convenience sampling method., Intervention: Postgraduate medical residents participating as TTCs received either multisource feedback or standard feedback following trauma cases., Main Outcome Measures: TTCs completed questionnaires designed to measure the self-reported intention to change practice (catalytic effect), immediately following a trauma case and 3 weeks later. Secondary outcomes included measures of perceived benefit, acceptability, and feasibility from TTCs and other trauma team members., Results: Data were collected following 24 trauma team activations: TTCs from 12 activations received multisource feedback and 12 received standard feedback. The self-reported intention for practice change was not significantly different between groups initially (4.0 vs 4.0, p=0.57) and at 3 weeks (4.0 vs 3.0, p=0.25). Multisource feedback was perceived to be helpful and superior to the existing feedback process. Feasibility was identified as a challenge., Conclusions: The self-reported intention for practice change was no different for TTCs who received multisource feedback and those who received standard feedback. Multisource feedback was favourably received by trauma team members, and TTCs perceived multisource feedback as useful for their development., Competing Interests: Competing interests: AKH reports receiving payment for working with the Royal College of Physicians and Surgeons of Canada on the implementation and evaluation of Competency By Design in Canada. No other conflicts of interest to declare., (© Author(s) (or their employer(s)) 2023. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.)

Objectives: In 2017, Queen's University launched Competency-Based Medical Education (CBME) across 29 programs simultaneously. Two years post-implementation, we asked key stakeholders (faculty, residents, and program leaders) within the Pediatrics program for their perspectives on and experiences with CBME so far., Methods: Program leadership explicitly described the intended outcomes of implementing CBME. Focus groups and interviews were conducted with all stakeholders to describe the enacted implementation. The intended versus enacted implementations were compared to provide insight into needed adaptations for program improvement., Results: Overall, stakeholders saw value in the concept of CBME. Residents felt they received more specific feedback and monthly Competence Committee (CC) meetings and Academic Advisors were helpful. Conversely, all stakeholders noted the increased expectations had led to a feeling of assessment fatigue. Faculty noted that direct observation and not knowing a resident's previous performance information was challenging. Residents wanted to see faculty initiate assessments and improved transparency around progress and promotion decisions., Discussion: The results provided insight into how well the intended outcomes had been achieved as well as areas for improvement. Proposed adaptations included a need for increased direct observation and exploration of faculty accessing residents' previous performance information. Education was provided on the performance expectations of residents and how progress and promotion decisions are made. As well, "flex blocks" were created to help residents customize their training experience to meet their learning needs. The results of this study can be used to inform and guide implementation and adaptations in other programs and institutions., Competing Interests: AKH is a Clinician Educator at the Royal College of Physicians and Surgeons of Canada (RCPSC) and is also responsible for leading Competency-Based Medical Education (CBME) Program Evaluation across multiple programs at the RCPSC. There are no other disclosures., (© The Author(s) 2023. Published by Oxford University Press on behalf of the Canadian Paediatric Society. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.)

Competing Interests: Dr. Brent Thoma has received payments for teaching, research, and administrative work from the University of Saskatchewan College of Medicine, payments for teaching and administrative work from the Royal College of Physicians and Surgeons of Canada, honoraria for teaching or writing from Harvard Medical School, the New England Journal of Medicine, the University of Cincinnati Children's Hospital, and NYC Health + Hospitals, and research grant funding from the Government of Ontario and the Canadian Association of Emergency Physicians. P. Alison Paprica receives funding from the Canadian Institute of Health Research and other provincial and national research funders. She is affiliated with the Institute for Health Policy, Management and Evaluation at the University of Toronto, ICES and Health Data Research Network Canada. Dr. Padma Kaul has received payments for teaching, research and administrative work from the University of Alberta. She holds a Canadian Institute of Health Research Chair in Sex and Gender Science and is supported by a Heart & Stroke Foundation Chair in Cardiovascular Research. Dr. Warren J. Cheung has received payments for teaching, research and administrative work from the University of Ottawa as well as payments for teaching and administrative work from the Royal College of Physicians and Surgeons of Canada. Dr. Andrew K. Hall has received payments for teaching, research and administrative work from the University of Ottawa and Queen’s University, as well as payments for teaching and administrative work from the Royal College of Physicians and Surgeons of Canada. Ewan Affleck has received honoraria for work from the Auditor General of Ontario, The Canadian Medical Association, and Health Excellence Canada

Background: Global Health opportunities are popular, with many reported benefits. There is a need however, to identify and situate Global Health competencies within postgraduate medical education. We sought to identify and map Global Health competencies to the CanMEDS framework to assess the degree of equivalency and uniqueness between them., Methods: JBI scoping review methodology was utilized to identify relevant papers searching MEDLINE, Embase, and Web of Science. Studies were reviewed independently by two of three researchers according to pre-determined eligibility criteria. Included studies identified competencies in Global Health training at the postgraduate medicine level, which were then mapped to the CanMEDS framework., Results: A total of 19 articles met criteria for inclusion (17 from literature search and two from manual reference review). We identified 36 Global Health competencies; the majority (23) aligned with CanMEDS competencies within the framework. Ten were mapped to CanMEDS roles but lacked specific key or enabling competencies, while three did not fit within the specific CanMEDS roles., Conclusions: We mapped the identified Global Health competencies, finding broad coverage of required CanMEDS competencies. We identified additional competencies for CanMEDS committee consideration and discuss the benefits of their inclusion in future physician competency frameworks., Competing Interests: None of the authors have any conflicts of interest to declare., (© 2023 Pritchard, Alavian, Soogoor, Bartels, Hall; licensee Synergies Partners.)

Background: The CanMEDS physician competency framework will be updated in 2025. The revision occurs during a time of disruption and transformation to society, healthcare, and medical education caused by the COVID-19 pandemic and growing acknowledgement of the impacts of colonialism, systemic discrimination, climate change, and emerging technologies on healthcare and training. To inform this revision, we sought to identify emerging concepts in the literature related to physician competencies., Methods: Emerging concepts were defined as ideas discussed in the literature related to the roles and competencies of physicians that are absent or underrepresented in the 2015 CanMEDS framework. We conducted a literature scan, title and abstract review, and thematic analysis to identify emerging concepts. Metadata for all articles published in five medical education journals between October 1, 2018 and October 1, 2021 were extracted. Fifteen authors performed a title and abstract review to identify and label underrepresented concepts. Two authors thematically analyzed the results to identify emerging concepts. A member check was conducted., Results: 1017 of 4973 (20.5%) of the included articles discussed an emerging concept. The thematic analysis identified ten themes: Equity, Diversity, Inclusion, and Social Justice; Anti-racism; Physician Humanism; Data-Informed Medicine; Complex Adaptive Systems; Clinical Learning Environment; Virtual Care; Clinical Reasoning; Adaptive Expertise; and Planetary Health. All themes were endorsed by the authorship team as emerging concepts., Conclusion: This literature scan identified ten emerging concepts to inform the 2025 revision of the CanMEDS physician competency framework. Open publication of this work will promote greater transparency in the revision process and support an ongoing dialogue on physician competence. Writing groups have been recruited to elaborate on each of the emerging concepts and how they could be further incorporated into CanMEDS 2025., Competing Interests: Thoma, Atkinson, Hall, Frank, Snell, Anderson, and Van Melle have received stipends from the Royal College of Physicians and Surgeons of Canada. Thoma also reports payments for teaching, research, and administrative work from the University of Saskatchewan College of Medicine and teaching honoraria from various institutions within the past 3 years (Harvard Medical School, the New England Journal of Medicine, the University of Cincinnati Children's Hospital, and NYC Health + Hospitals). Samson receives stipends from the Collège des médecins du Québec and the Université de Montréal. Giuliani has an unrelated conflict-of-interest with AstraZeneca and Bristol Myers Squibb. Chan reports honoraria from McMaster University for her education research work with the McMaster Education Research, Innovation, and Theory (MERIT) group and administrative stipend for her role of Associate Dean via the McMaster Faculty of Health Sciences Office of Continuing Professional Development. Chan also reports teaching honoraria from various institutions within the past three years (UBC, UNBC, Baylor College of Medicine, Harvard University, NOSM, Catholic University of Korea, Taiwan Veteran’s General Hospital, Prince of Songkla University). Waters reports honoraria and salary support for academic contributions from McMaster University. Chan and Waters have received educational research grant funding from the Royal College of Physicians and Surgeons of Canada. Fowler is a paid employee of the College of Family Physicians of Canada. Tourian receives a salary from McGill University for his administrative work as the Assistant Dean of Postgraduate Medical Education. Constantin received a stipend from the Collège des médecins du Québec as an expert advisor; she also receives a salary from McGill University for her administrative and education work within Postgraduate Medical Education as well as within the Office of International Affairs. Karwowska receives a stipend from the Association of Faculties of medicine of Canada., (© 2023 Thoma, Karwowska, Samson, Labine, Waters, Giuliani, Chan, Atkinson, Constantin, Hall, Gomez-Garibello, Fowler, Tourian, Frank Anderson, Snell, Van Melle; licensee Synergies Partners.)

Introduction Emergency medicine (EM) postgraduate medical education in Canada has transitioned from traditional time-based training to competency-based medical education (CBME). In order to promote residents through stages of training, simulated assessments are needed to evaluate residents in high-stakes but low-frequency medical emergencies. There remains a gap in the literature pertaining to the use of evaluative tools in simulation, such as the Resuscitation Assessment Tool (RAT) in the new CBME curriculum design. Methods We completed a pilot study of resident physicians in one Canadian EM training program to evaluate the effectiveness and reliability of a simulation-based RAT for pediatric resuscitation. We recorded 10 EM trainees completing simulated scenarios and had nine EM physicians use the RAT tool to evaluate their performances. Generalizability theory was used to evaluate the reliability of the RAT tool. Results The mean RAT score for the management of pediatric myocarditis, cardiac arrest, and septic shock (appendicitis) across raters was 3.70, 3.73, and 4.50, respectively. The overall generalizability coefficient for testing simulated pediatric performance competency was 0.77 for internal consistency and 0.75 for absolute agreement. The performance of senior participants was superior to that of junior participants in the management of pediatric myocarditis (p = 0.01) but not statistically significant in the management of pediatric septic shock (p=0.77) or cardiac arrest (p =0.61). Conclusion Overall, our findings suggest that with an appropriately chosen simulated scenario, the RAT tool can be used effectively for the simulation of high-stakes and low-frequency scenarios for practice to enhance the new CBME curriculum in emergency medicine training programs., Competing Interests: The authors have declared that no competing interests exist., (Copyright © 2023, Mackenzie et al.)

Importance: Prior studies have revealed gender differences in the milestone and clinical competency committee assessment of emergency medicine (EM) residents., Objective: To explore gender disparities and the reasons for such disparities in the narrative comments from EM attending physicians to EM residents., Design, Setting, and Participants: This multicenter qualitative analysis examined 10 488 narrative comments among EM faculty and EM residents between 2015 to 2018 in 5 EM training programs in the US. Data were analyzed from 2019 to 2021., Main Outcomes and Measures: Differences in narrative comments by gender and study site. Qualitative analysis included deidentification and iterative coding of the data set using an axial coding approach, with double coding of 20% of the comments at random to assess intercoder reliability (κ, 0.84). The authors reviewed the unmasked coded data set to identify emerging themes. Summary statistics were calculated for the number of narrative comments and their coded themes by gender and study site. χ2 tests were used to determine differences in the proportion of narrative comments by gender of faculty and resident., Results: In this study of 283 EM residents, of whom 113 (40%) identified as women, and 277 EM attending physicians, of whom 95 (34%) identified as women, there were notable gender differences in the content of the narrative comments from faculty to residents. Men faculty, compared with women faculty, were more likely to provide either nonspecific comments (115 of 182 [63.2%] vs 40 of 95 [42.1%]), or no comments (3387 of 10 496 [32.3%] vs 1169 of 4548 [25.7%]; P < .001) to men and women residents. Compared with men residents, more women residents were told that they were performing below level by men and women faculty (36 of 113 [31.9%] vs 43 of 170 [25.3%]), with the most common theme including lack of confidence with procedural skills., Conclusions and Relevance: In this qualitative study of narrative comments provided by EM attending physicians to residents, multiple modifiable contributors to gender disparities in assessment were identified, including the presence, content, and specificity of comments. Among women residents, procedural competency was associated with being conflated with procedural confidence. These findings can inform interventions to improve parity in assessment across graduate medical education.

Purpose: Organizational readiness is critical for successful implementation of an innovation. We evaluated program readiness to implement Competence by Design (CBD), a model of Competency-Based Medical Education (CBME), among Canadian postgraduate training programs., Methods: A survey of program directors was distributed 1 month prior to CBD implementation in 2019. Questions were informed by the R = MC 2 framework of organizational readiness and addressed: program motivation, general capacity for change, and innovation-specific capacity. An overall readiness score was calculated. An ANOVA was conducted to compare overall readiness between disciplines., Results: Survey response rate was 42% ( n = 79). The mean overall readiness score was 74% (30-98%). There was no difference in scores between disciplines. The majority of respondents agreed that successful implementation of CBD was a priority (74%), and that their leadership (94%) and faculty and residents (87%) were supportive of change. Fewer perceived that CBD was a move in the right direction (58%) and that implementation was a manageable change (53%). Curriculum mapping, competence committees and programmatic assessment activities were completed by >90% of programs, while <50% had engaged off-service disciplines., Conclusion: Our study highlights important areas where programs excelled in their preparation for CBD, as well as common challenges that serve as targets for future intervention to improve program readiness for CBD implementation.

Objective Emergency Medicine (EM) Sim Cases was initially developed in 2015 as a free open-access simulation resource. To ensure the future of EM Sim Cases remains relevant and up to date, we performed a needs assessment to better define our audience and facilitate long-term goals. Methods We delivered a survey using a modified massive-online-needs-assessment methodology through an iterative process with simulation experts from the EM Simulation Educators Research Collaborative. We distributed the survey via email and Twitter and analyzed the data using descriptive statistics and thematic analysis. Results We obtained 106 responses. EM Sim Cases is commonly used by physicians primarily as an educational resource for postgraduate level trainees. Perceived needs included resuscitation, pediatrics, trauma, and toxicology content. Prompted needs included non-simulation-case educational resources, increased case database, and improved website organization. Conclusions Data collected from our needs assessment has defined our audience allowing us to design our long-term goals and strategies., Competing Interests: The authors have declared that no competing interests exist., (Copyright © 2022, Dinh et al.)

Purpose: This study evaluated the fidelity of competence committee (CC) implementation in Canadian postgraduate specialist training programs during the transition to competency-based medical education (CBME)., Methods: A national survey of CC chairs was distributed to all CBME training programs in November 2019. Survey questions were derived from guiding documents published by the Royal College of Physicians and Surgeons of Canada reflecting intended processes and design., Results: Response rate was 39% (113/293) with representation from all eligible disciplines. Committee size ranged from 3 to 20 members, 42% of programs included external members, and 20% included a resident representative. Most programs (72%) reported that a primary review and synthesis of resident assessment data occurs prior to the meeting, with some data reviewed collectively during meetings. When determining entrustable professional activity (EPA) achievement, most programs followed the national specialty guidelines closely with some exceptions (53%). Documented concerns about professionalism, EPA narrative comments, and EPA entrustment scores were most highly weighted when determining resident progress decisions., Conclusions: Heterogeneity in CC implementation likely reflects local adaptations, but may also explain some of the variable challenges faced by programs during the transition to CBME. Our results offer educational leaders important fidelity data that can help inform the larger evaluation and transformation of CBME.

Program evaluation is an "essential responsibility" but is often not seen as a scholarly pursuit. While Boyer expanded what qualifies as educational scholarship, many still need to engage in processes that are rigorous and of a requisite academic standard to be labelled as scholarly. Many medical educators may feel that scholarly program evaluation is a daunting task due to the competing interests of curricular change, remediation, and clinical care. This paper explores how educators can take their questions around outcomes and efficacy of our programs and efficiently engage in education scholarship. The authors outline how educators can examine whether training programs have a desired impact and outcomes, and then how they might leverage this process into education scholarship., Competing Interests: Dr. Shera Hosseini has received funding for her postdoctoral fellowship from the McMaster Institute for Research in Aging (MIRA). Dr. Yilmaz is the recipient of a 2019 TUBITAK Postdoctoral Fellowship grant. Dr. Shah—none; and no grants. Dr. Gottlieb holds grants for unrelated work with the Centers for Disease Control and Prevention, Council of Residency Directors in Emergency Medicine, Society for Academic Emergency Medicine, and eCampus Ontario. Dr. Stehman—none, Dr. Hall—holds grants for unrelated work from the Royal College of Physicians and Surgeons of Canada, Queen’s University Center for Teaching and Learning, and the Physician Services Incorporated Foundation. Dr. Chan holds grants for unrelated work from McMaster University, the PSI foundation, Society for Academic Emergency Medicine, eCampus Ontario, the University of Saskatchewan, and Royal College of Physicians and Surgeons of Canada., (© 2022 by the Society for Academic Emergency Medicine.)

Objectives: Systematic program evaluation of the Queen's University diagnostic radiology residency program following transition to a competency-based medical education (CBME) curriculum., Methods: Rapid Evaluation methodology and the Core Components Framework were utilized to measure CBME implementation. A combination of interviews and focus groups were held with program leaders (n = 6), faculty (n = 10), both CBME stream and traditional stream residents (n = 6), and program staff (n = 2). Interviews and focus groups were transcribed and analyzed abductively. Study team met with program leaders to review common themes and plan potential adaptations., Results: Strengths of CBME implementation included more frequent and timely feedback as well as the role of the Academic Advisor. However, frontline faculty felt insufficiently supported with regards to the theory and practical implementation of the new curriculum and found assessment tools unintuitive. The circumstances surrounding the curricular implementation also resulted in some negative sentiment. Additional faculty and resident education workshops were identified as areas for improvement as well as changes to assessment tools for increased clarity. Residents overall viewed the changes favorably, with traditional stream residents indicating that they also had a desire for increased feedback., Conclusions: Rapid Evaluation is an effective method for program assessment following curricular change in diagnostic radiology. A departmental champion driving enthusiasm for change from within may be valuable. Adequate resident and faculty education is key to maximize change and smooth the transition. Advances in knowledge: This study provides insights for other radiology training programs transitioning to a CBME framework and provides a structure for programmatic assessment., (Copyright © 2021 Elsevier B.V. All rights reserved.)

Aim: Endoscopic discectomies provide several advantages over other techniques such as traditional open lumbar discectomy (OLD) including possibly decreased complications, shorter hospital stay and an earlier return to work. Methods: An electronic database search including MEDLINE/PubMed, EMBASE, Scopus, Cochrane Database of Systematic Reviews and Cochrane Controlled trials (CENTRAL) were reviewed for randomized controlled trials (RCTs) only. Results: A total of nine RCTs met inclusion criteria. Three showed benefit of endoscopic discectomy over the comparator with regards to pain relief, with the remaining six studies showing no difference in pain relief or function. Conclusion: Based on review of the nine included studies, we can conclude that endoscopic discectomy is as effective as other surgical techniques, and has additional benefits of lower complication rate and superior perioperative parameters.

New opportunities have arisen for development of therapies for rare diseases with the increased focus and progress in the field. However, standardised framework integrating individual initiatives has not been formed. We present lessons learned and best practice from a collaborative success case in developing a treatment for a rare genetic disease. Our unique consortium model incorporated several of the identified developments under one project, DevelopAKUre, truly bringing together academia, industry and patient organisations in clinical drug development. We found that the equal partnership between all parties in our consortium was a key success factor creating a momentum based on a strong organisational culture where all partners had high engagement and taking ownership of the entire programme. With an agreed mutual objective, this provided synergies through connecting the strengths of the individual parties. Another key success factor was the central role of the patient organisation within the management team, and their unique study participants' advocacy role securing the understanding and meeting the needs of the clinical study participants in real-time. This resulted in an accelerated enrolment into the clinical studies with a high retention rate allowing for delivery of the programme with significantly improved timelines. Our project was partly funded through an external EU research grant, enabling our model with equal partnership. Further attention within the community should be given to establishing a functional framework where sustainable funding and risk sharing between private and public organisations allow for our model to be replicated., (© 2021. The Author(s).)

Purpose: Competency-based medical education (CBME) has prompted widespread implementation of workplace-based assessment (WBA) tools using entrustment anchors. This study aimed to identify factors that influence faculty's rating choices immediately following assessment and explore their experiences using WBAs with entrustment anchors, specifically the Ottawa Surgical Competency Operating Room Evaluation scale., Method: A convenience sample of 50 semi-structured interviews with Emergency Medicine (EM) physicians from a single Canadian hospital were conducted between July and August 2019. All interviews occurred within two hours of faculty completing a WBA of a trainee. Faculty were asked what they considered when rating the trainee's performance and whether they considered an alternate rating. Two team members independently analysed interview transcripts using conventional content analysis with line-by-line coding to identify themes., Results: Interviews captured interactions between 70% (26/37) of full-time EM faculty and 86% (19/22) of EM trainees. Faculty most commonly identified the amount of guidance the trainee required as influencing their rating. Other variables such as clinical context, trainee experience, past experiences with the trainee, perceived competence and confidence were also identified. While most faculty did not struggle to assign ratings, some had difficulty interpreting the language of entrustment anchors, being unsure whether their assessment should be retrospective or prospective in nature, and if/how the assessment should change whether they were 'in the room' or not., Conclusions: By going to the frontline during WBA encounters, this study captured authentic and honest reflections from physicians immediately engaged in assessment using entrustment anchors. While many of the factors identified are consistent with previous retrospective work, we highlight how some faculty consider factors outside the prescribed approach and struggle with the language of entrustment anchors. These results further our understanding of 'in-the-moment' assessments using entrustment anchors and may facilitate effective faculty development regarding WBA in CBME., (© 2021 John Wiley & Sons Ltd and The Association for the Study of Medical Education.)

Background: In the emergency department (ED), chest radiographs (CXRs) provide essential information for clinical diagnostic reasoning. Errors in interpretation by emergency physicians can lead to negative patient outcomes. To aid in teaching this important skill, an understanding of cognitive processes and cognitive load (CL) in CXR interpretation in emergency medicine (EM) personnel is warranted., Methods: This study adopted a concurrent mixed-methods research design. Participant groups included medical students (M), junior (J) and senior (S) EM residents, and attending emergency physicians (P) in the ED at an academic hospital. To elucidate cognitive processes, a real-time cognitive task analysis during CXR interpretation was performed. Interviews were audio recorded, transcribed verbatim, and analyzed thematically. The interview was followed by a questionnaire, where participants rated their CL, stress, and confidence level., Results: Levels of CL (M vs. S and M vs. P, p = 0.002; J vs. S, p   = 0.004; J vs. P, p = 0.005) and stress (J vs. P, p = 0.002) decreased, while confidence levels increased (M vs. S, p = 0.006; J vs. S, p ≤ 0.001; J vs. P, p   = 0.003) as experience level increased. Qualitative analysis of interviews revealed four themes: checking behavior, information reduction, pattern recognition versus systematic viewing, and recognizing scope of practice. Experts commonly utilized checking behavior (e.g., comparison to prior radiographs) and deprioritized task irrelevant data. Experts used a general overview technique as their initial approach as opposed to a systematic viewing approach, and they more readily recognized an EM physicians' scope of practice in this task., Conclusion: This study characterized differences in cognition that led to increased CL, stress, and lower level of confidence in EM learners during CXR interpretation and provided insight into expertise development in this important skill., Competing Interests: The authors have no potential conflicts to disclose., (© 2021 Society for Academic Emergency Medicine.)

Background: Emergency physicians often experience a high cognitive load (CL) due to the inherent nature of working in acute care settings. CL has traditionally been measured in educational studies but has not been well studied in the clinical environment., Methods: Emergency medicine attending physicians and residents working in an academic urgent care center completed psychometric questionnaires while on shift to measure overall CL, intrinsic cognitive load (ICL), extraneous cognitive load (ECL), and acute stress. Data regarding the patient load, patient acuity, and the number of patients in the waiting room were also collected. Correlational analysis and simple linear regression were used to evaluate predictors of CL on shift., Results: Forty-two questionnaires were completed (26 by attending physicians, 16 by residents). Attending physicians carried a significantly higher patient load compared to residents ( p  < 0.001). No differences in mean overall CL, ICL, ECL, and acute stress were observed between attending physicians and residents. Bivariate analysis demonstrated associations between ICL, ECL, acute stress, and overall CL in attending physicians. In residents, acute stress was the only variable associated with overall CL and the number of high-acuity patients was associated with ICL., Conclusions: Factors influencing reported CL during clinical work are different between attending emergency physicians and residents. Further study to appreciate the impact of these differences is required and may help educators elucidate strategies to better manage CL, thereby improving clinical performance and potentially improving patient care., Competing Interests: The authors have no potential conflicts to disclose., (© 2021 Society for Academic Emergency Medicine.)

Introduction: Simulation is becoming a popular educational modality for physician continuing professional development (CPD). This study sought to characterize how simulation-based CPD (SBCPD) is being used in Canada and what academic emergency physicians (AEPs) desire in an SBCPD program., Methods: Two national surveys were conducted from March to June 2018. First, the SBCPD Needs Assessment Survey was administered online to all full-time AEPs across 9 Canadian academic emergency medicine (EM) sites. Second, the SBCPD Status Survey was administered by telephone to the department representatives (DRs)-simulation directors or equivalent-at 20 Canadian academic EM sites., Results: Response rates for the SBCPD Needs Assessment and the SBCPD Status Survey were 40% (252/635) and 100% (20/20) respectively. Sixty percent of Canadian academic EM sites reported using SBCPD, although only 30% reported dedicated funding support. Academic emergency physician responses demonstrated a median annual SBCPD of 3 hours. Reported incentivization for SBCPD participation varied with AEPs reporting less incentivization than DRs. Academic emergency physicians identified time commitments outside of shift, lack of opportunities, and lack of departmental funding as their top barriers to participation, whereas DRs thought AEPs fear of peer judgment and inexperience with simulation were substantial barriers. Content areas of interest for SBCPD were as follows: rare procedures, pediatric resuscitation, and neonatal resuscitation. Lastly, interprofessional involvement in SBCPD was valued by both DRs and AEPs., Conclusions: Simulation-based CPD programs are becoming common in Canadian academic EM sites. Our findings will guide program coordinators in addressing barriers to participation, selecting content, and determining the frequency of SBCPD events., Competing Interests: The authors declare no conflict of interest., (Copyright © 2020 Society for Simulation in Healthcare.)

The international movement to competency-based medical education (CBME) marks a major transition in medical education that requires a shift in educators' and learners' approach to clinical experiences, the way assessment data are collected and integrated, and in learners' mindsets. Learners entering a CBME curriculum must actively drive their learning experiences and education goals. For some, this expectation may be a significant change from their previous approach to learning in medicine. This paper highlights 12 tips to help learners succeed within a CBME model.

The ongoing adoption of competency-based medical education (CBME) across health professions training draws focus to learner-centred educational design and the importance of fostering a growth mindset in learners, teachers, and educational programs. An emerging body of literature addresses the instructional practices and features of learning environments that foster the skills and strategies necessary for trainees to be partners in their own learning and progression to competence and to develop skills for lifelong learning. Aligned with this emerging area is an interest in Dweck's self theory and the concept of the growth mindset. The growth mindset is an implicit belief held by an individual that intelligence and abilities are changeable, rather than fixed and immutable. In this paper, we present an overview of the growth mindset and how it aligns with the goals of CBME. We describe the challenges associated with shifting away from the fixed mindset of most traditional medical education assumptions and practices and discuss potential solutions and strategies at the individual, relational, and systems levels. Finally, we present future directions for research to better understand the growth mindset in the context of CBME.

Medical education is situated within health care and educational organizations that frequently lag in their use of data to learn, develop, and improve performance. How might we leverage competency-based medical education (CBME) assessment data at the individual, program, and system levels, with the goal of redefining CBME from an initiative that supports the development of physicians to one that also fosters the development of the faculty, administrators, and programs within our organizations? In this paper we review the Deliberately Developmental Organization (DDO) framework proposed by Robert Kegan and Lisa Lahey, a theoretical framework that explains how organizations can foster the development of their people. We then describe the DDO's conceptual alignment with CBME and outline how CBME assessment data could be used to spur the transformation of health care and educational organizations into digitally integrated DDOs. A DDO-oriented use of CBME assessment data will require intentional investment into both the digitalization of assessment data and the development of the people within our organizations. By reframing CBME in this light, we hope that educational and health care leaders will see their investments in CBME as an opportunity to spur the evolution of a developmental culture.

There is an urgent need to capture the outcomes of the ongoing global implementation of competency-based medical education (CBME). However, the measurement of downstream outcomes following educational innovations, such as CBME is fraught with challenges stemming from the complexities of medical training, the breadth and variability of inputs, and the difficulties attributing outcomes to specific educational elements. In this article, we present a logic model for CBME to conceptualize an impact pathway relating to CBME and facilitate outcomes evaluation. We further identify six strategies to mitigate the challenges of outcomes measurement: (1) clearly identify the outcome of interest, (2) distinguish between outputs and outcomes, (3) carefully consider attribution versus contribution, (4) connect outcomes to the fidelity and integrity of implementation, (5) pay attention to unanticipated outcomes, and (6) embrace methodological pluralism. Embracing these challenges, we argue that careful and thoughtful evaluation strategies will move us forward in answering the all-important question: Are the desired outcomes of CBME being achieved?

Programmatic assessment as a concept is still novel for many in clinical education, and there may be a disconnect between the academics who publish about programmatic assessment and the front-line clinical educators who must put theory into practice. In this paper, we clearly define programmatic assessment and present high-level guidelines about its implementation in competency-based medical education (CBME) programs. The guidelines are informed by literature and by lessons learned from established programmatic assessment approaches. We articulate five steps to consider when implementing programmatic assessment in CBME contexts: articulate the purpose of the program of assessment, determine what must be assessed, choose tools fit for purpose, consider the stakes of assessments, and define processes for interpreting assessment data. In the process, we seek to offer a helpful guide or template for front-line clinical educators. We dispel some myths about programmatic assessment to help training programs as they look to design-or redesign-programs of assessment. In particular, we highlight the notion that programmatic assessment is not 'one size fits all'; rather, it is a system of assessment that results when shared common principles are considered and applied by individual programs as they plan and design their own bespoke model of programmatic assessment for CBME in their unique context.

Competency-based medical education has been advocated as the future of medical education for nearly a half-century. Inherent to this is the promise that advancement and transitions in training would be defined by readiness to practice rather than by time. Of the logistical problems facing competency-based, time-variable (CBTV) training, enacting time variability may be the largest hurdle to clear. Although it is true that an 'all or nothing' approach to CBTV training would require massive overhauls of both medical education and health care systems, the authors propose that training institutions should gradually evolve within their current environments to incrementally move toward the best version of CBTV training for learners, supervisors, and patients. In support of this evolution, the authors seek to demonstrate the feasibility of advancing toward the goal of realistic CBTV training by detailing examples of successful CBTV training and describing key features of initial steps toward CBTV training implementation.

As the global transformation of postgraduate medical training continues, there are persistent calls for program evaluation efforts to understand the impact and outcomes of competency-based medical education (CBME) implementation. The measurement of a complex educational intervention such as CBME is challenging because of the multifaceted nature of activities and outcomes. What is needed, therefore, is an organizational taxonomy to both conceptualize and categorize multiple outcomes. In this manuscript we propose a taxonomy that builds on preceding works to organize CBME outcomes across three domains: focus (educational, clinical), level (micro, meso, macro), and timeline (training, transition to practice, practice). We also provide examples of how to conceptualize outcomes of educational interventions across medical specialties using this taxonomy. By proposing a shared language for outcomes of CBME, we hope that this taxonomy will help organize ongoing evaluation work and catalyze those seeking to engage in the evaluation effort to help understand the impact and outcomes of CBME.

Phenomenon: Visual expertise in medicine involves a complex interplay between expert visual behavior patterns and higher-level cognitive processes. Previous studies of visual expertise in medicine have centered around traditionally visually intensive disciplines such as radiology and pathology. However, there is limited study of visual expertise in electrocardiogram (ECG) interpretation, a common clinical task that is associated with high error rates. This qualitatively driven multi-methods study aimed to describe differences in cognitive approaches to ECG interpretation between medical students, emergency medicine (EM) residents, and EM attending physicians. Approach: Ten medical students, 10 EM residents, and 10 EM attending physicians were recruited from one tertiary academic center to participate in this study. Participants interpreted 10 ECGs with a screen-based eye-tracking device, then underwent a subjective re situ interview augmented by playback of the participants' own gaze scan-paths via eye-tracking. Interviews were transcribed verbatim and an emergent thematic analysis was performed across participant groups. Diagnostic speed, accuracy, and heat maps of fixation distribution were collected to supplement the qualitative findings. Findings: Qualitative analysis demonstrated differences among the cohorts in three major themes: dual-process reasoning, ability to prioritize, and clinical implications. These qualitative findings were aligned with differences in visual behavior demonstrated by heat maps of fixation distribution across each ECG. More experienced participants completed ECG interpretation significantly faster and more accurately than less experienced participants. Insights: The cognitive processes related to ECG interpretation differed between novices and more experienced providers in EM. Understanding the differences in cognitive approaches to ECG interpretation between these groups may help inform best practices in teaching this ubiquitous diagnostic skill.

Study Objective: Little is known about the cause or optimal treatment of hyperemesis in habitual cannabis users. Anecdotal evidence supports the use of haloperidol over traditional antiemetics for this newly recognized disorder. We compare haloperidol with ondansetron for cannabis hyperemesis syndrome., Methods: We randomized cannabis users with active emesis to either haloperidol (with a nested randomization to either 0.05 or 0.1 mg/kg) or ondansetron 8 mg intravenously in a triple-blind fashion. The primary outcome was the reduction from baseline in abdominal pain and nausea (each measured on a 10-cm visual analog scale) at 2 hours after treatment. Although the trial allowed for crossover, the primary analysis used only the first treatment period because few subjects crossed over., Results: We enrolled 33 subjects, of whom 30 (16 men, aged 29 years [SD 11 years] using 1.5 g/day [SD 0.9 g/day] since age 19 years [SD 2 years]) received at least 1 treatment (haloperidol 13, ondansetron 17). Haloperidol at either dose was superior to ondansetron (difference 2.3 cm [95% confidence interval 0.6 to 4.0 cm]; P=.01), with similar improvements in both pain and nausea, as well as less use of rescue antiemetics (31% versus 59%; difference -28% [95% confidence interval -61% to 13%]) and shorter time to emergency department (ED) departure (3.1 hours [SD 1.7] versus 5.6 hours [SD 4.5]; difference 2.5 hours [95% confidence interval 0.1 to 5.0 hours]; P=.03). There were 2 return visits for acute dystonia, both in the higher-dose haloperidol group., Conclusion: In this clinical trial, haloperidol was superior to ondansetron for the acute treatment of cannabis-associated hyperemesis. The efficacy of haloperidol over ondansetron provides insight into the pathophysiology of this now common diagnosis in many EDs., (Copyright © 2020 American College of Emergency Physicians. Published by Elsevier Inc. All rights reserved.)

Background: Free Open-Access Medical education (FOAM) use among residents continues to rise. However, it often lacks quality assurance processes and residents receive little guidance on quality assessment. The Academic Life in Emergency Medicine Approved Instructional Resources tool (AAT) was created for FOAM appraisal by and for expert educators and has demonstrated validity in this context. It has yet to be evaluated in other populations., Objectives: We assessed the AAT's usability in a diverse population of practicing emergency medicine (EM) physicians, residents, and medical students; solicited feedback; and developed a revised tool., Methods: As part of the Medical Education Translational Resources: Impact and Quality (METRIQ) study, we recruited medical students, EM residents, and EM attendings to evaluate five FOAM posts with the AAT and provide quantitative and qualitative feedback via an online survey. Two independent analysts performed a qualitative thematic analysis with discrepancies resolved through discussion and negotiated consensus. This analysis informed development of an initial revised AAT, which was then further refined after pilot testing among the author group. The final tool was reassessed for reliability., Results: Of 330 recruited international participants, 309 completed all ratings. The Best Evidence in Emergency Medicine (BEEM) score was the component most frequently reported as difficult to use. Several themes emerged from the qualitative analysis: for ease of use-understandable, logically structured, concise, and aligned with educational value. Limitations include deviation from questionnaire best practices, validity concerns, and challenges assessing evidence-based medicine. Themes supporting its use include evaluative utility and usability. The author group pilot tested the initial revised AAT, revealing a total score average measure intraclass correlation coefficient (ICC) of moderate reliability (ICC = 0.68, 95% confidence interval [CI] = 0 to 0.962). The final AAT's average measure ICC was 0.88 (95% CI = 0.77 to 0.95)., Conclusions: We developed the final revised AAT from usability feedback. The new score has significantly increased usability, but will need to be reassessed for reliability in a broad population., Competing Interests: The authors have no potential conflicts to disclose. All authors had full access to all the study data and had final responsibility for the decision to submit for publication., (© 2021 by the Society for Academic Emergency Medicine.)