Search

INTRODUCTION: The implementation of programs of assessment based on Entrustable Professional Activities (EPAs) offers an opportunity for students to obtain unique data to guide their ongoing learning and development. Although authors have explored factors that contribute to trust-based decisions, learners’ use of assessors’ decisions about the level of supervision they need has not been fully investigated. METHODS: In this study, we conducted semi-structured interviews of clerkship students who participated in the first year of our EPA program to determine how they interpret and use supervision ratings provided in EPA assessments. Content analysis was performed using concept-driven and open coding. RESULTS: Nine interviews were completed. Twenty-two codes derived from previous work describing factors involved in trust decisions and 12 novel codes were applied to the interview text. Analyses revealed that students focus on written and verbal feedback from assessors more so than on supervision ratings. Axial coding revealed a temporal organization that categorized how students considered the data from EPA assessments. While factors before, during, and after an assessment affected students’ use of information, the relationship between the student and the assessor had impact throughout. CONCLUSIONS: Although students reported varying use of the supervision ratings, their perspectives about how assessors and students interact and/or partner before, during, and after assessments provide insights into the importance of an educational alliance in making a program of assessment meaningful and acceptable to learners.

The rapid expansion of entrustable professional activity (EPA) assessment programs has led to calls to ensure fidelity in implementation and integrity in meeting the goals of competency-based medical education. Initiated in July 2017, in advance of the articulated core components of EPA implementation, this article describes the structure and outcomes of the University of Virginia (UVA) EPA Program and provides support for the identified essential components.The UVA EPA Program includes workplace assessments by residents/fellows, attending faculty, and master assessors (MAs), experienced clinicians who assess students across disciplines and clinical settings. All assessors participate in formal professional development and provide verbal and written comments to support their supervision ratings. The Entrustment Committee, composed of 12 MAs, uses a shared mental model and aggregates all assessor data to make a high-stakes summative entrustment decision about students' readiness to assume the role of an acting intern.Since 2017, over 2,000 assessors have completed 56,969 EPA assessments for 1,479 students. Ninety-four percent of assessments have been done during the clerkship phase. Residents/fellows have completed a mean of 18 assessments, attending faculty a mean of 27, and MAs a mean of 882. Seventy-four percent of observed encounters involved patients with acute concerns with or without a co-morbid condition. Fifty percent of assessments occurred in inpatient and 32% in ambulatory settings. Eighty-seven percent of assessments contained narrative comments with more than 100 characters.Planned next steps will include earlier identification of students who require individualized learning to promote the development of skills related to EPAs, expansion of the remediation program to enable more students to engage in a clinical performance mastery elective, and creation of targeted professional development for assessors to reinforce the tenets of the EPA program.

Leadership development programs often use institutional projects to activate learning. We explored how project work shaped leadership identity formation in senior women leaders from one academic health science center who enrolled in The Hedwig von Ameringen Executive Leadership in Academic Medicine (ELAM®) program. We interviewed ELAM Fellows and conducted a qualitative analysis of transcripts. Our primary analysis focused on the influences of projects on Fellows and institutions. Leadership identity formation emerged as a distinct pattern, so this narrative content was separated for secondary analysis. All authors approved the final assignment of themes and codes. Participants described a multi-dimensional process for developing a leadership identity. Themes encompassed participants’ View of Self and One’s Image as a Leader, Interpersonal Relationships, and Commitment to a Value-based Goal. These internal factors grounded external influences, such as interactions with colleagues and institutional leaders, and the world beyond the institution. We examined the process of leadership identity formation from the perspective of women leaders in academic health sciences who completed an institutional project during a leadership development program. Findings illustrate how internal and external forces, experienced in the context of project work, combine to influence leadership identity formation in women.

INTRODUCTION The Hedwig von Ameringen Executive Leadership in Academic Medicine program (ELAM) is a national professional development program for women that includes institutional action projects (IAPs). Although benefits of ELAM participation are well documented, the value of the IAPs has not been specifically evaluated. We explored the experience of ELAM Fellows and leaders from one institution to elucidate how institutional factors influence project implementation and outcomes. METHODS Fellows and deans participated in semistructured interviews. We analyzed the transcripts qualitatively to develop themes and describe factors that influenced IAP implementation and outcomes. We used the New World Kirkpatrick Model, an updated version of the widely used Kirkpatrick model of educational program evaluation, as a framework to elucidate how participants applied their leadership learning through project work, and to analyze early results of projects that indicated institutional impact. RESULTS Project work had bidirectional impact on the fellows in the program and on the institution itself. Project enablers included: focusing projects on institutional priorities, obtaining sustainable support, and navigating institutional complexity. Leading indicators of institutional outcomes included contributions to institutional leadership and culture, and mutual enhancement of the reputation of the fellow and of the institution. DISCUSSION By examining enablers and barriers for institutionally based projects conducted in a national leadership development program, we identified the drivers that facilitated application of leadership learning. Leading indicators of project outcomes reflected bidirectional impact on fellows and the institution, demonstrating outcomes at the highest levels of the New World Kirkpatrick Model.

Problem: Development of a novel, competency-based program of assessment requires creation of a plan to measure the processes that enable successful implementation. The principles of implementation science outline the importance of considering key drivers that support and sustain transformative change within an educational program. The introduction of Entrustable Professional Activities (EPAs) as a framework for assessment has underscored the need to create a structured plan to prepare assessors to engage in a new paradigm of assessment. Although approaches to rater training for workplace-based assessments have been described, specific strategies to prepare assessors to apply standards related to the level of supervision a student needs have not been documented. Intervention: We describe our systematic approach to prepare assessors, faculty and postgraduate trainees, to complete EPA assessments for medical students during the clerkship phase of our curriculum. This institution-wide program is designed to build assessors' skills in direct observation of learners during authentic patient encounters. Assessors apply new knowledge and practice skills in using established performance expectations to determine the level of supervision a learner needs to perform clinical tasks. Assessors also learn to provide feedback and narrative comments to coach students and promote their ongoing clinical development. Data visualizations for assessors facilitate reinforcement of the tenets learned during training. Collaborative learning and peer feedback during faculty development sessions promote the formation of a community of practice among assessors. Context: Faculty development for assessors was implemented in advance of implementation of the EPA program. Assessors in the program include residents/fellows who work closely with students, faculty with discipline-specific expertise and a group of experienced clinicians who were selected to serve as experts in competency-based EPA assessments, the Master Assessors. Training focused on creating a shared understanding about the application of criteria used to evaluate student performance. EPA assessments based on the AAMC's Core Entrustable Professional Activities for Entering Residency, were completed in nine core clerkships. EPA assessments included a supervision rating based on a modified scale for use in undergraduate medical education. Impact: Data from EPA assessments completed during the first year of the program were analyzed to evaluate the effectiveness of the faculty development activities implemented to prepare assessors to consistently apply standards for assessment. A systematic approach to training and attention to critical drivers that enabled institution-wide implementation, led to consistency in the supervision rating for students' first EPA assessment completed by any type of assessor, ratings by assessors done within a specific clinical context, and ratings assigned by a group of specific assessors across clinical settings. Lessons learned: A systematic approach to faculty development with a willingness to be flexible and reach potential participants using existing infrastructure, can facilitate assessors' engagement in a new culture of assessment. Interaction among participants during training sessions not only promotes learning but also contributes to community building. A leadership group responsible to oversee faculty development can ensure that the needs of stakeholders are addressed and that a change in assessment culture is sustained.

The goal of designing innovative curricula and programs in medical education is to create interventions that will change the attitudes, knowledge, skills, and behaviors of learners in order to prepare them to contribute to the health of patients and communities they serve. Systematic evaluation allows curriculum/program developers to optimize their curriculum/program and ensure that the goals of the curriculum/program are met. While curriculum/program evaluation is distinct from educational research, when approached in a rigorous manner, curriculum/program evaluation can be published as educational scholarship. The goal of our paper is to equip readers with tools to apply a scholarly approach to curriculum/program evaluation to produce findings that will advance the field. We start by defining curricula, educational programs, learner assessment, and evaluation. We then briefly describe examples of best practice models for curriculum/program development that incorporate the critical step of planning for evaluation. Building on published work, we distinguish the use of best practice models and conceptual frameworks to inform curriculum/program development and evaluation. More specifically, we outline steps to plan an evaluation that demonstrates WHAT was effective, describes HOW the curriculum/program contributed to the outcomes achieved, and points to WHY the curriculum/program led to the outcomes observed. We conclude with key considerations for publishing findings of an evaluation, including what to include in each section of a manuscript.

Surgical simulation has been used to facilitate the acquisition of vascular surgery skills. However, high cost and limited availability may restrict the use of this educational resource. We report how instruction using a low-cost, pulsatile, carotid endarterectomy (CEA) benchtop surgical simulation model can be used to enhance learners' procedure-specific knowledge, comfort, and confidence in performing the steps of a CEA procedure DESIGN: A single instructor engaged each participant in a one-on-one instructional session during which the instructor demonstrated, and then the participants performed, the steps of a CEA. Participants completed a pre- and postintervention assessment of knowledge and attitudes about preforming a CEA and use of simulation as a learning tool. Postintervention, participants rated the impact of the simulation model on their learning. A Related T-test and Wilcoxin signed Rank Test were used to compare pre- and postintervention results.University of Virginia Health System, Charlottesville, Virginia.Seventeen postgraduate trainees.A significant difference was observed in pre- and postknowledge scores (48% vs 91% correct, p0.01). Trainee confidence (1.65 vs 2.88, p0.01) and comfort (1.59 vs 2.82, p0.01) with doing the procedure also increased significantly. Sixteen (94%) responded that use of the simulator was extremely or very important as a tool for learning. All 17 trainees (100%) reported that the simulation experience was either essential or very useful in helping them learn how to perform a CEA. Sixty-five percent responded that they were extremely likely to apply the skills learned during the intervention the next time they performed a CEA.A low-cost, pulsatile CEA simulation model used as an educational tool increased procedure-specific knowledge, comfort, and confidence among trainees. Learner's increased confidence and affirmation that they are likely to apply the learned skills in a clinical setting support the use of this educational approach to impact trainee behaviors.

Coaching is a critical tool to guide student development of clinical competency and formation of professional identity in medicine, two inextricably linked concepts. Because progress toward clinical competence is linked to thinking, acting and feeling like a physician, a coach’s knowledge about a learner’s development of clinical skills is essential to promoting the learner’s professional identity formation. A longitudinal coaching program provides a foundation for the formation of coach-learner relationships built on trust. Trusting relationships can moderate the risk and vulnerability inherent in a hierarchical medical education system and allow coaching conversations to focus on the promotion of self-regulated learning and fostering skills for life-long learning. Herein, we describe a comprehensive, longitudinal clinical coaching program for medical students designed to support learners’ professional identify formation and effectively promote their emerging competence.

In the move toward competency-based medical education, leaders have called for standardization of learning outcomes and individualization of the learning process. Significant progress has been made in establishing defined expectations for the knowledge, skills, attitudes, and behaviors required for successful transition to residency training, but individualization of educational processes to assist learners in reaching these competencies has been predominantly conceptual to date. The traditional time-based structure of medical education has posed a challenge to individualization within the curriculum and has led to more attention on innovations that facilitate transition from medical school to residency. However, a shift of focus to the clerkship-to-postclerkship transition point in the undergraduate curriculum provides an opportunity to determine how longitudinal competency-based assessments can be used to facilitate intentional and individualized structuring of the long-debated fourth year.This Perspective demonstrates how 2 institutions-the University of Virginia School of Medicine and the University of Michigan Medical School-are using competency assessments and applying standardized outcomes in decisions about individualization of the postclerkship learning process. One institution assesses Core Entrustable Professional Activities for Entering Residency, whereas the other has incorporated Accreditation Council for Graduate Medical Education core competencies and student career interests to determine degrees of flexibility in the postclerkship phase. Individualization in addition to continued assessment of performance presents an opportunity for intentional use of curriculum time to develop each student to be competently prepared for the transition to residency.

Assessment of entrustable professional activities (EPAs) requires a supervisor to determine a learner’s need for supervision to perform the tasks. This study examines medical students’ perspectives, after completing their core clerkships, about the supervision they need to perform tasks defined by the core EPAs for entering residency. Compared with assertions at the start of the clerkship year, on retrospective self-assessment, students reported needing higher levels of supervision predominantly for activities commonly done during clinical rotations. Students remain confident in their ability to perform tasks with autonomy in the post-clerkship phase, however, raising concern about their abilities to discern limitations and appropriately request supervision.

Entrustable professional activities (EPAs) have been introduced as a framework for teaching and assessment in competency-based educational programs. With growing use, has come a call to examine the validity of EPA assessments. We sought to explore the correlation of EPA assessments with other clinical performance measures to support use of supervision ratings in decisions about medical students' curricular progression.Spearman rank coefficients were used to determine correlation of supervision ratings from EPA assessments with scores on clerkship evaluations and performance on an end-of-clerkship-year Objective Structured Clinical Examination (CPX).Both overall clinical evaluation items score (rho 0.40;Correlation of EPA supervision ratings with commonly used clinical performance measures offers support for their use in undergraduate medical education. Data supporting the validity of EPA assessments promotes stakeholders' acceptance of their use in summative decisions about students' readiness for increased patient care responsibility.

Historically, health sciences education has been guided by tradition and teacher preferences rather than by the application of practices supported by rigorous evidence of effectiveness. Although often underutilized, conceptual frameworks-theories that describe the complexities of educational and social phenomenon-are essential foundations for scholarly work in education. Conceptual frameworks provide a lens through which educators can develop research questions, design research studies and educational interventions, assess outcomes, and evaluate the impact of their work. Given this vital role, conceptual frameworks should be considered at the onset of an educational initiative. Use of different conceptual frameworks to address the same topic in medical education may provide distinctive approaches. Exploration of educational issues by employing differing, theory-based approaches advances the field through the identification of the most effective educational methods. Dissemination of sound educational research based on theory is similarly essential to spark future innovation. Ultimately, this rigorous approach to medical education scholarship is necessary to allow us to establish how our educational interventions impact the health and well-being of our patients.

BACKGROUND AND OBJECTIVES: To describe the supply, distribution, and characteristics of international medical graduates (IMGs) in pediatrics who provide patient care in the United States. METHODS: Cross-sectional study, combining data from the 2019 Physician Masterfile of the American Medical Association and the Educational Commission for Foreign Medical Graduates database. RESULTS: In total, 92 806 pediatric physicians were identified, comprising 9.4% of the entire US physician workforce. Over half are general pediatricians. IMGs account for 23.2% of all general pediatricians and pediatric subspecialists. Of all IMGs in pediatrics, 22.1% or 4775 are US citizens who obtained their medical degree outside the United States or Canada, and 15.4% (3246) attended medical school in the Caribbean. Fifteen non-US medical schools account for 29.9% of IMGs currently in active practice in pediatrics in the United States. IMGs are less likely to work in group practice or hospital-based practice and are more likely to be employed in solo practice (compared with US medical school graduates). CONCLUSIONS: With this study, we provide an overview of the pediatric workforce, quantifying the contribution of IMGs. Many IMGs are US citizens who attend medical school abroad and return to the United States for postgraduate training. Several factors, including the number of residency training positions, could affect future numbers of IMGs entering the United States. Longitudinal studies are needed to better understand the implications that workforce composition and distribution may have for the care of pediatric patients.

CONTEXT: Situativity theory posits that learning and the development of clinical reasoning skills are grounded in context. In case-based teaching, this context comes from recreating the clinical environment, through emulation, as with manikins, or description. In this study, we sought to understand the difference in student clinical reasoning abilities after facilitated patient case scenarios with or without a manikin. METHODS: Fourth-year medical students in an internship readiness course were randomized into patient case scenarios without manikin (control group) and with manikin (intervention group) for a chest pain session. The control and intervention groups had identical student-led case progression and faculty debriefing objectives. Clinical reasoning skills were assessed after the session using a 64-question script concordance test (SCT). The test was developed and piloted prior to administration. Hospitalist and emergency medicine faculty responses on the test items served as the expert standard for scoring. RESULTS: Ninety-six students were randomized to case-based sessions with (n = 48) or without (n = 48) manikin. Ninety students completed the SCT (with manikin n = 45, without manikin n = 45). A statistically significant mean difference on test performance between the two groups was found (t = 3.059, df = 88, p = .003), with the manikin group achieving higher SCT scores. CONCLUSION: Use of a manikin in simulated patient case discussion significantly improves students’ clinical reasoning skills, as measured by SCT. These results suggest that using a manikin to simulate a patient scenario situates learning, thereby enhancing skill development. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (10.1007/s40670-019-00904-0) contains supplementary material, which is available to authorized users.

Medical students entering the clerkship phase of the curriculum and clerkship directors were surveyed at the start of the clerkship year to assess perspectives related to the level of supervision required by students to perform the Core Entrustable Professional Activities (EPAs) for Entering Residency. In aggregate, students reported readiness to perform tasks with lower levels of supervision than the clerkship directors did. For eight of the core EPAs, there was a statistically significant difference in their perspectives. These results illustrate an important gap related to expectations for performance that could impact implementation of a trust-based assessment program.

INTRODUCTION Multisite and national professional development (PD) programs for educators are challenging to establish. Use of implementation science (IS) frameworks designed to convert evidence-based intervention methods into effective health care practice may help PD developers translate proven educational methods and models into successful, well-run programs. Implementation of the national Educational Scholars Program (ESP) is used to illustrate the value of the IS model. METHODS Four adaptable elements of IS are described: (1) replication of an evidence-based model, (2) systematic stages of implementation, (3) management of implementation using three implementation drivers, and (4) demonstration of program success through measures of fidelity to proven models and sustainability. RESULTS Implementation of the ESP was grounded on five established principles and methods for successful PD. The process was conducted in four IS stages over 10 years: Exploration, Installation, Initial Implementation, and Full Implementation. To ensure effective and efficient processes, attention to IS implementation drivers helped to manage organizational relationships, build competence in faculty and scholars, and address leadership challenges. We describe the ESP's fidelity to evidence-based structures and methods, and offer three examples of sustainability efforts that enabled achievement of targeted program outcomes, including academic productivity, strong networking, and career advancement of scholars. DISCUSSION Application of IS frameworks to program implementation may help other PD programs to translate evidence-based methods into interventions with enhanced impact. A PD program can follow systematic developmental stages and be operationalized by practical implementation drivers, thereby creating successful and sustainable interventions that promote the academic vitality of health professions educators.

Despite academic medicine's endorsement of professional development and mentoring, little is known about what junior faculty learn about mentoring in implicit curricula of professional development programs, and how their mentor identity evolves in this context. The authors explored what faculty-participants in the Educational Scholars Program implicitly learned about mentoring and how the implicit curriculum affected mentor identity transformation.Semistructured interviews with 19 of 36 former faculty-participants were conducted in 2016. Consistent with constructivist grounded theory, data collection and analysis overlapped. The authors created initial codes informed by Ibarra's model for identity transformation, iteratively revised codes based on incoming data patterns, and created visual representations of relationships amongst codes to gain a holistic, shared understanding of the data.In the implicit curriculum, faculty-participants learned the importance of having multiple mentors, the value of peer mentors, and the incremental process of becoming a mentor. The authors used Ibarra's model to understand how the implicit curriculum worked to transform mentor identity: Faculty-participants reported observing mentors, experimenting with different ways to mentor and to be a mentor, and evaluating themselves as mentors.The Educational Scholars Program's implicit curriculum facilitated faculty-participants taking on mentor identity via opportunities it afforded to watch mentors, experiment with mentoring, and evaluate self as mentor, key ingredients for identity construction. Leaders of professional development programs can develop faculty as mentors by capitalizing on what faculty-participants learn in the implicit curriculum and deliberately structuring postgraduation mentoring opportunities.

A sizeable literature describes the effectiveness of institution-based faculty development programs in nurturing faculty educators as scholars, but national programs are less common and seldom evaluated. To fill this role, the Educational Scholars Program (ESP) was created within the Academic Pediatric Association (APA) in 2006. It is a national, three-year, cohort-based certification program focused on fostering educational scholarship. This article describes the development and outcomes of an innovative program embedded within the framework of a national professional organization, and offers a model for potential adaptation by similar organizations to enhance their support of educators.After 10 years, 171 scholars have enrolled in the ESP, and 50 faculty have participated. Scholars are assigned a faculty advisor and participate in three full-day sessions at a national meeting; online, interactive learning modules; and a mentored, scholarly project. The program receives support from the APA in four organizational frames: structural, human resource, political, and symbolic. The self-perceived scholarly proficiency of the scholars in Cohort 1 increased significantly over time, and their productivity and collaborations increased during and after the program. Scholars wrote enthusiastically about their experience in yearly and postprogram evaluations. In interviews, eight past APA presidents explained that the ESP strengthened the APA's mission, created new leaders, and provided a new model for other APA programs. Outcomes of the ESP suggest that a longitudinal faculty development program embedded within a national professional organization can create a social enterprise not only within the organization but also within the broader national community of educator-scholars.

Clinical educators at U.S. academic health centers are frequently disadvantaged in the academic promotion system, lacking needed faculty development, mentoring, and networking support.In 2006, we implemented the national Educational Scholars Program to offer faculty development in educational scholarship for early career educators in pediatrics. We aimed to provide them with skills, experience, and initial success in educational scholarship and dissemination. The 3-year curriculum is delivered in interactive sessions at the annual pediatric academic meetings and online intersession modules. Curriculum content progresses from educational scholarship and implementing scholarly projects to dissemination and professional networking. Intersession modules address project planning, building an educator portfolio, reviewing the literature, using technology, authorship, and peer review. Concurrently, all scholars must complete a mentored educational project and demonstrate national dissemination of a peer-reviewed product to obtain a Certificate of Excellence in Educational Scholarship.The setting of this study was a national, longitudinal, cohort-based faculty development program built within the Academic Pediatric Association, a 2,000-member professional organization.In 10 years, the Educational Scholars Program has enrolled 172 scholars in 8 cohorts; 94 have graduated so far. We describe how formative evaluation guided curriculum refinement and process improvement. Summative evaluations show that faculty and scholars were satisfied with the program. Participant outcomes from Cohort 1, assessed at Kirkpatrick's four levels of evaluation, demonstrate increases in scholarly productivity, leadership activities, and academic promotions.Curriculum building is a dynamic process of ongoing evaluation and modification. Our program benefited from designing an integrated and focused curriculum, developing educational principles to guide program improvements, creating curricular tools to help learners organize and document their efforts, supporting project-based learning with expert mentoring, and facilitating peer and faculty networking and collaboration. A national, longitudinal faculty development program can support growth in academic knowledge and skills, promote professional networking, and thereby enrich educators' career opportunities.

This AMEE Guide provides an overview of medical education scholarship for early career scholars, based upon a summary of the existing literature and pragmatic advice derived from the experience of its authors. After providing an introduction to the principles of scholarship and describing questions that the Guide addresses, the authors offer a conceptual description of the complementary traditions of teaching and educational discovery, and advocate for the development of educational scholars with both traditions. They then describe the attributes of effective mentor-mentee relationships and how early career scholars can identify potential mentors who can fulfill this role. In the subsequent sections, they describe the appropriate development of scholarly questions and other components of a complete scholarly plan, including how to use conceptual frameworks in guiding such plans. From here, they describe methods that align with both the teaching and discovery traditions and provide concrete examples of each. They then provide guidelines for assessing the impact of scholarship, identify the various opportunities for sharing it, and how to effectively interpret and describe it. Additionally, they provide practical advice on how appropriately to demonstrate the scholarship in a promotional packet, including the principle of reflectivity in scholarship. Finally, they address the principles of applied research ethics for educational scholarship and when to consider soliciting approval for scholarly activities by a human research board.

Critical thinking is essential to a health professional's competence to assess, diagnose, and care for patients. Defined as the ability to apply higher-order cognitive skills (conceptualization, analysis, evaluation) and the disposition to be deliberate about thinking (being open-minded or intellectually honest) that lead to action that is logical and appropriate, critical thinking represents a "meta-competency" that transcends other knowledge, skills, abilities, and behaviors required in health care professions. Despite its importance, the developmental stages of critical thinking have not been delineated for nurses and physicians. As part of a task force of educators who considered different developmental stage theories, the authors have iteratively refined and proposed milestones in critical thinking. The attributes associated with unreflective, beginning, practicing, advanced, accomplished, and challenged critical thinkers are conceived as independent of an individual's level of training. Depending on circumstances and environmental factors, even the most experienced clinician may demonstrate attributes associated with a challenged thinker. The authors use the illustrative case of a patient with abdominal pain to demonstrate how critical thinking may manifest in learners at different stages of development, analyzing how the learner at each stage applies information obtained in the patient interaction to arrive at a differential diagnosis and plan for evaluation. The authors share important considerations and provide this work as a foundation for the development of effective approaches to teaching and promoting critical thinking and to establishing expectations for learners in this essential meta-competency.

Background: There is a paucity of literature focusing on what students and educators believe constitutes students' readiness for clinical clerkships. The purpose of this study is to gain insight into clinical faculty and students’ perceptions of student readiness for clerkship education. Methods: Educational experts conducted focus groups and semi-structured individual interviews with third-year medical students and faculty who teach in clinical clerkships, including directors of the required rotations. Two authors independently coded the transcribed notes from the interviews to identify themes. Discrepancies were resolved among all authors in an iterative process. Results: Eight students and 15 faculty members participated in the interviews. While faculty focused on the need for students to be able to apply knowledge, be active in their learning, and display professionalism, students discussed their beliefs about why readiness for a “completely new environment” was unattainable. Discussion: Issues that impact perceptions about readiness include: defining professionalism in the context of a hierarchical system, the challenge of transferring learning to apply knowledge in new environments, and the teacher’s role in facilitating and encouraging active learning. The differences in conceptualizations among faculty and students about readiness highlight the need for enhanced communication regarding expectations for students as they enter clinical clerkships.

The academic community needs a sound framework for the promotion and advancement of educators. The Group on Educational Affairs of the Association of American Medical Colleges organized a consensus conference that affirmed the use of five domains for documenting the quantity and quality of scholarly engagement in educational activities: teaching, curriculum, advising/mentoring, educational leadership/administration, and learner assessment.In this article, we offer detailed guidelines to evaluate these five domains of educator performance and the essential elements of scholarly activity. The guidelines are adapted from our developmental educator portfolio template and educator portfolio analysis tool, previously published in MedEdPORTAL. A short tool for educator performance evaluation that summarizes items in the guidelines is proposed for discussion.Our goal in this article is to itemize criteria for systematic faculty evaluation that can be applied in any institutional setting to assist promotion decision makers in their task of evaluating medical school faculty.

Introduction: Mentoring is increasingly recognized as central to career development. Less attention has been paid, however, to how mentoring relationships evolve over time. To provide a more complete picture of these complex relationships, the authors explored mentoring from a mentee’s perspective within the context of a three-year faculty development program in which the mentor provided specific expertise to assist the mentee in completing a scholarly educational project. Methods: Using an evolving focus group design, the authors interviewed mentee groups in 2007–2009 inclusive. Transcripts were coded inductively; codes were revised as data patterns became more apparent. Preliminary assertions about the answers to guiding questions were made; the trustworthiness of the assertions was assessed via member check. Results: Mentees offered a variety of reasons for choosing their project mentor, including proximity, familiarity, and mentor expertise. There was a dyadic relationship with the project mentor in year 1, a broader collaboration with multiple senior mentors in year 2, and mentoring among program peers in year 3. Mentees benefitted from mentors’ supportive behaviors and, to a lesser extent, mentors’ challenging behaviors. Conclusion: Mentoring relationships, in the context of this faculty development program, tended not to be an exclusive dyadic connection but rather a constellation of relationships that evolved over time and included peer mentoring. The complex reality of these relationships challenges the application of traditional mentoring models and suggests unique considerations in developing mentoring programs designed to meet the needs of faculty in academic medicine.

Projects--planned activities with specific goals and outcomes--have been used in faculty development programs to enhance participant learning and development. Projects have been employed most extensively in programs designed to develop faculty as educators. The authors review the literature and report the results of their 2008 study of the impact of projects within the Pennsylvania State University College of Medicine Junior Faculty Development Program, a comprehensive faculty development program. Using a mixed-methods approach, the products of project work, the academic productivity of program graduates, and the impact of projects on career development were analyzed. Faculty who achieved the most progress on their projects reported the highest number of academic products related to their project and the highest number of overall academic achievements. Faculty perceived that their project had three major effects on their professional development: production of a tangible outcome, development of a career focus, and development of relationships with mentors and peers. On the basis of these findings and a review of the literature, the authors conclude that projects are an essential element of a faculty development program. Projects provide a foundation for future academic success by enabling junior faculty to develop and hone knowledge and skills, identify a career focus and gain recognition within their community, generate scholarship, allocate time to academic work, and establish supportive relationships and collaborative networks. A list of best practices to successfully incorporate projects within faculty development programs is provided.

Traditional promotion standards rely heavily on quantification of research grants and publications in the curriculum vitae. The promotion and retention of educators is challenged by the lack of accepted standards to evaluate the depth, breadth, quality, and impact of educational activities. The authors sought to develop a practical analysis tool for the evaluation of educator portfolios (EPs), based on measurable outcomes that allow reproducible analysis of the quality and impact of educational activities.The authors, 10 veteran educators and an external expert evaluator, used a scholarly, iterative consensus-building process to develop the tool and test it using real EPs from educational scholars who followed an EP template. They revised the template in parallel with the analysis tool to ensure that EP data enabled valid and reliable evaluation. The authors created the EP template and analysis tool for scholar and program evaluation in the Educational Scholars Program, a three-year national certification program of the Academic Pediatric Association.The analysis tool combines 18 quantitative and 25 qualitative items, with specifications, for objective evaluation of educational activities and scholarship.The authors offer this comprehensive, yet practical tool as a method to enhance opportunities for faculty promotions and advancement, based on well-defined and documented educational outcome measures. It is relevant for clinical educators across disciplines and across institutions. Future studies will test the interrater reliability of the tool, using data from EPs written using the revised template.

Introduction: Mentoring is a central component of professional development. Evaluation of “successful” mentoring programs, however, has been limited and mainly focused on measures of satisfaction with the relationship. In today’s environment, mentoring programs must produce tangible outcomes to demonstrate success. To address this issue, the authors advance the framework of functional mentoring combined with measurement of outcomes at multiple levels. Methods: The mentoring program is embedded within an intensive, continuing medical education (CME) accredited faculty development program. Survey methodology is used to collect qualitative and quantitative data at the start, midpoint, and end of the program and longitudinally. Participants in 4 years of the program were surveyed. Results: In 4 years, 165 faculty participated in the program. Respondents were highly satisfied with the pairings: 85% of junior faculty believed their mentor had a significant effect on their projects. Junior faculty reported a significant enhancement of skills related to initiating and negotiating a new mentoring relationship (85%) and stated that their project would have a significant impact on their career (92%) and on the department or institution (86%). Discussion: The success of this mentoring program is demonstrated at multiple levels. The key outcome of functional mentoring is the project. Projects are aligned with professional responsibilities and with institutional missions. The project contributes to the individual’s dossier and adds value to the institution. Functional mentoring is a practical approach that allows measurable results at multiple levels.

Introduction Effective mentoring relationships are important contributors to career satisfaction, academic productivity, perceived self-efficacy, and successful networking. They are especially critical during times of transition in one's career. Unfortunately, such relationships can be difficult to find and challenging to sustain. In recent years, the concept of mentee-driven mentoring relationships has been proposed as a method of improving mentoring relationships. In this workshop, we apply the widely utilized concept of the individualized learning plan to examine and optimize the mentee-driven mentorship. Methods In small-group discussions, participants reflect on their unique mentoring needs as they transition into the next stage of their careers. Next, participants apply the corporate concept of managing up to their current mentoring relationships. Workshop leaders then introduce the optimal characteristics of mentors as well as the concept of a mentoring mosaic. Participants then create their own mentoring mosaics that describe the depth and breadth of their potential and of existing mentoring networks. With guidance from the workshop facilitators and other small-group discussants, participants then expand their mentoring mosaics by exploring possibilities for untapped mentoring resources (recognizing that mentors can come from the junior, peer, and senior levels). Finally, participants discuss in small groups strategies to overcome several challenging mentoring scenarios, including how to recruit new mentors and maximize the limited time of a mentor. Participants will leave with a better understanding of their roles as mentees and mentors. Results This workshop has been presented at the 2014 and 2015 Pediatric Academic Societies' Annual Meetings and at the Association of Pediatric Program Directors' Annual Spring Meeting in 2013 and 2014. Participants have included pediatric residents, chief residents, fellows, junior faculty, mid-level faculty, and senior-level faculty. Ninety-five percent of participants agreed or strongly agreed that their participation would improve the quality of their mentoring relationships. Discussion Although we have not collected data on the career outcomes of our workshop participants, literature suggests that mentees who utilize the principles we present will benefit greatly (e.g., attainment of leadership positions, academic promotion, career satisfaction).

Dialogue is essential for transforming the culture of academic health centers, yet organizational silence is common. Senior faculty and administrative leaders can play key roles in building a healthier environment by not only developing their own skills in discussing difficult issues, but also in coaching others to do so. These resources are designed for use in a professional development workshop intended to raise the level of understanding of senior faculty about the impact of organizational silence, generate discussion about practices to support dialogue around challenging issues, and increase their ability coach others to have difficult conversations. Brief didactic components present key background information and the activities allow participants to apply their learning and practice skills, thereby enhancing their confidence and coaching behaviors to effectively assist colleagues in engaging in challenging conversations. Two worksheets, a case example for role play, and a power point slide deck are included. This workshop was presented at a national conference (the 2013 Association of American Medical Colleges Group on Faculty Affairs Professional Development Conference). The audience was comprised primarily of faculty affairs deans at U.S. medical schools, a group of institutional leaders often called upon for advice about, and management of, difficult issues. Evaluations from attendees documented the usefulness of the hands-on activities and the value of the “take home” messages provided during the session. This proposal is limited by not having more robust program evaluation data. Unfortunately, only 10 attendees completed the formal evaluation for the conference. However, because there are so few resources available on the topic of coaching others to counteract organizational silence, we believe it is important to begin filling this gap by offering resources on the topic.

Barriers to influenza vaccination negatively impact immunization rates contributing to the morbidity and mortality from influenza. This study sought to determine if 1) the availability of year-round scheduling of annual autumn/winter influenza vaccination was associated with improved immunization rates for 2 high-risk populations of children and 2) this system was associated with early season vaccine administration.A retrospective cohort analysis was utilized to compare immunization rates during the 2003-2004 and 2004-2005 seasons. Billing records were used to determine eligible patients and vaccine receipt. A single, pediatric practice was studied, and two groups of patients were analyzed: 1) infants aged 6-23 months and 2) children21 years old with asthma. As opposed to Year 1, in Year 2 appointments in "flu clinics" for the following autumn became available 7 months prior to when vaccine became available. Patients and providers could therefore schedule an immunization appointment throughout the year.In Year 1, 552/1365 (40.4%) infants received at least 1 dose of influenza vaccine compared to 940/1265 (74.3%) in Year 2 (p0.001). For patients with asthma, 309/1332 (23.2%) received at least 1 dose of vaccine in Year 1 compared with 522/1489 (35.1%) in Year 2 (p0.001). Both groups also achieved higher immunization rates between September and November in Year 2 (p0.001).Year-round scheduling of influenza vaccination was associated with improved immunization rates in two high-risk populations, and may remove the barrier of scheduling difficulty, allow for a consistent year-round message from providers, and improve timing of vaccine administration.

Dialogue is essential for transforming institutions into learning organizations, yet many well-known characteristics of academic health centers (AHCs) interfere with open discussion. Rigid hierarchies, intense competition for resources, and the power of peer review in advancement processes all hamper difficult conversations, thereby contributing to organizational silence, and at great cost to the institution. Information necessary for critical decisions is not shared, individuals and the organization do not learn from mistakes, and diverse perspectives from those with less power are not entertained, or worse, are suppressed. When leaders become more skilled at inviting multiple perspectives and faculty more adept at broaching difficult conversations with those in power, differences are more effectively addressed and conflicts resolved. In this article, the authors frame why this skill is an essential competency for faculty and leaders alike and provide the following recommendations to institutions for increasing capacity in this area: (1) develop leaders to counteract organizational silence, (2) develop faculty members' skills in raising difficult issues with those in positions of power, and (3) train mentors to coach others in raising difficult conversations. The vitality of AHCs requires that faculty and institutional leaders develop relational communication skills and partner in learning through challenging conversations.

Valuing faculty as educators is essential for medical schools to fulfill their unique mission of educating physicians. The 2006 Consensus Conference on Educational Scholarship, sponsored by the Association of American Medical Colleges (AAMC) Group on Educational Affairs, provided educators seeking academic promotion with a portfolio-based format for documenting activities in five domains, using evidence of quantity, quality, a scholarly approach, and educational scholarship. Yet, the lack of a rigorous, widely accepted system to assess educator portfolio submissions during the promotion and tenure process continues to impede the ability to fully value educators and educational scholars.The AAMC Task Force on Educator Evaluation was formed in 2010 to establish consensus guidelines for use by those responsible for the rigorous evaluation of the educational contributions of faculty. The task force delineated the educational contributions currently valued by institutions and then fulfilled its charge by creating the Toolbox for Evaluating Educators, a resource which contains explicit evidence-based criteria to evaluate faculty in each of the five domains of educator activity. Adoption of such criteria is now the rate-limiting step in using a fair process to recognize educators through academic promotion. To inform institutional review and implementation of these criteria, this article describes the iterative, evidence- and stakeholder-based process to establish the criteria. The authors advocate institutional adoption of these criteria so that faculty seeking academic promotion as educators, like their researcher colleagues, can be judged and valued using established standards for the assessment of their work.

This leadership workshop addresses a leadership topic for which there is a paucity of information in the current literature: how to smoothly leave a position/organization and move effectively into a new position/organization. Thus, the purpose of this resource is to provide tools for a new or aspiring leader to enable him/her to strategically and successfully manage the transition to a new leadership role in a new organization. The workshop is specifically tailored to the leadership development of faculty in academic medicine. We utilize a case-based format to explore a real-world scenario that new leaders are likely to encounter. The case was developed from a compilation of the actual experiences of three leaders in academic medicine who, reflecting upon their own transitions to new, higher leadership positions in new institutions and their knowledge of the extant literature, created an illustrative case with essential content. The case study uncovers the major issues and opportunities facing a leader who is new to their position and/or the institution/organization. The workshop is organized into four sections or parts, presented as longitudinal “chapters” of the case. Questions at the end of each section allow workshop participants–new and aspiring leaders as well as more experienced leaders who have encountered similar situations–with an opportunity to share and learn from each other as well as from the workshop leader(s). For each section of the case, the core content of the section follows the section ‘title slide” slide portraying the specific chapter of the case. We developed and implemented this interactive workshop with hands-on activities, in order to engage leaders in the reflective and proactive work required to achieve the goal of a smooth, efficient, and successful transition to a new leadership position. In order to reach a substantial number of new and aspiring leaders in academic medicine, the workshop was presented at the Annual Meeting of the Association of American Medical Colleges in a plenary session cosponsored by multiple stakeholder groups of the association. Approximately 100 individuals attended the session. Evaluations completed by participants documented the usefulness of the content and approach, and comments underscored the value perceived by participants.

Objective A reliable and supportive mentor is indispensable to the career development of successful academic professionals. The Academic Pediatric Association (APA) utilized a speed mentoring format at the 2012 Pediatric Academic Societies meeting to enhance mentoring potential. We sought to evaluate the structure of the speed mentoring event and to determine the benefits and impact from the perspectives of the mentors and mentees. Methods Sixty mentees were matched with 60 mentors within various tracks. Each mentee met with 6 mentors for 10 minutes for each dyad. Participants were then asked to complete a survey 1 to 4 weeks after the event. Survey items included expectation, impact, and value of the experience along with potential for ongoing mentoring relationships. Results Fifty-four (90%) of the 60 mentees and 52 (87%) of 60 of the mentors completed the evaluation. Mentees stated that the event allowed them to receive advice from multiple mentors in a short time period. Mentors appreciated that they gained new insights, reflected on their own careers, and were able to give back to their field. Both mentees and mentors agreed that the time was well spent, would participate again, and identified chemistry as a major factor in pursuing an ongoing relationship. Conclusions This national speed mentoring event provided an innovative, fun, and time-efficient mechanism to establish connections, network, and determine whether chemistry existed for potential mentor–mentee relationships. Further study should evaluate whether it can be used in other venues and lead to the development of lasting mentor–mentee relationships.

Summary Background: The system used by academic health centres to evaluate teaching must be valued by the large number of faculty staff that teach in clinical settings. Peer review can be used to evaluate and enhance clinical teaching. The objective of this study was to determine the perceptions of clinical faculty about the effects of participating in peer review. Methods: Faculty members were observed teaching in a clinical setting by trained peer observers. Feedback was provided using a checklist of behaviours and descriptive comments. Afterwards, semi-structured interviews were conducted to assess the faculty member’s perception about the process. Notes from the interviews were analysed using a grounded theory approach. The study was approved by the institutional review boards of all the institutions involved. Results: Three themes emerged from the interviews with faculty members: (1) they found the process to be valuable – they received information that affirmed “good” teaching behaviours, and were prompted to be more focused on their teaching; (2) they were motivated to enhance their teaching by being more deliberate, interactive and learner-centred; and (3) they were inspired to explore other opportunities to improve their teaching skills. Discussion: Peer review is a process that promotes the open discussion and exchange of ideas. This conversation advances clinical teaching skills and allows high-quality teaching behaviours to be strengthened.

Introduction In the past two decades, significant progress has been made in defining and justifying the value of scholarship in education. However, educators at many academic health centers continue to struggle with advancement and promotion because the process for evaluating their contributions is often cumbersome and accepted standards for evaluation are vague or lacking. The basis for evaluating educators has been strengthened by defining educational scholarship and by developing templates for faculty to document their educational contributions using educator's portfolios. This toolbox presents a sound framework for evaluating educational contributions in a rigorous manner analogous to the peer review process used for assessment of a faculty member's work in research and other scholarly work. Methods This 90-minute workshop is designed to provide hands-on opportunities for members of promotions/tenure committees to utilize the AAMC Toolbox for Evaluating Educators resource in the assessment of faculty member performance whose careers focus across the following five domains of educator activity: learner assessment, curriculum development, mentoring and advising, education leadership and administration, and teaching activities. The exercises in the workshop allow participants to use the indicators to reach summative decisions through a rigorous and consistent application of clear yet flexible standards. The workshop consists of an introductory PowerPoint presentation and small-group activities followed by a facilitated large-group discussion that allows the participants to explore how the evidence-based standards in the toolbox can be integrated with existing institutional processes for the evaluation of the performance of educators. The workshop and its resources can be adapted and used for training/professional development sessions for other decision making committees/members (e.g., awards committees or selection committees for a teaching academy). Results A professional development workshop using the Toolbox has been presented at each of the regional Group on Educational Affairs meetings, at the Group on Faculty Affairs annual Professional Development Conference, and in plenary sessions at two annual AAMC meetings. The participants have reviewed, critiqued and applied the indicators in the Toolbox during interactive exercises. The authors used written and oral feedback to refine the structure and content of the Toolbox. Discussion The resource was created primarily for faculty and committees charged with faculty evaluation and decision making; however, it is also useful for educators who are seeking promotion and their mentors including faculty affairs and education administrative leaders. Knowledge of the standards by which one's performance will be judged helps an educator create a portfolio that documents his/her activities and the impact of their work, in a standardized format that can be readily assessed.

Too often, educators are disadvantaged in the promotion and tenure process because their roles are not well understood by committees, and methods to evaluate educator performance have not been standardized. These educator evaluation guidelines contribute to the construction of a framework for the academic promotion and advancement of educators. The guidelines were designed to assist members of promotion committees in the evaluation of medical school faculty by a systematic, objective, and criterion-based process, and to help educators understand the elements of performance by which they are evaluated. To make the tool as accessible and usable as possible we avoided technical educational jargon, provided concrete indicators for each item, and illustrated items with copious examples. The guidelines include two tables with a total of 24 evaluation items distilled from our previously published, 36-item research tool for analysis of educator portfolios. That research tool was created by an iterative consensus development process and tested with live educator portfolios, so the items included in these guidelines have been well validated. The first table (19 items) is organized around five domains of educator performance: teaching, curriculum, advising/mentoring, educational leadership/administration, and learner assessment. The second table (five items) addresses the two components of scholarly activity: scholarly approach and educational scholarship. The five domains and two definitions of scholarly activity were confirmed by a consensus conference convened by the AAMC Group on Educational Affairs in 2009. Each domain is defined and elaborated with items for evaluation selected to capture quantity, quality, and impact of performance. For each evaluation item, several suggested indicators describe what kinds of evidence might earn credits for the educator and specific examples of high-quality performance are provided. Evaluation of educators is not well captured by numerical measures only. Items in the guidelines are appropriate for both quantitative and qualitative assessment; quantitative items are labeled. Despite the difficulty of objectively evaluating qualitative measures, some elements of educational practice are best assessed based on their adherence to accepted educational principles, rather than by being counted. Thus, the guidelines encourage an educator evaluation that is flexible and takes a broad perspective on performance. The guidelines avoid a prescriptive approach. They do not include a rating system, because each institution needs to develop its own scoring approach based on the value it places on specific educator activities. We have left room for institutions to decide how many areas of excellence are expected of a faculty member in each promotion track and at each rank. However, the included User's Manual discusses possible approaches to creating a rating system to accompany the 24-item evaluation tool.

Today, faculty in academic medicine face challenges in all three mission areas--research, education, and patient care--and require a broad set of competencies to survive in this changing environment. To support faculty and to design assessments that match new expectations, the authors argue that it is essential to capture the full scope of skills, knowledge, and behaviors necessary for a successful faculty member. Thus, it is timely to explore and define competencies for faculty in academic medicine. The authors describe three approaches to identifying faculty competencies. Each reveals diverse but overlapping sets of competency domains, reflecting the breadth of activities expected of today's faculty. To organize these competencies into a coherent framework, the authors propose a model based on a typology of competency. A key feature of the model is the division between occupational competencies, which are largely role-specific, and personal competencies, which are necessary for all faculty. A competency framework also must be developmental, to reflect the growth in skills, knowledge, and behaviors from trainee to expert and to allow for an individual's changing roles over a career. Such a competency framework will inform professional development activities and require assessment of competence. The generation of competencies also will reveal areas of faculty practice that are poorly measured, requiring new tools to be incorporated into existing processes of faculty evaluation. The authors provide general principles to guide the identification of a competency framework for faculty and invite the academic medicine community to engage in further discussion.

Background: Developing and ensuring successful collaborative research in medical education is no small task, but the rewards to researchers can be great. Collaborative research in medical education offers significant opportunities for investigators who wish to pool limited resources and expand professional networks. Despite this, collaboration often occurs without advance planning for the logistical aspects of collaborative work. Aims and Methods: These 12 tips, derived from developing and presenting a session on strategies for effective collaboration conducted at a national meeting, will assist readers who are planning to collaborate or are already engaged in collaborative scholarship. The tips are organized into items to consider in three phases of collaboration: planning, implementation, and dissemination of outcomes. Results and Conclusions: From selecting a topic to recognizing when the collaboration should end, the tips underscore the importance of good communication, and clear expectations for participants.