Search

The rapid expansion of entrustable professional activity (EPA) assessment programs has led to calls to ensure fidelity in implementation and integrity in meeting the goals of competency-based medical education. Initiated in July 2017, in advance of the articulated core components of EPA implementation, this article describes the structure and outcomes of the University of Virginia (UVA) EPA Program and provides support for the identified essential components.The UVA EPA Program includes workplace assessments by residents/fellows, attending faculty, and master assessors (MAs), experienced clinicians who assess students across disciplines and clinical settings. All assessors participate in formal professional development and provide verbal and written comments to support their supervision ratings. The Entrustment Committee, composed of 12 MAs, uses a shared mental model and aggregates all assessor data to make a high-stakes summative entrustment decision about students' readiness to assume the role of an acting intern.Since 2017, over 2,000 assessors have completed 56,969 EPA assessments for 1,479 students. Ninety-four percent of assessments have been done during the clerkship phase. Residents/fellows have completed a mean of 18 assessments, attending faculty a mean of 27, and MAs a mean of 882. Seventy-four percent of observed encounters involved patients with acute concerns with or without a co-morbid condition. Fifty percent of assessments occurred in inpatient and 32% in ambulatory settings. Eighty-seven percent of assessments contained narrative comments with more than 100 characters.Planned next steps will include earlier identification of students who require individualized learning to promote the development of skills related to EPAs, expansion of the remediation program to enable more students to engage in a clinical performance mastery elective, and creation of targeted professional development for assessors to reinforce the tenets of the EPA program.

Leadership development programs often use institutional projects to activate learning. We explored how project work shaped leadership identity formation in senior women leaders from one academic health science center who enrolled in The Hedwig von Ameringen Executive Leadership in Academic Medicine (ELAM®) program. We interviewed ELAM Fellows and conducted a qualitative analysis of transcripts. Our primary analysis focused on the influences of projects on Fellows and institutions. Leadership identity formation emerged as a distinct pattern, so this narrative content was separated for secondary analysis. All authors approved the final assignment of themes and codes. Participants described a multi-dimensional process for developing a leadership identity. Themes encompassed participants’ View of Self and One’s Image as a Leader, Interpersonal Relationships, and Commitment to a Value-based Goal. These internal factors grounded external influences, such as interactions with colleagues and institutional leaders, and the world beyond the institution. We examined the process of leadership identity formation from the perspective of women leaders in academic health sciences who completed an institutional project during a leadership development program. Findings illustrate how internal and external forces, experienced in the context of project work, combine to influence leadership identity formation in women.

INTRODUCTION The Hedwig von Ameringen Executive Leadership in Academic Medicine program (ELAM) is a national professional development program for women that includes institutional action projects (IAPs). Although benefits of ELAM participation are well documented, the value of the IAPs has not been specifically evaluated. We explored the experience of ELAM Fellows and leaders from one institution to elucidate how institutional factors influence project implementation and outcomes. METHODS Fellows and deans participated in semistructured interviews. We analyzed the transcripts qualitatively to develop themes and describe factors that influenced IAP implementation and outcomes. We used the New World Kirkpatrick Model, an updated version of the widely used Kirkpatrick model of educational program evaluation, as a framework to elucidate how participants applied their leadership learning through project work, and to analyze early results of projects that indicated institutional impact. RESULTS Project work had bidirectional impact on the fellows in the program and on the institution itself. Project enablers included: focusing projects on institutional priorities, obtaining sustainable support, and navigating institutional complexity. Leading indicators of institutional outcomes included contributions to institutional leadership and culture, and mutual enhancement of the reputation of the fellow and of the institution. DISCUSSION By examining enablers and barriers for institutionally based projects conducted in a national leadership development program, we identified the drivers that facilitated application of leadership learning. Leading indicators of project outcomes reflected bidirectional impact on fellows and the institution, demonstrating outcomes at the highest levels of the New World Kirkpatrick Model.

Problem: Development of a novel, competency-based program of assessment requires creation of a plan to measure the processes that enable successful implementation. The principles of implementation science outline the importance of considering key drivers that support and sustain transformative change within an educational program. The introduction of Entrustable Professional Activities (EPAs) as a framework for assessment has underscored the need to create a structured plan to prepare assessors to engage in a new paradigm of assessment. Although approaches to rater training for workplace-based assessments have been described, specific strategies to prepare assessors to apply standards related to the level of supervision a student needs have not been documented. Intervention: We describe our systematic approach to prepare assessors, faculty and postgraduate trainees, to complete EPA assessments for medical students during the clerkship phase of our curriculum. This institution-wide program is designed to build assessors' skills in direct observation of learners during authentic patient encounters. Assessors apply new knowledge and practice skills in using established performance expectations to determine the level of supervision a learner needs to perform clinical tasks. Assessors also learn to provide feedback and narrative comments to coach students and promote their ongoing clinical development. Data visualizations for assessors facilitate reinforcement of the tenets learned during training. Collaborative learning and peer feedback during faculty development sessions promote the formation of a community of practice among assessors. Context: Faculty development for assessors was implemented in advance of implementation of the EPA program. Assessors in the program include residents/fellows who work closely with students, faculty with discipline-specific expertise and a group of experienced clinicians who were selected to serve as experts in competency-based EPA assessments, the Master Assessors. Training focused on creating a shared understanding about the application of criteria used to evaluate student performance. EPA assessments based on the AAMC's Core Entrustable Professional Activities for Entering Residency, were completed in nine core clerkships. EPA assessments included a supervision rating based on a modified scale for use in undergraduate medical education. Impact: Data from EPA assessments completed during the first year of the program were analyzed to evaluate the effectiveness of the faculty development activities implemented to prepare assessors to consistently apply standards for assessment. A systematic approach to training and attention to critical drivers that enabled institution-wide implementation, led to consistency in the supervision rating for students' first EPA assessment completed by any type of assessor, ratings by assessors done within a specific clinical context, and ratings assigned by a group of specific assessors across clinical settings. Lessons learned: A systematic approach to faculty development with a willingness to be flexible and reach potential participants using existing infrastructure, can facilitate assessors' engagement in a new culture of assessment. Interaction among participants during training sessions not only promotes learning but also contributes to community building. A leadership group responsible to oversee faculty development can ensure that the needs of stakeholders are addressed and that a change in assessment culture is sustained.

In the move toward competency-based medical education, leaders have called for standardization of learning outcomes and individualization of the learning process. Significant progress has been made in establishing defined expectations for the knowledge, skills, attitudes, and behaviors required for successful transition to residency training, but individualization of educational processes to assist learners in reaching these competencies has been predominantly conceptual to date. The traditional time-based structure of medical education has posed a challenge to individualization within the curriculum and has led to more attention on innovations that facilitate transition from medical school to residency. However, a shift of focus to the clerkship-to-postclerkship transition point in the undergraduate curriculum provides an opportunity to determine how longitudinal competency-based assessments can be used to facilitate intentional and individualized structuring of the long-debated fourth year.This Perspective demonstrates how 2 institutions-the University of Virginia School of Medicine and the University of Michigan Medical School-are using competency assessments and applying standardized outcomes in decisions about individualization of the postclerkship learning process. One institution assesses Core Entrustable Professional Activities for Entering Residency, whereas the other has incorporated Accreditation Council for Graduate Medical Education core competencies and student career interests to determine degrees of flexibility in the postclerkship phase. Individualization in addition to continued assessment of performance presents an opportunity for intentional use of curriculum time to develop each student to be competently prepared for the transition to residency.

Entrustable professional activities (EPAs) have been introduced as a framework for teaching and assessment in competency-based educational programs. With growing use, has come a call to examine the validity of EPA assessments. We sought to explore the correlation of EPA assessments with other clinical performance measures to support use of supervision ratings in decisions about medical students' curricular progression.Spearman rank coefficients were used to determine correlation of supervision ratings from EPA assessments with scores on clerkship evaluations and performance on an end-of-clerkship-year Objective Structured Clinical Examination (CPX).Both overall clinical evaluation items score (rho 0.40;Correlation of EPA supervision ratings with commonly used clinical performance measures offers support for their use in undergraduate medical education. Data supporting the validity of EPA assessments promotes stakeholders' acceptance of their use in summative decisions about students' readiness for increased patient care responsibility.

INTRODUCTION Multisite and national professional development (PD) programs for educators are challenging to establish. Use of implementation science (IS) frameworks designed to convert evidence-based intervention methods into effective health care practice may help PD developers translate proven educational methods and models into successful, well-run programs. Implementation of the national Educational Scholars Program (ESP) is used to illustrate the value of the IS model. METHODS Four adaptable elements of IS are described: (1) replication of an evidence-based model, (2) systematic stages of implementation, (3) management of implementation using three implementation drivers, and (4) demonstration of program success through measures of fidelity to proven models and sustainability. RESULTS Implementation of the ESP was grounded on five established principles and methods for successful PD. The process was conducted in four IS stages over 10 years: Exploration, Installation, Initial Implementation, and Full Implementation. To ensure effective and efficient processes, attention to IS implementation drivers helped to manage organizational relationships, build competence in faculty and scholars, and address leadership challenges. We describe the ESP's fidelity to evidence-based structures and methods, and offer three examples of sustainability efforts that enabled achievement of targeted program outcomes, including academic productivity, strong networking, and career advancement of scholars. DISCUSSION Application of IS frameworks to program implementation may help other PD programs to translate evidence-based methods into interventions with enhanced impact. A PD program can follow systematic developmental stages and be operationalized by practical implementation drivers, thereby creating successful and sustainable interventions that promote the academic vitality of health professions educators.

Despite academic medicine's endorsement of professional development and mentoring, little is known about what junior faculty learn about mentoring in implicit curricula of professional development programs, and how their mentor identity evolves in this context. The authors explored what faculty-participants in the Educational Scholars Program implicitly learned about mentoring and how the implicit curriculum affected mentor identity transformation.Semistructured interviews with 19 of 36 former faculty-participants were conducted in 2016. Consistent with constructivist grounded theory, data collection and analysis overlapped. The authors created initial codes informed by Ibarra's model for identity transformation, iteratively revised codes based on incoming data patterns, and created visual representations of relationships amongst codes to gain a holistic, shared understanding of the data.In the implicit curriculum, faculty-participants learned the importance of having multiple mentors, the value of peer mentors, and the incremental process of becoming a mentor. The authors used Ibarra's model to understand how the implicit curriculum worked to transform mentor identity: Faculty-participants reported observing mentors, experimenting with different ways to mentor and to be a mentor, and evaluating themselves as mentors.The Educational Scholars Program's implicit curriculum facilitated faculty-participants taking on mentor identity via opportunities it afforded to watch mentors, experiment with mentoring, and evaluate self as mentor, key ingredients for identity construction. Leaders of professional development programs can develop faculty as mentors by capitalizing on what faculty-participants learn in the implicit curriculum and deliberately structuring postgraduation mentoring opportunities.

A sizeable literature describes the effectiveness of institution-based faculty development programs in nurturing faculty educators as scholars, but national programs are less common and seldom evaluated. To fill this role, the Educational Scholars Program (ESP) was created within the Academic Pediatric Association (APA) in 2006. It is a national, three-year, cohort-based certification program focused on fostering educational scholarship. This article describes the development and outcomes of an innovative program embedded within the framework of a national professional organization, and offers a model for potential adaptation by similar organizations to enhance their support of educators.After 10 years, 171 scholars have enrolled in the ESP, and 50 faculty have participated. Scholars are assigned a faculty advisor and participate in three full-day sessions at a national meeting; online, interactive learning modules; and a mentored, scholarly project. The program receives support from the APA in four organizational frames: structural, human resource, political, and symbolic. The self-perceived scholarly proficiency of the scholars in Cohort 1 increased significantly over time, and their productivity and collaborations increased during and after the program. Scholars wrote enthusiastically about their experience in yearly and postprogram evaluations. In interviews, eight past APA presidents explained that the ESP strengthened the APA's mission, created new leaders, and provided a new model for other APA programs. Outcomes of the ESP suggest that a longitudinal faculty development program embedded within a national professional organization can create a social enterprise not only within the organization but also within the broader national community of educator-scholars.

Clinical educators at U.S. academic health centers are frequently disadvantaged in the academic promotion system, lacking needed faculty development, mentoring, and networking support.In 2006, we implemented the national Educational Scholars Program to offer faculty development in educational scholarship for early career educators in pediatrics. We aimed to provide them with skills, experience, and initial success in educational scholarship and dissemination. The 3-year curriculum is delivered in interactive sessions at the annual pediatric academic meetings and online intersession modules. Curriculum content progresses from educational scholarship and implementing scholarly projects to dissemination and professional networking. Intersession modules address project planning, building an educator portfolio, reviewing the literature, using technology, authorship, and peer review. Concurrently, all scholars must complete a mentored educational project and demonstrate national dissemination of a peer-reviewed product to obtain a Certificate of Excellence in Educational Scholarship.The setting of this study was a national, longitudinal, cohort-based faculty development program built within the Academic Pediatric Association, a 2,000-member professional organization.In 10 years, the Educational Scholars Program has enrolled 172 scholars in 8 cohorts; 94 have graduated so far. We describe how formative evaluation guided curriculum refinement and process improvement. Summative evaluations show that faculty and scholars were satisfied with the program. Participant outcomes from Cohort 1, assessed at Kirkpatrick's four levels of evaluation, demonstrate increases in scholarly productivity, leadership activities, and academic promotions.Curriculum building is a dynamic process of ongoing evaluation and modification. Our program benefited from designing an integrated and focused curriculum, developing educational principles to guide program improvements, creating curricular tools to help learners organize and document their efforts, supporting project-based learning with expert mentoring, and facilitating peer and faculty networking and collaboration. A national, longitudinal faculty development program can support growth in academic knowledge and skills, promote professional networking, and thereby enrich educators' career opportunities.

This AMEE Guide provides an overview of medical education scholarship for early career scholars, based upon a summary of the existing literature and pragmatic advice derived from the experience of its authors. After providing an introduction to the principles of scholarship and describing questions that the Guide addresses, the authors offer a conceptual description of the complementary traditions of teaching and educational discovery, and advocate for the development of educational scholars with both traditions. They then describe the attributes of effective mentor-mentee relationships and how early career scholars can identify potential mentors who can fulfill this role. In the subsequent sections, they describe the appropriate development of scholarly questions and other components of a complete scholarly plan, including how to use conceptual frameworks in guiding such plans. From here, they describe methods that align with both the teaching and discovery traditions and provide concrete examples of each. They then provide guidelines for assessing the impact of scholarship, identify the various opportunities for sharing it, and how to effectively interpret and describe it. Additionally, they provide practical advice on how appropriately to demonstrate the scholarship in a promotional packet, including the principle of reflectivity in scholarship. Finally, they address the principles of applied research ethics for educational scholarship and when to consider soliciting approval for scholarly activities by a human research board.

Critical thinking is essential to a health professional's competence to assess, diagnose, and care for patients. Defined as the ability to apply higher-order cognitive skills (conceptualization, analysis, evaluation) and the disposition to be deliberate about thinking (being open-minded or intellectually honest) that lead to action that is logical and appropriate, critical thinking represents a "meta-competency" that transcends other knowledge, skills, abilities, and behaviors required in health care professions. Despite its importance, the developmental stages of critical thinking have not been delineated for nurses and physicians. As part of a task force of educators who considered different developmental stage theories, the authors have iteratively refined and proposed milestones in critical thinking. The attributes associated with unreflective, beginning, practicing, advanced, accomplished, and challenged critical thinkers are conceived as independent of an individual's level of training. Depending on circumstances and environmental factors, even the most experienced clinician may demonstrate attributes associated with a challenged thinker. The authors use the illustrative case of a patient with abdominal pain to demonstrate how critical thinking may manifest in learners at different stages of development, analyzing how the learner at each stage applies information obtained in the patient interaction to arrive at a differential diagnosis and plan for evaluation. The authors share important considerations and provide this work as a foundation for the development of effective approaches to teaching and promoting critical thinking and to establishing expectations for learners in this essential meta-competency.

The academic community needs a sound framework for the promotion and advancement of educators. The Group on Educational Affairs of the Association of American Medical Colleges organized a consensus conference that affirmed the use of five domains for documenting the quantity and quality of scholarly engagement in educational activities: teaching, curriculum, advising/mentoring, educational leadership/administration, and learner assessment.In this article, we offer detailed guidelines to evaluate these five domains of educator performance and the essential elements of scholarly activity. The guidelines are adapted from our developmental educator portfolio template and educator portfolio analysis tool, previously published in MedEdPORTAL. A short tool for educator performance evaluation that summarizes items in the guidelines is proposed for discussion.Our goal in this article is to itemize criteria for systematic faculty evaluation that can be applied in any institutional setting to assist promotion decision makers in their task of evaluating medical school faculty.

Introduction: Mentoring is increasingly recognized as central to career development. Less attention has been paid, however, to how mentoring relationships evolve over time. To provide a more complete picture of these complex relationships, the authors explored mentoring from a mentee’s perspective within the context of a three-year faculty development program in which the mentor provided specific expertise to assist the mentee in completing a scholarly educational project. Methods: Using an evolving focus group design, the authors interviewed mentee groups in 2007–2009 inclusive. Transcripts were coded inductively; codes were revised as data patterns became more apparent. Preliminary assertions about the answers to guiding questions were made; the trustworthiness of the assertions was assessed via member check. Results: Mentees offered a variety of reasons for choosing their project mentor, including proximity, familiarity, and mentor expertise. There was a dyadic relationship with the project mentor in year 1, a broader collaboration with multiple senior mentors in year 2, and mentoring among program peers in year 3. Mentees benefitted from mentors’ supportive behaviors and, to a lesser extent, mentors’ challenging behaviors. Conclusion: Mentoring relationships, in the context of this faculty development program, tended not to be an exclusive dyadic connection but rather a constellation of relationships that evolved over time and included peer mentoring. The complex reality of these relationships challenges the application of traditional mentoring models and suggests unique considerations in developing mentoring programs designed to meet the needs of faculty in academic medicine.

Projects--planned activities with specific goals and outcomes--have been used in faculty development programs to enhance participant learning and development. Projects have been employed most extensively in programs designed to develop faculty as educators. The authors review the literature and report the results of their 2008 study of the impact of projects within the Pennsylvania State University College of Medicine Junior Faculty Development Program, a comprehensive faculty development program. Using a mixed-methods approach, the products of project work, the academic productivity of program graduates, and the impact of projects on career development were analyzed. Faculty who achieved the most progress on their projects reported the highest number of academic products related to their project and the highest number of overall academic achievements. Faculty perceived that their project had three major effects on their professional development: production of a tangible outcome, development of a career focus, and development of relationships with mentors and peers. On the basis of these findings and a review of the literature, the authors conclude that projects are an essential element of a faculty development program. Projects provide a foundation for future academic success by enabling junior faculty to develop and hone knowledge and skills, identify a career focus and gain recognition within their community, generate scholarship, allocate time to academic work, and establish supportive relationships and collaborative networks. A list of best practices to successfully incorporate projects within faculty development programs is provided.

Traditional promotion standards rely heavily on quantification of research grants and publications in the curriculum vitae. The promotion and retention of educators is challenged by the lack of accepted standards to evaluate the depth, breadth, quality, and impact of educational activities. The authors sought to develop a practical analysis tool for the evaluation of educator portfolios (EPs), based on measurable outcomes that allow reproducible analysis of the quality and impact of educational activities.The authors, 10 veteran educators and an external expert evaluator, used a scholarly, iterative consensus-building process to develop the tool and test it using real EPs from educational scholars who followed an EP template. They revised the template in parallel with the analysis tool to ensure that EP data enabled valid and reliable evaluation. The authors created the EP template and analysis tool for scholar and program evaluation in the Educational Scholars Program, a three-year national certification program of the Academic Pediatric Association.The analysis tool combines 18 quantitative and 25 qualitative items, with specifications, for objective evaluation of educational activities and scholarship.The authors offer this comprehensive, yet practical tool as a method to enhance opportunities for faculty promotions and advancement, based on well-defined and documented educational outcome measures. It is relevant for clinical educators across disciplines and across institutions. Future studies will test the interrater reliability of the tool, using data from EPs written using the revised template.

Introduction: Mentoring is a central component of professional development. Evaluation of “successful” mentoring programs, however, has been limited and mainly focused on measures of satisfaction with the relationship. In today’s environment, mentoring programs must produce tangible outcomes to demonstrate success. To address this issue, the authors advance the framework of functional mentoring combined with measurement of outcomes at multiple levels. Methods: The mentoring program is embedded within an intensive, continuing medical education (CME) accredited faculty development program. Survey methodology is used to collect qualitative and quantitative data at the start, midpoint, and end of the program and longitudinally. Participants in 4 years of the program were surveyed. Results: In 4 years, 165 faculty participated in the program. Respondents were highly satisfied with the pairings: 85% of junior faculty believed their mentor had a significant effect on their projects. Junior faculty reported a significant enhancement of skills related to initiating and negotiating a new mentoring relationship (85%) and stated that their project would have a significant impact on their career (92%) and on the department or institution (86%). Discussion: The success of this mentoring program is demonstrated at multiple levels. The key outcome of functional mentoring is the project. Projects are aligned with professional responsibilities and with institutional missions. The project contributes to the individual’s dossier and adds value to the institution. Functional mentoring is a practical approach that allows measurable results at multiple levels.

Introduction Effective mentoring relationships are important contributors to career satisfaction, academic productivity, perceived self-efficacy, and successful networking. They are especially critical during times of transition in one's career. Unfortunately, such relationships can be difficult to find and challenging to sustain. In recent years, the concept of mentee-driven mentoring relationships has been proposed as a method of improving mentoring relationships. In this workshop, we apply the widely utilized concept of the individualized learning plan to examine and optimize the mentee-driven mentorship. Methods In small-group discussions, participants reflect on their unique mentoring needs as they transition into the next stage of their careers. Next, participants apply the corporate concept of managing up to their current mentoring relationships. Workshop leaders then introduce the optimal characteristics of mentors as well as the concept of a mentoring mosaic. Participants then create their own mentoring mosaics that describe the depth and breadth of their potential and of existing mentoring networks. With guidance from the workshop facilitators and other small-group discussants, participants then expand their mentoring mosaics by exploring possibilities for untapped mentoring resources (recognizing that mentors can come from the junior, peer, and senior levels). Finally, participants discuss in small groups strategies to overcome several challenging mentoring scenarios, including how to recruit new mentors and maximize the limited time of a mentor. Participants will leave with a better understanding of their roles as mentees and mentors. Results This workshop has been presented at the 2014 and 2015 Pediatric Academic Societies' Annual Meetings and at the Association of Pediatric Program Directors' Annual Spring Meeting in 2013 and 2014. Participants have included pediatric residents, chief residents, fellows, junior faculty, mid-level faculty, and senior-level faculty. Ninety-five percent of participants agreed or strongly agreed that their participation would improve the quality of their mentoring relationships. Discussion Although we have not collected data on the career outcomes of our workshop participants, literature suggests that mentees who utilize the principles we present will benefit greatly (e.g., attainment of leadership positions, academic promotion, career satisfaction).

Dialogue is essential for transforming the culture of academic health centers, yet organizational silence is common. Senior faculty and administrative leaders can play key roles in building a healthier environment by not only developing their own skills in discussing difficult issues, but also in coaching others to do so. These resources are designed for use in a professional development workshop intended to raise the level of understanding of senior faculty about the impact of organizational silence, generate discussion about practices to support dialogue around challenging issues, and increase their ability coach others to have difficult conversations. Brief didactic components present key background information and the activities allow participants to apply their learning and practice skills, thereby enhancing their confidence and coaching behaviors to effectively assist colleagues in engaging in challenging conversations. Two worksheets, a case example for role play, and a power point slide deck are included. This workshop was presented at a national conference (the 2013 Association of American Medical Colleges Group on Faculty Affairs Professional Development Conference). The audience was comprised primarily of faculty affairs deans at U.S. medical schools, a group of institutional leaders often called upon for advice about, and management of, difficult issues. Evaluations from attendees documented the usefulness of the hands-on activities and the value of the “take home” messages provided during the session. This proposal is limited by not having more robust program evaluation data. Unfortunately, only 10 attendees completed the formal evaluation for the conference. However, because there are so few resources available on the topic of coaching others to counteract organizational silence, we believe it is important to begin filling this gap by offering resources on the topic.

Barriers to influenza vaccination negatively impact immunization rates contributing to the morbidity and mortality from influenza. This study sought to determine if 1) the availability of year-round scheduling of annual autumn/winter influenza vaccination was associated with improved immunization rates for 2 high-risk populations of children and 2) this system was associated with early season vaccine administration.A retrospective cohort analysis was utilized to compare immunization rates during the 2003-2004 and 2004-2005 seasons. Billing records were used to determine eligible patients and vaccine receipt. A single, pediatric practice was studied, and two groups of patients were analyzed: 1) infants aged 6-23 months and 2) children21 years old with asthma. As opposed to Year 1, in Year 2 appointments in "flu clinics" for the following autumn became available 7 months prior to when vaccine became available. Patients and providers could therefore schedule an immunization appointment throughout the year.In Year 1, 552/1365 (40.4%) infants received at least 1 dose of influenza vaccine compared to 940/1265 (74.3%) in Year 2 (p0.001). For patients with asthma, 309/1332 (23.2%) received at least 1 dose of vaccine in Year 1 compared with 522/1489 (35.1%) in Year 2 (p0.001). Both groups also achieved higher immunization rates between September and November in Year 2 (p0.001).Year-round scheduling of influenza vaccination was associated with improved immunization rates in two high-risk populations, and may remove the barrier of scheduling difficulty, allow for a consistent year-round message from providers, and improve timing of vaccine administration.

Dialogue is essential for transforming institutions into learning organizations, yet many well-known characteristics of academic health centers (AHCs) interfere with open discussion. Rigid hierarchies, intense competition for resources, and the power of peer review in advancement processes all hamper difficult conversations, thereby contributing to organizational silence, and at great cost to the institution. Information necessary for critical decisions is not shared, individuals and the organization do not learn from mistakes, and diverse perspectives from those with less power are not entertained, or worse, are suppressed. When leaders become more skilled at inviting multiple perspectives and faculty more adept at broaching difficult conversations with those in power, differences are more effectively addressed and conflicts resolved. In this article, the authors frame why this skill is an essential competency for faculty and leaders alike and provide the following recommendations to institutions for increasing capacity in this area: (1) develop leaders to counteract organizational silence, (2) develop faculty members' skills in raising difficult issues with those in positions of power, and (3) train mentors to coach others in raising difficult conversations. The vitality of AHCs requires that faculty and institutional leaders develop relational communication skills and partner in learning through challenging conversations.

Valuing faculty as educators is essential for medical schools to fulfill their unique mission of educating physicians. The 2006 Consensus Conference on Educational Scholarship, sponsored by the Association of American Medical Colleges (AAMC) Group on Educational Affairs, provided educators seeking academic promotion with a portfolio-based format for documenting activities in five domains, using evidence of quantity, quality, a scholarly approach, and educational scholarship. Yet, the lack of a rigorous, widely accepted system to assess educator portfolio submissions during the promotion and tenure process continues to impede the ability to fully value educators and educational scholars.The AAMC Task Force on Educator Evaluation was formed in 2010 to establish consensus guidelines for use by those responsible for the rigorous evaluation of the educational contributions of faculty. The task force delineated the educational contributions currently valued by institutions and then fulfilled its charge by creating the Toolbox for Evaluating Educators, a resource which contains explicit evidence-based criteria to evaluate faculty in each of the five domains of educator activity. Adoption of such criteria is now the rate-limiting step in using a fair process to recognize educators through academic promotion. To inform institutional review and implementation of these criteria, this article describes the iterative, evidence- and stakeholder-based process to establish the criteria. The authors advocate institutional adoption of these criteria so that faculty seeking academic promotion as educators, like their researcher colleagues, can be judged and valued using established standards for the assessment of their work.

This leadership workshop addresses a leadership topic for which there is a paucity of information in the current literature: how to smoothly leave a position/organization and move effectively into a new position/organization. Thus, the purpose of this resource is to provide tools for a new or aspiring leader to enable him/her to strategically and successfully manage the transition to a new leadership role in a new organization. The workshop is specifically tailored to the leadership development of faculty in academic medicine. We utilize a case-based format to explore a real-world scenario that new leaders are likely to encounter. The case was developed from a compilation of the actual experiences of three leaders in academic medicine who, reflecting upon their own transitions to new, higher leadership positions in new institutions and their knowledge of the extant literature, created an illustrative case with essential content. The case study uncovers the major issues and opportunities facing a leader who is new to their position and/or the institution/organization. The workshop is organized into four sections or parts, presented as longitudinal “chapters” of the case. Questions at the end of each section allow workshop participants–new and aspiring leaders as well as more experienced leaders who have encountered similar situations–with an opportunity to share and learn from each other as well as from the workshop leader(s). For each section of the case, the core content of the section follows the section ‘title slide” slide portraying the specific chapter of the case. We developed and implemented this interactive workshop with hands-on activities, in order to engage leaders in the reflective and proactive work required to achieve the goal of a smooth, efficient, and successful transition to a new leadership position. In order to reach a substantial number of new and aspiring leaders in academic medicine, the workshop was presented at the Annual Meeting of the Association of American Medical Colleges in a plenary session cosponsored by multiple stakeholder groups of the association. Approximately 100 individuals attended the session. Evaluations completed by participants documented the usefulness of the content and approach, and comments underscored the value perceived by participants.

Summary Background: The system used by academic health centres to evaluate teaching must be valued by the large number of faculty staff that teach in clinical settings. Peer review can be used to evaluate and enhance clinical teaching. The objective of this study was to determine the perceptions of clinical faculty about the effects of participating in peer review. Methods: Faculty members were observed teaching in a clinical setting by trained peer observers. Feedback was provided using a checklist of behaviours and descriptive comments. Afterwards, semi-structured interviews were conducted to assess the faculty member’s perception about the process. Notes from the interviews were analysed using a grounded theory approach. The study was approved by the institutional review boards of all the institutions involved. Results: Three themes emerged from the interviews with faculty members: (1) they found the process to be valuable – they received information that affirmed “good” teaching behaviours, and were prompted to be more focused on their teaching; (2) they were motivated to enhance their teaching by being more deliberate, interactive and learner-centred; and (3) they were inspired to explore other opportunities to improve their teaching skills. Discussion: Peer review is a process that promotes the open discussion and exchange of ideas. This conversation advances clinical teaching skills and allows high-quality teaching behaviours to be strengthened.

Introduction In the past two decades, significant progress has been made in defining and justifying the value of scholarship in education. However, educators at many academic health centers continue to struggle with advancement and promotion because the process for evaluating their contributions is often cumbersome and accepted standards for evaluation are vague or lacking. The basis for evaluating educators has been strengthened by defining educational scholarship and by developing templates for faculty to document their educational contributions using educator's portfolios. This toolbox presents a sound framework for evaluating educational contributions in a rigorous manner analogous to the peer review process used for assessment of a faculty member's work in research and other scholarly work. Methods This 90-minute workshop is designed to provide hands-on opportunities for members of promotions/tenure committees to utilize the AAMC Toolbox for Evaluating Educators resource in the assessment of faculty member performance whose careers focus across the following five domains of educator activity: learner assessment, curriculum development, mentoring and advising, education leadership and administration, and teaching activities. The exercises in the workshop allow participants to use the indicators to reach summative decisions through a rigorous and consistent application of clear yet flexible standards. The workshop consists of an introductory PowerPoint presentation and small-group activities followed by a facilitated large-group discussion that allows the participants to explore how the evidence-based standards in the toolbox can be integrated with existing institutional processes for the evaluation of the performance of educators. The workshop and its resources can be adapted and used for training/professional development sessions for other decision making committees/members (e.g., awards committees or selection committees for a teaching academy). Results A professional development workshop using the Toolbox has been presented at each of the regional Group on Educational Affairs meetings, at the Group on Faculty Affairs annual Professional Development Conference, and in plenary sessions at two annual AAMC meetings. The participants have reviewed, critiqued and applied the indicators in the Toolbox during interactive exercises. The authors used written and oral feedback to refine the structure and content of the Toolbox. Discussion The resource was created primarily for faculty and committees charged with faculty evaluation and decision making; however, it is also useful for educators who are seeking promotion and their mentors including faculty affairs and education administrative leaders. Knowledge of the standards by which one's performance will be judged helps an educator create a portfolio that documents his/her activities and the impact of their work, in a standardized format that can be readily assessed.

Too often, educators are disadvantaged in the promotion and tenure process because their roles are not well understood by committees, and methods to evaluate educator performance have not been standardized. These educator evaluation guidelines contribute to the construction of a framework for the academic promotion and advancement of educators. The guidelines were designed to assist members of promotion committees in the evaluation of medical school faculty by a systematic, objective, and criterion-based process, and to help educators understand the elements of performance by which they are evaluated. To make the tool as accessible and usable as possible we avoided technical educational jargon, provided concrete indicators for each item, and illustrated items with copious examples. The guidelines include two tables with a total of 24 evaluation items distilled from our previously published, 36-item research tool for analysis of educator portfolios. That research tool was created by an iterative consensus development process and tested with live educator portfolios, so the items included in these guidelines have been well validated. The first table (19 items) is organized around five domains of educator performance: teaching, curriculum, advising/mentoring, educational leadership/administration, and learner assessment. The second table (five items) addresses the two components of scholarly activity: scholarly approach and educational scholarship. The five domains and two definitions of scholarly activity were confirmed by a consensus conference convened by the AAMC Group on Educational Affairs in 2009. Each domain is defined and elaborated with items for evaluation selected to capture quantity, quality, and impact of performance. For each evaluation item, several suggested indicators describe what kinds of evidence might earn credits for the educator and specific examples of high-quality performance are provided. Evaluation of educators is not well captured by numerical measures only. Items in the guidelines are appropriate for both quantitative and qualitative assessment; quantitative items are labeled. Despite the difficulty of objectively evaluating qualitative measures, some elements of educational practice are best assessed based on their adherence to accepted educational principles, rather than by being counted. Thus, the guidelines encourage an educator evaluation that is flexible and takes a broad perspective on performance. The guidelines avoid a prescriptive approach. They do not include a rating system, because each institution needs to develop its own scoring approach based on the value it places on specific educator activities. We have left room for institutions to decide how many areas of excellence are expected of a faculty member in each promotion track and at each rank. However, the included User's Manual discusses possible approaches to creating a rating system to accompany the 24-item evaluation tool.

Today, faculty in academic medicine face challenges in all three mission areas--research, education, and patient care--and require a broad set of competencies to survive in this changing environment. To support faculty and to design assessments that match new expectations, the authors argue that it is essential to capture the full scope of skills, knowledge, and behaviors necessary for a successful faculty member. Thus, it is timely to explore and define competencies for faculty in academic medicine. The authors describe three approaches to identifying faculty competencies. Each reveals diverse but overlapping sets of competency domains, reflecting the breadth of activities expected of today's faculty. To organize these competencies into a coherent framework, the authors propose a model based on a typology of competency. A key feature of the model is the division between occupational competencies, which are largely role-specific, and personal competencies, which are necessary for all faculty. A competency framework also must be developmental, to reflect the growth in skills, knowledge, and behaviors from trainee to expert and to allow for an individual's changing roles over a career. Such a competency framework will inform professional development activities and require assessment of competence. The generation of competencies also will reveal areas of faculty practice that are poorly measured, requiring new tools to be incorporated into existing processes of faculty evaluation. The authors provide general principles to guide the identification of a competency framework for faculty and invite the academic medicine community to engage in further discussion.

Background: Developing and ensuring successful collaborative research in medical education is no small task, but the rewards to researchers can be great. Collaborative research in medical education offers significant opportunities for investigators who wish to pool limited resources and expand professional networks. Despite this, collaboration often occurs without advance planning for the logistical aspects of collaborative work. Aims and Methods: These 12 tips, derived from developing and presenting a session on strategies for effective collaboration conducted at a national meeting, will assist readers who are planning to collaborate or are already engaged in collaborative scholarship. The tips are organized into items to consider in three phases of collaboration: planning, implementation, and dissemination of outcomes. Results and Conclusions: From selecting a topic to recognizing when the collaboration should end, the tips underscore the importance of good communication, and clear expectations for participants.

This resource is a tool for the evaluation of educator portfolios, based on measurable outcomes designed to allow reproducible analysis of the quality and impact of educational activities. The analysis tool is organized in tabular format and combines 18 quantitative, 25 qualitative, and five holistic ratings. We sought a balance between qualitative and quantitative items in a tool that captured important nuances of educational performance but was not too long to preclude practical use by a promotions committee. A unique feature of this tool is the inclusion of seven index scores that combine several quantitative measures to measure depth and impact of an educational activity. For example, the curriculum impact index, which measures the impact of all the curricula developed by a faculty member, combines for each curriculum the number of learners taught, a geographic impact score, and an implementation score. Qualitative items are scored on a 3-point rating scale; verbal specifications for an intermediate rating are provided. The accompanying instructions for use provide step-by-step guidance on how to score both quantitative and qualitative items with specific examples. Use of this comprehensive-yet-practical tool is offered as a method to enhance opportunities for the promotion and advancement of educators, based on well-defined and documented educational outcome measures. It is relevant for clinical educators across disciplines and across institutions. Our lessons learned as a consensus development team may prove useful to other raters who use or adapt this tool. Evaluation of quantitative items taught us repeatedly to keep it simple. Proliferation of items is tempting, but a focused selection of essential items makes the tool practical. Evaluation of qualitative items needs to be grounded in sound principles that are clear to the rater. Specification of ratings for each item was critical to achieve concordance in our qualitative ratings.

Most who teach in clinical settings see themselves primarily as clinicians or physicians responsible for patient care and only secondarily as educators. The education literature suggests that teaching predominantly operates at a tacit level, where teachers rely on core beliefs to guide their practice, and actually spend little time in reflective practice. Given the lack of research on how medical educators in clinical settings view their teaching, the purpose of this qualitative study was to explore the teaching beliefs of faculty in a pediatrics department in a college of medicine.Using a Teaching Perspectives Inventory, observations and in-depth interviews, a complex picture was revealed about teaching beliefs of medical educators.Due to contextual constraints of the clinical setting (e.g., time, competing stakeholders) that requires primary attention to patient care, they describe 'teaching on the fly'. There is a strong emphasis on: delivering content; encouraging thinking among students; providing questioning and engaging learning experiences; and respecting students as learners.The implications of these beliefs are significant and indicate that faculty can benefit from opportunities that make their beliefs about teaching more conscious, particularly in determining how best to prepare future physicians to teach in clinical settings.

The educator portfolio (EP) is a collection of information that documents educational efforts, activities, and scholarship. This tool provides a template for collecting EP information in a standardized manner. Standardization makes the EP suitable for internal peer review in promotion decisions and also allows for external review. We have experience both in education and in developing and using EPs in our academic careers. This template was created using a scholarly approach. The first step involved a critical appraisal of existing formats for portfolios and of the literature regarding their use. Following a modified Delphi process, we conducted multiple iterations of review and refinement of the template. In our national faculty development program, this template is used as a developmental document. Throughout the program, scholars are expected to build their EPs with mentoring and feedback from faculty. Ultimately, these materials are used to create a high-stakes EP for promotion review. This comprehensive template contains both quantitative and qualitative components; to our knowledge, it is the first to do so. The qualitative elements promote reflective self-assessment by users. Questions throughout stimulate users to consider criteria for scholarship. The purpose of each section is clearly defined. The instructions embedded in each section discuss important underlying educational principles to be considered during the development of the document. During the pilot of this template, we realized that young educators needed specific guidelines to enable them to complete their EPs successfully. The instructions were enhanced by the description of educational principles and of specific references from the literature. The guidelines were also augmented by specific examples of activities, teaching strategies, assessment tools, and evaluation strategies. Throughout completion of their EPs, scholars are asked to use the principles of educational scholarship in their analysis of the results and impact of their work. This template encourages users to reflect on their work and to use what they learn to guide their future educational efforts. The template is generalizable to all institutions and can be used by all types of educators within an organization.

Empowerment of faculty is essential for academic success. The Junior Faculty Development Program (JFDP), sponsored by the Office of Professional Development of the Penn State College of Medicine, was established in 2003 with the goal of promoting the development and advancement of junior faculty so they can achieve success in their academic careers. The program consists of two components: a curriculum in research, education, clinical practice, and career development, and an individual project completed under the guidance of a senior faculty mentor. The curriculum provides faculty with knowledge, skills, and resources. Mentoring provides relationships and support. Together, these elements combine to empower junior faculty to better manage their careers. The effectiveness of the program has been demonstrated by several measures: participants evaluated the program highly, demonstrated increases in their perceptions of their own abilities, and completed tasks important to the advancement of their careers. Participants stated they were better prepared to advance their academic careers and that the individual projects would contribute to their career advancement. On the basis of this experience, the authors suggest that faculty development programs should empower faculty so that they can more effectively chart a successful career in academic medicine. This report describes an empowerment model, and the design, implementation, and evaluation of the Junior Faculty Development Program in 2003-04 and 2004-05. The authors offer this program as a model for the benefit of other institutions and for one of their most valuable assets: junior faculty.

Objective To assess the quality of communication and education about asthma in Pennsylvania public schools. Methods Survey of a stratified random sample of school nurses in rural and urban Pennsylvania public schools (n = 996) concerning communication with school nurses about asthma by physicians and parents, nurses' perceived obstacles to asthma management at school, and utilization of and need for education about asthma. Results A total of 757 surveys were received (response rate 76%). Thirty-nine percent of school nurses rated their communication with physicians about asthma as either poor or very poor. Urban nurses were significantly more likely to report poor/very poor physician communication ( P = .09). Fifty-two percent of the nurses overall (43% rural, 56% urban) also cited lack of communication with parents as an important obstacle to asthma management. Forty-nine percent of school nurses (43% rural, 52% urban) reported attending an asthma education program during the previous year, and 75% (83% rural, 71% urban) expressed interest in additional education. Education about asthma was provided for classroom teachers in 54% of schools (56% rural, 54% urban) and provided for students in 58% of schools (54% rural, 60% urban). Conclusions These findings document need for improvement in communication about children's asthma between school nurses and physicians. Although communication appears better in rural relative to urban schools, it is a salient issue in both settings. Study findings also indicate the need for expanded professional education opportunities for school nurses and improved access to appropriate curricular materials for school staff, parents, and students.