Search

BACKGROUND: Entrustable Professional Activities (EPA) and competencies represent components of a competency-based education framework. EPAs are assessed based on the level of supervision (LOS) necessary to perform the activity safely and effectively. The broad competencies, broken down into narrower subcompetencies, are assessed using milestones, observable behaviors of ones abilities along a developmental spectrum. Integration of the two methods, accomplished by mapping the most relevant subcompetencies to each EPA, may provide a cross check between the two forms of assessment and uncover those subcompetencies that have the greatest influence on the EPA assessment. OBJECTIVES: We hypothesized that 1) there would be a strong correlation between EPA LOS ratings with the milestone levels for the subcompetencies mapped to the EPA; 2) some subcompetencies would be more critical in determining entrustment decisions than others, and 3) the correlation would be weaker if the analysis included only milestones reported to the Accreditation Council for Graduate Medical Education (ACGME). METHODS: In fall 2014 and spring 2015, the Subspecialty Pediatrics Investigator Network asked Clinical Competency Committees to assign milestone levels to each trainee enrolled in a pediatric fellowship for all subcompetencies mapped to 6 Common Pediatric Subspecialty EPAs as well as provide a rating for each EPA based upon a 5-point LOS scale. RESULTS: One-thousand forty fellows were assessed in fall and 1048 in spring, representing about 27% of all fellows. For each EPA and in both periods, the average milestone level was highly correlated with LOS (rho range 0.59-0.74; p  0.05). CONCLUSIONS: We found a strong relationship between milestone level and EPA LOS rating but no difference if the subcompetencies were weighted, or if only milestones reported to the ACGME were used. Our results suggest that representative behaviors needed to effectively perform the EPA, such as key subcompetencies and milestones, allow for future language adaptations while still supporting the current model of assessment. In addition, these data provide additional validity evidence for using these complementary tools in building a program of assessment.

Abstract The Milestones were initiated by the Accreditation Council for Graduate Medical Education (ACGME) to provide a framework for monitoring a trainee’s progression throughout residency/fellowship. The Milestones describe stepwise skill progression through six core domains of clinical competency: Patient Care, Medical Knowledge, Interpersonal and Communication Skills, Practice-based Learning and Improvement, Professionalism, and Systems-based Practice. Since their introduction in 2013, several barriers to implementation have emerged. Thus, the ACGME launched the Milestones 2.0 project to develop updated specialty-specific milestones. The Pediatric Endocrinology Milestones 2.0 project aimed to improve upon Milestones 1.0 by addressing common limitations, providing resources for faculty to easily incorporate milestones into their assessment of trainees, and adding sub-competencies in health disparities, patient safety, and physician well-being. This paper reviews the development of the Pediatric Endocrinology Milestones 2.0 including the major changes from Milestones 1.0, development of the Supplemental Guide, and how Milestones 2.0 can be applied at the program level. Although use of the Milestones are required only for ACGME programs, the tools provided in Milestones 2.0 are applicable to fellowship programs worldwide.

The Milestones were initiated by the Accreditation Council for Graduate Medical Education (ACGME) to provide a framework for monitoring a trainee's progression throughout residency/fellowship. The Milestones describe stepwise skill progression through six core domains of clinical competency: Patient Care, Medical Knowledge, Interpersonal and Communication Skills, Practice-based Learning and Improvement, Professionalism, and Systems-based Practice. Since their introduction in 2013, several barriers to implementation have emerged. Thus, the ACGME launched the Milestones 2.0 project to develop updated specialty-specific milestones. The Pediatric Endocrinology Milestones 2.0 project aimed to improve upon Milestones 1.0 by addressing common limitations, providing resources for faculty to easily incorporate milestones into their assessment of trainees, and adding sub-competencies in health disparities, patient safety, and physician well-being. This paper reviews the development of the Pediatric Endocrinology Milestones 2.0 including the major changes from Milestones 1.0, development of the Supplemental Guide, and how Milestones 2.0 can be applied at the program level. Although use of the Milestones are required only for ACGME programs, the tools provided in Milestones 2.0 are applicable to fellowship programs worldwide. [ABSTRACT FROM AUTHOR]

Abstract Background Entrustable Professional Activities (EPA) and competencies represent components of a competency-based education framework. EPAs are assessed based on the level of supervision (LOS) necessary to perform the activity safely and effectively. The broad competencies, broken down into narrower subcompetencies, are assessed using milestones, observable behaviors of one’s abilities along a developmental spectrum. Integration of the two methods, accomplished by mapping the most relevant subcompetencies to each EPA, may provide a cross check between the two forms of assessment and uncover those subcompetencies that have the greatest influence on the EPA assessment. Objectives We hypothesized that 1) there would be a strong correlation between EPA LOS ratings with the milestone levels for the subcompetencies mapped to the EPA; 2) some subcompetencies would be more critical in determining entrustment decisions than others, and 3) the correlation would be weaker if the analysis included only milestones reported to the Accreditation Council for Graduate Medical Education (ACGME). Methods In fall 2014 and spring 2015, the Subspecialty Pediatrics Investigator Network asked Clinical Competency Committees to assign milestone levels to each trainee enrolled in a pediatric fellowship for all subcompetencies mapped to 6 Common Pediatric Subspecialty EPAs as well as provide a rating for each EPA based upon a 5-point LOS scale. Results One-thousand forty fellows were assessed in fall and 1048 in spring, representing about 27% of all fellows. For each EPA and in both periods, the average milestone level was highly correlated with LOS (rho range 0.59–0.74; p 0.05). Conclusions We found a strong relationship between milestone level and EPA LOS rating but no difference if the subcompetencies were weighted, or if only milestones reported to the ACGME were used. Our results suggest that representative behaviors needed to effectively perform the EPA, such as key subcompetencies and milestones, allow for future language adaptations while still supporting the current model of assessment. In addition, these data provide additional validity evidence for using these complementary tools in building a program of assessment.

Background: Entrustable Professional Activities (EPA) and competencies represent components of a competency-based education framework. EPAs are assessed based on the level of supervision (LOS) necessary to perform the activity safely and effectively. The broad competencies, broken down into narrower subcompetencies, are assessed using milestones, observable behaviors of one's abilities along a developmental spectrum. Integration of the two methods, accomplished by mapping the most relevant subcompetencies to each EPA, may provide a cross check between the two forms of assessment and uncover those subcompetencies that have the greatest influence on the EPA assessment. Objectives: We hypothesized that 1) there would be a strong correlation between EPA LOS ratings with the milestone levels for the subcompetencies mapped to the EPA; 2) some subcompetencies would be more critical in determining entrustment decisions than others, and 3) the correlation would be weaker if the analysis included only milestones reported to the Accreditation Council for Graduate Medical Education (ACGME). Methods: In fall 2014 and spring 2015, the Subspecialty Pediatrics Investigator Network asked Clinical Competency Committees to assign milestone levels to each trainee enrolled in a pediatric fellowship for all subcompetencies mapped to 6 Common Pediatric Subspecialty EPAs as well as provide a rating for each EPA based upon a 5-point LOS scale. Results: One-thousand forty fellows were assessed in fall and 1048 in spring, representing about 27% of all fellows. For each EPA and in both periods, the average milestone level was highly correlated with LOS (rho range 0.59–0.74; p < 0.001). Correlations were similar when using a weighted versus unweighted milestone score or using only the ACGME reported milestones (p > 0.05). Conclusions: We found a strong relationship between milestone level and EPA LOS rating but no difference if the subcompetencies were weighted, or if only milestones reported to the ACGME were used. Our results suggest that representative behaviors needed to effectively perform the EPA, such as key subcompetencies and milestones, allow for future language adaptations while still supporting the current model of assessment. In addition, these data provide additional validity evidence for using these complementary tools in building a program of assessment. [ABSTRACT FROM AUTHOR]

Abstract Background Interpersonal and Communication Skills (ICS) and Professionalism milestones are challenging to evaluate during medical training. Paucity in proficiency, direction and validity evidence of assessment tools of these milestones warrants further research. We validated the reliability of the previously-piloted Instrument for Communication skills and Professionalism Assessment (InCoPrA) in medical learners. Methods This validity approach was guided by the rigorous Kane’s Framework. Faculty-raters and standardized patients (SPs) used their respective InCoPrA sub-component to assess distinctive domains pertinent to ICS and Professionalism through multiple expert-built simulated-scenarios comparable to usual care. Evaluations included; inter-rater reliability of the faculty total score; the correlation between the total score by the SPs; and the average of the total score by two-faculty members. Participants were surveyed regarding acceptability, realism, and applicability of this experience. Results Eighty trainees and 25 faculty-raters from five medical residency training sites participated. ICC of the total score between faculty-raters was generally moderate (ICC range 0.44–0.58). There was on average a moderate linear relationship between the SPs and faculty total scores (Pearson correlations range 0.23–0.44). Majority of participants ascertained receiving a meaningful, immediate, and comprehensive patient-faculty feedback. Conclusions This work substantiated that InCoPrA was a reliable, standardized, evidence-based, and user-friendly assessment tool for ICS and Professionalism milestones. Validating InCoPrA showed generally-moderate agreeability and high acceptability. Using InCoPrA also promoted engaging all stakeholders in medical education and training–faculty, learners, and SPs—using simulation-media as pathway for comprehensive feedback of milestones growth.

Background: Interpersonal and Communication Skills (ICS) and Professionalism milestones are challenging to evaluate during medical training. Paucity in proficiency, direction and validity evidence of assessment tools of these milestones warrants further research. We validated the reliability of the previously-piloted Instrument for Communication skills and Professionalism Assessment (InCoPrA) in medical learners. Methods: This validity approach was guided by the rigorous Kane's Framework. Faculty-raters and standardized patients (SPs) used their respective InCoPrA sub-component to assess distinctive domains pertinent to ICS and Professionalism through multiple expert-built simulated-scenarios comparable to usual care. Evaluations included; inter-rater reliability of the faculty total score; the correlation between the total score by the SPs; and the average of the total score by two-faculty members. Participants were surveyed regarding acceptability, realism, and applicability of this experience. Results: Eighty trainees and 25 faculty-raters from five medical residency training sites participated. ICC of the total score between faculty-raters was generally moderate (ICC range 0.44–0.58). There was on average a moderate linear relationship between the SPs and faculty total scores (Pearson correlations range 0.23–0.44). Majority of participants ascertained receiving a meaningful, immediate, and comprehensive patient-faculty feedback. Conclusions: This work substantiated that InCoPrA was a reliable, standardized, evidence-based, and user-friendly assessment tool for ICS and Professionalism milestones. Validating InCoPrA showed generally-moderate agreeability and high acceptability. Using InCoPrA also promoted engaging all stakeholders in medical education and training–faculty, learners, and SPs—using simulation-media as pathway for comprehensive feedback of milestones growth. [ABSTRACT FROM AUTHOR]

Abstract Background The changing healthcare landscape requires physicians to develop new knowledge and skills such as high-value care, systems improvement, population health, and team-based care, which together may be referred to as the Science of Health Care Delivery (SHCD). To engender public trust and confidence, educators must be able to meaningfully assess physicians’ abilities in SHCD. We aimed to develop a novel set of SHCD milestones based on published Accreditation Council for Graduate Medical Education (ACGME) milestones that can be used by medical schools to assess medical students’ competence in SHCD. Methods We reviewed all ACGME milestones for 25 specialties available in September 2013. We used an iterative, qualitative process to group the ACGME milestones into SHCD content domains, from which SHCD milestones were derived. The SHCD milestones were categorized within the current ACGME core competencies and were also mapped to Association of American Medical Colleges’ Entrustable Professional Activities (AAMC EPAs). Results Fifteen SHCD sub-competencies and corresponding milestones are provided, grouped within ACGME core competencies and mapped to multiple AAMC EPAs. Conclusions This novel set of milestones, grounded within the existing ACGME competencies, defines fundamental expectations within SHCD that can be used and adapted by medical schools in the assessment of medical students in this emerging curricular area. These milestones provide a blueprint for SHCD content and assessment as ongoing revisions to milestones and curricula occur.

Background: The changing healthcare landscape requires physicians to develop new knowledge and skills such as high-value care, systems improvement, population health, and team-based care, which together may be referred to as the Science of Health Care Delivery (SHCD). To engender public trust and confidence, educators must be able to meaningfully assess physicians' abilities in SHCD. We aimed to develop a novel set of SHCD milestones based on published Accreditation Council for Graduate Medical Education (ACGME) milestones that can be used by medical schools to assess medical students' competence in SHCD. Methods: We reviewed all ACGME milestones for 25 specialties available in September 2013. We used an iterative, qualitative process to group the ACGME milestones into SHCD content domains, from which SHCD milestones were derived. The SHCD milestones were categorized within the current ACGME core competencies and were also mapped to Association of American Medical Colleges' Entrustable Professional Activities (AAMC EPAs). Results: Fifteen SHCD sub-competencies and corresponding milestones are provided, grouped within ACGME core competencies and mapped to multiple AAMC EPAs. Conclusions: This novel set of milestones, grounded within the existing ACGME competencies, defines fundamental expectations within SHCD that can be used and adapted by medical schools in the assessment of medical students in this emerging curricular area. These milestones provide a blueprint for SHCD content and assessment as ongoing revisions to milestones and curricula occur. [ABSTRACT FROM AUTHOR]

Background: Mistreatment of trainees remains a frequently reported phenomenon in medical education. One barrier to creating an educational culture of respect and professionalism may be a lack of alignment in the perceptions of mistreatment among different learners. Through the use of clinical vignettes, our aim was to assess the perceptions of trainees toward themes of potential mistreatment at different stages of training. Methods: Based on observations from external experts embedded in the clinical learning environment, six thematic areas of potential mistreatment were identified: verbal abuse, specialty-choice discrimination, non-educational tasks, withholding/denying learning opportunities, neglect and gender/racial insensitivity. Corresponding clinical vignettes were created and distributed to 1) medical students, 2) incoming interns, 3) residents/fellows. Perceptions of the appropriateness of the interactions depicted in the vignettes were measured on a 5-point Likert scale. Scores were categorized into neutral or appropriate (⩽3) or inappropriate (i.e. mistreatment) (>3) and compared using chi-squared tests. Results: Four hundred twenty seven trainees participated (182 students, 120 interns, 125 residents/fellows). Proportions of students perceiving mistreatment differed significantly from those of interns and residents/fellows in domains of verbal abuse, specialty discrimination and gender/racial insensitivity (p < 0.05). In scenarios comparing interns to residents/ fellows, no significant differences were noted in perceptions of mistreatment in the domains of non-educational tasks, withholding learning and neglect. Conclusions: Perceptions of mistreatment differ at different developmental stages of medical training. After exposure to the clinical learning environment, perceptions of incoming interns did not differ from those of residents/fellows, implicating clinical rotations as a key period in indoctrinating students into the prevailing culture. More longitudinal studies are needed to confirm or better examine this phenomenon. [ABSTRACT FROM AUTHOR]