Search

PURPOSE To examine whether there are group differences in milestone ratings submitted by program directors working with clinical competency committees (CCCs) based on gender for internal medicine (IM) residents and whether women and men rated similarly on milestones perform comparably on subsequent in-training and certification examinations. METHOD This national retrospective study examined end-of-year medical knowledge (MK) and patient care (PC) milestone ratings and IM In-Training Examination (IM-ITE) and IM Certification Examination (IM-CE) scores for 2 cohorts (2014-2017, 2015-2018) of U.S. IM residents at ACGME-accredited programs. It included 20,098/21,440 (94%) residents, with 9,424 women (47%) and 10,674 men (53%). Descriptive statistics and differential prediction techniques using hierarchical linear models were performed. RESULTS For MK milestone ratings in PGY-1, men and women showed no statistical difference at a significance level of .01 (P = .02). In PGY-2 and PGY-3, men received statistically higher average MK ratings than women (P = .002 and P < .001, respectively). In contrast, men and women received equivalent average PC ratings in each PGY (P = .47, P = .72, and P = .80, for PGY-1, PGY-2, and PGY-3, respectively). Men slightly outperformed women with similar MK or PC ratings in PGY-1 and PGY-2 on the IM-ITE by about 1.7 and 1.5 percentage points, respectively, after adjusting for covariates. For PGY-3 ratings, women and men with similar milestone ratings performed equivalently on the IM-CE. CONCLUSIONS Milestone ratings were largely similar for women and men. Generally, women and men with similar MK or PC milestone ratings performed similarly on future examinations. Although there were small differences favoring men on earlier examinations, these differences disappeared by the final training year. It is questionable whether these small differences are educationally or clinically meaningful. The findings suggest fair, unbiased milestone ratings generated by program directors and CCCs assessing residents.

Purpose: To assess the correlations between United States Medical Licensing Examination (USMLE) performance, American College of Physicians (ACP) Internal Medicine In-Training Examination (IM-ITE) performance, American Board of Internal Medicine (ABIM) Internal Medicine Certification Exam (IM-CE) performance, and other medical knowledge and demographic variables. Method: The study included 9,676 postgraduate year (PGY) 1, 11,424 PGY2, and 10,239 PGY3 internal medicine (IM) residents from any Accreditation Council for Graduate Medical Education-accredited IM residency program who took the IM-ITE (2014 or 2015) and the IM-CE (2015-2018). USMLE scores, IM-ITE percent correct scores, and IM-CE scores were analyzed using multiple linear regression and IM-CE pass/fail status was analyzed using multiple logistic regression, controlling for USMLE Step 1, Step 2 Clinical Knowledge, and Step 3 scores; averaged medical knowledge milestones; age at IM-ITE; gender; and medical school location (United States or Canada vs international). Results: All variables were significant predictors of passing the IM-CE with IM-ITE scores having the strongest association and USMLE Step scores being the next strongest predictors. Prediction curves for the probability of passing the IM-CE based solely on IM-ITE score for each PGY show that residents must score higher on the IM-ITE with each subsequent administration to maintain the same estimated probability of passing the IM-CE. Conclusions: The findings from this study should support residents and program directors in their efforts to more precisely identify and evaluate knowledge gaps for both personal learning and program improvement. While no individual USMLE Step score was as strongly predictive of IM-CE score as IM-ITE score, the combined relative contribution of all 3 USMLE Step scores was of a magnitude similar to that of IM-ITE score.

The Accreditation Council for Graduate Medical Education (ACGME) has surveyed residents since 2003, and faculty since 2012. Surveys are designed to assess program functioning and specify areas for improvement. The purpose of this study was to assess the association of the ACGME's resident and faculty surveys with residency-program-specific performance on the American Board of Internal Medicine (ABIM) certification exam.Data were available from residents and faculty in 375 U.S. ACGME-accredited internal medicine programs from the 2012-2013, 2013-2014, and 2014-2015 academic years. Analysis of variance and correlations were used to examine the relationship between noncompliance with ACGME program requirements as assessed by the resident and faculty surveys, and ABIM program pass rates.Noncompliance reported on the resident and faculty surveys was highest for programs not meeting the ACGME program requirement of an 80% pass rate on the ABIM certification examination. This relationship was significant for overall noncompliance, both within the resident (P.001) and faculty (P.05) surveys, for many areas within the two surveys (correlations ranged between -.07 and -.25, and P values ranged between .20 and.001), and for the highest levels of noncompliance across areas of the resident (P.001) and faculty (P.04) surveys.ACGME resident and faculty surveys were significantly associated with ABIM program pass rates, supporting the importance of these surveys within the ACGME's Next Accreditation System.

To report findings from a national effort initiated by three primary care certifying boards to catalyze change in primary care training.In this mixed-method pilot study (2012-2014), 36 faculty in 12 primary care residencies (family medicine, internal medicine, pediatrics) from four institutions participated in a professional development program designed to prepare faculty to accelerate change in primary care residency training by uniting them in a common mission to create effective ambulatory clinical learning environments. Surveys administered at baseline and 12 months after initial training measured changes in faculty members' confidence and skills, continuity clinics, and residency training programs. Feasibility evaluation involved assessing participation. The authors compared quantitative data using Wilcoxon signed-rank and Bhapkar tests. Observational field notes underwent narrative analysis.Most participants attended two in-person training sessions (92% and 72%, respectively). Between baseline and 12 months, faculty members' confidence in leadership improved significantly for 15/19 (79%) variables assessed; their self-assessed skills improved significantly for 21/22 (95%) competencies. Two medical home domains ("Continuity of Care," "Support/Care Coordination") improved significantly (P.05) between the two time periods. Analyses of qualitative data revealed that interdisciplinary learning communities formed during the program and served to catalyze transformational change.Results suggest that improvements in faculty perceptions of confidence and skills occurred and that the creation of interdisciplinary learning communities catalyzed transformation. Lengthening the intervention period, engaging other professions involved in training the primary care workforce, and a more discriminating evaluation design are needed to scale this model nationally.

Problem The transition from medical student to intern may cause stress and burnout in new interns and the delivery of suboptimal patient care. Despite a formal set of subinternship curriculum guidelines, program directors have expressed concern regarding the skill set of new interns and the lack of standardization in that skill set among interns from different medical schools. To address these issues, the Accreditation Council for Graduate Medical Education’s Next Accreditation System focuses on the development of a competency-based education continuum spanning undergraduate, graduate, and continuing medical education. Approach In 2010, the Clerkship Directors in Internal Medicine subinternship task force, in collaboration with the Association of Program Directors in Internal Medicine survey committee, surveyed internal medicine residency program directors to determine which competencies or skills they expected from new medical school graduates. The authors summarized the results using categories of interest. Outcomes In both an item rank list and free-text responses, program directors were nearly uniform in ranking the skills they deemed most important for new interns—organization and time management and prioritization skills; effective communication skills; basic clinical skills; and knowing when to ask for assistance. Next Steps Stakeholders should use the results of this survey as they develop a milestone-based curriculum for the fourth year of medical school and for the internal medicine subinternship. By doing so, they should develop a standardized set of skills that meet program directors’ expectations, reduce the stress of transitions across the educational continuum, and improve the quality of patient care.

PURPOSE Residents' reflections on quality improvement (QI) opportunities are poorly understood. The authors used the Mayo Evaluation of Reflection on Improvement Tool (MERIT) to measure residents' reflection scores across three years and to determine associations between reflection scores and resident and adverse patient event characteristics. METHOD From 2006 to 2009, 48 Mayo Clinic internal medicine residents completed biannual reflections on adverse events and classified event severity and preventability. Faculty assessed residents' reflections using MERIT, which contains 18 Likert-scaled items and measures three factors-personal reflection, systems reflection, and event merit. ANOVA was used to identify changes in MERIT scores across three years of training and among factors, paired t tests were used to identify differences between MERIT factor scores, and generalized estimating equations were used to examine associations between MERIT scores and resident and adverse event characteristics. RESULTS The residents completed 240 reflections. MERIT reflection scores were stable over time. Individual factor scores differed significantly (P < .0001), with event merit being the highest and systems reflection the lowest. Event preventability was significantly associated with MERIT factor scores and overall scores (beta = 0.415; CI = 0.186-0.643; P = .0004). No significant associations between MERIT scores and resident characteristics or event severity were identified. CONCLUSIONS Residents' reflections on adverse events remained constant over time, were lowest for systems factors, and were associated with adverse event preventability. Future research should explore learners' emphasis on systems aspects of QI and the relationship between QI and event preventability.

OBJECTIVE: To accurately model residents' work hours and assess options to forthrightly meet Residency Review Committee-Internal Medicine (RRC-IM) requirements. DESCRIPTION: The requirements limiting residents' work hours are clearly defined by the Accreditation Council for Graduate Medical Education (ACGME) and the RRC-IM: "When averaged over any four-week rotation or assignment, residents must not spend more than 80 hours per week in patient care duties."(1) The call for the profession to realistically address work-hours violations is of paramount importance.(2) Unfortunately, work hours are hard to calculate. We developed an electronic model of residents' work-hours scenarios using Microsoft Excel 97. This model allows the input of multiple parameters (i.e., call frequency, call position, days off, short-call, weeks per rotation, outpatient weeks, clinic day of the week, additional time due to clinic) and start and stop times for post-call, non-call, short-call, and weekend days. For each resident on a rotation, the model graphically demonstrates call schedules, plots clinic days, and portrays all possible and preferred days off. We tested the model for accuracy in several scenarios. For example, the model predicted average work hours of 85.1 hours per week for fourth-night-call rotations. This was compared with logs of actual work hours of 84.6 hours per week. Model accuracy for this scenario was 99.4% (95% CI 96.2%-100%). The model prospectively predicted work hours of 89.9 hours/week in the cardiac intensive care unit (CCU). Subsequent surveys found mean CCU work hours of 88, 1 hours per week. Model accuracy for this scenario was 98% (95% CI 93.2-100%). Thus validated, we then used the model to test proposed scenarios for complying with RRC-IM limits. The flexibility of the model allowed demonstration of the full range of work-hours scenarios in every rotation of our 36-month program. Demonstrations of status-quo work-hours scenarios were presented to faculty as well as real-time demonstrations of the feasibility, or unfeasibility, of their proposed solutions. The model clearly demonstrated that non-call (i.e., short-call) admissions without concomitant decreases in overnight call frequency resulted in substantial increases in total work hours. Attempts to "get the resident out" an hour or two earlier each day had negligible effects on total hours and were unrealistic paper solutions. For fourth-night-call rotations, the addition of a "golden weekend" (i.e., a fifth day off per month) was found to significantly reduce work hours. The electronic model allowed the development of creative schedules for previously third-night-call rotations that limit resident work hours without decreasing continuity of care by scheduling overnight call every sixth night alternating with sixth-night-short-call rotations. DISCUSSION: Our electronic model is sufficiently robust to accurately estimate work hours on multiple and varied rotations. This model clearly demonstrates that it is very difficult to meet the RRC-IM work-hours limitations under standard fourth-night-call schedules with only four days off per month. We are successfully using our model to test proposed alternative scenarios, to overcome faculty misconceptions about resident work-hours "solutions," and to make changes to our call schedules that both are realistic for residents to accomplish and truly diminish total resident work hours toward the requirements of the RRC-IM.