73. ETIOLOGY OF UNABLE TO ASSESS ENTRUSTABLE PROFESSIONAL ACTIVITIES IN A NATIONAL STUDY

Background Assessment using entrustable professional activities (EPAs) is currently under study as a joint effort of the American Board of Pediatrics and program director community. However, limited evidence exists for the feasibility of EPA-based assessment in pediatrics. Aim Among pediatric residency programs that implemented EPA-based assessment over 3 academic years (2015-16, 2016-17, 2017-18), determine how often programs were “unable to assess” a resident on a given EPA. Methods 23 programs reported clinical competency committee (CCC) determined EPA supervision level assignments for all residents in their program for a subset of 5-6 of the 17 general pediatrics EPAs biannually (Fall and Spring), except interns who were only assessed at the end of the academic year (Spring). They indicated “unable to assess” whenever applicable. Results Across the 5 data collection cycles, there were 29,643 opportunities to assign an EPA supervision level to 1987 residents from all three post-graduate years. Of these, 4140 assignments (14%) were deemed “unable to assess.” Across all EPAs, this determination was made less often in mid-year reports than year-end reports (p=.005). Post-graduate year 3 (PGY3) residents and PGY2 residents had significantly lower frequencies of “unable to assess” assignments when compared with PGY1 residents (Figure). EPAs with the lowest rates of “unable to assess” included: EPA 4 (patients with acute, common dx) at 0.4%, EPAs 10 (initial resuscitation/stabilization) and 16 (handovers) at 0.9%, EPA 11 (manage information) at 4%, and EPA 13 (practice management) at 5%. EPAs with the highest rates of “unable to assess” included EPA 8 (transition to adult care) at 43%, EPA 17 (procedures) at 27%, EPA 14 (quality improvement) at 20%, and EPAs 12 (make referrals) and 9 (behavioral and mental health) at 14%. Conclusions “Unable to assess” assignments are related to the given EPA, training year, and timing of assessment within the academic year. CCCs may be less likely to assign EPA supervision levels at timepoints viewed as higher stakes, such as transition from one training year to the next. Future work should explore this further and consider the etiology behind high levels of inability to assess some EPAs in particular.