1. Surgical Skills Feedback and myTIPreport
- Author
-
AnnaMarie Connolly, Richard Uribe, Kimberly Kenton, Elizabeth Buys, Alice R. Goepfert, Nicole Donnellan, and Anita K. Blanchard
- Subjects
Medical education ,030219 obstetrics & reproductive medicine ,business.industry ,education ,Obstetric Surgical Procedures ,MEDLINE ,Reproducibility of Results ,Obstetrics and Gynecology ,Construct validity ,Spearman's rank correlation coefficient ,Test (assessment) ,Dreyfus model of skill acquisition ,03 medical and health sciences ,Gynecologic Surgical Procedures ,0302 clinical medicine ,Obstetrics and gynaecology ,Rating scale ,Humans ,Medicine ,Female ,Educational Measurement ,Prospective Studies ,030212 general & internal medicine ,business ,Inclusion (education) - Abstract
Objective To initiate construct validity testing of myTIPreport for procedural skill assessment in a prospective multicenter evaluation study. Methods Teachers and learners from a convenience-based site selection of obstetrics and gynecology (OBGYN) and female pelvic medicine and reconstructive surgery (FPMRS) training programs performed procedural assessments in myTIPreport. The specifically defined 5-point Dreyfus rating scale describing ability levels from novice to expert was used. Defined as the degree to which a test or measure assesses what it was designed to measure, construct validity of myTIPreport was tested by comparing the medians of procedure-specific overall assessments, by both teachers and learners themselves, of senior learners with junior learners. To minimize type I error, comparisons were performed only when a threshold of 10 or greater feedback encounters per learner group was met. Correlation of teacher assessments and learner self-assessments was examined for myTIPreport. Results From November 2014 to May 2016, 12 OBGYN and 7 FPMRS training programs participated. There were 440 learners and 443 teachers. Feedback was recorded on 5,093 surgical procedures; 4,567 for OBGYN residents and 526 for FPMRS fellows. Each OBGYN procedure had two categories of teacher and learner assessments comparing postgraduate year (PGY)-4 with PGY-1 learner performance. This yielded 48 possible assessment comparisons for the included 24 OBGYN procedures. In all, 28 of these 48 (58%) met the threshold number of observations per learner group. In 28 of these 28 (100%) comparison categories, PGY-4s rated significantly higher than PGY-1s. Similarly, in 16 of 18 (89%) comparison categories meeting inclusion criteria, FPMRS PGY-7s rated significantly higher than FPMRS PGY-5s. Strong correlation was noted of teacher assessments and learner self-assessments in myTIPreport with a Spearman correlation coefficient of 0.89 (P Conclusion As noted for the majority of compared teacher assessments and learner self-assessments, myTIPreport appeared to detect differences between senior and junior learners. These data support the emerging construct validity of myTIPreport for procedural skills assessment.
- Published
- 2017
- Full Text
- View/download PDF