Search

ObjectiveWorkplace-based assessments (WBAs) are used in multiple surgical specialties to facilitate feedback to residents as a form of formative assessment. The validity evidence to support this purpose is limited and has yet to include investigations of how users interpret the assessment and make rating decisions (response processes). This study aimed to explore the validity evidence based on response processes for a WBA in surgery.DesignSemi-structured interviews explored the reasonings and strategies used when answering questions in a surgical WBA, the System for Improving and Measuring Procedural Learning (SIMPL). Interview questions probed the interpretation of the three assessment questions and their respective answer categories (level of autonomy, operative performance, case complexity). Researchers analyzed transcripts using directed qualitative content analysis to generate themes.SettingSingle tertiary academic medical center.ParticipantsEight residents and 13 faculty within the Department of Otolaryngology-Head and Neck Surgery participating in a 6-month pilot of SIMPL.ResultsWe identified four overarching themes that that characterized faculty and resident response processes while completing SIMPL: (1) Faculty and resident users had similar content-level interpretations of the questions and corresponding answer choices; (2) Users employed a variety of cognitive, behavioral, and emotional processes to make rating decisions; (3) Contextual factors influenced ratings; and (4) Tensions during interpretation contributed to rating uncertainty.ConclusionsResponse processes are a key source of evidence to support the validity for the formative use of WBAs. Evaluating response process evidence should go beyond basic content-level analysis as contextual factors and tensions that arise during interpretation also play a large role in rating decisions. Additional work and a continued critical lens are needed to ensure that WBAs can truly meet the needs for formative assessment.

Additional file 1.

This webinar, A Roadmap for Publishing Health Professions Education Scholarship, was delivered on March 9, 2020 by Drs. Lauren Maggio and Bridget O'Brien as one installment of a four-part series on fostering scholarship in medical education offered by the AAMC (aamc.org).Over the past decade, the number of venues for publishing health professions education (HPE) scholarship has grown. With this increase has come a variety of publication types ranging from one-page Last Pages to comprehensive Literature Reviews as well as a number of excellent resources to help scholars write up their work effectively for specific publication types. In this webinar, the speakers will introduce participants to the evolving landscape of publishing in HPE, provide examples of how to think through the available publishing opportunities, and highlight some resources authors can use to prepare their work for submission. By the end of this webinar, participants will be able to: Describe a variety of dissemination venues for publishing scholarship in HPE; Select appropriate publication types for their scholarship; and Identify resources available to guide their publication preparation.

ContextTransitions of patient care responsibility occur frequently between physicians. Resultant discontinuities make it difficult for physicians to observe clinical outcomes. Little is known about what physicians do to overcome the practical challenges to learning these discontinuities create. This study explored physicians' activities in practice as they sought follow-up information about patients.MethodsUsing a constructivist grounded theory approach, semi-structured interviews with 18 internal medicine hospitalist and resident physicians at a single tertiary care academic medical center explored participants' strategies when deliberately conducting follow-up after they transitioned responsibility for patients to other physicians. Following open coding, the authors used activity theory (AT) to explore interactions among the social, cultural and material influences related to follow-up.ResultsThe authors identified three themes related to follow-up: (i) keeping lists to track patients, (ii) learning to create tracking systems and (iii) conducting follow-up. Analysis of participants' follow-up processes as an activity system highlighted key tensions in the system and participants' work adaptations. Tension within functionality of electronic health records for keeping lists (tools) to find information about patients' outcomes (object) resulted in using paper lists as workarounds. Tension between paper lists (tools) and protecting patients' health information (rules) led to rule-breaking or abandoning activities of locating information. Finding time to conduct desired follow-up produced tension between this and other activity systems.ConclusionIn clinical environments characterised by discontinuity, lists of patients served as tools for guiding patient care follow-up. The authors offer four recommendations to address the tensions identified through AT: (i) optimise electronic health record tracking systems to eliminate the need for paper lists; (ii) support physicians' skill development in developing and maintaining tracking systems for follow-up; (iii) dedicate time in physicians' work schedules for conducting follow-up; and (iv) engage physicians and patients in determining guidelines for longitudinal tracking that optimise physicians' learning and respect patients' privacy.