Standardized Interview Scoring Methodology for Neurosurgical Residency Applicant Selection

Interviews are critical to the neurosurgery resident application process. The COVID-19 pandemic forced residency interview activities are conducted virtually. To maintain a degree of control during a period of uncertainty, our department implemented a standardized survey for interviewers to evaluate the noncognitive attributes and program compatibility of applicants. Our objective was to assess the reliability and biases associated with our standardized interviewer survey implemented in neurosurgical residency interviews. A 5-question interviewer survey to assess applicant interview performance and program compatibility was implemented during the 2020 to 2021 interview season. After the application cycle, survey metrics were retrospectively reviewed. Multiple cohort analyses were performed by dividing interviewers into cohorts based on status (faculty or resident) and sex. Applicant scores were assessed within sex subgroups for each aforementioned interviewer cohort. Intraclass correlation coefficients (ICCs) were calculated to assess survey reliability. Fifteen interviewers (8 faculty and 7 residents) and 35 applicants were included. Female applicants (17%) and interviewers (20%) comprised the minority. There were no differences between resident and faculty reviewer scores; however, female reviewers gave higher overall scores than male reviewers (P= .003). There was no difference in total scores between female and male applicants when evaluating all reviewers or subgroups of faculty, residents, females, or males. ICC analysis demonstrated good (ICC 0.75-0.90) or excellent (ICC > 0.90) reliability for all questions and overall score. The standardized interviewer survey was a feasible and reliable method for evaluating noncognitive attributes during neurosurgery residency interviews. There was no perceptible evidence of sex bias in our single-program experience.