Asking applicants what they would do versus what they did do: a meta-analytic comparison of situational and past behaviour employment interview questions

Criterion-related validities and inter-rater reliabilities for structured employment interview studies using situational questions (e.g. 'Assume that you were faced with the following situation ... what would you do?') were compared meta-analytically with studies using past behaviour questions (e.g. 'Can you think of a time when ... what did you do?'). Validities and reliabilities were further analysed in terms of whether descriptively-anchored rating scales were used to judge interviewees' answers, and validities for each question type were also assessed across three levels of job complexity. While both question formats yielded high validity estimates, studies using past behaviour questions, when used with discriptively anchored answer rating scales, yielded a substantially higher mean validity estimate than studies using the situational question format with descriptively-anchored answer rating scales (.63 versus .47). Question type (situational versus past behaviour) was found to moderate interview validity, after controlling for whether studies used answer rating scales. No support was found for the hypothesis that situational questions are less valid for predicting job performance in high-complexity jobs. Sample-weighted mean inter-rater reliabilities were similar for both situational and past behaviour questions, provided that descriptively-anchored rating scales were used (.79 and .77, respectively), although they were slightly lower (.73) for past behaviour question studies lacking such rating scales.
Considerable research has been conducted with the aim of identifying procedures which can improve the criterion-related validity of the employment interview. One of the most profound and consistent findings from [...]