Testing the value of expert insight: Comparing local versus general expert judgment models.

Expert judges often claim to utilize expert insight to tailor judgments to maximize predictive validity for a specific context. We evaluated multi‐organizational assessment data regarding the prediction of supervisory ratings of job performance from ratings on individual assessment dimensions, finding no evidence that the average expert assessor effectively tailored judgments to specific organizations to maximize prediction. Expert judgment was outperformed in all organizational contexts by linear models of expert judgment, optimal weighted regression models, as well as simple sum composites. Critically, the dimension weighting policies of the expert assessors were not consistent with optimal weights for predicting job performance at any organization. We discuss why expertise tends not to contribute to predictive validity and describe methods for improving overall judgmental accuracy. Practitioner points: Expert judges often claim to utilize expert insight to tailor judgments to maximize predictive validity for a specific context.Research has shown that mechanical methods of data combination (e.g., algorithms) consistently match or outperform the judgmental accuracy of subjective data combinations (e.g., subjective expert judgment).We evaluated expert judgment across three managerial hiring samples, and found no evidence that the average expert assessor effectively tailored judgments to specific organizations to maximize prediction.Expert judgment was outperformed by mechanical methods, and expert judges data combination policies were not consistent with optimal weights for predicting job performance at any organization.Reliance on subjective methods of judgment even when exercised by individuals with experience and expertise leads to predictions that are less accurate than those made using mechanical methods.Procedures should be implemented to minimize human error such as reducing the amount of information presented, providing decision aids, and integrating expert judgment with mechanical methods. [ABSTRACT FROM AUTHOR]