An Analysis of the External Validity and Applied Relevance of Three Rating Formats.

Three rating instruments (Behaviorally Anchored Rating Scales, Numerically Anchored Rating Scales, and Trait Ratings) were compared to an external criterion of police patrolman job performance. The external criterion was five vignettes of hypothetical patrolmen's job behavior based on an inclusive job analysis. Each of the five vignettes was rated by 28 first-level supervisors using each of the three instruments. Rater perceptions of each instrument's relevance for various organizational purposes were also gathered. Analysis of the three rating instruments centered on their respective criterion sensitivity. Results indicated that the three comparison instruments were approximately equal in terms of their resistance to the errors of leniency and central tendency. None of the three instruments was shown to be resistant to halo error. The NARS instrument was preferred for applied usage. Explanations for the results center on scale ambiguity, rater opportunities to observe behavior, and the nature of rater perceptions of performance. [ABSTRACT FROM AUTHOR]