Search

This paper discusses the roles of validity, cut score choice, and adverse impact on selection system utility using data from two concurrent validation studies. We contrast an assessment center and published aptitude test on several metrics, including validity, testing costs, adverse impact, and utility. The assessment center produced slightly lower validity than the aptitude test while costing roughly 10 times as much per candidate. In spite of these advantages for the aptitude test, the assessment center produced so much less adverse impact its operational utility would be higher given cut scores likely to be chosen in this organization. Potential concerns with applying net utility models to this type of situation are discussed in comparison to gross utility models. (Reprinted by permission of the publisher.)

Overt integrity tests used by psychologists appear to be very susceptible to coaching of the patient, but covert tests seem to be less affected by coaching. Faking was not an effective way to bias the results of either cover or over integrity tests. The implications for the debate over the possible prediction of counterproductive behavior are dsicussed.

John E. Exner has inadequately calculated the percentage of agreement for Rorschach in his defense of the Comprehensive System. The reliability figures should be computed for the total number of responses in a protocol and not for a single response. Contrary to his claim that only 4 scores were selected in a study pointing out the weaknesses of the system, 8 scores of validity were used. Finally, there is misunderstanding regarding the number of articles published.

The Comprehensive System for the Rorschach is unreliable, despite its wide acceptance, and its validity has yet to be established. Basic issues related to the validity and interrater reliability of the Comprehensive System needs to be resolved, and there are not published studies on the system to justify its widespread acceptance. Field studies of the administration, scoring and recording practices of psychologists are necessary to validate the Comprehensive System. Existing empirical evidence for the validity of indices that directly influence clinical decision making is negligible.

Four studies examined the dimensionality, reliability and construct validity of the Driving Appraisal Inventory (DAI), a self-report measure of driving habits and skills. In Study 1, analysis of 127 items led to the formation of four unidimensional, internally reliable scales: Carelessness, Drunken Driving, Vehicle Safety, and Self-Evaluation. In Study 2, all were found to have acceptable test-retest reliability. In Study 3, convergent validity support was obtained for the Carelessness and Vehicle Safety scales. In Study 4, concurrent criterion-related validity support was obtained for the Carelessness and Drunken Driving scales. Scores on the Self-Evaluation scale appear to be more a function of self-confidence and/or ego-involvement in driving skills and less a function of actual driving skills. Men score higher than women do on the Carelessness, Drunken Driving, and Self-Evaluation scales, and in Study 4, criterion-related validity of the Self-Evaluation scale was stronger among women than among men.

J. M. Wood, M. T. Nezworski and W. J. Stejskal's comment that the Comprehensive System for the Rorschach lacks interrater reliability is incorrect. The method of percentage of agreement approach provides empirical support for the system. Their criticism on validity of the measure is based only on 4 of the 54 variables that form the Structural Summary Rorschach data. Of the 48 articles published on Rorschach test, 34 are research reports and 24 of these involve data and findings from 36 unpublished studies. Thus, contrary to their belief, this system has a strong research base.