Search

Responds to Austin, Humphreys, and Hulin's (1989) critique of Barrett, Caldwell, and Alexander, suggesting that the burden of proof still rests on the advocates of the concept of dynamic criteria, and that empirical support is lacking for the existence of dynamic criteria as a simplex. Contrary evidence from educational, organizations, and experimental psychology is reviewed. (TE)

Evaluates a generalization of James' second-order approximation to the test for regression slope equality in psychological measurement. Adaptability of the approximation for testing; Equality of k independent means; Availability of the program.

Setting critical scores on content valid tests may involve making judgments about test content. A conclusion which has been reached in the testing literature is that the Angoff method is preferred as this type of judgmental method of setting standards or critical scores. The cur- rent paper reflects this conclusion, but also discusses several limitations of the Angoff method, and outlines why less than optimal data may be obtained using this preferred method or similar methods. Several techniques are reviewed and critiqued that may be used in conjunction with the Angoff method or similar methods to attempt improvement of the psychometric quality of the critical score data. Components from two psychometric frameworks (generalizability theory and Cronbach's, 1955, accuracy scores) are integrated and applied in assessing the im- pact that each of the techniques may have on Angoff critical score data. The techniques reviewed can be roughly divided into: (a) procedural methods (involved in the judgment process), and (b) psychometric methods (involved in data analysis following judgment). The present review is intended as a resource to personnel testing specialists who maybe interested in methods of bolstering the process of judgmentally deriving critical scores. [ABSTRACT FROM AUTHOR]

The present study examined several psychometric issues relevant to the use of a favored technique (the Angoff method) used to set standards in criterion referenced testing. The research was conducted within a setting which allowed (a) confident identification of expert and non-expert judges, and (b) estimation of ‘true’ scores for items judged so that accuracy of judgments in addition to reliability could be examined. Results suggested that expertise of judges does make a difference in producing more accurate and reliable data, underscoring the importance of using true subject matter experts (SMEs) in the judgment process. A rater analysis technique (rater-total correlations) was illustrated, which might prove useful in improving the quality of data obtained using the Angoff method, particularly when there is some question regarding the internal consistency of ratings and expertise of some of the raters. Finally, a rater accuracy adjustment/calibration technique was examined and proved to be a potentially useful method to maximize accuracy of a standard derived using the Angoff method in settings where archival normative test data can be obtained. Other methods that could potentially be used to improve Angoff data were discussed. [ABSTRACT FROM AUTHOR]

Cutoff scores are used in a variety of settings to help managers make personnel decisions. Although cutoff scores are used widely, information about their derivation and appropriate use is scattered in the legal, psychometric, and professional literatures. The purpose of this paper is to analyze critically and to integrate these diverse literatures, to summarize what is known and what is unknown about the use and misuse of cutoff scores. AlTernative methods are examined in relation to legal precedents set by the courts, to psychometric principles, and to the various sets of professional guidelines. Directions for future research are suggested, and we conclude with a set of recommendations regarding acceptable professional practice in this area. [ABSTRACT FROM AUTHOR]

In clinical research, test data are often encountered where groups differ in terms of total number of responses. When this occurs, conversion of component scores into percentages of R for each individual is recommended before multivariate clustering techniques are applied. An illustration, of increased prediction resulting from percentage conversions is presented and other advantages of this approach are discussed. [ABSTRACT FROM AUTHOR]