Criterion-Referenced Testing: A Critical Analysis of Selected Models. Technical Paper 306. Final Report

Alternative mathematical models for scoring and decision making with criterion referenced tests are described, especially as they concern appropriate test length and methods of establishing statistically valid cutting scores. Several of these approaches are reviewed and compared on formal-analytic and empirical grounds: (1) Block's approach to setting mastery standards, involving student performance and academic requirements; (2) Crehan's classification, comparing scores of students who have and who have not completed training; (3) the probabilistic models of Macready, Dayton, and Emrick, which assume an equal proportion of masters and nonmasters; (4) the binomial distribution model, which allows for partial acquisition of proficiency; (5) the Bayesian model, which considers prior experience; (6) Rasch's one-parameter logistic model, which yields person-free test calibrations and item-free person measurements; and (7) the regression approach of classical test theory, which enables the estimation of true scores to be made from observed scores. Examples of these approaches are given, as well as their advantages, disadvantages, and ambiguities. (GDC)