Multistage Adaptive Testing for a Large-Scale Classification Test: Design, Heuristic Assembly, and Comparison with Other Testing Modes. ACT Research Report Series, 2012 (6)

Multistage adaptive tests (MSTs) have gained increasing popularity in recent years. MST is a balanced compromise between linear test forms (i.e., paper-and-pencil testing and computer-based testing) and traditional item-level computer-adaptive testing (CAT). It combines the advantages of both. On one hand, MST is adaptive (and therefore more efficient than linear tests). On the other hand, unlike CAT, it allows test developers to review test forms before administration, and it allows examinees to review and revise answers. Despite the advantages of MST, there is little literature on the details of heuristic automated assembly of MST and on the investigation of MST in the context of classification tests. In this study, we designed a MST for a large-scale classification test and performed the automated test assembly using a heuristic method. We then compared the performance of the MST with that of a linear test form and a CAT using computer simulation. The automated test assembly was successful. In comparing MST and CAT, we did observe a trade-off in measurement accuracy and item bank usage. For classification purposes, however, MST provided classification accuracy as good as that from CAT, with more efficient item bank usage. (Contains 5 tables, 5 figures and 2 footnotes.)