Search

Since the onset of the pandemic in 2020, many credentialing organizations have incorporated online remote administration of their examinations to enable continuity of their programs. This paper describes a research study examining several high stakes credentialing examination programs that utilized mixed delivery modes, including online remote testing at home, as well as testing in test centers. Candidates were monitored in real time by a test proctor, either remotely by video camera, or in person. The study examined the comparability of test scores, instances of irregular candidate testing behavior (potential cheating), and candidate test taking experience ratings across modalities. Overall, results of the study indicated that test scores were psychometrically sound and comparable across modes; rates of suspect test taking behavior were low and not significantly different across modes; and candidate experience ratings were favorable and unrelated to testing modality. Implications for future practice and research are discussed. [ABSTRACT FROM AUTHOR]

Preventing cheating on tests is crucial, but in the event of failure to prevent it, detecting its presence and impact on scores is equally important. Many statistical indices have been developed for this purpose in nearly 100 years of research, often in a quest for the "best" index. The premise of this study is that: (a) Cheating is manifested in different patterns of aberrant responding, (b) No single index can effectively detect all such patterns, and thus (c) A suite of different indices working together should be more effective at identifying anomalous responding indicative of potential cheating. Two methods for simultaneous consideration of a suite of 7 indices were investigated for detecting three patterns of cheating that were manipulated in random subsamples of actual data from two high stakes credentialing tests. Discriminant function analysis distinguished presence and impact of cheating through different linear combinations of the indices. An alternative approach based on testing the fit of individuals' profiles across the 7 indices to heuristic model profiles for each cheating pattern led to similarly accurate classifications of test-taker's into their manipulated cheating groups. This profile similarity method provides a means for theory development and model testing, in aggregating any number of individual statistical indices for detecting fit to any modeled pattern of interest. [ABSTRACT FROM AUTHOR]