Reverse Engineering a Multiple-Choice Test Blueprint to Improve Course Alignment

Large introductory classes, with their expansive curriculum, demand assessment strategies that blend efficiency with reliability, prompting the consideration of multiple-choice (MC) tests as a viable option. Crafting a high-quality MC test, however, necessitates a meticulous process involving reflection on assessment format appropriateness, test blueprint design, and adherence to item-writing guidelines aligned with learning objectives and teaching strategies. This inherently time-consuming undertaking ideally requires a collaborative effort from a team of writers who possess expertise in both the subject domain and the specific course context -- an aspiration complicated by the multifaceted demands of higher education instruction. Given these challenges, educators often seek pragmatic solutions, including the adoption or adaptation of existing MC tests. However, the utility of these tests is ambiguous if the original test blueprint and the classification of questions are unknown. This paper introduces a structured four-step "reverse engineering" test blueprint process and proposes a systematic approach to identify test questions that align with the targeted learning objectives. One crucial step incorporates the Taxonomy Table (Anderson & Krathwohl, 2001) facilitating the classification of questions in the cognitive process dimension. As we delve into the intricacies of this analytical journey, we aim to provide a valuable resource for educators seeking to optimize the effectiveness and relevance of MC tests as a high-stakes assessment option.