1. Validity assumptions for a multiple-choice test of medical knowledge with open-books and web access. A known groups comparison study.
- Author
-
Lotte Dyhrberg O'Neill, Eivind Ortind Simonsen, Ulla Breth Knudsen, Jesper Stentoft, Anders Bonde Jensen, Charlotte Green Carlsen, and Anne Mette Mørcke
- Subjects
Special aspects of education ,LC8-6691 - Abstract
Relatively little evidence about the validity threats in open-book multiple-choice tests exist. The aim of this study was to examine validity aspects relating to gener-alization, extrapolation and decision of a multiple-choice test of medical knowledge with aids (open-book and internet access). The theoretical framework was modern validity theory, and the study was designed as a ‘known groups com-parison’ study. Test performances of three known groups of test takers hypothe-sized to have different knowledge levels of the test content were compared, and analysis of pass/fail decisions was used to examine implications of decisions based on test scores. Results indicated that it was possible to discriminate between expert and non-expert test taker groups even with the access to aids. In contrast, an inde-fensible passing score was found to be the largest potential threat to test validity. Relatively little evidence about the validity threats in open-book multiple-choice tests exist. The aim of this study was to examine validity aspects relating to gener-alization, extrapolation and decision of a multiple-choice test of medical knowledge with aids (open-book and internet access). The theoretical framework was modern validity theory, and the study was designed as a ‘known groups com-parison’ study. Test performances of three known groups of test takers hypothe-sized to have different knowledge levels of the test content were compared, and analysis of pass/fail decisions was used to examine implications of decisions based on test scores. Results indicated that it was possible to discriminate between expert and non-expert test taker groups even with the access to aids. In contrast, an inde-fensible passing score was found to be the largest potential threat to test validity.
- Published
- 2018