1. The Optimal Setting of A/B Exam Papers without Item Pools: A Hybrid Approach of IRT and BGP
- Author
-
Jia Ming Ying, Chi Kit Ho, Paul Juinn Bing Tan, Jin Hua Chen, and Zheng-Yun Zhuang
- Subjects
Computer science ,General Mathematics ,assessment ,education ,0211 other engineering and technologies ,02 engineering and technology ,Consistency (negotiation) ,Item response theory ,Computer Science (miscellaneous) ,Mathematics education ,ComputingMilieux_COMPUTERSANDEDUCATION ,question bank ,Engineering (miscellaneous) ,binary goal programing ,Item pool ,021103 operations research ,evaluation ,lcsh:Mathematics ,05 social sciences ,050301 education ,item response theory ,Subject (documents) ,Hybrid approach ,lcsh:QA1-939 ,Test (assessment) ,A/B exam papers setting ,data-driven decision-making ,Term test ,item pool ,0503 education - Abstract
The administration of A/B exams usually involves the use of items. Issues arise when the pre-establishment of a question bank is necessary and the inconsistency in the knowledge points to be tested (in the two exams) reduces the exams &lsquo, fairness&rsquo, These are critical for a large multi-teacher course wherein the teachers are changed such that the course and examination content are altered every few years. However, a fair test with randomly participating students should still be a guaranteed subject with no item pool. Through data-driven decision-making, this study collected data related to a term test for a compulsory general course for empirical assessments, pre-processed the data and used item response theory to statistically estimate the difficulty, discrimination and lower asymptotic for each item in the two exam papers. Binary goal programing was finally used to analyze and balance the fairness of A/B exams without an item pool. As a result, pairs of associated questions in the two exam papers were optimized in terms of their overall balance in three dimensions (as the goals) through the paired exchanges of items. These exam papers guarantee their consistency (in the tested knowledge points) and also ensure the fairness of the term test (a key psychological factor that motivates continued studies). Such an application is novel as the teacher(s) did not have a pre-set question bank and could formulate the fairest strategy for the A/B exam papers. The model can be employed to address similar teaching practice issues.
- Published
- 2020