The Art of Test Construction: Can You Make a Good Physics Exam by Selecting Questions from a Bank?

In the traditional approach to constructing a GCSE or A Level examination paper, a single person writes the whole paper. In some other contexts, tests are constructed by selecting questions from a bank of questions. In this research, we asked experts to evaluate the quality of Physics exam papers constructed in the traditional way, constructed by expert selection of items from a bank, and constructed by computer selection of items from a bank. Anecdotal evidence suggested a "compilation" process would be detrimental to the quality of this kind of exam. We wanted to test whether in fact assessment experts could distinguish between tests that had been created in the traditional way, and those that had been compiled by selection from a bank, when they were unaware of the method of construction.