Search

Item banking involves tests being constructed by selecting from a bank of pre-written questions. There are various examples of multiple-choice tests where item banking is used, but few examples inv...

This paper analyses the psychometric paradigm for understanding educational measurement and standard setting by considering the extent to which that understanding is based on metaphors. I argue tha...

Comparative Judgement (CJ) is an increasingly widely investigated method in assessment for creating a scale, for example of the quality of essays. One area that has attracted attention in C...

The five chapters that make up the second section of the Handbook are grouped under the heading ‘Content’. At first sight, the fifth of these (Chapter 11) about ‘Setting performance standards on te...

For many years, question choice has been used in some UK public examinations, with students free to choose which questions they answer from a selection (within certain parameters). There has been little published research on choice of exam questions in recent years in the UK. In this article we distinguish different scenarios in which choice arises, explore the arguments for and against using optional questions, and exploit the item level data that has recently become available from on-screen marking of examinations to exemplify methods for investigating the (statistical) comparability of optional questions. We conclude that unless there is a very good reason for allowing question choice it should be avoided.

This research investigated the accuracy (agreement with the original marking and grading) of examiners’ holistic judgements of the quality of examination scripts that were close together in overall mark. For a History and a Physics exam, examiners considered pairs of scripts (with marks removed) and made three types of judgement: (1) Absolute – which grade each script was worth; (2) Relative – which of the pair was better in terms of overall quality; (3) Confidence – how confident they were about judgements (1) and (2). In both subjects, relative judgements were more accurate than absolute judgements, and judgements rated as ‘very confident’ were more accurate than other judgements. In Physics, the further apart the two scripts in terms of overall mark the greater was the likelihood of a correct relative judgement, but in History this expected pattern was not found. Despite differences between the research setting and the use of expert judgement in grading the live examinations, these results suggest that...

In this paper we develop an evidence-based framework for considering many of the factors affecting marker agreement in GCSEs and A levels. A logical analysis of the demands of the marking task sugg...