A Culture of Question Writing: Professional Examination Question Writers' Practices

Background: The development of a set of questions is a central element of examination development, with the validity of an examination resting to a large extent on the quality of the questions that it comprises. This paper reports on the methods and findings of a project that explores how educational examination question writers engage in the practice of question setting. Although the practice of examination question writing is ubiquitous, limited research has been carried out on how writers actually draft individual examination questions when writing complete examination papers. Purpose: The main aim of this study was to understand more deeply the process of examination question writing. Sample: The study involved 12 experienced educational examination question writers. Design and methods: The first data collection phase involved observing writers whilst they wrote the first draft of a question paper (using a 'think aloud' process). Following the observation, the writers took part in an hour-long interview. This led to the construction of a model of question writing. Phase 2 of the data collection process involved validating the proposed question writing model with six additional question writers by means of telephone interviews. Analysis and findings: A model was developed that outlined the key stages of writing that all of the writers moved through in the completion of their question writing task. The question writing model situates the process of writing individual questions in relation to the broader cognitive tasks required in writing a complete paper. It also situates the writing process within the social dimensions of question writing. Conclusions: Iteration in the writing process appears to have an important function. These iterations represent a form of decision-making that is common to the examination question writing culture and appears to be logical and rule bound. Whilst the recognition of iterative behaviour reveals a highly personalised element of question writing, it also reveals a common expectation that the ideas across a subject area are highly inter-related, and that an examination paper should reflect this.