Search

Background: As assessment has been shown to direct learning, it is critical that the examinations developed to test clinical competence in medical undergraduates are valid and reliable. The use of extended matching questions (EMQ) has been advocated to overcome some of the criticisms of using multiple-choice questions to test factual and applied knowledge., Methods: We analysed the results from the Extended Matching Questions Examination taken by 4th year undergraduate medical students in the academic year 2001 to 2002. Rasch analysis was used to examine whether the set of questions used in the examination mapped on to a unidimensional scale, the degree of difficulty of questions within and between the various medical and surgical specialties and the pattern of responses within individual questions to assess the impact of the distractor options., Results: Analysis of a subset of items and of the full examination demonstrated internal construct validity and the absence of bias on the majority of questions. Three main patterns of response selection were identified., Conclusion: Modern psychometric methods based upon the work of Rasch provide a useful approach to the calibration and analysis of EMQ undergraduate medical assessments. The approach allows for a formal test of the unidimensionality of the questions and thus the validity of the summed score. Given the metric calibration which follows fit to the model, it also allows for the establishment of items banks to facilitate continuity and equity in exam standards.

This study examines the Likert-style successive integer scoring of Goldberg's (1972, 1978) General Health Questionnaire (GHQ) with a psychometric model in which the thresholds between successive categories within each item can be estimated. The model is particularly appropriate because the scoring of the successive categories, which are not named in the same way across items, by successive integers has received substantial discussion in the literature. Results from 1967 teachers in Western Australia who completed the 30-item form of the GHQ show that the items conform reasonably well to the model at a general or macro-level of analysis. In particular, the original ordering of categories is supported. However, as expected, there are systematic differences between distances among threshold within items and systematic differences among thresholds between items. The differences between positively and negatively orientated items confirm a suggestion in the literature that these two classes of items form sufficiently different scales so that they could be treated as separate, though reasonably correlated, scales.