Search

Context: A new student selection instrument has been designed to assess candidate suitability for a problem-based learning, small group curriculum., Objective: To evaluate the performance of the new teamwork selection instrument in terms of its discriminatory power, fairness, validity, reliability and acceptability among candidates., Sample: A sample of 69 volunteer candidates attending for interview formed 13 teams of 5 or 6 candidates each. Each candidate was assessed independently by 2 assessors. Candidate performance in the exercise was used for instrument evaluation purposes only., Results: The instrument demonstrated good item discrimination (item-total correlations r = between 0.75 and 0.83, P <0.01); the potential for good agreement between raters (63% agreement, weighted kappa = 0.38, P <0.01); strong internal consistency reliability (Cronbach's alpha = 0.93), and good acceptability among candidates. No sources of assessment bias were identified on the basis of candidates' age (univariate anova F = 0.43, P >0.05), gender (unrelated samples t-test F = 1.2, P >0.05) or socioeconomic background (univariate anova F = 0.85, P >0.05). There was no statistically significant relationship between the candidates' performance in the new exercise and their performance in the standardised formal interview (r = - 0.37, P >0.05); the instrument had limited predictive validity, and some of the measured attributes require conceptual clarification. Discussion Statistical and conceptual analysis highlights the scope for development in the teamwork exercise. The exercise appears to be well suited to assessing candidate suitability for a problem-based learning curriculum.

Background: Progress testing is a form of longitudinal examination which, in principle, samples at regular intervals from the complete domain of knowledge considered a requirement for medical students on completion of the undergraduate programme. Over the course of the programme students improve their scores on the test, enabling them, as well as staff, to monitor their progress., Aim: We aimed to review methods which have been used to assess the results of individual tests, and to make recommendations on best practice., Discussion: In assessing progress tests, there are a variety of choices that must be made. These include whether the test is norm- or criterion-referenced; whether marking is negative or "number-right"; whether the grades are reported on a continuous or a discontinuous scale, and whether the grades are weighted towards the most recent observations, or the entire set of grades is used to determine the final grade. Grade boundary setting in the context of progress tests is also considered, using a mathematical model to predict the consequences of different approaches. The relationships between boundary setting, progression and remediation rules are considered., Conclusions: We concluded that norm referencing is preferable to criterion referencing, negative marking preferable to number-right marking, a discontinuous scale preferable to a continuous scale and that grades should be weighted to favour the most recent outcomes, although there should still be a degree of persistence (earlier grades should not disappear all together). Grade boundaries should be established with regard to rules on remediation and progression.

Background: Anatomy learning is generally seen as essential to medicine, and exposure to cadavers is generally seen as essential to anatomy learning around the world. Few voices dissenting from these propositions can be identified., Aims: This paper aims to consider arguments relating to the use of cadavers in anatomy teaching, and to describe the rationale behind the decision of a new UK medical school not to use cadaveric material., Discussion: First, the background to use of cadavers in anatomy learning is explored, and some general educational principles are explored. Next, arguments for the use of human cadaveric material are summarised. Then, possible arguments against use of cadavers, including educational principles as well as costs, hazards and practicality, are considered. These are much less well explored in the existing literature. Next, the rationale behind the decision of a new UK medical school not to use cadaveric material is indicated, and the programme of anatomy teaching to be employed in the absence of the use of human remains is described. Curriculum design and development, and evaluation procedures, are briefly described. Issues surrounding pathology training by autopsy, and postgraduate training in surgical anatomy, are not addressed in this paper., Future Directions: Evidence relating to the effect on medical learning by students not exposed to cadavers is scant, and plainly opportunities will now arise through our programme to gather such evidence. We anticipate that this discussion paper will contribute to an ongoing debate, in which virtually all previous papers on this topic have concluded that use of cadavers is essential to medical learning.

Background/aims: The new Peninsula Medical School (PMS) admitted its first undergraduate cohort in September 2002. Development of a new school has given rise to opportunities for critical appraisal of best practice in selecting students for medical school and development of new ways forward., Methods: The authors developed a selection strategy and principles grounded in the objectives of the PMS programme and drawing extensively on best practice in assessment. Critical appraisal of custom and practice amongst other UK medical schools led to the rejection of hitherto commonly used instruments and procedures and the development of new ones. Common assumptions about ways to ensure quality were also challenged., Results: A process for development of an evidence-based selection strategy is described. In addition, selection instruments and criteria to be used in a new medical school are discussed. Quality performance policies informed by psychometric and qualitative assessment performance measures are recommended., Discussion and Conclusion: Critical appraisal of custom and practice and the development of new strategies, criteria and selection instruments go only part way to informing best practice. As with other UK medical schools grappling with these issues, the test of these selection processes lies in their product.

Context: The responsibility to determine just who is competent to practice medicine, and at what standard, is great. Whilst there is still a period available for potential remediation, examinations at the completion of year three of the four-year Graduate Entry Medical Programme (GEMP) at Flinders University of South Australia (FUSA) are high stakes and contain the majority of final summative assessment for the certification of student to doctor. Therefore, the medical school has recently examined its methods for certification, the clinical practice standards sought in its programme and how to determine these standards., Design: For all assessments a standard was documented and methods employed to set these standards using specific measures of performance. A modification of the Angoff method was applied to the written examination and the Rothman method, using two criteria, was used to determine competency in the objective structured clinical examination (OSCE). These methods were used for the first time in 1998. Both methods used trained 'experts' as standard setters and both methods used the notion of the 'borderline candidate' to determine the passing standard. This paper describes these two criterion-referenced standard-setting procedures as used in this school and related examination performance., Conclusions: Whilst the use of standard-setting procedures goes part way to defining and measuring competence, it is time consuming and requires significant examiner training and acceptance. Using 50% to determine who is and isn't competent is simpler but not transparent, fair nor defensible.