Search

ContextSingle-best-answer questions (SBAQs) have been widely used to test knowledge because they are easy to mark and demonstrate high reliability. However, SBAQs have been criticised for being subject to cueing. ObjectivesWe used a novel assessment tool that facilitates efficient marking of open-ended very-short-answer questions (VSAQs). We compared VSAQs with SBAQs with regard to reliability, discrimination and student performance, and evaluated the acceptability of VSAQs. MethodsMedical students were randomised to sit a 60-question assessment administered in either VSAQ and then SBAQ format (Group 1, n = 155) or the reverse (Group 2, n = 144). The VSAQs were delivered on a tablet; responses were computer-marked and subsequently reviewed by two examiners. The standard error of measurement (SEM) across the ability spectrum was estimated using item response theory. ResultsThe review of machine-marked questions took an average of 1 minute, 36 seconds per question for all students. The VSAQs had high reliability (alpha: 0.91), a significantly lower SEM than the SBAQs (p < 0.001) and higher mean item-total point biserial correlations (p < 0.001). The VSAQ scores were significantly lower than the SBAQ scores (p < 0.001). The difference in scores between VSAQs and SBAQs was attenuated in Group 2. Although 80.4% of students found the VSAQs more difficult, 69.2% found them more authentic. ConclusionsThe VSAQ format demonstrated high reliability and discrimination and items were perceived as more authentic. The SBAQ format was associated with significant cueing. The present results suggest the VSAQ format has a higher degree of validity. The authors present a novel method of marking short-answer questions and demonstrate high reliability, discrimination and acceptability

Objectives Given the absence of a common passing standard for students at UK medical schools, this paper compares independently set standards for common ‘one from five’ single-best-answer (multiple-choice) items used in graduation-level applied knowledge examinations and explores potential reasons for any differences. Methods A repeated cross-sectional study was conducted. Participating schools were sent a common set of graduation-level items (55 in 2013–2014; 60 in 2014–2015). Items were selected against a blueprint and subjected to a quality review process. Each school employed its own standard-setting process for the common items. The primary outcome was the passing standard for the common items by each medical school set using the Angoff or Ebel methods. Results Of 31 invited medical schools, 22 participated in 2013–2014 (71%) and 30 (97%) in 2014–2015. Schools used a mean of 49 and 53 common items in 2013–2014 and 2014–2015, respectively, representing around one-third of the items in the examinations in which they were embedded. Data from 19 (61%) and 26 (84%) schools, respectively, met the inclusion criteria for comparison of standards. There were statistically significant differences in the passing standards set by schools in both years (effect sizes (f2): 0.041 in 2013–2014 and 0.218 in 2014–2015; both p < 0.001). The interquartile range of standards was 5.7 percentage points in 2013–2014 and 6.5 percentage points in 2014–2015. There was a positive correlation between the relative standards set by schools in the 2 years (Pearson's r = 0.57, n = 18, p = 0.014). Time allowed per item, method of standard setting and timing of examination in the curriculum did not have a statistically significant impact on standards. Conclusions Independently set standards for common single-best-answer items used in graduation-level examinations vary across UK medical schools. Further work to examine standard-setting processes in more detail is needed to help explain this variab

Background The deleterious effects of climate change mean that environmental sustainability is increasingly becoming a moral and economic necessity. Consequently, clinicians will increasingly be called upon to manage the effects of health care on climate change, and they must therefore do as much as is practically possible to limit the negative effects of their practice on the environment. As medical educators we have the opportunity not only to reduce the environmental impact of our own clinical practice, but also that of those who we teach, through innovation. Such novelty can be explored during student‐selected components (SSCs). Clinicians will increasingly be called upon to manage effects of health care on climate change Context The project, entitled ‘Can we introduce sustainability to clinical skills teaching?’ was led by two third‐year medical students during their SSC periods. New ways to make existing skills more sustainable were explored by surveying existing practice in the workplace, analysing selected skills in a lab‐based setting and through discussions with sustainability champions. Innovation Cannulation and intravenous (IV) antibiotic preparation were chosen by the students as prototype skills. These skills were observed by the students in the workplace and adapted by them to appease the ‘triple bottom line’ of sustainability: environmental, social and economic factors were addressed. The revised skills were taught by the students to their peers in a sustainably conscious fashion. Implications Provided that such innovations in sustainable skills teaching are deemed appropriate by clinical skills directors, such methods could be adopted across medical schools and expanded to cover a wider range of skills.