Search

Objectives Despite significant evidence supporting the use of three-option multiple-choice questions ( MCQs), these are rarely used in written examinations for health professions students. The purpose of this study was to examine the effects of reducing four- and five-option MCQs to three-option MCQs on response times, psychometric characteristics, and absolute standard setting judgements in a pharmacology examination administered to health professions students. Methods We administered two versions of a computerised examination containing 98 MCQs to 38 Year 2 medical students and 39 Year 3 pharmacy students. Four- and five-option MCQs were converted into three-option MCQs to create two versions of the examination. Differences in response time, item difficulty and discrimination, and reliability were evaluated. Medical and pharmacy faculty judges provided three-level Angoff ( TLA) ratings for all MCQs for both versions of the examination to allow the assessment of differences in cut scores. Results Students answered three-option MCQs an average of 5 seconds faster than they answered four- and five-option MCQs (36 seconds versus 41 seconds; p = 0.008). There were no significant differences in item difficulty and discrimination, or test reliability. Overall, the cut scores generated for three-option MCQs using the TLA ratings were 8 percentage points higher (p = 0.04). Conclusions The use of three-option MCQs in a health professions examination resulted in a time saving equivalent to the completion of 16% more MCQs per 1-hour testing period, which may increase content validity and test score reliability, and minimise construct under-representation. The higher cut scores may result in higher failure rates if an absolute standard setting method, such as the TLA method, is used. The results from this study provide a cautious indication to health professions educators that using three-option MCQs does not threaten validity and may strengthen it by allowing additional MCQs to be tested in a fixed amount of testing time with no deleterious effect on the reliability of the test scores. [ABSTRACT FROM AUTHOR]

Medical Education 2011: : 87-94 Although firmly grounded in Flexner's legacy of ideas, today's medical curriculum, as both an entity and a process, has become more and more complex. The curriculum as an entity is portrayed according to five key elements: the expected competencies and roles; the learners at the centre of the enterprise; assessment linking competencies and learners; the conditions and resources for learning; and a multifaceted socio-politico-cultural context in which the learning occurs. Significant developments have also occurred in the disciplines of curriculum studies, cognitive psychology and organisational change over the past century, as well as in institutional best practices, that help us to better understand and plan curricular innovations. Practical advice is offered to help curriculum developers in designing or reforming the medical curriculum. The key points of this are: (i) while focusing reform and innovation on specific elements of the curriculum, consider how those elements affect other elements and vice versa, in positive and negative ways; (ii) while grounding the reform or innovation in sound conceptual frameworks, seize any opportunities to formulate a research agenda that can build upon and advance our understanding of curricular innovations, and, (iii) moving beyond considering the curriculum as an entity, use deliberative and leadership processes that can lead to enduring curriculum reform. [ABSTRACT FROM AUTHOR]

Context Authors have questioned the degree to which medical education research informs practice and advances the science of medical education. Objective This study aims to propose a framework for classifying the purposes of education research and to quantify the frequencies of purposes among medical education experiments. Methods We looked at articles published in 2003 and 2004 in Academic Medicine, Advances in Health Sciences Education, American Journal of Surgery, Journal of General Internal Medicine, Medical Education and Teaching and Learning in Medicine (1459 articles). From the 185 articles describing education experiments, a random sample of 110 was selected. The purpose of each study was classified as description (‘What was done?’), justification (‘Did it work?’) or clarification (‘Why or how did it work?’). Educational topics were identified inductively and each study was classified accordingly. Results Of the 105 articles suitable for review, 75 (72%) were justification studies, 17 (16%) were description studies, and 13 (12%) were clarification studies. Experimental studies of assessment methods (5/6, 83%) and interventions aimed at knowledge and attitudes (5/28, 18%) were more likely to be clarification studies than were studies addressing other educational topics (< 8%). Conclusions Clarification studies are uncommon in experimental studies in medical education. Studies with this purpose (i.e. studies asking: ‘How and why does it work?’) are needed to deepen our understanding and advance the art and science of medical education. We hope that this framework stimulates education scholars to reflect on the purpose of their inquiry and the research questions they ask, and to strive to ask more clarification questions. [ABSTRACT FROM AUTHOR]

Context Informative titles and abstracts facilitate reading and searching the literature. Objective To evaluate the quality of titles and abstracts of full-length reports of experimental studies in medical education. Methods We used a random sample of 110 articles (of 185 eligible articles) describing education experiments. Articles were published in 2003 and 2004 in Academic Medicine, Advances in Health Sciences Education, American Journal of Surgery, Journal of General Internal Medicine, Medical Education and Teaching and Learning in Medicine. Titles were categorised as informative, indicative, neither, or both. Abstracts were evaluated for the presence of a rationale, objective, descriptions of study design, setting, participants, study intervention and comparison group, main outcomes, results and conclusions. Results Of the 105 articles suitable for review, 86 (82%) had an indicative title and 10 (10%) had a title that was both indicative and informative. A rationale was present in 66 abstracts (63%), objectives were present in 84 (80%), descriptions of study design in 20 (19%), setting in 29 (28%), and number and stage of training of participants in 42 (40%). The study intervention was defined in 55 (52%) abstracts. Among the 48 studies with a control or comparison group, this group was defined in 21 abstracts (44%). Study outcomes were defined in 64 abstracts (61%). Data were presented in 48 (46%) abstracts. Conclusions were presented in 97 abstracts (92%). Conclusions Reports of experimental studies in medical education frequently lack the essential elements of informative titles and abstracts. More informative reporting is needed. [ABSTRACT FROM AUTHOR]

Objective Determine the prevalence of essential elements of reporting in experimental studies in medical education. Design Systematic review. Data Sources Articles published in 2003 and 2004 in Academic Medicine, Advances in Health Sciences Education, American Journal of Surgery, Journal of General Internal Medicine, Medical Education, and Teaching and Learning in Medicine. Review Methods Articles describing education experiments, including evaluation studies with experimental designs, were identified ( n = 185) by reviewing titles and abstracts. A random sample ( n = 110) was selected for full review. The full text of each article was evaluated for the presence of guideline-based features of quality reporting: a critical literature review, conceptual framework, statement of study intent (e.g. aim, research question, or hypothesis), statement of study design, definition of main intervention and comparison intervention or control group, and consideration of human subject rights. Results Of the 105 articles suitable for review, 47 (45%) contained a critical literature review and 58 (55%) presented a conceptual framework. A statement of study intent was present in 80 articles (76%), among which the independent and dependent variables were operationally defined in 38 (47%) and 26 articles (32%), respectively. A total of 17 articles (16%) contained an explicit study design statement. Among the 48 studies with a comparison group, 35 (73%) clearly defined the comparison intervention or control group. Institutional review board approval or participant consent was reported in 44 articles (42%). Conclusions The quality of reporting of experimental studies in medical education was generally poor. Criteria are proposed as a starting point for establishing reporting standards for medical education research. [ABSTRACT FROM AUTHOR]

A letter to the editor is presented by the authors of the article "Threats to validity in the use and interpretation of script concordance test scores"in response to the article "Scoring the scrip concordance test: not a black and white issue" by S. Lunbarsky, R. Gagnon, and B. Charlin from a 2013 issue.