Search

We aimed to determine the quality of AI-generated (ChatGPT-4 and Claude 3) Script Concordance Test (SCT) items through an expert panel. We generated SCT items on abdominal radiology using a complex prompt in large language model (LLM) chatbots (ChatGPT-4 and Claude 3 (Sonnet) in April 2024) and evaluated the items’ quality through an expert panel of 16 radiologists. Expert panel, which was blind to the origin of the items provided without modifications, independently answered each item and assessed them using 12 quality indicators. Data analysis included descriptive statistics, bar charts to compare responses against accepted forms, and a heatmap to show performance in terms of the quality indicators. SCT items generated by chatbots assess clinical reasoning rather than only factual recall (ChatGPT: 92.50%, Claude: 85.00%). The heatmap indicated that the items were generally acceptable, with most responses favorable across quality indicators (ChatGPT: 71.77%, Claude: 64.23%). The comparison of the bar charts with acceptable and unacceptable forms revealed that 73.33% and 53.33% of the questions in the items can be considered acceptable, respectively, for ChatGPT and Claude. The use of LLMs to generate SCT items can be helpful for medical educators by reducing the required time and effort. Although the prompt provides a good starting point, it remains crucial to review and revise AI-generated SCT items before educational use. The prompt and the custom GPT, “Script Concordance Test Generator”, available at https://chatgpt.com/g/g-RlzW5xdc1-script-concordance-test-generator, can streamline SCT item development.

Abstract Background Anatomy is a crucial part of medical education, and there have been attempts to improve this field by utilizing various methods. With the advancement of technology, three-dimensional (3D) materials have gained popularity and become a matter of debate about their effectiveness compared to two-dimensional (2D) sources. This research aims to analyze the effectiveness of 3D PDFs compared to 2D atlases. Methods This study is a randomized controlled trial involving 87 Year-1 and Year-2 medical students at Gazi University Faculty of Medicine, Turkey. The study was conducted in two steps. In Step-1, students were randomized to watch lecture videos on liver anatomy and male genitalia anatomy supplemented with either a 3D PDF (intervention group) or 2D atlas (control group) images. Following the video lectures, a test (immediate test) was administered. In Step-2, the same test (delayed test) was administered 10 days after the immediate test. The test scores were compared between the intervention and control groups. In addition to the descriptive analyses, Chi-square and Mann-Whitney U tests were performed. Results In the immediate test, while there was no significant difference between the groups for the liver test (p > 0.05), 3D PDF group’s scores (Median = 24.50) was significantly higher than the 2D atlas group’s in the genitalia test (Median = 21.00), (p = 0.017). The effect size (Cohen’s d) was 0.57. In the delayed test, there was no significant difference between the groups in the liver and genitalia tests (p > 0.05). However, the effect size in the immediate genitalia test was 0.40. Year-1 students’ immediate test of genitalia performances were significantly higher in the 3D PDF group (Median = 24.00) than the 2D atlas group (Median = 19.00), (p = 0.016). The effect size was 0.76. Also, Year-1 students’ 3D PDF group (Median = 20.50) presented with significantly higher performance than the 2D atlas group (Median = 12.00), (p = 0.044) in the delayed test of genitalia, with the 0.63 effect size. Conclusion 3D PDF is more effective than 2D atlases in teaching anatomy, especially to initial learners. It is particularly useful for teaching complex anatomical structures, such as male genitalia, compared to the liver. Hence, it may be a valuable tool for medical teachers to utilize during lectures.

Medical education always evolves to incorporate more tools for specific needs in assessing clinical reasoning skills. Among these tools, Script Concordance Test (SCT) has a particular importance due to its focus on assessing decision-making in uncertain clinical situations. However, development of SCT items is effortful. Artificial intelligence tools, such as large language models, offer significant benefits. These models are already used for generating multiple-choice questions, and their use in generating SCTs offers great promise. However, this requires well-designed prompts to generate SCTs. This article proposes a generic prompt for the ChatGPT-4, Claude 3, Llama 3, and ChatGPT-4o large language model chatbots to generate SCTs, which can be tailored to various fields of medicine and different stages of medical education. It can help to streamline the development process of SCTs. Initial findings are promising, and there is a need for generating SCTs using large language models and conducting research to assess the quality of SCTs.

Leadership can be defined as a combination of position, responsibilities, attitude, skills, and behaviours contributing to sustainable success. Despite extensive research on the representation of various groups, including genders and personal qualities, in leadership and management positions in medical education, there remains a lack of clarity regarding what these positions exactly are. In this scoping review, we aimed to determine these positions by following Levac et al.'s framework. We searched four databases; PubMed, ERIC, Scopus, and Web of Science until March 23, 2023, resulting in 558 records for screening after removing duplicates and applying some exclusions. The review included 266 studies for data extraction and synthesis. In addition to the leadership positions, we revealed the studies’ countries, main context, discipline, and year. We identified 396 leadership and management positions in medical education under four categories: academic, administrative, journal- related, and society-related positions. Additionally, the majority of the studies (59.77%) were on gender-related topics, followed by those focusing on underrepresented minorities. The articles covered 40 different disciplines, with the surgical fields exhibiting the highest frequencies, such as orthopaedic surgery, neurosurgery, and plastic and reconstructive surgery. The systematically developed list of positions in our study may be considered as a valuable resource for future research, especially in studies that aim to determine the representation of women and underrepresented minorities in leadership and management positions in medical education.

Abstract Background IFMSA Social Accountability Assessment Tool has been developed for medical students by medical students to assess medical schools. However, its psychometric characteristics are unknown since it was developed without any analysis. We aimed to reveal its reliability and validity. Methods 1122 undergraduate medical students from various years in Gazi University Faculty of Medicine have participated in the study. They have answered the Turkish version of IFMSA Social Accountability Assessment Tool created through a translation process by experts. Exploratory and confirmatory factor analyses were carried out. Results Exploratory factor analysis showed that factor loadings were between 0.46 and 0.73 for Factor 1, 0.68 and 0.87 for Factor 2. The two-factor model, which consists of “Community Centeredness” and “Socio-Demographic Characteristics”, was evaluated through confirmatory factor analysis. The goodness-of-fit statistics of the model showed well-fit: CMIN/df 4.46, GFI 0.96, CFI 0.95, RMSEA 0.05, SRMR 0.03. Standardized regression weights were between 0.43 and 0.77. Conclusion The tool has acceptable psychometric characteristics, with good reliability and validity. It could be considered as a point of departure for the change in the way of being socially accountable since it enables medical students to explore the weak areas of their medical schools in terms of social accountability.

The significant challenge faced by medical schools is the effortful process of writing a high quantity of high-quality case-based multiple-choice questions (MCQs) to assess the higher-order skills of medical students. The demand for a high volume of MCQs in education has led to the development of Automatic Item Generation (AIG), specifically template-based AIG, which involves creating cognitive and item models by subject matter experts to generate hundreds of MCQs at once using software. It demonstrated significant success in various languages and even being incorporated into national medical licensure exams. However, this method still heavily depends on the efforts of subject matter experts. This paper introduces a detailed ChatGPT prompt for quickly generating case-based MCQs and provides important research questions for future exploration into ChatGPT's potential in generating items, signaling the beginning of the artificial intelligence era in medical education, encouraging health professions education researchers to delve deeper into its potential.

Abstract Background: The aim of this study was to find out specialty training preferences of senior medical students from three medical schools in Turkey, Spain, and Pakistan. Methods: A Discrete Choice Experiment was carried out using an electronic form for students in three countries in 2021–2022 term. Each choice set in the form consisted of two hypothetical specialty training positions. The attributes were location, earnings, working conditions, personal perspective, quality of education, probability of malpractice, and prestige. Conditional logit model was used to estimate participants’ preferences and “willingness to accept” values. Results: The most valued attribute was “personal perspective on specialty area” for Turkish and Spanish students, while this attribute was not meaningful for Pakistani students. Turkish students needed a 204% of change in their income for a swap between the specialty that they like and not like. This tradeoff necessitated a 300% change for Spanish students. The most valued attribute for Pakistani students, which was “working conditions”, necessitated a 97% increase in income to switch from working in good conditions to working in poor conditions. Conclusion: In this first multinational DCE study in the medical education literature, we found the preferences of medical students in Turkey, Spain, and Pakistan are affected to various extents by several factors.

As we are transitioning from the age of information to the age of artificial intelligence, this study aims to explore the potential of blockchain and Holochain technologies as solutions to the data privacy, security, and integrity concerns in medical education, and to provide recommendations for their use that also minimize negative impacts regarding planetary health and climate change. While these technologies offer promising solutions, their environmental impact has not been discussed in the literature with regards to medical education. This study offers two key recommendations for medical education researchers, educators, and policy makers to ensure the responsible use of Web3 technologies in this field. Recommendation 1: If the use of blockchain technology is necessary, avoid using proof-of-work blockchains. Recommendation 2: Use Holochain instead of blockchain.

The COVID-19 (Coronavirus Disease of 2019) pandemic, as the modern world’s gravest health crisis, has rapidly required more health facilities and health care providers (HCPs). The education system in general and medical education, in particular, has also changed dramatically with the school closures. Teaching and learning activities are undertaken remotely and on digital platforms. Academicians, HCPs, and medical students have gained new skills and experienced to manage their own learning process online. Many sources became easily reachable, open, and free. Open and democratized medicine, as we have seen during this pandemic, offers an interactive, egalitarian educational model. Governments, global organizations, and companies collaborated in order to keep all students learning and all instructors teaching. Many societies are faced with the digital divide and exclusion of learners. In this paper, we discussed the effect of COVID-19 on medical education at all levels and its’ democratization and shared our recommendations for sustaining these achievements after the pandemic.

This study introduces ContExtended Questions (CEQ), which is a tool both to teach and assess clinical reasoning particularly in the preclinical years, and the web-based program to implement. CEQ consists of text-based case-based multiple-choice questions that provide patient data in a fixed and predetermined sequence. It enables the examinees to develop and reshape their illness scripts by using feedback after every question. Feedback operates to transform the examinee’s failure into a “productive failure”. The preliminary results of the randomized controlled experiment of teaching clinical reasoning to preclinical students through CEQ is quite satisfactory. In the medical education literature, this would be the first time that students, who have no or very limited clinical experience, developed their illness scripts just by taking formative multiple-choice tests. The approach would be named “test-only learning”. The complete results of the experiment and then more experiments in other contexts and domains are necessary to establish a more powerful assessment tool and software. Furthermore, by changing the content of the questions, it is possible to use CEQ in every period of medical education and health professions education.