Search

Background: The creation of high-quality multiple-choice questions (MCQs) is essential for medical education assessments but is resource-intensive and time-consuming when done by human experts. Large language models (LLMs) like ChatGPT-4o offer a promising alternative, but their efficacy remains unclear, particularly in high-stakes exams. Objective: This study aimed to evaluate the quality and psychometric properties of ChatGPT-4o-generated MCQs compared to human-created MCQs in a high-stakes medical licensing exam. Methods: A prospective cohort study was conducted among medical doctors preparing for the Primary Examination on Emergency Medicine (PEEM) organised by the Hong Kong College of Emergency Medicine in August 2024. Participants attempted two sets of 100 MCQs—one AI-generated and one human-generated. Expert reviewers assessed MCQs for factual correctness, relevance, difficulty, alignment with Bloom's taxonomy (remember, understand, apply and analyse), and item writing flaws. Psychometric analyses were performed, including difficulty and discrimination indices and KR-20 reliability. Candidate performance and time efficiency were also evaluated. Results: Among 24 participants, AI-generated MCQs were easier (mean difficulty index = 0.78 ± 0.22 vs. 0.69 ± 0.23, p < 0.01) but showed similar discrimination indices to human MCQs (mean = 0.22 ± 0.23 vs. 0.26 ± 0.26). Agreement was moderate (ICC = 0.62, p = 0.01, 95% CI: 0.12–0.84). Expert reviews identified more factual inaccuracies (6% vs. 4%), irrelevance (6% vs. 0%), and inappropriate difficulty levels (14% vs. 1%) in AI MCQs. AI questions primarily tested lower-order cognitive skills, while human MCQs better assessed higher-order skills (χ² = 14.27, p = 0.003). AI significantly reduced time spent on question generation (24.5 vs. 96 person-hours). Conclusion: ChatGPT-4o demonstrates the potential for efficiently generating MCQs but lacks the depth needed for complex assessments. Human review remains essential to ensure quality. Combining AI efficiency with expert oversight could optimise question creation for high-stakes exams, offering a scalable model for medical education that balances time efficiency and content quality. [ABSTRACT FROM AUTHOR]

Background Low back pain (LBP) is a leading cause of disability worldwide and has posed numerous health and socioeconomic challenges. This study compared whether nonsteroidal anti-inflammatory drugs (NSAIDs) in combination with tramadol, tizanidine or placebo would be the best treatment regime to improve the Roland Morris Disability Questionnaire (RMDQ) scores at 1 week. Methods This was a multi-center, double-blind, randomized, and placebo-controlled trial including adult patients with acute LBP and sciatica in three emergency departments in Hong Kong. Patients were randomized to the receive tramadol 50 mg, tizanidine 2 mg, or placebo every 6 hours for 2 weeks in a 1:1:1 ratio. The RMDQ and other secondary outcomes were measured at baseline, Day 2, 7, 14, 21, and 28. Data were analyzed on an intention to treat basis. Crude and adjusted mean differences in the changes of RMDQ and NRS scores from baseline to Day 7 between tizanidine/tramadol and placebo were determined with 95% confidence intervals. Results Two hundred and ninety-one patients were analyzed with the mean age of 47.4 years and 57.7% were male. The primary outcome of mean difference in RMDQs on Day 7 (compared with baseline) was non-significant for tizanidine compared with placebo (adjusted mean difference − 0.56, 95% CI -2.48 to 1.37) and tramadol compared with placebo (adjusted mean difference − 0.85, 95% CI -2.80 to 1.10). Only 23.7% were fully compliant to the treatment allocated. Complier Average Causal Effect analysis also showed no difference in the primary outcome for the tizanidine and tramadol versus placebo. Conclusion Among patients with acute LBP and sciatica presenting to the ED, adding tramadol or tizanidine to diclofenac did not improve functional recovery. What is already known on this topic There was no clear evidence of efficacy for cyclobenzaprine, methocarbamol, and metaxalone in addition to nonsteroidal anti-inflammatory drugs alone in patients with acute low back pain in the emergency department setting. What this study adds Among Chinese patients with acute low back pain and sciatica presenting to the emergency department, adding tramadol or tizanidine to diclofenac did not improve functional recovery at one week. The compliance to the analgesic drugs were low in this trial, with only 23.7% has taken at least 75% of the recommended doses, and 53.6% of patients has taken less than 25% of the recommended doses. Significant adverse events related to tramadol and tizanidine were found, with the most common being sleepiness and dizziness. How this study might affect research, practice or policy The lack of compliance would drive the effect estimate towards the null hypothesis (i.e. no difference), and further studies on relevant drugs should take into account of these low compliance rates. [ABSTRACT FROM AUTHOR]