Search

Background ChatGPT is a powerful artificial intelligence large language model with great potential as a tool in medical practice and education, but its performance in radiology remains unclear. Purpose To assess the performance of ChatGPT on radiology board-style examination questions without images and to explore its strengths and limitations. Materials and Methods In this exploratory prospective study performed from February 25 to March 3, 2023, 150 multiple-choice questions designed to match the style, content, and difficulty of the Canadian Royal College and American Board of Radiology examinations were grouped by question type (lower-order [recall, understanding] and higher-order [apply, analyze, synthesize] thinking) and topic (physics, clinical). The higher-order thinking questions were further subclassified by type (description of imaging findings, clinical management, application of concepts, calculation and classification, disease associations). ChatGPT performance was evaluated overall, by question type, and by topic. Confidence of language in responses was assessed. Univariable analysis was performed. Results ChatGPT answered 69% of questions correctly (104 of 150). The model performed better on questions requiring lower-order thinking (84%, 51 of 61) than on those requiring higher-order thinking (60%, 53 of 89) ( P = .002). When compared with lower-order questions, the model performed worse on questions involving description of imaging findings (61%, 28 of 46; P = .04), calculation and classification (25%, two of eight; P = .01), and application of concepts (30%, three of 10; P = .01). ChatGPT performed as well on higher-order clinical management questions (89%, 16 of 18) as on lower-order questions ( P = .88). It performed worse on physics questions (40%, six of 15) than on clinical questions (73%, 98 of 135) ( P = .02). ChatGPT used confident language consistently, even when incorrect (100%, 46 of 46). Conclusion Despite no radiology-specific pretraining, ChatGPT nearly passed a radiology board-style examination without images; it performed well on lower-order thinking questions and clinical management questions but struggled with higher-order thinking questions involving description of imaging findings, calculation and classification, and application of concepts. © RSNA, 2023 See also the editorial by Lourenco et al and the article by Bhayana et al in this issue.

Background: Our goal is to conduct a multicenter randomized controlled trial (RCT) to investigate whether exercise can reduce incident fractures compared with no intervention among women aged ≥65 years with a vertebral fracture., Objectives: This pilot study will determine the feasibility of recruitment, retention, and adherence for the proposed trial., Design: The proposed RCT will be a pilot feasibility study with 1:1 randomization to exercise or attentional control groups., Setting: Five Canadian sites (1 community hospital partnered with an academic center and 4 academic hospitals or centers affiliated with an academic center) and 2 Australian centers (1 academic hospital and 1 center for community primary care, geriatric, and rehabilitation services)., Participants: One hundred sixty women aged ≥65 years with vertebral fracture at 5 Canadian and 2 Australian centers will be recruited., Intervention: The Build Better Bones With Exercise (B3E) intervention includes exercise and behavioral counseling, delivered by a physical therapist in 6 home visits over 8 months, and monthly calls; participants are to exercise ≥3 times weekly. Controls will receive equal attention., Measurements: Primary outcomes will include recruitment, retention, and adherence. Adherence to exercise will be assessed via calendar diary. Secondary outcomes will include physical function (lower extremity strength, mobility, and balance), posture, and falls. Additional secondary outcomes will include quality of life, pain, fall self-efficacy, behavior change variables, intervention cost, fractures, and adverse events. Analyses of feasibility objectives will be descriptive or based on estimates with 95% confidence intervals, where feasibility will be assessed relative to a priori criteria. Differences in secondary outcomes will be evaluated in intention-to-treat analyses via independent Student t tests, chi-square tests, or logistic regression. The Bonferroni method will be used to adjust the level of significance for secondary outcomes so the overall alpha level is .05., Limitations: No assessment of bone mineral density will be conducted. The proposed definitive trial will require a large sample size., Conclusions: The viability of a large-scale exercise trial in women with vertebral fractures will be evaluated, as well as the effects of a home exercise program on important secondary outcomes., (© 2014 American Physical Therapy Association.)