Search

Background and Objectives: Interest surrounding generative large language models (LLMs) has rapidly grown. Although ChatGPT (GPT-3.5), a general LLM, has shown near-passing performance on medical student board examinations, the performance of ChatGPT or its successor GPT-4 on specialized examinations and the factors affecting accuracy remain unclear. This study aims to assess the performance of ChatGPT and GPT-4 on a 500-question mock neurosurgical written board examination., Methods: The Self-Assessment Neurosurgery Examinations (SANS) American Board of Neurological Surgery Self-Assessment Examination 1 was used to evaluate ChatGPT and GPT-4. Questions were in single best answer, multiple-choice format. χ 2 , Fisher exact, and univariable logistic regression tests were used to assess performance differences in relation to question characteristics., Results: ChatGPT (GPT-3.5) and GPT-4 achieved scores of 73.4% (95% CI: 69.3%-77.2%) and 83.4% (95% CI: 79.8%-86.5%), respectively, relative to the user average of 72.8% (95% CI: 68.6%-76.6%). Both LLMs exceeded last year's passing threshold of 69%. Although scores between ChatGPT and question bank users were equivalent ( P = .963), GPT-4 outperformed both (both P < .001). GPT-4 answered every question answered correctly by ChatGPT and 37.6% (50/133) of remaining incorrect questions correctly. Among 12 question categories, GPT-4 significantly outperformed users in each but performed comparably with ChatGPT in 3 (functional, other general, and spine) and outperformed both users and ChatGPT for tumor questions. Increased word count (odds ratio = 0.89 of answering a question correctly per +10 words) and higher-order problem-solving (odds ratio = 0.40, P = .009) were associated with lower accuracy for ChatGPT, but not for GPT-4 (both P > .005). Multimodal input was not available at the time of this study; hence, on questions with image content, ChatGPT and GPT-4 answered 49.5% and 56.8% of questions correctly based on contextual context clues alone., Conclusion: LLMs achieved passing scores on a mock 500-question neurosurgical written board examination, with GPT-4 significantly outperforming ChatGPT., (Copyright © Congress of Neurological Surgeons 2023. All rights reserved.)

Background and Objectives: General large language models (LLMs), such as ChatGPT (GPT-3.5), have demonstrated the capability to pass multiple-choice medical board examinations. However, comparative accuracy of different LLMs and LLM performance on assessments of predominantly higher-order management questions is poorly understood. We aimed to assess the performance of 3 LLMs (GPT-3.5, GPT-4, and Google Bard) on a question bank designed specifically for neurosurgery oral boards examination preparation., Methods: The 149-question Self-Assessment Neurosurgery Examination Indications Examination was used to query LLM accuracy. Questions were inputted in a single best answer, multiple-choice format. χ 2 , Fisher exact, and univariable logistic regression tests assessed differences in performance by question characteristics., Results: On a question bank with predominantly higher-order questions (85.2%), ChatGPT (GPT-3.5) and GPT-4 answered 62.4% (95% CI: 54.1%-70.1%) and 82.6% (95% CI: 75.2%-88.1%) of questions correctly, respectively. By contrast, Bard scored 44.2% (66/149, 95% CI: 36.2%-52.6%). GPT-3.5 and GPT-4 demonstrated significantly higher scores than Bard (both P < .01), and GPT-4 outperformed GPT-3.5 ( P = .023). Among 6 subspecialties, GPT-4 had significantly higher accuracy in the Spine category relative to GPT-3.5 and in 4 categories relative to Bard (all P < .01). Incorporation of higher-order problem solving was associated with lower question accuracy for GPT-3.5 (odds ratio [OR] = 0.80, P = .042) and Bard (OR = 0.76, P = .014), but not GPT-4 (OR = 0.86, P = .085). GPT-4's performance on imaging-related questions surpassed GPT-3.5's (68.6% vs 47.1%, P = .044) and was comparable with Bard's (68.6% vs 66.7%, P = 1.000). However, GPT-4 demonstrated significantly lower rates of "hallucination" on imaging-related questions than both GPT-3.5 (2.3% vs 57.1%, P < .001) and Bard (2.3% vs 27.3%, P = .002). Lack of question text description for questions predicted significantly higher odds of hallucination for GPT-3.5 (OR = 1.45, P = .012) and Bard (OR = 2.09, P < .001)., Conclusion: On a question bank of predominantly higher-order management case scenarios for neurosurgery oral boards preparation, GPT-4 achieved a score of 82.6%, outperforming ChatGPT and Google Bard., (Copyright © Congress of Neurological Surgeons 2023. All rights reserved.)

Background: Evidence regarding the consequence of efforts to increase patient throughput and decrease length of stay in the context of elective spine surgery is limited., Objective: To evaluate whether early time of discharge results in increased rates of hospital readmission or return to emergency department for patients admitted after elective, posterior, lumbar decompression surgery., Methods: We conducted a retrospective cohort study of 779 patients admitted to hospital after undergoing elective, posterior, lumbar decompression surgery. Multiple logistic regression evaluated the relationship between time of discharge and the primary outcome of return to acute care within 30 days, while controlling for sociodemographic, procedural, and discharge characteristics., Results: In multiple logistic regression, time of discharge earlier in the day was not associated with increased odds of return to acute care within 30 days (odds ratio [OR] 1.18, 95% CI 0.92-1.52, P = .19). Weekend discharge (OR 1.99, 95% CI 1.04-3.79, P = .04) increased the likelihood of return to acute care. Surgeon experience (<1 year of attending practice, OR 0.43, 95% CI 0.19-1.00, P = .05 and 2-5 years of attending practice, OR 0.50, 95% CI 0.25-1.01, P = .054), weekend discharge (OR 0.49, 95% CI 0.27-0.89, P = .02), and physical therapy evaluation (OR 0.20, 95% CI 0.12-0.33, P < .001) decreased the likelihood of discharge before noon., Conclusion: Time of discharge is not associated with risk of readmission or presentation to the emergency department after elective lumbar decompression. Weekend discharge is independently associated with increased risk of readmission and decreased likelihood of prenoon discharge., (Copyright © Congress of Neurological Surgeons 2022. All rights reserved.)

Background: Encouraging early time of discharge (TOD) for medical inpatients is commonplace and may potentially improve patient throughput. It is unclear, however, whether early TOD after elective spine surgery achieves this goal without a consequent increase in re-presentations to the hospital., Objective: To evaluate whether early TOD results in increased rates of hospital readmission or return to the emergency department after elective anterior cervical spine surgery., Methods: We analyzed 686 patients who underwent elective uncomplicated anterior cervical spine surgery at a single institution. Logistic regression was used to evaluate the relationship between sociodemographic, procedural, and discharge characteristics, and the outcomes of readmission or return to the emergency department and TOD., Results: In multiple logistic regression, TOD was not associated with increased risk of readmission or return to the emergency department within 30 days of surgery. Weekend discharge (odds ratio [OR] 0.33, 95% CI 0.21-0.53), physical therapy evaluation (OR 0.44, 95% CI 0.28-0.71), and occupational therapy evaluation (OR 0.32, 95% CI 0.17-0.63) were all significantly associated with decreased odds of discharge before noon. Disadvantaged status, as measured by area of deprivation index, was associated with increased odds of readmission or re-presentation (OR 1.86, 95% CI 0.95-3.66), although this result did not achieve statistical significance., Conclusion: There does not appear to be an association between readmission or return to the emergency department and early TOD after elective spine surgery. Overuse of inpatient physical and occupational therapy consultations may contribute to decreased patient throughput in surgical admissions., (Copyright © Congress of Neurological Surgeons 2022. All rights reserved.)

Background: Ventriculoperitoneal shunting is one of the most commonly performed neurosurgical procedures. Typically, for insertion of the peritoneal catheter, a mini-laparotomy technique is used. Although generally safe, it can be cosmetically undesirable and time consuming. Complications include malpositioning, bowel injury, and delayed hernias. Laparoscopic techniques have been advocated to address these issues, but have been slow to gain traction with neurosurgeons., Objective: To describe our experience with single port optical access laparoscopy for placement of ventriculoperitoneal shunts. Our technique simplifies adoption of a laparoscopic technique for neurosurgeons looking to incorporate its benefits., Methods: All ventriculoperitoneal shunts placed by the senior author since April 2011 were retrospectively reviewed. Surgical and perioperative complications, length of postoperative stay, and need for revisions were analyzed., Results: Fifty-six patients were included in the study. There were no cases of peritoneal catheter misplacement. One intraoperative complication occurred early in the series, in which there was an injury to the gallbladder necessitating cholecystectomy. There were 7 cases followed by shunt revision inclusive of the abdomen. In 3 cases, pseudocysts were noted., Conclusion: Single port optical access laparoscopy is a fast and minimally invasive technique that allows direct visualization of the layers of the abdominal wall as they are traversed and visualization of the peritoneal catheter during placement. It uses a small cosmetic incision and obviates the need for postoperative abdominal radiographic studies. The procedure has a modest learning curve, but can be safely used without the assistance of an assist surgeon after the skills are acquired.