Search

Background: The widespread diffusion of Artificial Intelligence (AI) platforms is revolutionizing how health-related information is disseminated, thereby highlighting the need for tools to evaluate the quality of such information. This study aimed to propose and validate the Quality Assessment of Medical Artificial Intelligence (QAMAI), a tool specifically designed to assess the quality of health information provided by AI platforms., Methods: The QAMAI tool has been developed by a panel of experts following guidelines for the development of new questionnaires. A total of 30 responses from ChatGPT4, addressing patient queries, theoretical questions, and clinical head and neck surgery scenarios were assessed by 27 reviewers from 25 academic centers worldwide. Construct validity, internal consistency, inter-rater and test-retest reliability were assessed to validate the tool., Results: The validation was conducted on the basis of 792 assessments for the 30 responses given by ChatGPT4. The results of the exploratory factor analysis revealed a unidimensional structure of the QAMAI with a single factor comprising all the items that explained 51.1% of the variance with factor loadings ranging from 0.449 to 0.856. Overall internal consistency was high (Cronbach's alpha = 0.837). The Interclass Correlation Coefficient was 0.983 (95% CI 0.973-0.991; F (29,542) = 68.3; p < 0.001), indicating excellent reliability. Test-retest reliability analysis revealed a moderate-to-strong correlation with a Pearson's coefficient of 0.876 (95% CI 0.859-0.891; p < 0.001)., Conclusions: The QAMAI tool demonstrated significant reliability and validity in assessing the quality of health information provided by AI platforms. Such a tool might become particularly important/useful for physicians as patients increasingly seek medical information on AI platforms., (© 2024. The Author(s).)

Objectives: To evaluate the reliability and validity of the Artificial Intelligence Performance Instrument (AIPI)., Methods: Medical records of patients consulting in otolaryngology were evaluated by physicians and ChatGPT for differential diagnosis, management, and treatment. The ChatGPT performance was rated twice using AIPI within a 7-day period to assess test-retest reliability. Internal consistency was evaluated using Cronbach's α. Internal validity was evaluated by comparing the AIPI scores of the clinical cases rated by ChatGPT and 2 blinded practitioners. Convergent validity was measured by comparing the AIPI score with a modified version of the Ottawa Clinical Assessment Tool (OCAT). Interrater reliability was assessed using Kendall's tau., Results: Forty-five patients completed the evaluations (28 females). The AIPI Cronbach's alpha analysis suggested an adequate internal consistency (α = 0.754). The test-retest reliability was moderate-to-strong for items and the total score of AIPI (r s  = 0.486, p = 0.001). The mean AIPI score of the senior otolaryngologist was significantly higher compared to the score of ChatGPT, supporting adequate internal validity (p = 0.001). Convergent validity reported a moderate and significant correlation between AIPI and modified OCAT (r s  = 0.319; p = 0.044). The interrater reliability reported significant positive concordance between both otolaryngologists for the patient feature, diagnostic, additional examination, and treatment subscores as well as for the AIPI total score., Conclusions: AIPI is a valid and reliable instrument in assessing the performance of ChatGPT in ear, nose and throat conditions. Future studies are needed to investigate the usefulness of AIPI in medicine and surgery, and to evaluate the psychometric properties in these fields., (© 2023. The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature.)

Introduction: Sialendoscopy has emerged in the last decades as a groundbreaking technique, offering a minimally invasive approach for exploring and managing salivary gland disorders. More recently, the advent of chatbots, powered by advanced natural processing language and artificial intelligence algorithms, has revolutionized the way healthcare professionals and patients access and analyze medical information and potentially will support soon the clinical decision-making process., Materials and Methods: A prospective, cross-sectional study was designed to assess the level of agreement between Chat-GPT and 10 expert sialendoscopists aiming the capabilities of Chat-GPT to further improve the management of salivary gland disorders., Results: The mean level of agreement was 3.4 (SD: 0.69; Min: 2, Max: 4) for Chat-GPT's answers while it was 4.1 (SD: 0.56; Min: 3, Max: 5) for the group of EESS (p < 0.015). The overall Wilcoxon signed-rank test yielded a significance level of p < 0.026 when comparing the level of agreement between Chat-GPT and EESS. The mean number of therapeutic alternatives suggested by Chat-GPT was 3.33 (SD: 1.2; Min: 2, Max: 5), while it was 2.6 (SD: 0.51; Min: 2, Max: 3) for the group of EESS; p = 0.286 (95% CI - 0.385 to 1.320)., Conclusion: Chat-GPT represents a promising tool in the clinical decision-making process within the salivary gland clinic, particularly for patients who are candidates for sialendoscopy treatment. Additionally, it serves as a valuable source of information for patients. However, further development is necessary to enhance the reliability of these tools and ensure their safety and optimal use in the clinical setting., (© 2023. The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature.)

Objective: To investigate the consistency of Chatbot Generative Pretrained Transformer (ChatGPT)‐4 in the analysis of clinical pictures of common laryngological conditions. Study Design: Prospective uncontrolled study. Setting: Multicenter study. Methods: Patient history and clinical videolaryngostroboscopic images were presented to ChatGPT‐4 for differential diagnoses, management, and treatment(s). ChatGPT‐4 responses were assessed by 3 blinded laryngologists with the artificial intelligence performance instrument (AIPI). The complexity of cases and the consistency between practitioners and ChatGPT‐4 for interpreting clinical images were evaluated with a 5‐point Likert Scale. The intraclass correlation coefficient (ICC) was used to measure the strength of interrater agreement. Results: Forty patients with a mean complexity score of 2.60 ± 1.15. were included. The mean consistency score for ChatGPT‐4 image interpretation was 2.46 ± 1.42. ChatGPT‐4 perfectly analyzed the clinical images in 6 cases (15%; 5/5), while the consistency between GPT‐4 and judges was high in 5 cases (12.5%; 4/5). Judges reported an ICC of 0.965 for the consistency score (P =.001). ChatGPT‐4 erroneously documented vocal fold irregularity (mass or lesion), glottic insufficiency, and vocal cord paralysis in 21 (52.5%), 2 (0.05%), and 5 (12.5%) cases, respectively. ChatGPT‐4 and practitioners indicated 153 and 63 additional examinations, respectively (P =.001). The ChatGPT‐4 primary diagnosis was correct in 20.0% to 25.0% of cases. The clinical image consistency score was significantly associated with the AIPI score (rs = 0.830; P =.001). Conclusion: The ChatGPT‐4 is more efficient in primary diagnosis, rather than in the image analysis, selecting the most adequate additional examinations and treatments. [ABSTRACT FROM AUTHOR]

Objective: To study the performance of Chatbot Generative Pretrained Transformer‐4 (ChatGPT‐4) in the management of cases in otolaryngology–head and neck surgery. Study Design: Prospective case series. Setting: Multicenter University Hospitals. Methods: History, clinical, physical, and additional examinations of adult outpatients consulting in otolaryngology departments of CHU Saint‐Pierre and Dour Medical Center were presented to ChatGPT‐4, which was interrogated for differential diagnoses, management, and treatment(s). According to specialty, the ChatGPT‐4 responses were assessed by 2 distinct, blinded board‐certified otolaryngologists with the Artificial Intelligence Performance Instrument. Results: One hundred cases were presented to ChatGPT‐4. ChaGPT‐4 indicated a mean of 3.34 (95% confidence interval [CI]: 3.09, 3.59) additional examinations per patient versus 2.10 (95% CI: 1.76, 2.34; P =.001) for the practitioners. There was strong consistency (k > 0.600) between otolaryngologists and ChatGPT‐4 for the indication of upper aerodigestive tract endoscopy, positron emission tomography and computed tomography, audiometry, tympanometry, and psychophysical evaluations. Primary diagnosis was correctly performed by ChatGPT‐4 in 38% to 86% of cases depending on subspecialty. Additional examinations indicated by ChatGPT‐4 were pertinent and necessary in 8% to 31% of cases, while the treatment regimen was pertinent in 12% to 44% of cases. The performance of ChatGPT‐4 was not influenced by the human‐reported level of difficulty of clinical cases. Conclusion: ChatGPT‐4 may be a promising adjunctive tool in otolaryngology, providing extensive documentation about additional examinations, primary and differential diagnoses, and treatments. The ChatGPT‐4 is more effective in providing a primary diagnosis, and less effective in the selection of additional examinations and treatments. [ABSTRACT FROM AUTHOR]

Introduction: Despite multiple prognostic indicators described for oral cavity squamous cell carcinoma (OCSCC), its management still continues to be a matter of debate. Machine learning is a subset of artificial intelligence that enables computers to learn from historical data, gather insights, and make predictions about new data using the model learned. Therefore, it can be a potential tool in the field of head and neck cancer. Methods: We conducted a systematic review. Results: A total of 81 manuscripts were revised, and 46 studies met the inclusion criteria. Of these, 38 were excluded for the following reasons: use of a classical statistical method (N = 16), nonspecific for OCSCC (N = 15), and not being related to OCSCC survival (N = 7). In total, 8 studies were included in the final analysis. Conclusions: ML has the potential to significantly advance research in the field of OCSCC. Advantages are related to the use and training of ML models because of their capability to continue training continuously when more data become available. Future ML research will allow us to improve and democratize the application of algorithms to improve the prediction of cancer prognosis and its management worldwide. [ABSTRACT FROM AUTHOR]

This document is a correction notice for an article titled "Exploring the potential of Chat-GPT as a supportive tool for sialendoscopy clinical decision making and patient information support" published in the European Archives of Oto-Rhino-Laryngology. The correction states that the ethics declaration statement was missing from the original article and should have included information about the author Jerome R. Lechien's involvement as a guest editor of a special issue on 'ChatGPT and Artificial Intelligence in Otolaryngology-Head and Neck Surgery'. The original article has been updated to include this information. The publisher, Springer Nature, remains neutral regarding jurisdictional claims and institutional affiliations. The authors of the article are Carlos M. Chiesa-Estomba, Jerome R. Lechien, Luigi A. Vaira, Aina Brunet, Giovanni Cammaroto, Miguel Mayo-Yanez, Alvaro Sanchez-Barrueco, and Carlos Saga-Gutierrez. [Extracted from the article]