1. GPT Models Can Perform Thematic Analysis in Public Health Studies, Akin to Qualitative Researchers
- Author
-
Yuyi Yang, Charles Alba, Chenyu Wang, Xi Wang, Jami Anderson, and Ruopeng An
- Subjects
social computing applications in healthcare and public health ,ethnographic and qualitative methodologies ,machine learning ,data mining ,computational linguistics ,Electronic computers. Computer science ,QA75.5-76.95 ,Social sciences (General) ,H1-99 - Abstract
Conducting thematic analysis in qualitative research can be laborious and time-consuming. We propose and evaluate the feasibility of using Generative Pre-trained Transformer (GPT) models to assist public health researchers in extracting themes from interview transcripts. Carefully engineered prompts were used to sequentially extract and synthesize transcripts into a concise set of study-level themes relevant to the study’s goals. An evaluation using a 5-point Likert scale (0−4) assessed GPT-generated themes across 11 published studies based on four criteria: succinctness, alignment with researcher-identified themes, quality of explanations, and relevance of quotes. Across all four criteria, the scores averaged 3.05 (95% Confidence Interval (CI): [2.93, 3.16]). Our findings indicate that at least half of the GPT-generated themes align with those in published studies, exhibiting succinctness with minimal repetition, substantial depth of explanations, and relevant quotations. Despite these promising results, practices such as complementing outputs with field-specific knowledge are recommended.
- Published
- 2024
- Full Text
- View/download PDF