1. Generating Social and Emotional Skill Items: Humans vs. ChatGPT. ACT Research. Issue Brief
- Author
-
ACT, Inc., Kate E. Walton, and Cristina Anguiano-Carrasco
- Abstract
Large language models (LLMs), such as ChatGPT, are becoming increasingly prominent. Their use is becoming more and more popular to assist with simple tasks, such as summarizing documents, translating languages, rephrasing sentences, or answering questions. Reports like McKinsey's (Chui, & Yee, 2023) estimate that by implementing LLMs, corporations could see a potential growth of $4.4 trillion annually in corporate benefits, while Nielsen (2023) estimates a 66% increase in employee productivity when using LLMs and other forms of generative artificial intelligence (AI). Can we use ChatGPT in the field of social and emotional learning assessment development to enhance our productivity? Some have examined how social and emotional (SE) skills are related to ChatGPT usage, such as cheating in the academic domain (Greitemeyer & Kastenmüller, 2023). In another study, researchers (de Winter et al., 2023) had ChatGPT generate a large number of personas and complete several SE skill measures. They then carried out several analyses such as a factor analysis and correlations with outcome measures and determined how similar the results were to previous research using human-completed SE skill measures. In the current study, rather than have ChatGPT complete SE skill measures, we sought to have ChatGPT create SE skill measures. Ultimately, we will compare a ChatGPT-generated assessment with a human-generated assessment in terms of reliability and validity.
- Published
- 2024