Search

The increasing use of machine learning and Large Language Models (LLMs) opens up opportunities to use these artificially intelligent algorithms in novel ways. This article proposes a methodology using LLMs to support traditional deductive coding in qualitative research. We began our analysis with three different sample texts taken from existing interviews. Next, we created a codebook and inputted the sample text and codebook into an LLM. We asked the LLM to determine if the codes were present in a sample text provided and requested evidence to support the coding. The sample texts were inputted 160 times to record changes between iterations of the LLM response. Each iteration was analogous to a new coder deductively analyzing the text with the codebook information. In our results, we present the outputs for these recursive analyses, along with a comparison of the LLM coding to evaluations made by human coders using traditional coding methods. We argue that LLM analysis can aid qualitative researchers by deductively coding transcripts, providing a systematic and reliable platform for code identification, and offering a means of avoiding analysis misalignment. Implications of using LLM in research praxis are discussed, along with current limitations.

Abstract Digital educational games exhibit substantial promise in advancing STEM education. Nevertheless, the empirical evidence on both the efficacy of digital game-based learning and its designs in STEM education is characterized by notable inconsistencies. Therefore, the current study aimed to investigate (1) the general effect of digital game-based STEM learning over STEM learning without digital game, and (2) the enhancement effect of added game-design elements against base game versions in STEM learning. Two meta-analyses were conducted in this study. Based on the 136 effect sizes extracted from 86 studies, the first meta-analysis revealed a medium to large general effect of digital game-based STEM learning over conventional STEM learning (g = 0.624, 95% CI [0.457, 0.790]). In addition, digital game-based STEM learning appeared to be differentially effective for different learning outcome, different types of game, and different subject. A total of 44 primary studies and 81 effect sizes were identified in the second meta-analysis. The results revealed a small to medium enhancement effect of added game-design elements over base game versions (g = 0.301, 95% CI [0.163, 0.438]). Furthermore, our results indicated that the game-design elements added for content learning were more effective than those added for gaming experience. Possible explanations for these findings, as well as the limitations and directions for future research were discussed.

Abstract Background Capturing measures of students’ attitudes toward science has long been a focus within the field of science education. The resulting interest has led to the development of many instruments over the years. There is considerable disagreement about how attitudes should be measured, and especially whether students’ attitudes toward science can or should be measured unidimensionally, or whether separate attitude dimensions or subscales should be considered. When it is agreed upon that the attitudes toward science construct should be measured along separate subscales, there is no consensus about which subscales should be used. Methods A streamlined version of the modified Attitudes Towards Science Inventory (mATSI), a widely used science measurement instrument, was validated for a more diverse sample as compared to the original study (Weinburgh and Steele in Journal of Women and Minorities in Science and Engineering 6:87–94, 2000). The analytical approach used factor analyses and longitudinal measurement invariance. The study used a sample of 2016 self-reported responses from 6 and 7th grade students. The factor analysis elucidated the factor structure of students’ attitudes toward science, and some modifications were made in accordance with the results. Measurement invariance analysis was used to confirm the stability of the measure. Results Our results support that the subscales, anxiety toward science and value and enjoyment of science, are two factors and stable over time. Conclusions Our results suggest that our proposed modified factor structure for students’ attitudes toward science is reliable, valid, and appropriate for use in longitudinal studies. This study and its resulting streamlined mATSI survey could be of value to those interested in studying student engagement and measuring middle-school students' attitudes toward science.