1. Understanding validity criteria in technology-enhanced learning: A systematic literature review.
- Author
-
van Haastrecht, Max, Haas, Marcel, Brinkhuis, Matthieu, and Spruit, Marco
- Subjects
- *
ARTIFICIAL intelligence , *MOBILE learning , *DISTANCE education , *EVALUATION methodology , *SCHOOL environment - Abstract
Technological aids are ubiquitous in today's educational environments. Whereas much of the dust has settled in the debate on how to validate traditional educational solutions, in the area of technology-enhanced learning (TEL) many questions still remain. Technologies often abstract away student behaviour by condensing actions into numbers, meaning teachers have to assess student data rather than observing students directly. With the rapid adoption of artificial intelligence in education, it is timely to obtain a clear image of the landscape of validity criteria relevant to TEL. In this paper, we conduct a systematic review of research on TEL interventions, where we combine active learning for title and abstract screening with a backward snowballing phase. We extract information on the validity criteria used to evaluate TEL solutions, along with the methods employed to measure these criteria. By combining data on the research methods (qualitative versus quantitative) and knowledge source (theory versus practice) used to inform validity criteria, we ground our results epistemologically. We find that validity criteria tend to be assessed more positively when quantitative methods are used and that validation framework usage is both rare and fragmented. Yet, we also find that the prevalence of different validity criteria and the research methods used to assess them are relatively stable over time, implying that a strong foundation exists to design holistic validation frameworks with the potential to become commonplace in TEL research. • The first technology-enhanced learning (TEL) review with a pre-published protocol. • Usage of validation frameworks in TEL is rare and fragmented. • Criteria tend to be assessed more positively when quantitative methods are used. • The TEL validity criteria landscape has remained remarkably stable over time. • A strong foundation exists to design holistic validation frameworks for TEL. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF