Search

Background: Many learners struggle to productively self-regulate their learning. To support the learners' self-regulated learning (SRL) and boost their achievement, it is essential to understand the cognitive and metacognitive processes that underlie SRL. To measure these processes, contemporary SRL researchers have largely utilized think aloud or trace data, however, not without challenges. Objectives: In this paper, we present the findings of a study that investigated how concurrent analysis and integration of think aloud and trace data could advance the measurement of SRL and assist in better understanding the mechanisms of SRL processes, especially those details that remain obscured by observing each data channel individually. Methods: We concurrently collected think aloud and trace data generated by 44 university students in a laboratory setting and analysed those data relative to the same timeline. Results: We found that the two data channels could be interchangeably used to measure SRL processes for only 17.18% of all the time segments identified in a learning task. Moreover, SRL processes for around 45% of all the time segments could be detected via either trace data or think aloud data. For another 27.17% of all the time segments, different SRL processes were detected in both data channels. Conclusions: Our results largely suggest that the two data collection methods can be used to complement each other in measuring SRL. In particular, we found that think aloud and trace data could provide different perspectives on SRL. The integration of the two methods further allowed us to reveal a more complex and more comprehensive temporal associations among SRL processes compared to using a single data collection method. In future research, the integrated measurement of SRL can be used to improve the detection of SRL processes and provide a fuller picture of SRL.

Background: Assignments that involve writing based on several texts are challenging to many learners. Formative feedback supporting learners in these tasks should be informed by the characteristics of evolving written product and by the characteristics of learning processes learners enacted while developing the product. However, formative feedback in writing tasks based on multiple texts has almost exclusively focused on essay product and rarely included SRL processes. Objectives: We explored the viability of using product and process features to develop machine learning classifiers that identify low- and high-performing essays in a multi-text writing task. Methods: We examined learning processes and essay submissions of 163 graduate students working on an authentic multi-text writing assignment. We utilised learners' trace data to obtain process features and state-of-the-art natural language processing methods to obtain product features for our classifiers. Results and Conclusions: Of four popular classifiers examined in this study, Random Forest achieved the best performance (accuracy = 0.80 and recall = 0.77). The analysis of important features identified in the Random Forest classification model revealed one product (coverage of reading topics) and three process (elaboration/organisation, re-reading and planning) features as important predictors of writing quality. Major Takeaways: The classifier can be used as a part of a future automated writing evaluation system that will support at scale formative assessment in writing tasks based on multiple texts in different courses. Based on important predictors of essay performance, a guidance can be tailored to learners at the outset of a multi-text writing task to help them do well in the task.

Self-regulation is an essential skill for lifelong learning. Research has shown that self-regulated learning (SRL) leads to greater academic achievement and sustainable education, but students often struggle with SRL. Scaffolds are widely reported as an effective and efficient support method for SRL. To further improve digital scaffolds' effectiveness, real-time detection of learning behavior can be used to personalize scaffolds. Therefore, the present study aimed to inform the field of scaffolding SRL by reporting on the design and evaluation of digital scaffolds. We present decisions made during the design process of personalized scaffolds to inform future scaffold designs. We evaluated how scaffolds were personalized based on real-time detection SRL, how university students respond to the scaffolds (i.e., compliance), and how this response is related to learning outcomes (i.e., quality of an essay). The research design was a pre-posttest with three conditions (no, generalized (same for all), or personalized scaffolds). A 45-minute reading and writing task was used, during which SRL processes were recorded in real-time. Findings revealed that different real-time SRL processes could be used to personalize scaffolds, meaning that we were able to personalize the content of scaffold based on students' actual learning behavior. In addition, students in the personalized condition complied more with the scaffolds than students in the other conditions. This compliance with the scaffolds was generally associated with better learning outcomes. To conclude, our approach showed how design decisions could be evaluated and provided insight into the personalization of scaffolds.

Contemporary research that looks at self-regulated learning (SRL) as processes of learning events derived from trace data has attracted increasing interest over the past decade. However, limited research has been conducted that looks into the validity of trace-based measurement protocols. In order to fill this gap in the literature, we propose a novel validation approach that combines "theory-driven" and "data-driven" perspectives to increase the validity of interpretations of SRL processes extracted from trace-data. The main contribution of this approach consists of three alignments between trace data and think aloud data to improve measurement validity. In addition, we define the match rate between SRL processes extracted from trace data and think aloud as a quantitative indicator together with other three indicators (sensitivity, specificity and trace coverage), to evaluate the "degree" of validity. We tested this validation approach in a laboratory study that involved 44 learners who learned individually about the topic of artificial intelligence in education with the use of a technology-enhanced learning environment for 45 minutes. Following this new validation approach, we achieved an improved match rate between SRL processes extracted from trace-data and think aloud data (training set: 54.24%; testing set: 55.09%) compared to the match rate before applying the validation approach (training set: 38.97%; test set: 34.54%). By considering think aloud data as "reference point", this improvement of the match rate quantified the extent to which validity can be improved by using our validation approach. In conclusion, the novel validation approach presented in this study used both empirical evidence from think aloud data and rationale from our theoretical framework of SRL, which now, allows testing and improvement of the validity of trace-based SRL measurements.

Self-regulated learning (SRL) has been linked to improved learning and corresponding learning outcomes. However, there is a need for more precise insights into how SRL during learning contributes to specific learning outcomes. We operationalised four learning outcomes that varied on two dimensions: structure/connectedness and level/deepness of knowledge. Specifically, we assessed how surface knowledge measured with a domain knowledge test (independent concepts) and a concept map (connected concepts), and deep knowledge measured with a transfer test (independent concepts) and an essay (connected concepts) were associated with frequencies of SRL activities during learning, assessed by concurrent think aloud, while taking into account students' metacognitive and prior knowledge. Forty-four university students performed a 45-minute problem-solving task integrating information about three topics to write a vision essay on the future of education. Results of the pre-/post-test analysis revealed a learning gain in domain knowledge and concept maps. Low cognitive activities were associated with all knowledge measures, except the concept maps and transfer. Furthermore, specific low cognitive activities showed either a positive or negative association; for example, processing showed a positive association with essay, but a negative association with domain knowledge. High cognitive activities were associated with the essay (connected concepts), but not with the concept map. Both metacognitive activities and knowledge were related to transfer. To conclude, taking the level and structure of knowledge into account helps specify the association between SRL activities during learning and the related learning outcomes.

In recent years, unobtrusive measures of self-regulated learning (SRL) processes based on log data recorded by digital learning environments have attracted increasing attention. However, researchers have also recognised that simple navigational log data or time spent on pages are often not fine-grained enough to study complex SRL processes. Recent advances in data-capturing technologies enabled researchers to go beyond simple navigational logs to measure SRL processes with multi-channel data. What multi-channel data can reveal about SRL processes, and to what extent can the addition of peripheral and eye-tracking data with navigational log data change and improve the measurement of SRL are key questions that require further investigation. Hence, we conducted a study and collected learning trace data generated by 25 university students in a laboratory setting, that aimed to address this problem by enhancing navigational log data with peripheral and eye-tracking data. We developed a trace-based measurement protocol of SRL, which interpreted raw trace data from multi-channel data into SRL processes. Specifically, the study compared the frequency and duration of SRL processes detected, how much duration and times of occurrences of the detected SRL processes were affected or refined. We also used a process mining technique to analyses how temporal sequencing of the detected SRL processes changed by enriching navigational log data with peripheral and eye-tracking data. The results revealed that by adding new data channels, we improved the capture of learning actions and detected SRL processes while enhancing the granularity of the measurement. In comparison to the use of navigational logs only, the completeness of temporal sequencing relationships between SRL processes with multi-channel data improved. In addition, we concluded that eye-tracking data is valuable for measuring and extracting SRL processes, and it should receive more attention in the future.

Scaffolds that support self‐regulated learning (SRL) have been found to improve learning outcomes. The effects of scaffolds can differ depending on how learners use them and how specific scaffolds might influence learning processes differently. Personalized scaffolds have been proposed to be more beneficial for learning due to their adaptivity to learning progress and individualized content to learning needs. The present study investigated finer‐grained effects of how personalized scaffolds driven by a rule‐based artificial intelligence system influenced SRL processes, especially how students learned with them. Using a pre‐post experimental design, we investigated personalized scaffolds based on university students' real‐time learning processes in a technologically enhanced learning environment. Students in the experimental group (n = 30) received personalized scaffolds, while the control group (n = 29) learned without scaffolds. All students completed a 45‐minute learning task with trace data recorded. Findings indicated scaffold effects on students' subsequent learning behaviour. Additionally, only scaffold interaction correlated to essay performance and suggests that the increase in frequencies of SRL activities alone does not contribute directly to learning outcomes. As guidelines for real‐time SRL support are lacking, this study provides valuable insights to enhance SRL support with adaptive learning technologies. Practitioner notesWhat is already known about this topic Self‐regulated learning scaffolds, especially adaptive scaffolds, improve learning.Personalized scaffolds have effects on self‐regulated learning activities.Past research focused on aggregated effects of scaffolds.What this paper adds Investigates how students learn with personalized scaffolds in terms of frequencies of learning activities and scaffold interaction.Takes a closer look at which learning activities and when the effects of personalized scaffolds occur.Examines how finer‐grained effects of personalized scaffolds correspond to learning outcomes.Implications for practice and/or policy Personalized scaffold effects vary across learning, and future research should consider finer‐grained investigations of SRL support in order to better understand their influence on learning.The number of personalized scaffolds provided should be reconsidered in the future as students only use some of the support provided, especially when task demands increase.Personalized scaffold interaction is linked to improvement in task performance, so future research should also focus on students' appropriate use of self‐regulated learning support. [ABSTRACT FROM AUTHOR]

Self‐regulated learning (SRL) is an essential skill to achieve one's learning goals. This is particularly true for online learning environments (OLEs) where the support system is often limited compared to a traditional classroom setting. Likewise, existing research has found that learners often struggle to adapt their behaviour to the self‐regulatory demands of OLEs. Even so, existing SRL analysis tools have limited utility for real‐time or individualised support of a learner's SRL strategy during a study session. Accordingly, we explore a reinforcement learning based approach to learning optimal SRL strategies for a specific learning task. Specifically, we utilise the sequences of SRL processes acted by 44 participants, and their assessment scores for a prescribed learning task, in a purpose‐built OLE to develop a long short‐term memory (LSTM) network based reward function. This is used to train a reinforcement learning agent to find the optimal sequence of SRL processes for the learning task. Our findings indicate that the developed agents were able to effectively select SRL processes so as to maximise a prescribed learning goal as measured by predicted assessment score and predicted knowledge gains. The contributions of this work will facilitate the development of a tool which can detect sub‐optimal SRL strategy in real‐time and enable individualised SRL focused scaffolding. Practitioner notesWhat is already known about this topic Learners often fail to adequately adapt their behavior to the self‐regulatory demands of e‐Learning environments.In order to promote effective Self‐regulated learning (SRL) capabilities, researchers and educators need tools that are able to analyze and diagnose a learner's SRL strategy use.Current methods for SRL analysis are more often descriptive as opposed to prescriptive and have limited utility for real‐time analysis or support of a learner's SRL behavior.What this paper adds This paper proposes the use of Reinforcement Learning for prescriptive analytics of SRL. We train a Reinforcement Learning agent on sequences of SRL processes acted by learners in order to learn the optimal SRL strategy for a given learning task.Implications for practice and/or policy Our work will facilitate the development of a tool which can detect sub‐optimal SRL strategy in real‐time and enable individualized SRL focused scaffolding.The implications of our work can aid in course design by predicting the self‐regulatory load imposed by a given task.The ability to model SRL strategies using Reinforcement Learning can be extended to simulate or test SRL theories. [ABSTRACT FROM AUTHOR]