Search

Researchers have demonstrated that dialogue‐based intelligent tutoring systems (ITS) can be effective in assisting students in learning. However, little research has attempted to explore the necessity of equipping dialogue‐based ITS with one of the most important capabilities of human tutors, that is, maintaining polite interactions with students, which is essential to provide students with a pleasant learning experience. In this study, we examined the role of politeness by analysing a large‐scale real‐world dataset consisting of over 14K online human–human tutorial dialogues. Specifically, we employed linguistic theories of politeness to characterise the politeness levels of tutor–student‐generated utterances, investigated the correlation between the politeness levels of tutors' utterances and students' problem‐solving performance and quantified the power of politeness in predicting students' problem‐solving performance by applying Gradient Tree Boosting. The study results showed that: (i) in the effective tutorial sessions (ie, sessions in which students successfully solved problems), tutors tended to be very polite at the start of a tutorial session and become more direct to guide students as the session progressed; (ii) students with better performance in solving problems tended to be more polite at the beginning and the end of a tutorial session than their counterparts who failed to solve problems; (iii) the correlation between tutors' polite expressions and students' performance was not evident in non‐instructional communication; and (iv) politeness alone cannot adequately reveal students' problem‐solving performance, and thus other factors (eg, sentiment contained in utterances) should also be taken into account. Practitioner notesWhat is already known about this topic Human–human tutoring is acknowledged as an effective instructional method.Polite expression can help strengthen the relationship between tutors and students.Polite expression can promote students' learning achievements in many educational contexts.What this paper adds By considering the students' prior progress on a problem‐based learning task, we demonstrated the extent to which tutors and students display politeness in tutoring dialogues.Tutors' polite expressions might not correlate with students' problem‐solving performance in online human–human tutoring dialogues.Politeness alone was insufficient to predict the students' performance.Implications for practice Tutors might consider using words with positive sentiment values to express politeness to students with prior progress, which might encourage those students to make a further effort.The polite strategy of expressing indirect requests could help tutors mitigate the sense of directness, but this strategy should be carefully used in delivering instructional hints, especially for students without prior progress.To better assist students without prior progress, tutors might consider using more direct expression to explicitly guide students. [ABSTRACT FROM AUTHOR]

Background: Assignments that involve writing based on several texts are challenging to many learners. Formative feedback supporting learners in these tasks should be informed by the characteristics of evolving written product and by the characteristics of learning processes learners enacted while developing the product. However, formative feedback in writing tasks based on multiple texts has almost exclusively focused on essay product and rarely included SRL processes. Objectives: We explored the viability of using product and process features to develop machine learning classifiers that identify low‐ and high‐performing essays in a multi‐text writing task. Methods: We examined learning processes and essay submissions of 163 graduate students working on an authentic multi‐text writing assignment. We utilised learners' trace data to obtain process features and state‐of‐the‐art natural language processing methods to obtain product features for our classifiers. Results and Conclusions: Of four popular classifiers examined in this study, Random Forest achieved the best performance (accuracy = 0.80 and recall = 0.77). The analysis of important features identified in the Random Forest classification model revealed one product (coverage of reading topics) and three process (elaboration/organisation, re‐reading and planning) features as important predictors of writing quality. Major Takeaways: The classifier can be used as a part of a future automated writing evaluation system that will support at scale formative assessment in writing tasks based on multiple texts in different courses. Based on important predictors of essay performance, a guidance can be tailored to learners at the outset of a multi‐text writing task to help them do well in the task. Lay Description: What is already known about this topic?: Both product and process features should be used to inform formative feedback on writing.Providing product‐ and process‐oriented feedback to learners is challenging.Automatic writing evaluation systems have mainly relied upon product features.Automated analysis of learners' trace data and their essay drafts is a promising venue. What this paper adds?: An accurate machine learning classifier that identifies low‐ and high‐scoring essays.The classifier utilized both product and process features.We obtained process features from learners' trace data in digital learning environment.We computed product features using state‐of‐the‐art text analytical methods. Implications for practice and/or policy: The classifier can be used as a part of a future automated writing evaluation system.We revealed learning processes and essay characteristics that influence performance.Based on important predictors of performance, formative feedback can be given to learners. [ABSTRACT FROM AUTHOR]

An introduction is presented wherein the editor discusses various reports within the issue on topics, including the role of learning analytics in advancing formative and summative assessment and the use of trace-based learning analytics methods to assess learning processes.

Developing knowledge‐transforming skills in writing may help students increase learning by actively building knowledge, regardless of the domain. However, many undergraduate students struggle to transform knowledge when drafting essays based on multiple sources. Writing analytics can be used to scaffold knowledge transforming as writers bring evidence to bear in supporting claims. We investigated how to automatically identify sentences representing knowledge transformation in argumentative essays. A synthesis of cognitive theories of writing and Bloom's typology identified 22 linguistic features to model processes of knowledge transforming in a corpus of 38 undergraduates' essays. Findings indicate undergraduates mostly paraphrase or copy information from multiple sources rather than engage deeply with sources' content. Eight linguistic features were important for discriminating evidential sentences as telling versus transforming source knowledge. We trained a machine learning algorithm that accurately classified nearly three of four evidential sentences as knowledge‐telling or knowledge‐transforming, offering potential for use in future research. Lay Description: What is already known about this topic: Engagement in knowledge transforming in multi‐source writing benefits learning.Post‐secondary writers rarely succeed in knowledge transforming.Computer‐generated formative feedback may promote knowledge‐transforming processes.Computational tools so far developed focused on a composition's rhetorical features only. What this paper adds: We proposed a novel methodology to identify knowledge transforming in argumentative essays.Computational approach involved both rhetorical and content characteristics of evidential sentences.Machine learning algorithm was developed to classify text as knowledge‐transforming versus telling.Eight linguistic features predicted evidential sentences as telling or transforming source knowledge. Implications for practice: There is a clear need to teach post‐secondary students tactics for knowledge transforming.The results can inform the development of writing analytics tool to scaffold knowledge‐transforming revisions. [ABSTRACT FROM AUTHOR]

Using traces of behaviors to predict outcomes is useful in varied contexts ranging from buyer behaviors to behaviors collected from smart-home devices. Increasingly, higher education systems have been using Learning Management System (LMS) digital data to capture and understand students' learning and well-being. Researchers in the social sciences are increasingly interested in the potential of using digital log data to predict outcomes and design interventions. Using LMS data for predicting the likelihood of students' success in for-credit college courses provides a useful example of how social scientists can use these techniques on a variety of data types. Here, we provide a primer on how LMS data can be feature-mapped and analyzed to accomplish these goals. We begin with a literature review summarizing current approaches to analyzing LMS data, then discuss ethical issues of privacy when using demographic data and equitable model building. In the second part of the paper, we provide an overview of popular machine learning algorithms and review analytic considerations such as feature generation, assessment of model performance, and sampling techniques. Finally, we conclude with an empirical example demonstrating the ability of LMS data to predict student success, summarizing important features and assessing model performance across different model specifications. [ABSTRACT FROM AUTHOR]

• Many students struggle to overcome poor performances early in STEM courses. • Evaluation of prior studying predicts plans to adapt in next learning cycle. • Plans to adapt predict an increase of desirable learning behaviors. • Desirable learning behaviors positively relate to subsequent exam score. • Natural language processing was used to measure evaluation and plans to adapt. Researchers and many educators agree that the ability to self-regulate learning is important for academic success. Yet, many students struggle to anticipate learning difficulties and adjust accordingly. Further, despite theorizing that self-regulated learning involves adaptation across learning cycles, few researchers have examined students' evaluative judgments, their implications for students' behavior in a subsequent learning cycle, or their effects on achievement. Utilizing data from a large, introductory college biology course, we examined how struggling students' evaluative judgments made after a first unit exam predicted changes in learning behaviors as well as how those changes predicted performance on a subsequent exam. We used natural language processing to analyze data from a reflective essay written after a first unit exam, identifying language that reflected evaluation of prior studying and plans to adapt learning. Then, we utilized digital traces of learning behaviors to measure students' actual adaptation of their use of learning resources. Results from a path analysis revealed students' evaluations predicted how extensively they discussed plans to adapt their learning process. Plans to adapt described in written reflections predicted an increase in the frequency of desirable learning behaviors, which in turn predicted higher subsequent exam scores, after controlling for previous exam performance. These findings provide empirical evidence of multiple theorized self-regulated learning processes, including how evaluations of learning at the end of a learning cycle can inform planning and behavior changes in a subsequent learning cycle, and that increases in the enactment of effective learning strategies predict improved performance in complex learning tasks. [ABSTRACT FROM AUTHOR]

Background: Select and enact appropriate learning tactics that advance learning has been considered a critical set of skills to successfully complete highly flexible online courses, such as Massive open online courses (MOOCs). However, limited by analytic methods that have been used in the past, such as frequency distribution, sequence mining and process mining, we lack a deep, complete and detailed understanding of the learning tactics used by MOOC learners. Objectives: In the present study, we proposed four major dimensions to better interpret and understand learning tactics, which are frequency, continuity, sequentiality and role of learning actions within tactics. The aim of this study was to examine to what extent can a new analytic technique, the ordered network analysis (ONA), deepen the understanding of MOOC learning tactics compared to using other methods. Methods: In particular, we performed a fine‐grained analysis of learning tactics detected from more than 4 million learning events in the behavioural trace data of 8788 learners who participated in a large‐scale MOOC 'Flipped Classroom'. Results and Conclusions: We detected eight learning tactics, and then chose one typical tactic as an example to demonstrate how the ONA technique revealed all four dimensions and provided deeper insights into this MOOC learning tactic. Most importantly, based on the comparison with different methods such as process mining, we found that the ONA method provided a unique opportunity and novel insight into the roles of different learning actions in tactics which was neglected in the past. Takeaway: In summary, we conclude that ONA is a promising technique that can benefit the research on learning tactics, and ultimately benefit MOOC learners by strengthening the strategic support. Lay Description: What is already known about this topic?: Since external support from instructors or peers are usually limited in MOOCs, the ability to self‐regulate learning (SRL), that is, select and enact goal‐oriented learning tactics that advance learning comes to be a critical set of skills to successfully complete coursework.Researchers have utilised analytic techniques that go beyond frequency‐based approaches, for example, process mining and sequence mining, to detect and understand learning tactics.However, limited by previous analytic methods that have been used in the past, we still lack a deep, complete and detailed understanding of the learning tactics used by MOOC learners. What this paper adds?: We proposed four major dimensions to better interpret and understand learning tactics, which are frequency, continuity, sequentiality and role of learning actions within tactics.We examined to what extent can a new analytic technique, the ordered network analysis (ONA), deepen the understanding of learning tactics compared to other methods.We found that the ONA method provided a unique opportunity and novel insight into the roles of different learning actions in tactics which was neglected in the past. Implications for practice and/or policy: Researchers who study learning tactics unpacked using the ONA technique can gain a deeper insight into a theorised learning processes that interplay within and across learning tactics and, in that way, improve their understanding of how self‐regulated learners enact and monitor learning tactics.The analytics on learning tactics use in a form of ONA graphs has the potential to provide learners with a detailed overview of their learning engagement over the selected learning period in MOOCs.The ONA graphs of learning tactics, combined with the information about learning performance and course requirements, instructors or learners (prompted by instructors) can evaluate whether the way learners have studied in a MOOC was beneficial to learners' learning success. [ABSTRACT FROM AUTHOR]

Even highly motivated undergraduates drift off their STEM career pathways. In large introductory STEM classes, instructors struggle to identify and support these students. To address these issues, we developed co‐redesign methods in partnership with disciplinary experts to create high‐structure STEM courses that better support students and produce informative digital event data. To those data, we applied theory‐ and context‐relevant labels to reflect active and self‐regulated learning processes involving LMS‐hosted course materials, formative assessments, and help‐seeking tools. We illustrate the predictive benefits of this process across two cycles of model creation and reapplication. In cycle 1, we used theory‐relevant features from 3 weeks of data to inform a prediction model that accurately identified struggling students and sustained its accuracy when reapplied in future semesters. In cycle 2, we refit a model with temporally contextualized features that achieved superior accuracy using data from just two class meetings. This modelling approach can produce durable learning analytics solutions that afford scaled and sustained prediction and intervention opportunities that involve explainable artificial intelligence products. Those same products that inform prediction can also guide intervention approaches and inform future instructional design and delivery.Practitioner notesWhat is already known about this topic Learning analytics includes an evolving collection of methods for tracing and understanding student learning through their engagements with learning technologies. Prediction models based on demographic data can perpetuate systemic biases. Prediction models based on behavioural event data can produce accurate predictions of academic success, and validation efforts can enrich those data to reflect students' self‐regulated learning processes within learning tasks. What this paper adds Learning analytics can be successfully applied to predict performance in an authentic postsecondary STEM context, and the use of context and theory as guides for feature engineering can ensure sustained predictive accuracy upon reapplication. The consistent types of learning resources and cyclical nature of their provisioning from lesson to lesson are hallmarks of high‐structure active learning designs that are known to benefit learners. These designs also provide opportunities for observing and modelling contextually grounded, theory‐aligned and temporally positioned learning events that informed prediction models that accurately classified students upon initial and later reapplications in subsequent semesters. Co‐design relationships where researchers and instructors work together toward pedagogical implementation and course instrumentation are essential to developing unique insights for feature engineering and producing explainable artificial intelligence approaches to predictive modelling. Implications for practice and/or policy High‐structure course designs can scaffold student engagement with course materials to make learning more effective and products of feature engineering more explainable. Learning analytics initiatives can avoid perpetuation of systemic biases when methods prioritize theory‐informed behavioural data that reflect learning processes, sensitivity to instructional context and development of explainable predictors of success rather than relying on students' demographic characteristics as predictors. Prioritizing behaviours as predictors improves explainability in ways that can inform the redesign of courses and design of learning supports, which further informs the refinement of learning theories and their applications. Learning analytics includes an evolving collection of methods for tracing and understanding student learning through their engagements with learning technologies. Prediction models based on demographic data can perpetuate systemic biases. Prediction models based on behavioural event data can produce accurate predictions of academic success, and validation efforts can enrich those data to reflect students' self‐regulated learning processes within learning tasks. Learning analytics can be successfully applied to predict performance in an authentic postsecondary STEM context, and the use of context and theory as guides for feature engineering can ensure sustained predictive accuracy upon reapplication. The consistent types of learning resources and cyclical nature of their provisioning from lesson to lesson are hallmarks of high‐structure active learning designs that are known to benefit learners. These designs also provide opportunities for observing and modelling contextually grounded, theory‐aligned and temporally positioned learning events that informed prediction models that accurately classified students upon initial and later reapplications in subsequent semesters. Co‐design relationships where researchers and instructors work together toward pedagogical implementation and course instrumentation are essential to developing unique insights for feature engineering and producing explainable artificial intelligence approaches to predictive modelling. High‐structure course designs can scaffold student engagement with course materials to make learning more effective and products of feature engineering more explainable. Learning analytics initiatives can avoid perpetuation of systemic biases when methods prioritize theory‐informed behavioural data that reflect learning processes, sensitivity to instructional context and development of explainable predictors of success rather than relying on students' demographic characteristics as predictors. Prioritizing behaviours as predictors improves explainability in ways that can inform the redesign of courses and design of learning supports, which further informs the refinement of learning theories and their applications. [ABSTRACT FROM AUTHOR]

Collaborative discussions should engage all students, not just a few who dominate ("leaders") while others participate as "followers" (Zhu, 2006). Cunningham (1991) noted that collaborating learners bring, discuss and debate multiple perspectives to develop their own position while acknowledging others' views. Higher levels of knowledge construction emerged when posts stimulated frequent reply by multiple participants (Aviv, Erlich, Ravid, & Geva, 2003) and were strongly content- and task-oriented (Rovai, 2007). So, to help students more actively and productively engage in knowledge-constructing discussions, an instructor needs to detect students' posts that do not stimulate replies, identify content those posts introduce, and guide students to revise posts to encourage peers' responses. However, such monitoring would be very time- and energy-consuming, especially in large-enrolment courses (Hura, 2010). To set a stage for developing a classifier to automate these tasks, we proposed 10 rhetorical moves characteristic of the interactive mode of Chi and Wylie's ICAP framework (2014) and categorized fine-grained content in discussion posts using these moves. We then identified attributes of posts that triggered a greater number of responses. Rhetorical moves of "asking questions," "requesting justification," "building-on," "giving a reason" and "making a claim" triggered more peer responses. Posts with moves of "disagreeing," "comparing" and "making claims" predicted students' achievement on a test and an argumentative writing task. We propose analytics for learners and instructors about forming and revising posts to promote constructive discussions and subsequent achievements. • Providing both frequent and on topic posts is important to advance knowledge construction in online discussions. • We partitioned students' posts into idea units and developed a 10-category coding scheme inspired by the ICAP framework. • Asking questions, requesting justification and building-on trigger more peer responses that contribute to the discussion. • Disagreeing, comparing and making claims predict students' achievement on argumentative writing task and achievement test. [ABSTRACT FROM AUTHOR]