Search

Computer-Supported Collaborative Learning (CSCL) is known to be productive if well structured. In CSCL, students construct knowledge by performing learning tasks while communicating about their work. This communication is most often done through online written chats. Understanding what is happening in chats is important from both research and practical perspectives. From a research perspective, insight into chat content offers a window into student interaction and learning. From a more practical standpoint, insight into chat content can (potentially) be used to trigger supportive elements in CSCL environments (e.g., context-sensitive tips or conversational agents). The latter requires real-time, and therefore automated, analysis of the chats. Such an automated analysis is also helpful from the research perspective, since hand-coding of chats is a very time and labour-consuming activity. In this article, we propose a new machine learning-based system for automated coding of student chats, which we labelled ConSent. The core of ConSent is an algorithm that uses contextual information and sentence encoding to produce a reliable estimation of chat message content (i.e. code). To optimize usability, ConSent was designed in such a way that it can cover various topics and various languages. To evaluate our approach, we used two sets of chats coming from different topics (within the domain of physics) and different languages (Dutch and Portuguese). We tested different algorithm configurations, including two multilingual sentence encoders, to find the model that yields the best reliability. As a result, analysis revealed that ConSent models can perform with substantial reliability levels and are able to transfer reliable coding of chats in a similar topic and different language. Finally, we discuss how ConSent can form the basis for a conversational agent, we explain the limitations of our approach, and we indicate possible paths for future work to contribute towards reliable and transferable models.

Background: Sustaining productive student-student dialogue in online collaborative inquiry learning is challenging, and teacher support is limited when needed in multiple groups simultaneously. Collaborative conversational agents (CCAs) have been used in the past to support student dialogue. Yet, research is needed to reveal the characteristics and effectiveness of such agents. Objectives: To investigate the extent to which our analytics-based Collaborative Learning Agent for Interactive Reasoning (Clair) can improve the productivity of student dialogue, we assessed both the levels at which students shared thoughts, listened to each other, deepened reasoning, and engaged with peer's reasoning, as well as their perceived productivity in terms of their learning community, accurate knowledge, and rigorous thinking. Method: In two separate studies, 19 and 27 dyads of secondary school students from Brazil and the Netherlands, respectively, participated in digital inquiry-based science lessons. The dyads were assigned to two conditions: with Clair present (treatment) or absent (control) in the chat. Sequential pattern mining of chat logs and the student's responses to a questionnaire were used to evaluate Clair's impact. Results: Analysis revealed that in both studies, Clair's presence resulted in dyads sharing their thoughts at a higher frequency compared to dyads that did not have Clair. Additionally, in the Netherlands' study, Clair's presence led to a higher frequency of students engaging with each other's reasoning. No differences were observed in students' perceived productivity. Conclusion: This work deepens our understanding of how CCAs impact student dialogue and illustrates the importance of a multidimensional perspective in analysing the role of CCAs in guiding student dialogue.

Asking questions in classrooms can produce metacognitive judgements in students about their confidence in being able to answer correctly. In audience response systems (ARSs), these judgements can be elicited and used as additional feedback metrics. This study (n = 79) explores how online concurrent item-by-item judgments (OCJ) and retrospective composite judgments of performance accuracy (RJPA) can enhance students’ performance and self-assessing accuracy (i.e., calibration – as measured by sensitivity, specificity, and absolute accuracy index). In each of eight weeks, the students answered a multiple-choice quiz and had to denote their level of confidence that their answers were correct (OCJ) and estimate their final score (RJPA). The quizzes followed the voting/revoting paradigm according to which students answer all the quiz questions, receive feedback, and answer the same questions again before the correct answers are shown. The students were randomly grouped into two conditions based on the feedback they received in the ARS: the OCJ group (n = 41) received the percentage distribution and peers’ OCJs as feedback metrics, while the RJPA group (n = 38) received the percentage distribution and peers’ RJPAs. Data analysis showed a systemic underconfidence that affected students’ OCJ judgments. As a result, students in the RJPA group scored significantly higher than the ones in the OCJ one, were more accurate in self-assessing in the revoting phase, and felt overall more confident in the revoting phase. The study also discusses the relationship between the two judgments employed and the calibration variability between the two study phases.