Search

Combining individual and collaborative learning is common, but dynamic combinations (which happen as-the-need arises, rather than in preplanned ways, and may happen on an individual basis) are rare. This work reports findings from a technology probe study exploring alternative designs for classroom co-orchestration support for dynamically transitioning between individual and collaborative learning. The study involved (1) a technology-probe classroom study in an authentic, AI-supported classroom to understand teachers' and students' needs for co-orchestration support over dynamic transitions; and (2) workshops and interviews with students and teachers to get informed feedback about their lived experiences. About 118 students and 3 teachers from a middle school in the US experienced a pairing policy--student, teacher, and AI-controlled pairing policy--(i.e., identifying students needing help and potential helpers) for switching from individual to a peer-tutoring activity. This work aims to answer the following questions: (1) How did students and teachers react to these pairing policies? and (2) What are students' and teachers' desires for sharing control over the orchestration of dynamic transitions? Findings suggest the need for a form of hybrid control between students, teachers, and AI systems over transitions, as well as for adaptivity and adaptability for different classroom characteristics, teachers, and students' prior knowledge.

Using data to generate a deeper understanding of collaborative learning is not new, but automatically analyzing log data has enabled new means of identifying key indicators of effective collaboration and teamwork that can be used to predict outcomes and personalize feedback. Collaboration analytics is emerging as a new term to refer to computational methods for identifying salient aspects of collaboration from multiple group data sources for learners, educators, or other stakeholders to gain and act upon insights. Yet, it remains unclear how collaboration analytics go beyond previous work focused on modelling group interactions for the purpose of adapting instruction. This paper provides a conceptual model of collaboration analytics to help researchers and designers identify the opportunities enabled by such innovations to advance knowledge in, and provide enhanced support for, collaborative learning and teamwork. We argue that mapping from low-level data to higher-order constructs that are educationally meaningful, and that can be understood by educators and learners, is essential to assessing the validity of collaboration analytics. Through four cases, the paper illustrates the critical role of theory, task design, and human factors in the design of interfaces that inform actionable insights for improving collaboration and group learning.

Teachers' spatial behaviours in the classroom can strongly influence students' engagement, motivation and other behaviours that shape their learning. However, classroom teaching behaviour is ephemeral, and has largely remained opaque to computational analysis. Inspired by the notion of Spatial Pedagogy, this paper presents a system called 'Moodoo' that automatically tracks and models how teachers make use of the classroom space by analysing indoor positioning traces. We illustrate the potential of the system through an authentic study with seven teachers enacting three distinct learning designs with more than 200 undergraduate students in the context of science education. The system automatically extracts spatial metrics (e.g. teacher-student ratios, frequency of visits to students' personal spaces, presence in classroom spaces of interest, index of dispersion and entropy), mapping from the teachers' low-level positioning data to higher-order spatial constructs. We illustrate how these spatial metrics can be used to generate a deeper understanding of how the pedagogical commitments embedded in the learning design, and personal teaching strategies, are reflected in the ways teachers use the learning space to provide support to students.

Learning Analytics (LA) systems can offer new insights into learners' behaviours through analysis of multiple data streams. There remains however a dearth of research about how LA interfaces can enable effective communication of educationally meaningful insights to teachers and learners. This highlights the need for a participatory, horizontal co-design process for LA systems. Inspired by the notion of translucence, this paper presents LAT-EP (Learning Analytics Translucence Elicitation Process), a five-step process to design for the effective use of translucent LA systems. LAT-EP was operationalised in an authentic multimodal learning analytics (MMLA) study in the context of teamwork in clinical simulation. Results of this process are illustrated through a series of visual proxies co-designed with teachers, each presenting traces of social, physical, affective and epistemic evidence captured while teams of student nurses practised clinical skills in a simulated hospital setting.

From a human-centred computing perspective, supporting the interpretation of educational dashboards and visualizations by the people intended to use them exposes critical design challenges that may often be trivialized. Empirical evidence already shows that "usable" visualizations are not necessarily effective from an educational perspective. Since an educator's interpretation of visualized data is essentially the construction of a narrative about student progress, in this paper, we propose the concept of "Educational Data Storytelling" as an approach for explaining student data by aligning educational visualizations with the intended learning design. We present a pilot study that explores the effectiveness of these data storytelling elements based on educator responses to prototypes by analyzing the kinds of stories they articulate, their eye-tracking behaviour, and their preferences after inspecting a series of student data visualizations. The dual purpose is to understand the contribution of each visual element for data storytelling, as well as the effectiveness of the enhancements when combined.

There is growing interest in creating learning analytics feedback interfaces that support students directly. While dashboards and other visualizations are proliferating, the evidence is that many fail to provide meaningful insights that help students reflect productively. The contribution of this article is qualitative and quantitative evidence from two studies evaluating a multimodal teamwork analytics tool in authentic clinical teamwork simulations. Collocated activity data are rendered to help nursing students reflect on errors and stress-related incidents during simulations. The user interface explicitly guides student reflection using data storytelling principles, tuned to the intended learning outcomes. The results demonstrate the potential of interfaces that "tell one data story at a time," by helping students to identify misconceptions and errors; think about strategies they might use to address errors, and reflect on their arousal levels. The results also illuminate broader issues around automated formative assessment, and the intelligibility and accountability of learning analytics.

Background Objective Methods Results and Conclusions Dashboards play a prominent role in learning analytics (LA) research. In collaboration activities, dashboards can show traces of team participation. They are often evaluated based on students' perceived satisfaction and engagement with the dashboard. However, there is a notable methodological gap in understanding how these dashboards support the nuanced process of student reflection.This paper presents empirical evidence on how students from high and low‐performing groups reflect individually on their performance while using a Learning Analytics Dashboard (LAD).We address this in the context of education in healthcare, wherein we captured actions and positioning data from a simulation‐based collaborative activity and generated a collaborative LAD. A total of 41 nursing students were invited to participate in a post‐hoc semi‐structured individual interview to use a collaborative LAD while answering a set of prompts to reflect on their individual and group performance. Students' reflections were coded and analysed using Bain's 5R reflection framework. We used epistemic network analysis to capture the dynamic reflection process and to understand the connections between the reflection stages (from low to high). We compared how different these connections were for students in high and low‐performing groups.Our results revealed that most students were only able to achieve low and middle stages of reflection. Yet, students in low‐performing groups predominantly followed low‐to‐middle stages of reflection. In contrast, students from high‐performing groups demonstrated the ability to transition between low‐to‐middle and low‐to‐high stages of reflection. Based on these findings, we discuss implications for both research and practice, particularly emphasising the necessity to scaffold reflection when using LADs. [ABSTRACT FROM AUTHOR]

Multimodal learning analytics (MMLA) offers the potential to provide evidence‐based insights into complex learning phenomena such as collaborative learning. Yet, few MMLA applications have closed the learning analytics loop by being evaluated in real‐world educational settings. This study evaluates the effectiveness of an MMLA solution in enhancing feedback and reflection within a complex and highly dynamic collaborative learning environment. A two‐year longitudinal study was conducted with 399 students and 17 teachers, utilising an MMLA system in reflective debriefings in the context of healthcare education. We analysed the survey data of 74 students and 11 teachers regarding their perceptions of the MMLA system. We applied the Evaluation Framework for Learning Analytics, augmented by complexity, accuracy and trust measures, to assess both teachers' and students' perspectives. The findings illustrated that teachers and students both had generally positive perceptions of the MMLA solution. Teachers found the MMLA solution helpful in facilitating feedback provision and reflection during debriefing sessions. Similarly, students found the MMLA solution effective in providing clarity on the data collected, stimulating reflection on their learning behaviours, and prompting considerations for adaptation in their learning behaviours. However, the complexity of the MMLA solution and the need for qualitative measures of communication emerged as areas for improvement. Additionally, the study highlighted the importance of data accuracy, transparency, and privacy protection to maintain user trust. The findings provide valuable contributions to advancing our understanding of the use of MMLA in supporting feedback and reflection practices in intricate collaborative learning while identifying avenues for further research and improvement. We also provided several insights and practical recommendations for successful MMLA implementation in authentic learning contexts. Practitioner notesWhat is currently known about this topic Multimodal learning analytics (MMLA) seeks to generate data‐informed insights about learners' metacognitive and emotional states as well as their learning behaviours, by utilising intricate physical and physiological signals.MMLA has not only pioneered novel data analytic methods but also aspired to complete the learning analytics loop by crafting innovative, tangible solutions that relay these insights to the concerned stakeholders.A prominent direction within MMLA research has been the formulation of tools to support feedback and reflection in collaborative learning scenarios, given MMLA's capacity to discern intricate and dynamic learning behaviours.What this paper adds Teachers' and students' positive perceptions of an MMLA implementation in stimulating considerations of adaptations in their pedagogical practices and learning behaviours, respectively.Empirical evidence supporting the potential of MMLA in assisting teachers to facilitate students' reflective practices during intricate collaborative learning scenarios.The importance of addressing issues related to design complexity, interpretability for users with disabilities, aggregated data representation, and concerns related to trust for building a practical MMLA solution in real learning settings.Implications for practice and/or policy The MMLA solution can provide teachers with a comprehensive view of student performance, illuminate areas for improvement, and confirm learning scenario outcomes.The MMLA solution can stimulate students' reflections on their learning behaviours and promote considerations of adaptation in their learning behaviours.Providing clear explanations and guidance on how to interpret analytics, as well as addressing concerns related to data completeness and representation, are essential to maximising utility. [ABSTRACT FROM AUTHOR]

The term "Classroom Proxemics" refers to how teachers and students use classroom space, and the impact of this and the spatial design on learning and teaching. This study addresses the divide between, on the one hand, substantial work on proxemics based on classroom observations and, on the other hand, emerging work to design automated feedback that helps teachers identify salient patterns in their use of the classroom space. This study documents how digital analytics were designed in service of a senior teacher's practice-based inquiry into classroom proxemics. Indoor positioning data from four teachers were analysed, visualized and used as evidence to compare three distinct learning designs enacted in a physics classroom. This study demonstrates how teachers can make effective use of such visualizations, to gain insight into their classroom practice. This is evidenced by (a) documenting teachers' reflections on visualizations of positioning data, both their own and that of peers and (b) identifying the types of indicator (operationalized as analytical metrics) that foreground the most useful information for teachers to gain insight into their practice.

Artificial intelligence (AI) can enhance teachers' capabilities by sharing control over different parts of learning activities. This is especially true for complex learning activities, such as dynamic learning transitions where students move between individual and collaborative learning in un‐planned ways, as the need arises. Yet, few initiatives have emerged considering how shared responsibility between teachers and AI can support learning and how teachers' voices might be included to inform design decisions. The goal of our article is twofold. First, we describe a secondary analysis of our co‐design process comprising six design methods to understand how teachers conceptualise sharing control with an AI co‐orchestration tool, called Pair‐Up. We worked with 76 middle school math teachers, each taking part in one to three methods, to create a co‐orchestration tool that supports dynamic combinations of individual and collaborative learning using two AI‐based tutoring systems. We leveraged qualitative content analysis to examine teachers' views about sharing control with Pair‐Up, and we describe high‐level insights about the human‐AI interaction, including control, trust, responsibility, efficiency, and accuracy. Secondly, we use our results as an example showcasing how human‐centred learning analytics can be applied to the design of human‐AI technologies and share reflections for human‐AI technology designers regarding the methods that might be fruitful to elicit teacher feedback and ideas. Our findings illustrate the design of a novel co‐orchestration tool to facilitate the transitions between individual and collaborative learning and highlight considerations and reflections for designers of similar systems. Practitioner notesWhat is already known about this topic: Artificial Intelligence (AI) can help teachers facilitate complex classroom activities, such as having students move between individual and collaborative learning in unplanned ways.Designers should use human‐centred design approaches to give teachers a voice in deciding what AI might do in the classroom and if or how they want to share control with it.What this paper adds: Presents teacher views about how they want to share control with AI to support students moving between individual and collaborative learning.Describes how we adapted six design methods to design AI features.Illustrates a complete, iterative process to create human‐AI interactions to support teachers as they facilitate students moving from individual to collaborative learning.Implications for practice: We share five implications for designers that teachers highlighted as necessary when designing AI‐features, including control, trust, responsibility, efficiency and accuracy.Our work also includes a reflection on our design process and implications for future design processes. [ABSTRACT FROM AUTHOR]