Search

Access to body-based resources has been shown to augment cognitive processes, but not all movements equally aid reasoning. Interactive technologies, like dynamic geometry systems (DGS), potentially amplify the link between movement and geometric representation, thereby deepening students' understanding of geometric properties. This study investigated the influence of access to a DGS on students' geometric reasoning and gesture production compared to the impact of only spontaneous gestures. Access to simulations did not significantly increase the likelihood of students producing correct mathematical insights compared to spontaneous gestures. However, when controlling gesture production, DGS access became a significant predictor for transformational proof production. We also found that access to DGS influenced the type of gestures students produced during geometric reasoning tasks. This study underlines the vital role of gestures in geometric reasoning and suggests that while DGS offers benefits, spontaneous gestures remain a crucial embodied resource for exploring geometric concepts. [This paper was published in: "Proceedings of the 18th International Conference of the Learning Sciences - ICLS 2024," edited by R. Lindgren et al., International Society of the Learning Sciences, 2024, pp. 131-38.]

The use of technology as a facilitator in learning environments has become increasingly prevalent with the global pandemic caused by COVID-19. As such, computer-supported collaborative learning (CSCL) gains a wider adoption in contrast to traditional learning methods. At the same time, the need for automated tools capable of assessing and stimulating collaboration between participants has become more stringent, as human monitoring of the increasing volume of conversations becomes overwhelming. This paper introduces a method grounded in dialogism for evaluating students' involvement in chat conversations based on semantic chains computed using language models. These semantic chains reflect emergent voices from dialogism that span and interact throughout the conversation. Our integrated method uses contextual information captured by BERT transformer models to identify links in a chain that connects semantically related concepts from a voice uttered by one or more participants. Two types of visualizations were generated to depict the longitudinal propagation and the transversal inter-animation of voices within the conversation. In addition, a list of handcrafted features derived from the constructed chains and computed for each participant is introduced. Several machine learning algorithms were tested using these features to evaluate the extent to which semantic chains are predictive of student involvement in chat conversations. [This paper was published in: "Ludic, Co-design and Tools Supporting Smart Learning Ecosystems and Smart Education, Proceedings of the 6th International Conference on Smart Learning Ecosystems and Regional Development," edited by Ó. Mealha et al., Springer Nature Singapore Pte Ltd., 2022, pp. 67-78.]

Reading comprehension is essential for students' ability to build knowledge. Students' comprehension abilities can be enhanced by providing students with deliberate practice and formative feedback on reading comprehension strategies. iSTART is an Intelligent Tutoring System (ITS) that is designed to provide instruction in reading strategies with minimal teacher supervision -- affording the ability to teach reading strategies at scale. In the current study, undergraduate students received reading strategy instruction and opportunities for deliberate practice via the iSTART intelligent tutoring system or not (i.e., no-treatment control group). Participants' reading comprehension and psychology knowledge were assessed. The iSTART group demonstrated substantially greater scores than the control group on a post-training reading comprehension measure (Cohen's d > 1.0). The average psychology knowledge scores did not differ between iSTART (post-training) and control groups, but overall these scores were unexpectedly low. Within the iSTART group, there was no difference in reading comprehension and knowledge scores as a function of students' different behaviors in the system. Overall, the results indicate that iSTART is an effective tool to teach reading strategies at large scale. However, further work is required to test the extent to which iSTART supports knowledge building. [This paper was published in: "Proceedings of Learning @ Scale conference (L@S '24)," ACM, 2024.]

As evidence grows supporting the importance of non-cognitive factors in learning, computer-assisted learning platforms increasingly incorporate non-academic interventions to influence student learning and learning related-behaviors. Non-cognitive interventions often attempt to influence students' mindset, motivation, or metacognitive reflection to impact learning behaviors and outcomes. In the current paper, we analyze data from five experiments, involving seven treatment conditions embedded in mastery-based learning activities hosted on a computer-assisted learning platform focused on middle school mathematics. Each treatment condition embodied a specific non-cognitive theoretical perspective. Over seven school years, 20,472 students participated in the experiments. We estimated the effects of each treatment condition on students' response time, hint usage, likelihood of mastering knowledge components, learning efficiency, and post-tests performance. Our analyses reveal a mix of both positive and negative treatment effects on student learning behaviors and performance. Few interventions impacted learning as assessed by the post-tests. These findings highlight the difficulty in positively influencing student learning behaviors and outcomes using non-cognitive interventions. [This paper was published in: "LAK23: 13th International Learning Analytics and Knowledge Conference Proceedings," March 13-17, 2023.]

This paper demonstrates how to apply Machine Learning (ML) techniques to analyze student interaction data collected in an online mathematics game. We examined: (1) how different ML algorithms influenced the precision of middle-school students' (N = 359) performance prediction; and (2) what types of in-game features were associated with student math knowledge scores. The results indicated that the Random Forest algorithm showed the best performance in predicting posttest math knowledge scores among the seven algorithms employed. Out of 37 features included in the model, the validity of the students' first mathematical transformation was the most predictive of their math knowledge scores. Implications for game learning analytics and supporting students' algebraic learning are discussed based on the findings.

Computer-assisted learning platforms (CALPS) increasingly include gamified elements to improve student outcomes by enhancing their engagement with content. Although evidence exists that gamified programs increase engagement and learning outcomes, there is little causal research on what programmatic mechanisms drive the effect between engagement and learning. In the following paper, we explore this relationship through a method of causal moderation known as fully latent principal stratification. Using data from a large-scale randomized control trial assessing gamified and traditional CALP systems' effects on algebraic knowledge, we estimate the impact of using the gamified CALP on students who engage with one of its key gamification elements--replaying a problem after a suboptimal attempt. The gamified CALP asks students to manipulate algebraic expressions from start to goal states and provides feedback based on the efficiency of these manipulations, allowing students to replay the problems when their efficiency can be improved. We find that the effect of gamification is greater for students with a higher propensity to replay problems. This finding suggests that gamification elements that provide students with opportunities to retry problems are driving the game's efficacy and provide evidence for a scalable mechanism of gamification that can improve students' learning. [This paper was published in: "Proceedings of the Tenth ACM Conference on Learning @ Scale (L@S '23), July 20-22, 2023, Copenhagen, Denmark," ACM, 2023, pp.164-173.]

This work proposes Dynamic Linear Epsilon-Greedy, a novel contextual multi-armed bandit algorithm that can adaptively assign personalized content to users while enabling unbiased statistical analysis. Traditional A/B testing and reinforcement learning approaches have trade-offs between empirical investigation and maximal impact on users. Our algorithm seeks to balance these objectives, allowing platforms to personalize content effectively while still gathering valuable data. Dynamic Linear Epsilon-Greedy was evaluated via simulation and an empirical study in the ASSISTments online learning platform. In simulation, Dynamic Linear Epsilon-Greedy performed comparably to existing algorithms and in ASSISTments, slightly increased students' learning compared to A/B testing. Data collected from its recommendations allowed for the identification of qualitative interactions, which showed high and low knowledge students benefited from different content. Dynamic Linear Epsilon-Greedy holds promise as a method to balance personalization with unbiased statistical analysis. All the data collected during the simulation and empirical study are publicly available at https://osf.io/zuwf7/. [This paper was published in: "UMAP '23: Proceedings of the 31st ACM Conference on User Modeling, Adaptation and Personalization," June 26-29, 2023.]

We sought to identify the extent to which an innovative educational technology called ELCII (Early Language Comprehension Individualized Instruction) improved inference making for various populations of kindergarten students. Analyses examined student performance based on individual-level demographic characteristics (i.e., gender, race/ethnicity, home language, English Language Learner status, and Free or Reduced Lunch eligibility) to shed light on whether ELCII addresses the needs of learners from marginalized communities. Results showed that all students, on average, improved in their inference making performance over time and that growth in inference making performance did not differ across subgroups. However, the results also highlighted persistent gaps that were present at the onset of the program, and unfortunately continued to persist. To bridge such gaps, the standard of 'working for all' is a necessary first step, but not sufficient. Future research should identify ways to accelerate growth of core skill development among students who need additional support.

Math performance continues to be an important focus for improvement. The most recent National Report Card in the U.S. suggested student math scores declined in the past two years possibly due to COVID-19 pandemic and related school closures. We report on the implementation of a math homework program that leverages AI-based one-to-one technology, in 32 schools for two years as a part of a randomized controlled trial in diverse settings of the state of North Carolina in the US. The program, called "ASSISTments," provides feedback to students as they solve homework problems and automatically prepares reports for teachers about student performance on daily assignments. The paper describes the sample, the study design, the implementation of the intervention, including the recruitment effort, the training and support provided to teachers, and the approaches taken to assess teacher's progress and improve implementation fidelity. Analysis of data collected during the study suggest that (a) treatment teachers changed their homework review practices as they used ASSISTments, and (b) the usage of ASSISTments was positively correlated with student learning outcome.

Large-scale, classroom-based experiments using adaptive instructional software pose somewhat unique challenges for experimental design and deployment. One reason for this is that adaptive software allows students to advance through the curriculum at different rates and encounter content at different times, meaning that content targeted for experimentation is often reached asynchronously by students within the same classroom. In addition, many pedagogical approaches subject to experimentation will require multiple "touch points" with students throughout a course. Nimble experimental methods necessitate that experiments are able to take place at any time during the school year, so a robust experimentation system may need to be aware of the student's educational experience(s) prior to the experiment, both to permit students to be excluded from the experiment (if desired by the researcher) if they have previously been taught the target content, and to ensure that the pedagogical approach taken during the experiment is consistent with that which has been experienced by the student before and after the experimental period. The UpGrade A/B testing platform (Ritter et al., 2020) is designed to help experimenters navigate these issues. This paper discusses several key design considerations for conducting digital experiments within adaptive educational software: ordering and sequencing, coordination of experimental activities, and exclusion criteria. Following this is an illustration of how these principles were applied in two recently-conducted large-scale experiments.