Search

Understanding reader behaviors such as skimming, deep reading, and scanning is essential for improving educational instruction. While prior eye-tracking studies have trained models to recognize reading behaviors, they often rely on instructed reading tasks, which can alter natural behaviors and limit the applicability of these findings to in-the-wild settings. Additionally, there is a lack of clear definitions for reading behavior archetypes in the literature. We conducted a classroom study to address these issues by collecting instructed and in-the-wild reading data. We developed a mixed-method framework, including a human-driven theoretical model, statistical analyses, and an AI classifier, to differentiate reading behaviors based on their velocity, density, and sequentiality. Our lightweight 2D CNN achieved an F1 score of 0.8 for behavior recognition, providing a robust approach for understanding in-the-wild reading. This work advances our ability to provide detailed behavioral insights to educators, supporting more targeted and effective assessment and instruction., Comment: 24 pages, 16 figures, 6 tables, conference

The workshop 'AI-based Planning for Cyber-Physical Systems', which took place on February 26, 2024, as part of the 38th Annual AAAI Conference on Artificial Intelligence in Vancouver, Canada, brought together researchers to discuss recent advances in AI planning methods for Cyber-Physical Systems (CPS). CPS pose a major challenge due to their complexity and data-intensive nature, which often exceeds the capabilities of traditional planning algorithms. The workshop highlighted new approaches such as neuro-symbolic architectures, large language models (LLMs), deep reinforcement learning and advances in symbolic planning. These techniques are promising when it comes to managing the complexity of CPS and have potential for real-world applications., Comment: This is the Proceedings of the AAAI Workshop on AI Planning for Cyber-Physical Systems - CAIPI24, which was held in Vancouver, CA, February 26, 2024

Recent technological advancements have enhanced our ability to collect and analyze rich multimodal data (e.g., speech, video, and eye gaze) to better inform learning and training experiences. While previous reviews have focused on parts of the multimodal pipeline (e.g., conceptual models and data fusion), a comprehensive literature review on the methods informing multimodal learning and training environments has not been conducted. This literature review provides an in-depth analysis of research methods in these environments, proposing a taxonomy and framework that encapsulates recent methodological advances in this field and characterizes the multimodal domain in terms of five modality groups: Natural Language, Video, Sensors, Human-Centered, and Environment Logs. We introduce a novel data fusion category -- mid fusion -- and a graph-based technique for refining literature reviews, termed citation graph pruning. Our analysis reveals that leveraging multiple modalities offers a more holistic understanding of the behaviors and outcomes of learners and trainees. Even when multimodality does not enhance predictive accuracy, it often uncovers patterns that contextualize and elucidate unimodal data, revealing subtleties that a single modality may miss. However, there remains a need for further research to bridge the divide between multimodal learning and training studies and foundational AI research., Comment: Submitted to ACM Computing Surveys. Currently under review

This article proposes a novel approach to traffic signal control that combines phase re-service with reinforcement learning (RL). The RL agent directly determines the duration of the next phase in a pre-defined sequence. Before the RL agent's decision is executed, we use the shock wave theory to estimate queue expansion at the designated movement allowed for re-service and decide if phase re-service is necessary. If necessary, a temporary phase re-service is inserted before the next regular phase. We formulate the RL problem as a semi-Markov decision process (SMDP) and solve it with proximal policy optimization (PPO). We conducted a series of experiments that showed significant improvements thanks to the introduction of phase re-service. Vehicle delays are reduced by up to 29.95% of the average and up to 59.21% of the standard deviation. The number of stops is reduced by 26.05% on average with 45.77% less standard deviation., Comment: Accepted to IEEE ITSC 2024

This article presents the first field deployment of a multi-agent reinforcement-learning (MARL) based variable speed limit (VSL) control system on the I-24 freeway near Nashville, Tennessee. We describe how we train MARL agents in a traffic simulator and directly deploy the simulation-based policy on a 17-mile stretch of Interstate 24 with 67 VSL controllers. We use invalid action masking and several safety guards to ensure the posted speed limits satisfy the real-world constraints from the traffic management center and the Tennessee Department of Transportation. Since the time of launch of the system through April, 2024, the system has made approximately 10,000,000 decisions on 8,000,000 trips. The analysis of the controller shows that the MARL policy takes control for up to 98% of the time without intervention from safety guards. The time-space diagrams of traffic speed and control commands illustrate how the algorithm behaves during rush hour. Finally, we quantify the domain mismatch between the simulation and real-world data and demonstrate the robustness of the MARL policy to this mismatch.

Early and accurate detection of anomalous events on the freeway, such as accidents, can improve emergency response and clearance. However, existing delays and errors in event identification and reporting make it a difficult problem to solve. Current large-scale freeway traffic datasets are not designed for anomaly detection and ignore these challenges. In this paper, we introduce the first large-scale lane-level freeway traffic dataset for anomaly detection. Our dataset consists of a month of weekday radar detection sensor data collected in 4 lanes along an 18-mile stretch of Interstate 24 heading toward Nashville, TN, comprising over 3.7 million sensor measurements. We also collect official crash reports from the Nashville Traffic Management Center and manually label all other potential anomalies in the dataset. To show the potential for our dataset to be used in future machine learning and traffic research, we benchmark numerous deep learning anomaly detection models on our dataset. We find that unsupervised graph neural network autoencoders are a promising solution for this problem and that ignoring spatial relationships leads to decreased performance. We demonstrate that our methods can reduce reporting delays by over 10 minutes on average while detecting 75% of crashes. Our dataset and all preprocessing code needed to get started are publicly released at https://vu.edu/ft-aed/ to facilitate future research.

Investigating children's embodied learning in mixed-reality environments, where they collaboratively simulate scientific processes, requires analyzing complex multimodal data to interpret their learning and coordination behaviors. Learning scientists have developed Interaction Analysis (IA) methodologies for analyzing such data, but this requires researchers to watch hours of videos to extract and interpret students' learning patterns. Our study aims to simplify researchers' tasks, using Machine Learning and Multimodal Learning Analytics to support the IA processes. Our study combines machine learning algorithms and multimodal analyses to support and streamline researcher efforts in developing a comprehensive understanding of students' scientific engagement through their movements, gaze, and affective responses in a simulated scenario. To facilitate an effective researcher-AI partnership, we present an initial case study to determine the feasibility of visually representing students' states, actions, gaze, affect, and movement on a timeline. Our case study focuses on a specific science scenario where students learn about photosynthesis. The timeline allows us to investigate the alignment of critical learning moments identified by multimodal and interaction analysis, and uncover insights into students' temporal learning progressions.

LLMs have demonstrated proficiency in contextualizing their outputs using human input, often matching or beating human-level performance on a variety of tasks. However, LLMs have not yet been used to characterize synergistic learning in students' collaborative discourse. In this exploratory work, we take a first step towards adopting a human-in-the-loop prompt engineering approach with GPT-4-Turbo to summarize and categorize students' synergistic learning during collaborative discourse. Our preliminary findings suggest GPT-4-Turbo may be able to characterize students' synergistic learning in a manner comparable to humans and that our approach warrants further investigation., Comment: In press at the 25th international conference on Artificial Intelligence in Education (AIED) Late-Breaking Results (LBR) track

This paper explores the use of large language models (LLMs) to score and explain short-answer assessments in K-12 science. While existing methods can score more structured math and computer science assessments, they often do not provide explanations for the scores. Our study focuses on employing GPT-4 for automated assessment in middle school Earth Science, combining few-shot and active learning with chain-of-thought reasoning. Using a human-in-the-loop approach, we successfully score and provide meaningful explanations for formative assessment responses. A systematic analysis of our method's pros and cons sheds light on the potential for human-in-the-loop techniques to enhance automated grading for open-ended science assessments., Comment: In press at EAAI-24: The 14th Symposium on Educational Advances in Artificial Intelligence

The collision dynamics of two drops of the same liquid moving in the same direction has been studied numerically. A wide range of radius ratios of trailing drop and leading drop ($R_r$) and the velocity ratios ($U_r$) have been deployed to understand the collision outcomes. A volume of fluid (VOF) based open-source fluid flow solver, Basilisk, has been used with its adaptive mesh refinement feature to capture the nuances of the interface morphology. The simulations are analyzed for the evolving time instances. Different collision outcomes, such as coalescence and reflexive separation with and without the formation of satellite drops, have been observed for various combinations of $U_r$ and $R_r$. The study analyzes the evolution of kinetic energy and surface energy before and after the collision for plausible outcomes. The collision outcomes are depicted on a regime map with $U_r-R_r$ space, highlighting distinct regimes formed due to variations in relevant governing parameters., Comment: 11 pages, 10 figurs, accepted in Physics of Fluids

We investigate the interactions between two drops in a heated environment and analyze the effect of evaporation on bouncing, coalescence and reflexive separation phenomena. A reliable mass transfer model is incorporated in a coupled level-set and volume-of-fluid framework to accurately model the evaporation process and the evolution of drop interfaces during the interactions. The numerical technique is extensively validated against the benchmark problems involving the evaporation of a single drop. We analyze the contour plots of temperature and vapor mass fraction fields for each collision outcome. Our numerical simulations reveal that vapor entrapment during the separation process, with high-velocity vapor manages to escape. Increasing evaporation rates result in slower post-collision drop separation. Furthermore, the differences in kinetic energy and surface energy are analyzed for different Stefan numbers. The coalescence of drops exhibits energy oscillations until dissipation, while the bouncing and reflexive separations lack such oscillations. In the reflexive separation regime, the kinetic energy of the drops becomes zero after detachment., Comment: 33 pages, 19 figures, Accepted in International Journal of Heat and Mass Transfer

Variable Speed Limit (VSL) control acts as a promising highway traffic management strategy with worldwide deployment, which can enhance traffic safety by dynamically adjusting speed limits according to real-time traffic conditions. Most of the deployed VSL control algorithms so far are rule-based, lacking generalizability under varying and complex traffic scenarios. In this work, we propose MARVEL (Multi-Agent Reinforcement-learning for large-scale Variable spEed Limits), a novel framework for large-scale VSL control on highway corridors with real-world deployment settings. MARVEL utilizes only sensing information observable in the real world as state input and learns through a reward structure that incorporates adaptability to traffic conditions, safety, and mobility, thereby enabling multi-agent coordination. With parameter sharing among all VSL agents, the proposed framework scales to cover corridors with many agents. The policies are trained in a microscopic traffic simulation environment, focusing on a short freeway stretch with 8 VSL agents spanning 7 miles. For testing, these policies are applied to a more extensive network with 34 VSL agents spanning 17 miles of I-24 near Nashville, TN, USA. MARVEL-based method improves traffic safety by 63.4% compared to the no control scenario and enhances traffic mobility by 58.6% compared to a state-of-the-practice algorithm that has been deployed on I-24. Besides, we conduct an explainability analysis to examine the decision-making process of the agents and explore the learned policy under different traffic conditions. Finally, we test the response of the policy learned from the simulation-based experiments with real-world data collected from I-24 and illustrate its deployment capability.

The study analyzed the impact of financial inclusion on the effectiveness of monetary policy in developing countries. By using a panel data set of 10 developing countries during 2004-2020, the study revealed that the financial inclusion measured by the number of ATM per 100,000 adults had a significant negative effect on monetary policy, whereas the other measure of financial inclusion i.e. the number of bank accounts per 100,000 adults had a positive impact on monetary policy, which is not statistically significant. The study also revealed that foreign direct investment (FDI), lending rate and exchange rate had a positive impact on inflation, but only the effect of lending rate is statistically significant. Therefore, the governments of these countries should make necessary drives to increase the level of financial inclusion as it stabilizes the price level by reducing the inflation in the economy.

The dynamics of two vertically coalescing drops and a pool of the same liquid have been investigated using a Coupled Level Set and Volume of Fluid (CLSVOF) method. Such a configuration enables us to study the dynamic interaction of an arbitrary-shaped liquid conglomerate, formed owing to drop-drop coalescence, with a pool. Similar to drop-pool and drop-drop interactions, partial coalescence is observed when a conglomerate interacts with a pool. The presence of the pool below the father drop is found to influence the coalescence characteristic of the two drops. At the same time, the movement of the capillary waves resulting from the interaction of two drops governs the coalescence dynamics of the conglomerate with the pool. As liquid interfaces interact and generate capillary waves at multiple locations, complex trajectories of capillary waves are observed, which play a crucial role in determining the pinch-off characteristics of the satellite during conglomerate-pool interaction. We examine the effect of the ratio of the diameters of the lower/father drop to the upper/mother drop (D_r) on the coalescence dynamics while maintaining the size of the mother drop constant. The variation in the coalescence dynamics due to change in $D_r$ is quantified in terms of the residence time (tau_r), pinch-off time (tau_p) and the satellite diameter to conglomerate diameter ratio (Ds/Dc). The coalescence dynamics of the conglomerate is then compared with that of an equivalent spherical drop of the same volume and also with that of a drop initialized with the same shape as that of the conglomerate. Finally, the regions of complete and partial coalescence for the conglomerate-pool interactions are demarcated on the Weber number - diameter ratio (We-Dr) space., Comment: 36 pages, 14 figures, Accepted in International Journal of Multiphase Flow

The evaporation of a liquid drop of initial diameter (Ddrop) migrating in a tube of diameter (D0) is investigated using the coupled level set and volume of fluid (CLSVOF) method focusing on determining the heat and mass transfer coefficients for a deforming drop. A robust phase change model is developed using an embedded boundary method under a finite difference framework to handle vaporizing flows. The model is extensively validated through simulations of benchmark problems such as arbitrary evaporation of a static drop and reproduction of psychrometric data. The results show that the Sherwood number (Sh) and the Nusselt number (Nu) reach a steady value after an initial transient period for the drop subjected to Hagen-Poiseuille flow. A parametric study is conducted to investigate the effect of drop deformation on the rate of evaporation. It is observed that Stefan flow due to evaporation has a negligible impact on the drop deformation dynamics. We also observed that, for different values of Ddrop/D0, the Sh follows a linear correlation with Re^{1/2}Sc^{1/3}., Comment: 14 pages, 10 figures, ASME Journal of Heat and Mass Transfer (Accepted)

In this work, we present an approach to supervisory reinforcement learning control for unmanned aerial vehicles (UAVs). UAVs are dynamic systems where control decisions in response to disturbances in the environment have to be made in the order of milliseconds. We formulate a supervisory control architecture that interleaves with extant embedded control and demonstrates robustness to environmental disturbances in the form of adverse wind conditions. We run case studies with a Tarot T-18 Octorotor to demonstrate the effectiveness of our approach and compare it against a classic cascade control architecture used in most vehicles. While the results show the performance difference is marginal for nominal operations, substantial performance improvement is obtained with the supervisory RL approach under unseen wind conditions., Comment: In review (2023-05-16)

In this paper, we leverage ideas from model-based control to address the sample efficiency problem of reinforcement learning (RL) algorithms. Accelerating learning is an active field of RL highly relevant in the context of time-varying systems. Traditional transfer learning methods propose to use prior knowledge of the system behavior to devise a gradual or immediate data-driven transformation of the control policy obtained through RL. Such transformation is usually computed by estimating the performance of previous control policies based on measurements recently collected from the system. However, such retrospective measures have debatable utility with no guarantees of positive transfer in most cases. Instead, we propose a model-based transformation, such that when actions from a control policy are applied to the target system, a positive transfer is achieved. The transformation can be used as an initialization for the reinforcement learning process to converge to a new optimum. We validate the performance of our approach through four benchmark examples. We demonstrate that our approach is more sample-efficient than fine-tuning with reinforcement learning alone and achieves comparable performance to linear-quadratic-regulators and model-predictive control when an accurate linear model is known in the three cases. If an accurate model is not known, we empirically show that the proposed approach still guarantees positive transfer with jump-start improvement., Comment: Published to IEEE CoDiT 2023

In this work, we address the problem of solving complex collaborative robotic tasks subject to multiple varying parameters. Our approach combines simultaneous policy blending with system identification to create generalized policies that are robust to changes in system parameters. We employ a blending network whose state space relies solely on parameter estimates from a system identification technique. As a result, this blending network learns how to handle parameter changes instead of trying to learn how to solve the task for a generalized parameter set simultaneously. We demonstrate our scheme's ability on a collaborative robot and human itching task in which the human has motor impairments. We then showcase our approach's efficiency with a variety of system identification techniques when compared to standard domain randomization., Comment: Accepted to ICRA 2022

Interest continues to be high in technology-based interventions for individuals with Autism Spectrum Disorder (ASD). Understanding the preferences and challenges of technology use among individuals with ASD can inform the design of such interventions. Through 18 interviews with parents, we used an iterative inductive-deductive approach to qualitative analysis and explored uses of technology for social skills development among adolescents with ASD. Our findings include parents' observations about their adolescent's preferences in types of technology devices and digital content, as well as both positive and negative effects of technology use on mood and behavior. Parents highlighted several avenues of technological preferences and risks that may inform intervention design, enhance user engagement, and capitalize on users' strengths while buttressing areas for growth. [The paper will be published in "Journal of Autism and Developmental Disorders" (EJ1349623).]

Self-regulated learning (SRL) is a critical 21st -century skill. In this paper, we examine SRL through the lens of the searching, monitoring, assessing, rehearsing, and translating (SMART) schema for learning operations. We use microanalysis to measure SRL behaviors as students interact with a computer-based learning environment, Betty's Brain. We leverage interaction data, survey data, "in situ" student interviews, and supervised machine learning techniques to predict the proportion of time spent on each of the SMART schema facets, developing models with prediction accuracy ranging from "rho" = 0.19 for translating to "rho" = 0.66 for assembling. We examine key interactions between variables in our models and discuss the implications for future SRL research. Finally, we show that both ground truth and predicted values can be used to predict future learning in the system. In fact, the inferred models of SRL outperform the ground truth versions, demonstrating both their generalizability and their potential for using these models to improve adaptive scaffolding for students who are still developing SRL skills. [For the full proceedings, see ED615472.]

Providing adaptive scaffolds to help learners develop self-regulated learning (SRL) processes has been an important goal for intelligent learning environments. Adaptive scaffolding is especially important in open-ended learning environments (OELE), where novice learners often face difficulties in completing their learning tasks. This paper presents a systematic framework for adaptive scaffolding in Betty's Brain, a learning-by-teaching OELE for middle school science, where students construct a causal model to teach a virtual agent, generically named Betty. We evaluate the adaptive scaffolding framework and discuss its implications on the development of more effective scaffolds for SRL in OELEs. We detect key cognitive/metacognitive inflection points, i.e., instances where students' behaviors and performance change as they work on their learning tasks. At such inflection points, Mr. Davis (a mentor agent) or Betty (the teachable agent) provide conversational feedback, focused on strategies to help students become productive learners. We conduct a classroom study with 98 middle schoolers to analyze the impact of adaptive scaffolds on students' learning behaviors and performance. Adaptive scaffolding produced mixed results, with some scaffolds (viz., strategic hints that supported debugging and assessment of causal models) being generally more useful to students than others (viz., encouragement prompts). We also note differences in learning behaviors of High and Low performers after receiving scaffolds. Overall, our findings suggest how adaptive scaffolding in OELEs like Betty's Brain can be further improved to narrow the gap between High and Low performers.

This paper reports initial results from a usability study conducted in the formative and user-centered design phase of a larger project to translate an existing, science-focused educational technology for neurotypical middle school students into a new, social-reasoning-focused educational technology for students on the autism spectrum. Participants in our study included both adolescents on the autism spectrum and typically developing adolescents, who were asked to complete the Betty's Brain educational-technology-based science activity as well as a social-reasoning movie question-answering activity. Results include qualitative observations of general student engagement and challenges as well as quantitative measures of performance and eye gaze, including key differences observed across our two sample groups, with the goal of informing the design and adaptation of future technology-based interventions. Our findings suggest specific considerations for designing educational technologies for adolescents on the autism spectrum, including: (1) finding ways to help students follow instructional/tutorial portions of new technologies, especially when lengthy instructions and/or complex interfaces are involved; (2) proactively anticipating and finding ways to mitigate potential student episodes of frustration / dysregulation while using the technology; and (3) capitalizing on features of the technology found to be engaging/motivating for students. [This paper was published in: "Proceedings of the 42nd Annual Meeting of the Cognitive Science Society," Cognitive Science Society, 2020, pp. 1922-1928.]

This paper discusses an Enhanced Model-Agnostic Meta-Learning (E-MAML) algorithm that generates fast convergence of the policy function from a small number of training examples when applied to new learning tasks. Built on top of Model-Agnostic Meta-Learning (MAML), E-MAML maintains a set of policy parameters learned in the environment for previous tasks. We apply E-MAML to developing reinforcement learning (RL)-based online fault tolerant control schemes for dynamic systems. The enhancement is applied when a new fault occurs, to re-initialize the parameters of a new RL policy that achieves faster adaption with a small number of samples of system behavior with the new fault. This replaces the random task sampling step in MAML. Instead, it exploits the extant previously generated experiences of the controller. The enhancement is sampled to maximally span the parameter space to facilitate adaption to the new fault. We demonstrate the performance of our approach combining E-MAML with proximal policy optimization (PPO) on the well-known cart pole example, and then on the fuel transfer system of an aircraft.

Faults are endemic to all systems. Adaptive fault-tolerant control maintains degraded performance when faults occur as opposed to unsafe conditions or catastrophic events. In systems with abrupt faults and strict time constraints, it is imperative for control to adapt quickly to system changes to maintain system operations. We present a meta-reinforcement learning approach that quickly adapts its control policy to changing conditions. The approach builds upon model-agnostic meta learning (MAML). The controller maintains a complement of prior policies learned under system faults. This "library" is evaluated on a system after a new fault to initialize the new policy. This contrasts with MAML, where the controller derives intermediate policies anew, sampled from a distribution of similar systems, to initialize a new policy. Our approach improves sample efficiency of the reinforcement learning process. We evaluate our approach on an aircraft fuel transfer system under abrupt faults., Comment: Accepted to PHM Conference 2020

A desirable property in fault-tolerant controllers is adaptability to system changes as they evolve during systems operations. An adaptive controller does not require optimal control policies to be enumerated for possible faults. Instead it can approximate one in real-time. We present two adaptive fault-tolerant control schemes for a discrete time system based on hierarchical reinforcement learning. We compare their performance against a model predictive controller in presence of sensor noise and persistent faults. The controllers are tested on a fuel tank model of a C-130 plane. Our experiments demonstrate that reinforcement learning-based controllers perform more robustly than model predictive controllers under faults, partially observable system models, and varying sensor noise levels., Comment: Published in IFAC SAFEPROCESS 2018

We propose a novel adaptive reinforcement learning control approach for fault tolerant control of degrading systems that is not preceded by a fault detection and diagnosis step. Therefore, \textit{a priori} knowledge of faults that may occur in the system is not required. The adaptive scheme combines online and offline learning of the on-policy control method to improve exploration and sample efficiency, while guaranteeing stable learning. The offline learning phase is performed using a data-driven model of the system, which is frequently updated to track the system's operating conditions. We conduct experiments on an aircraft fuel transfer system to demonstrate the effectiveness of our approach., Comment: Published in IFAC World Congress 2020

This paper demonstrates that continual relearning of control policies using incremental deep reinforcement learning (RL) can improve policy learning for non-stationary processes. We demonstrate this approach for a data-driven 'smart building environment' that we use as a test-bed for developing HVAC controllers for reducing energy consumption of large buildings on our university campus. The non-stationarity in building operations and weather patterns makes it imperative to develop control strategies that are adaptive to changing conditions. On-policy RL algorithms, such as Proximal Policy Optimization (PPO) represent an approach for addressing this non-stationarity, but exploration on the actual system is not an option for safety-critical systems. As an alternative, we develop an incremental RL technique that simultaneously reduces building energy consumption without sacrificing overall comfort. We compare the performance of our incremental RL controller to that of a static RL controller that does not implement the relearning function. The performance of the static controller diminishes significantly over time, but the relearning controller adjusts to changing conditions while ensuring comfort and optimal energy performance.