Search

The widespread shift in higher education (HE) from in-person instruction to pre-recorded video lectures means that many instructors have lost access to real-time student feedback for the duration of any given lecture (a ‘sea of faces’ that express struggle, comprehension, etc.). We hypothesized that this feedback could be partially restored by analyzing student facial movement data gathered during recorded lecture viewing and visualizing it on a common lecture timeline. Our approach builds on computer vision research on engagement and affect in facial expression, and education research on student struggle. Here, we focus on individual student struggle (the effortful attempt to grasp new concepts and ideas) and its group-level visualization as student feedback to support human instructors. Research suggests that instructor supported student struggle can help students develop conceptual understanding, while unsupported struggle can lead to disengagement. Studies of online learning in higher education found that when students struggle with recorded video lecture content, questions and confusion often remain unreported and thus unsupported by instructors. In a pilot study, we sought to identify group-level student struggle by analyzing individual student facial movement during asynchronous video lecture viewing and mapping cohort data to annotated lecture segments (e.g. when a new concept is introduced). We gathered real-time webcam data of 10 student participants and their self-paced intermittent click feedback on personal struggle state, along with retrospective self-reports. We analyzed participant video with computer vision techniques to identify facial movement and correlated the data with independent human observer inferences about struggle-related states. We plotted all participants’ data (computer vision analysis, self-report, observer annotation) along the lecture timeline. The visualization exposed group-level struggle patterns in relation to lecture content, which could help instructors identify content areas where students need additional support, e.g. through student-centered interventions or lecture revisions.

This paper offers a formal account of emotional inference and stress-related behaviour, using the notion of active inference. We formulate responses to stressful scenarios in terms of Bayesian belief-updating and subsequent policy selection; namely, planning as (active) inference. Using a minimal model of how creatures or subjects account for their sensations (and subsequent action), we deconstruct the sequences of belief updating and behaviour that underwrite stress-related responses – and simulate the aberrant responses of the sort seen in post-traumatic stress disorder (PTSD). Crucially, the model used for belief-updating generates predictions in multiple (exteroceptive, proprioceptive and interoceptive) modalities, to provide an integrated account of evidence accumulation and multimodal integration that has consequences for both motor and autonomic responses. The ensuing phenomenology speaks to many constructs in the ecological and clinical literature on stress, which we unpack with reference to simulated inference processes and accompanying neuronal responses. A key insight afforded by this formal approach rests on the trade-off between the epistemic affordance of certain cues (that resolve uncertainty about states of affairs in the environment) and the consequences of epistemic foraging (that may be in conflict with the instrumental or pragmatic value of ‘fleeing’ or ‘freezing’). Starting from first principles, we show how this trade-off is nuanced by prior (subpersonal) beliefs about the outcomes of behaviour – beliefs that, when held with unduly high precision, can lead to (Bayes optimal) responses that closely resemble PTSD.

The enormous range of possibilities for digital musical instrument (DMI) design is often limited by the adoption of unnecessary conceptual constraints. When considered in relation to DMIs, a careful analysis of the underlying concepts makes it possible to reject certain assumptions and thereby to expand the current range of acceptable possibilities for future designs.

The field of behaviour-based artificial intelligence (AI), with its roots in the robotics research of Rodney Brooks, is not predominantly tied to linguistic interaction in the sense of the classic Turing test (or, "imitation game"). Yet, it is worth noting, both are centred on a behavioural model of intelligence. Similarly, there is no intrinsic connection between musical AI and the language-based Turing test, though there have been many attempts to forge connections between them. Nonetheless, there are aspects of musical AI and the Turing test that can be considered in the context of non-language-based interactive environments–-in particular, when dealing with real-time musical AI, especially interactive improvisation software. This paper draws out the threads of intentional agency and human indistinguishability from Turing’s original 1950 characterisation of AI. On the basis of this distinction, it considers different approaches to musical AI. In doing so, it highlights possibilities for non-hierarchical interplay between human and computer agents.