Search

The availability of educational data in novel ways and formats brings new opportunities to students with special education needs (SEN), whose behaviour and learning are highly sensitive to their body conditions and surrounding environments. Multimodal learning analytics (MMLA) captures learner and learning environment data in various modalities and analyses them to explain the underlying educational insights. In this work, we applied MMLA to predict SEN students’ behaviour change upon their participation in applied behaviour analysis (ABA) therapies, where ABA therapy is an intervention in special education that aims at treating behavioural problems and fostering positive behaviour changes. Here we show that by inputting multimodal educational data, our machine learning models and deep neural network can predict SEN students’ behaviour change with optimum performance of 98% accuracy and 97% precision. We also demonstrate how environmental, psychological, and motion sensor data can significantly improve the statistical performance of predictive models with only traditional educational data. Our work has been applied to the Integrated Intelligent Intervention Learning (3I Learning) System, enhancing intensive ABA therapies for over 500 SEN students in Hong Kong and Singapore since 2020.

Abstract Background Autism spectrum disorder (ASD) is a developmental impairment characterized by persistent deficits in social communication and interactions, and over half of children with ASD possess below average intellectual ability (IQ

Previous studies have shown that graphic-based augmentative and alternative communication (AAC) output tend to be short and simple in structure with non-canonical word order, and that AAC users may show differences when communicating with peers compared to professionals such as speech therapists (STs). However, there was a lack of report for graphic-based AAC in the Chinese context, and the effect of communication partners had not been investigated systematically. In this study with 34 AAC users and 10 STs, we reported common and distinct features of free conversations in Cantonese graphic-based AAC, relative to AAC in other languages. We also found that AAC users were sensitive to different types of communication partners. In particular, when conversing with peers, AAC users produced long messages with equal proportion of questions and responses, which suggested active and bi-directional exchanges. In conversations with STs, AAC users showed high diversity in expressive vocabulary, indicating access to more semantic concepts. Results suggested that the base language and the communication partner are both influential factors that should be considered in studies of graphic-based AAC. The mobile AAC system facilitated free conversations in users with complex communication needs, affording an additional channel for social participation.

Applied behavior analysis (ABA) has become a popular behavioral therapy in the special education needs (SEN) community. ABA is used to manage SEN students’ behaviors by solving problems in socially important settings, and puts emphasis on having precise measurements on physical and observable events. In this work, we present how Internet of Things (IoT) technologies can be applied to enhance ABA therapy in normal SEN classroom settings. We measured (1) learning performance data, (2) learners’ physiological data, and (3) learning environment sensors’ data. Upon preliminary analysis, we have found that learners’ physiological data is highly diverse, while learner performance seems to be related to learners’ electrodermal activity. Our preliminary findings suggest the possibility of enhancing ABA for SEN with IoT technologies.