Automatic Classification of Difficulty of Texts From Eye Gaze and Physiological Measures of L2 English Speakers

Reading is an essential method for adults to learn new languages, but difficulty reading texts in a foreign language can increase learners’ anxiety. Identifying text difficulty from the reader’s perspective can aid language learning by tailoring texts to readers’ needs. There is little research focusing on L2 speakers or using a multimodal approach, i.e., using multiple sensors, to detect subjective difficulty. In this study ( $N=30$ ) we determined L2 speakers’ subjective difficulty while reading using language proficiency and objective text difficulty, combined with physiological data. We compared machine learning classifiers combining eye, skin and heart sensor data against models using each modality separately. Additionally, we assessed the effect on model performance of shifting the data to account for delayed physiological responses. The models detected 3 levels of subjective difficulty (low, medium, high) and were evaluated using leave-one-participant-out (LoPo) and leave-one-document-out (LoDo) cross-validation. The results showed acceptable levels of generalization to new participants ( $Acc_{LoPo} = 0.434$ ) and documents ( $Acc_{LoDo} = 0.521$ ). Combining sensor data from all modalities improved predictions in both LoDo and LoPo cross-validation, compared to each modality in isolation. Shifting the data to account for physiological response delay did not improve model performance compared to not shifting the data. These findings support refining subjective difficulty detection models and their implementation in adaptive language learning systems. Finally, this work contributes to the field of cognitive science and technology by laying the foundation for innovative approaches to cognitive state detection.