Back to Search Start Over

CoRe-Sleep: A Multimodal Fusion Framework for Time Series Robust to Imperfect Modalities

CoRe-Sleep: A Multimodal Fusion Framework for Time Series Robust to Imperfect Modalities

Authors :
Konstantinos Kontras
Christos Chatzichristos
Huy Phan
Johan Suykens
Maarten De Vos
Source :
IEEE Transactions on Neural Systems and Rehabilitation Engineering, Vol 32, Pp 840-849 (2024)
Publication Year :
2024
Publisher :
IEEE, 2024.

Abstract

Sleep abnormalities can have severe health consequences. Automated sleep staging, i.e. labelling the sequence of sleep stages from the patient’s physiological recordings, could simplify the diagnostic process. Previous work on automated sleep staging has achieved great results, mainly relying on the EEG signal. However, often multiple sources of information are available beyond EEG. This can be particularly beneficial when the EEG recordings are noisy or even missing completely. In this paper, we propose CoRe-Sleep, a Coordinated Representation multimodal fusion network that is particularly focused on improving the robustness of signal analysis on imperfect data. We demonstrate how appropriately handling multimodal information can be the key to achieving such robustness. CoRe-Sleep tolerates noisy or missing modalities segments, allowing training on incomplete data. Additionally, it shows state-of-the-art performance when testing on both multimodal and unimodal data using a single model on SHHS-1, the largest publicly available study that includes sleep stage labels. The results indicate that training the model on multimodal data does positively influence performance when tested on unimodal data. This work aims at bridging the gap between automated analysis tools and their clinical utility.

Details

Language :
English
ISSN :
15580210
Volume :
32
Database :
Directory of Open Access Journals
Journal :
IEEE Transactions on Neural Systems and Rehabilitation Engineering
Publication Type :
Academic Journal
Accession number :
edsdoj.2c5503d35c61408b9d19c82df55cad95
Document Type :
article
Full Text :
https://doi.org/10.1109/TNSRE.2024.3354388