Deep rhythm and long short term memory-based drowsiness detection.

• An automated Deep Network based drowsiness detection system has been developed. • In this study, the input EEG signals converted into time-frequency images by using Short-Time-Fourier-Transform (STFT). • A novel Deep Neural Network model based on CNN-LSTM including Multi-rhythms is presented. • The proposed model achieved 97.92% classification accuracy for the detection of drowsiness based on EEG recordings. In this paper, a deep-rhythm-based approach is proposed for the efficient detection of drowsiness based on EEG recordings. In the proposed approach, EEG images are used instead of signals where the time and frequency information of the EEG signals are incorporated. The EEG signals are converted to EEG images using the time-frequency transformation method. The Short-Time-Fourier-Transform (STFT) is used for this transformation due to its simplicity. The rhythm images are then extracted by dividing the EEG images based on frequency intervals. EEG signals contain five rhythms, namely Delta rhythm (0–4 Hz), Theta rhythm (4–8 Hz), Alpha rhythm (8–12 Hz), Beta rhythm (12–30 Hz), and Gamma rhythm (30–50 Hz). From each rhythm image, deep features are extracted based on a pre-trained convolutional neural network (CNN) model, with pre-trained residual network (ResNet) models such as ResNet18, ResNet50, and ResNet101. The obtained deep features from each rhythm image are fed into the Long-Short-Term-Memory (LSTM) layer, and the LSTM layers are then sequentially connected to each other. After the last LSTM layer, a fully-connected layer, a softmax layer, and a classification layer are employed in order to detect the class labels of the input EEG signals. Various experiments were conducted with the MIT/BIH Polysomnographic Dataset. The experiments showed that the concatenated ResNet features achieved an accuracy score of 97.92%. The obtained accuracy score was also compared with the state-of-the-art scores and, to the best of our knowledge, the proposed method achieved the best accuracy score among the methods compared. [ABSTRACT FROM AUTHOR]