Convolutional neural network with transfer learning approach for detection of unfavorable driving state using phase coherence image.

• The combination of deep learning model and functional brain network is effective. • Using deep learning approach, there is no need for hand-crafted feature steps. • Pre-trained deep learning model achieves better results on a smaller training set. Driving in an unfavorable driving state (UDS) is a safety issue and also an important cause of frequent traffic accidents. Finding corresponding countermeasures to decrease traffic accident rates and improve public road safety is critical but challenging. For the purpose of drawing on deep learning (DL) or transfer learning approach to automatically find features in images without hand-crafted feature extraction steps to re-mine the new synchronization and interaction mode between brain regions. In this study, the acquired multi-channel EEG records while performing driving experiments are innovatively converted into a two-dimensional phase coherence matrix (PC image), which is used as an input to a deep learning network (convolutional neural network) to accomplish the combination of deep learning models and functional brain network methods for the detection of UDS. With the motivation of enhancing the classification performance, two approaches, namely the extract image features approach using pre-trained deep learning models (Alexnet and Resnet 18) combined with support vector machine (SVM) and the transfer learning method based on pre-trained networks after fine-tuning are evaluated, respectively. The experimental results indicate that a highest accuracy of 98.44%, highest sensitivity of 100%, highest specificity of 97.02% and highest precision of 96.80% are obtained for the detection of driving-related unfavorable states by using the extract image features approach based on Alexnet network with the 'relu4′ layer activation combined with SVM. Additionally, transfer learning method based on Resnet 18 network after fine-tuning with a learning rate of 1e-3 and adam solver achieves an average classification accuracy of 97.90% outperforming other transfer learning methods considered in this work. It can be hence concluded that the innovative combination method of functional brain network and deep learning is effective for unfavorable driving state detection. [ABSTRACT FROM AUTHOR]