Search

• A novel dynamic graph attention network is proposed for EEG emotion recognition. • The high accuracy can be obtained from the DGAT model in EEG emotion recognition. • The channel connectivity can reflect the synergistic relationships within the brain. Deep learning classification models based on electroencephalogram (EEG) emotion recognition have demonstrated considerable proficiency in the categorization of emotional states. However, these models have limitations in their capability to analyze the active states and cooperative relationships among distinct brain regions. This study proposes a dynamic graph attention network (DGAT) for EEG emotion recognition, which learns the features of each channel and leverages multiple-head self-attention mechanisms to capture non-Euclidean relationships between channels. Then, we use differential entropy features of emotions signals on the SJTU emotion EEG dataset (SEED). The DGAT model achieved improved subject-dependent and cross-subject classification accuracy compared to previous models. Moreover, ablation studies show that the channel weight matrix(CWM) and appropriate hyper-parameters can improve the performance of the DGAT model significantly. Furthermore, by conducting interpretable analysis of the new connections and electrode weights learned by the model, we find that these connection weight relationships reflect a certain degree of coordination within the brain for EEG-based emotion recognition. These findings provide a new method for EEG emotion recognition and highlights the potential for using deep learning models to analyze the active state and synergistic relationships among brain regions. [ABSTRACT FROM AUTHOR]

Electroencephalography (EEG), as a physiological cue, is more objective and reliable in identifying emotions than non-physiological cues. Previous methods only consider one or two relationships among frequency, time and spatial domain features of EEG signals, and the designed models may still be relatively large in terms of parameters. Meanwhile, the training process of the previous networks is troublesome during algorithm optimization. To address these challenges, we design a simple and efficient feature preprocessing method to obtain a 3D feature structure that contains EEG signal information in the frequency, time and spatial domains simultaneously. Then, we propose a multiscale frequency–time–spatial convolutional model, MS-FTSCNN, which is able to capture frequency, time and spatial features from the input signals and fuse three features more efficiently. Moreover, the multi-scale one-dimensional convolutional kernel in our method can reduce network parameters, providing possibilities for real-time online applications. Finally, the recognition accuracies of arousal and valence of our proposed model are 93.82%, 94.48% on DEAP dataset and 92.64%, 92.15% on MOHNOB-HCI dataset, which is higher than most existing methods. • We put a novel method for EEG feature processing to get a 3D feature map which incorporates frequency, temporal and spatial feature of EEG signals simultaneously. • We design a network (MS-FTSCNN) to capture and fuse the frequency, time and space domain features of EEG signals with good generalization ability. • Through comprehensive experiments, we analyze the influence of different features on the final classification results of our model. [ABSTRACT FROM AUTHOR]

Electroencephalogram (EEG)-based emotion recognition is getting a lot of attention in Brain–Computer Interface (BCI) research. However, the recognition model of cross-subject emotions requires effectively represented EEG data to capture diverse features corresponding to the spatial, frequency and temporal of information. In contrast, by calculating Power Spectral Density (PSD), Differential Entropy (DE), and the spaced interpolation algorithm, brain map construction has the ability to enhance the quality of frequency and spatial data features. In addition, DEAP is a dimensional dataset. We used the multi-task learning method while considering the characteristics of the dimensional dataset. Therefore, we proposed a novel hybrid emotion recognition deep learning network called the multi-task hybrid emotion recognition network(MTHNet), which fuses Bi-LSTM(Bidirectional long short-term memory) and improved UNet. In the hybrid network, improved UNet is utilized to extract spatial features. Then Bi-LSTM cells are applied to extract temporal features and memorize the change of sequential data in forward and backward directions. The proposed method is applied to the well-known 5 public databases and has given outperforming results than the current state of the art. • The proposed method can capture heterogeneous features representing spatial, temporal and frequency of EEG. • The improved UNet address model degradation during cross-subject training. • The size of the time window affects the connection between various EEG segments. • The spatial interpolation algorithm makes the brain map less affected by electrode position. • Conversion method from single-task to multi-task is proposed for single task or discrete datasets. [ABSTRACT FROM AUTHOR]

In the past, Electroencephalography (EEG)-based emotion identification research mainly focused on healthy, disordered, or depressed subjects. Due to the lack of an emotion acquisition and expression channel, there may be a bias in affective cognition for the hearing impaired. In this work, 15 hearing-impaired subjects were recruited to participate in an emotional movie-induced experiment, while their EEG signals were collected for emotion recognition. Four types of movie clips, which contained happiness, calmness, sadness, and fear, were selected. We fused three kinds of features, differential entropy (DE), power spectral density (PSD), and functional brain network to integrate the frequency domain and the spatial domain information of EEG signals. To find the brain region connections of different emotions, different coupling methods, and binarization methods were explored. Moreover, the combination of different brain network attributes was used to find the representativeness of emotional information in the spatial domain. The experimental results show that the combination of phase-locked value (PLV) & 20% sparsity with four brain network attributes (clustering coefficient (CC), node degree (ND), node betweenness centrality (NBC), and node strength (NS)) achieved a higher accuracy of 94.17% for functional brain network-based emotion recognition. In addition, the method of feature fusion obtained a classification accuracy of 95.91% with an SVM-Linear classifier. Moreover, by constructing functional brain network, the key connectivity paths of different emotions are explored, which may provide insights into understanding the emotional processing of the hearing-impaired. [ABSTRACT FROM AUTHOR]

Physiological signals can more realistically reflect human emotional states. To overcome the limitations imposed in single-modal emotion recognition, emotion recognition of multimodal physiological signals has received increasingly widespread attention. However, the original fusion models usually ignore the different distributions of multiple signals and how to capture complementary features from multimodal information effectively. This paper proposes an effective classification model for multimodal physiological signals to address the above issues based on modeling the heterogeneity and correlation between multimodal signals. First, differential entropy features are extracted from Electroencephalography (EEG) signals and peripheral physiological signals (PPS) such as Electrocardiographic (ECG) signals, Electromyographic (EMG) signals, and other physiological signals. Then, according to the different distributions and frequency characteristics of the acquired signals, the EEG signal features are made into a three-dimensional feature map and input to the neural network to extract the frequency spatial dimension features. Further temporal features are extracted from the peripheral physiological signals using a long and short-term memory network. Finally, the EEG and peripheral physiological signal features were fused and input to a multimodal long and short-term memory network to extract the association between different modalities and perform classification. The experiments were conducted on the benchmark DEAP dataset, and the results showed that the classification accuracy of the proposed model in this paper was 95.89% and 94.99% in the arousal dimension and the valence dimension, respectively, which were 2.77% and 3.11% higher compared to the unimodal EEG model, respectively. This paper also analyzed the effects of different peripheral physiological signals on emotion recognition. [ABSTRACT FROM AUTHOR]

• A global to local feature aggregation network is put forward for EEG emotion recognition. • An effective topological graph construction method is put forward to map the actual EEG electrode positions into a spatial coordinate system. • Experimental validation of the model is carried out in the emotion datasets, the proposed algorithm obtains good performance. Recently, emotion recognition technology based on electroencephalogram (EEG) signals is widely used in areas such as human–computer interaction and disease diagnosis. Traditional deep learning models rarely focus on the topological features of EEG electrodes, and often focus only on the local features of EEG signals, which makes it difficult to enhance the effectiveness of emotion recognition. In order to improve the accuracy and robustness of EEG-based emotion recognition algorithms, we propose an EEG emotion recognition algorithm based on a global to local feature aggregation network (GLFANet). This algorithm firstly uses the spatial location of the channels of EEG signals and the frequency domain features of each channel to construct an undirected topological graph to represent the spatial connection relationship between channels. Then, the GLFANet can learn deeper features of the undirected topology graph for emotion recognition. GLFANet mainly consists of a global learner composed of multiple graph convolution blocks and a local learner composed of multiple convolution blocks, which can learn both global and local features of EEG signals. The experiment results show that the proposed algorithm achieves higher accuracy on DEAP, SEED and DREAMER contrasted to other advanced algorithms. [ABSTRACT FROM AUTHOR]

• The predicted pseudo-labels of samples were used to obtain subdomains in the target domain. • Differential Entropy (DE) features extracted from various frequency bands were represented as a set of characteristic matrixes. • Subdomain Associate Loop (SAL) was proposed as a domain adaptation loss criterion. Developing robust cross-subject or cross-session EEG-based affective models is a key issue in affective brain-computer interfaces, which often suffer from the individual differences and non-stationarity of EEG. Aiming at generalizing the affective model across subjects and sessions, this paper proposes a novel transfer learning strategy with Deep Subdomain Associate Adaptation Network (DSAAN) for EEG emotion recognition. Domain was divided into subdomains according to the sample labels, and the source domain use the true sample labels while the target domain use the predicted pseudo-labels. DSAAN was established as a transfer network by aligning the relevant subdomain distributions based on Subdomain Associate Loop (SAL). The adaptation of networks was achieved by minimizing the summation of source domain classification loss and SAL loss. For the purpose of verifying the generalization of DSAAN, we carried out the cross-session and cross-subject EEG emotion recognition experiments on benchmark SEED and DEAP. Compared with existing domain adaptation methods, the DSAAN achieved outstanding classification results. [ABSTRACT FROM AUTHOR]

• Provide DRS-Net, an end-to-end affective computing model using multichannel EEG data. • Automatically extract the EEG data' spatial–temporal features with a dynamic reservoir state encoder. • Integrating Reservoir Computing with the neural network to deal with EEG data processing. Affective computing based on electroencephalography (EEG) is a promising field that highly integrates research and technology. A critical challenge is effectively extracting and integrating the temporal and spatial information to form a better representation for multichannel EEG data. Most existing studies use hand-selected features from each channel, which neglect high-dimensional dynamic temporal features and interplay of data from different electrodes. This study proposed a Dynamic Reservoir State Network (DRS-Net) to recognize the subject's emotional states. The novel end-to-end model constructs a dynamic reservoir state encoder to extract multi-channel EEG data's dynamic high dimension non-linear spatial–temporal information with high speed and low complexity. Then, a Long-Short Term Memory-dense decoder model is devised to detect emotional states. The effectiveness of the proposed DRS-Net model was evaluated on SEED, SEED-IV, and DEAP datasets. To validate the performance of the proposed method, we first combined the hand-selected features (differential entropy, power spectra density, fractal dimension, and statistics features) and classic machine learning classifiers methods (support vector machine, random forest, and k-nearest neighbor). Then, we compare them with the proposed method and other state-of-the-art deep learning methods. The experimental results generated by our method outperform all other methods in terms of accuracy and F1 score. [ABSTRACT FROM AUTHOR]

• Subject contingent and noncontingent experiment are conducted on SEED and DEAP database for emotion detection. • Proposed LF-D f E feature extraction method detects nonlinearity and non Gaussianity of EEG signal. • BiLSTM network is adapted to learn spatial features from different brain region and temporal dependency of the EEG signal. • The experimental results using LF-D f E and BiLSTM network outperform the classical methods. This paper proposes emotion detection using Electroencephalography (EEG) signal based on Linear Formulation of Differential Entropy (LF-D f E) feature extractor and BiLSTM network classifier. LF-D f E effectively detects nonlinearity and non-Gaussianity of the EEG signal. BiLSTM network captures long term dependency of the EEG signal and learns spatial information from different brain regions. Proposed model is used to discriminate positive, negative, and neutral emotions on SEED database, valence and arousal on DEAP database. To assess the proposed model subject contingent, noncontingent and inter-dependent (cross-session) experiments are performed on the SEED database. The average accuracy of emotion detection on SEED database for subject contingent approach is improved by 4.12 %, for noncontingent approach by 4.5 % and for inter-dependent approach it is improved by 1.3 %. To reconfirm the above findings, one more experiment is conducted for subject noncontingent approach on DEAP database. On DEAP database for subject noncontingent experiment average accuracy is improved by 7.04 %. Experimental results of the proposed feature extractor LF-D f E with the BiLSTM network found to be improved over existing methods. [ABSTRACT FROM AUTHOR]