Search

A common but easily overlooked affective overlap problem has not been received enough attention in electroencephalogram (EEG)-based emotion recognition research. In real life, affective overlap refers to the current emotional state of human being is sometimes influenced easily by his/her historical mood. In stimulus-evoked EEG collection experiment, due to the short rest interval in consecutive trials, the inner mechanisms of neural responses make subjects cannot switch their emotion state easily and quickly, which might lead to the affective overlap. For example, we might be still in sad state to some extent even if we are watching a comedy because we just saw a tragedy before. In pattern recognition, affective overlap usually means that there exists the feature-label inconsistency in EEG data. To alleviate the impact of inconsistent EEG data, we introduce a variable to adaptively explore the sample inconsistency in emotion recognition model development. Then, we propose a semi-supervised emotion recognition model for joint sample inconsistency and feature importance exploration (SIFIAE). Accordingly, an efficient optimization method to SIFIAE model is proposed. Extensive experiments on the SEED-V dataset demonstrate the effectiveness of SIFIAE. Specifically, SIFIAE achieves 69.10%, 67.01%, 71.50%, 73.26%, 72.07% and 71.35% average accuracies in six cross-session emotion recognition tasks. The results illustrated that the sample weights have a rising trend in the beginning of most trials, which coincides with the affective overlap hypothesis. The feature importance factor indicated the critical bands and channels are more obvious compared with some models without considering EEG feature-label inconsistency. • Affective overlap problem was considered in emotion recognition. • Feature-label inconsistency and feature importance were utilized jointly. • SIFIAE obtained a significant improvement in cross-session emotion recognition. • Obtained sample inconsistency demonstrated affective overlap hypothesis. • Affective activation pattern could be adaptively obtained in our model. [ABSTRACT FROM AUTHOR]

Due to its advantages of high temporal and spatial resolution, the technology of simultaneous electroencephalogram-functional magnetic resonance imaging (EEG-fMRI) acquisition and analysis has attracted much attention, and has been widely used in various research fields of brain science. However, during the fMRI of the brain, ballistocardiogram (BCG) artifacts can seriously contaminate the EEG. As an unpaired problem, BCG artifact removal now remains a considerable challenge. Aiming to provide a solution, this paper proposed a novel modular generative adversarial network (GAN) and corresponding training strategy to improve the network performance by optimizing the parameters of each module. In this manner, we hope to improve the local representation ability of the network model, thereby improving its overall performance and obtaining a reliable generator for BCG artifact removal. Moreover, the proposed method does not rely on additional reference signal or complex hardware equipment. Experimental results show that, compared with multiple methods, the technique presented in this paper can remove the BCG artifact more effectively while retaining essential EEG information. • A novel GAN-based model is designed (BCGGAN) to remove the BCG artifact in simultaneous EEG-fMRI. • A modular training strategy is proposed to optimize the generator network in the BCGGAN model. • The proposed method does not require additional hardware or reference signal, such as carbon fiber sling or ECG signals. • The proposed method can remove the BCG artifact more effectively while retaining useful physiological information. [ABSTRACT FROM AUTHOR]

Background. Rapid serial visual presentation (RSVP) based brain-computer interface (BCI) is widely used to categorize the target and non-target images. The available information limits the prediction accuracy of single-trial using single-subject electroencephalography (EEG) signals. New Method. Hyperscanning is a new manner to record two or more subjects' signals simultaneously. So we designed a multi-level information fusion model for target image detection based on dual-subject RSVP, namely HyperscanNet. The two modules of this model fuse the data and features of the two subjects at the data and feature layers. A chunked long and short-term memory artificial neural network (LSTM) was used in the time dimension to extract features at different periods separately, completing fine-grained underlying feature extraction. While the feature layer is fused, some plain operations are used to complete the fusion of the data layer to ensure that important information is not missed. Results. Experimental results show that the F1-score (the harmonic mean of precision and recall) of this method with best group of channels and segment length is 82.76%. Comparison with existing methods. This method improves the F1-score by at least 5% compared to single-subject target detection. Conclusions. Target detection can be accomplished by the two subjects' collaboration to achieve a higher and more stable F1-score than a single subject. [Display omitted] • The ratio between target and non-target images is 1:9. • Designed a multi-level information fusion model for target image detection based on dual-subject RSVP. • Accomplished higher and more stable F1-score than single subject. [ABSTRACT FROM AUTHOR]

Emotions play a crucial role in human communication and affect all aspects of human life. However, to date, there have been few studies conducted on how movements under different emotions influence human brain activity and cortico-muscular coupling (CMC). In this study, for the first time, electroencephalogram (EEG) and electromyogram physiological electrical signals were used to explore this relationship. We performed frequency domain and nonlinear dynamics analyses on EEG signals and used transfer entropy to explore the CMC associated with the emotion-movement relationship. To study the transmission of information between different brain regions, we also constructed a functional brain network and calculated various network metrics using graph theory. We found that, compared with a neutral emotional state, movements made during happy and sad emotions had increased CMC strength and EEG power and complexity. The functional brain network metrics of these three emotional states were also different. Much of the emotion-movement relationship research has been based on subjective expression and external performance. Our research method, however, focused on the processing of physiological electrical signals, which contain a wealth of information and can objectively reveal the inner mechanisms of the emotion-movement relationship. Different emotional states can have a significant influence on human movement. This study presents a detailed introduction to brain activity and CMC. • Combine EEG and EMG to study the relationship between emotion and movement. • Movement associated with happy emotions can increase the complexity of the left hemisphere. • Movement associated with sad emotions can increase the complexity of the right hemisphere. • Increased grip strength leads to increased the CMC strength of EEG→EMG. • The network topology of movement under different emotions is significantly different. [ABSTRACT FROM AUTHOR]

Sparse representation-based classification (SRC) has more advantages in motor imagery EEG pattern recognition, and the quality of dictionary construction directly determines the performance of SRC. In this paper, we proposed a two-dimensional dictionary optimization (TDDO) method to directly improve the performance of SRC. Firstly, an initial dictionary was constructed with multi-band features extracted by filter band common spatial pattern (FBCSP). Then Lasso regression is used to select significant features in each atom synchronously in the horizontal direction, and the KNN-based method is used to clean up noise atoms in the vertical direction. Finally, an SRC method by training samples linearly representing test samples was implemented in classification. The results show the necessity and rationality of TDDO-SRC method. The highest average classification accuracy of 86.5% and 92.4% is obtained on two public datasets. The proposed method has more superior classification accuracy compared to traditional methods and existing winners' methods. The quality of dictionary construction has a great impact on the robustness of SRC. And compared with the original SRC, the classification accuracy of the optimized TDDO-SRC is greatly improved. • The selected features are distributed in frequency bands relevant to the MI tasks. • The atoms cleaning method satisfies the principle of dictionary construction. • A novel two-dimensional dictionary optimization algorithm is proposed. • Compared with SRC, TDDO-SRC significantly improves the classification accuracy. [ABSTRACT FROM AUTHOR]