Search

Neuroelectric signals recorded by micro-electrodes reflect the spontaneous and rhythmic activities of brain neurons. Numerous deep learning frameworks have been designed for various neuroelectric signal decoding tasks, most of which are based on convolutional neural network (CNN) and recurrent neural network (RNN). However, neither CNNs or RNNs can perceive the global dependencies of neural activities in both time and channel dimensions. To address this issue, this paper presents a temporal-channel cascaded transformer network to decode the neural activities of imagined handwriting movements, which can perform imagined handwriting character recognition from spiking activity recorded by two micro-electrode arrays (MEAs). Specifically, we design a temporal-channel cascaded framework and a dense residual transformer encoder structure, which can promote the hierarchical learning and fusion of the temporal and channel features. In addition, a mutual learning strategy of multiple class tokens is proposed to improve classification performance. We conduct performance evaluation experiments on a single-character handwriting-imagination dataset and a sentence handwriting-imagination dataset, which are collected from the public Handwriting BCI dataset. The comparison results demonstrate the superiority of the proposed framework and strategy. Especially in the imagined single-character recognition task, the recognition accuracy of our model can achieve 95.78%, which provides an improvement of + 2 % over the existing state of the art models. [ABSTRACT FROM AUTHOR]

• A bidirectional distillation strategy is proposed to realize the information interaction between hybrid networks. • BIHN optimizes the performance of both CogN and ComN in EEG cognitive recognition by using a bidirectional feedback mode. • Experimental results show the superior performance of BIHN in cognitive recognition compared to many SOTA models. • The BIHN architecture is suitable for various hybrid network pairs related to cognitive and computational networks. Background and objective: Extracting cognitive representation and computational representation information simultaneously from electroencephalography (EEG) data and constructing corresponding information interaction models can effectively improve the recognition capability of brain cognitive status. However, due to the huge gap in the interaction between the two types of information, existing studies have yet to consider the advantages of the interaction of both. Methods: This paper introduces a novel architecture named the bidirectional interaction-based hybrid network (BIHN) for EEG cognitive recognition. BIHN consists of two networks: a cognitive-based network named CogN (e.g., graph convolution network, GCN; capsule network, CapsNet) and a computing-based network named ComN (e.g., EEGNet). CogN is responsible for extracting cognitive representation features from EEG data, while ComN is responsible for extracting computational representation features. Additionally, a bidirectional distillation-based coadaptation (BDC) algorithm is proposed to facilitate information interaction between CogN and ComN to realize the coadaptation of the two networks through bidirectional closed-loop feedback. Results: Cross-subject cognitive recognition experiments were performed on the Fatigue-Awake EEG dataset (FAAD, 2-class classification) and SEED dataset (3-class classification), and hybrid network pairs of GCN + EEGNet and CapsNet + EEGNet were verified. The proposed method achieved average accuracies of 78.76% (GCN + EEGNet) and 77.58% (CapsNet + EEGNet) on FAAD and 55.38% (GCN + EEGNet) and 55.10% (CapsNet + EEGNet) on SEED, outperforming the hybrid networks without the bidirectional interaction strategy. Conclusions: Experimental results show that BIHN can achieve superior performance on two EEG datasets and enhance the ability of both CogN and ComN in EEG processing as well as cognitive recognition. We also validated its effectiveness with different hybrid network pairs. The proposed method could greatly promote the development of brain-computer collaborative intelligence. [ABSTRACT FROM AUTHOR]

A common but easily overlooked affective overlap problem has not been received enough attention in electroencephalogram (EEG)-based emotion recognition research. In real life, affective overlap refers to the current emotional state of human being is sometimes influenced easily by his/her historical mood. In stimulus-evoked EEG collection experiment, due to the short rest interval in consecutive trials, the inner mechanisms of neural responses make subjects cannot switch their emotion state easily and quickly, which might lead to the affective overlap. For example, we might be still in sad state to some extent even if we are watching a comedy because we just saw a tragedy before. In pattern recognition, affective overlap usually means that there exists the feature-label inconsistency in EEG data. To alleviate the impact of inconsistent EEG data, we introduce a variable to adaptively explore the sample inconsistency in emotion recognition model development. Then, we propose a semi-supervised emotion recognition model for joint sample inconsistency and feature importance exploration (SIFIAE). Accordingly, an efficient optimization method to SIFIAE model is proposed. Extensive experiments on the SEED-V dataset demonstrate the effectiveness of SIFIAE. Specifically, SIFIAE achieves 69.10%, 67.01%, 71.50%, 73.26%, 72.07% and 71.35% average accuracies in six cross-session emotion recognition tasks. The results illustrated that the sample weights have a rising trend in the beginning of most trials, which coincides with the affective overlap hypothesis. The feature importance factor indicated the critical bands and channels are more obvious compared with some models without considering EEG feature-label inconsistency. • Affective overlap problem was considered in emotion recognition. • Feature-label inconsistency and feature importance were utilized jointly. • SIFIAE obtained a significant improvement in cross-session emotion recognition. • Obtained sample inconsistency demonstrated affective overlap hypothesis. • Affective activation pattern could be adaptively obtained in our model. [ABSTRACT FROM AUTHOR]