Search

Objective . Motor-related brain-computer interface (BCI) have a broad range of applications, with the detection of premovement intentions being a prominent use case. However, the electroencephalography (EEG) features during the premovement phase are not distinctly evident and are susceptible to attentional influences. These limitations impede the enhancement of performance in motor-based BCI. The objective of this study is to establish a premovement BCI encoding paradigm that integrates the preparatory movement state and validates its feasibility in improving the detection of movement intentions. Methods . Two button tasks were designed to induce subjects into a preparation state for two movement intentions (left and right) based on visual guidance, in contrast to spontaneous premovement. The low frequency movement-related cortical potentials (MRCPs) and high frequency event-related desynchronization (ERD) EEG data of 14 subjects were recorded. Extracted features were fused and classified using task related common spatial patterns (CSP) and CSP algorithms. Differences between prepared premovement and spontaneous premovement were compared in terms of time domain, frequency domain, and classification accuracy. Results . In the time domain, MRCPs features reveal that prepared premovement induce lower amplitude and earlier latency on both contralateral and ipsilateral motor cortex compared to spontaneous premovement, with susceptibility to the dominant hand's influence. Frequency domain ERD features indicate that prepared premovement induce lower ERD values bilaterally, and the ERD recovery speed after button press is the fastest. By using the fusion approach, the classification accuracy increased from 78.92% for spontaneous premovement to 83.59% for prepared premovement ( p < 0.05). Along with the 4.67% improvement in classification accuracy, the standard deviation decreased by 0.95. Significance . The research findings confirm that incorporating a preparatory state into premovement enhances neural representations related to movement. This encoding enhancement paradigm effectively improves the performance of motor-based BCI. Additionally, this concept has the potential to broaden the range of decodable movement intentions and related information in motor-related BCI., (© 2024 IOP Publishing Ltd.)

Background: Motor-Imagery-based Brain-Computer Interface (MI-BCI) is a promising technology to assist communication, movement, and neurological rehabilitation for motor-impaired individuals. Electroencephalography (EEG) decoding techniques using deep learning (DL) possess noteworthy advantages due to automatic feature extraction and end-to-end learning. However, the DL-based EEG decoding models tend to show large variations due to intersubject variability of EEG, which results from inconsistencies of different subjects' optimal hyperparameters., New Methods: This study proposes a multi-branch multi-attention mechanism EEGNet model (MBMANet) for robust decoding. It applies the multi-branch EEGNet structure to achieve various feature extractions. Further, the different attention mechanisms introduced in each branch attain diverse adaptive weight adjustments. This combination of multi-branch and multi-attention mechanisms allows for multi-level feature fusion to provide robust decoding for different subjects., Results: The MBMANet model has a four-classification accuracy of 83.18% and kappa of 0.776 on the BCI Competition IV-2a dataset, which outperforms other eight CNN-based decoding models. This consistently satisfactory performance across all nine subjects indicates that the proposed model is robust., Conclusions: The combine of multi-branch and multi-attention mechanisms empowers the DL-based models to adaptively learn different EEG features, which provides a feasible solution for dealing with data variability. It also gives the MBMANet model more accurate decoding of motion intentions and lower training costs, thus improving the MI-BCI's utility and robustness., Competing Interests: Declaration of Competing Interest The authors were responsible for all aspects of the work and had no competing interests to declare., (Copyright © 2024 Elsevier B.V. All rights reserved.)

Deep learning has great potential on decoding EEG in brain-computer interface. While common deep learning algorithms cannot directly train models with data from multiple individuals because of the inter-individual differences in EEG. Collecting enough data for each subject to satisfy the training of deep learning would result in an increase in training cost. This study proposes a novel transfer learning, EEGNet-based multi-source domain filter for transfer learning (EEGNet-MDFTL), to reduce the amount of training data and improve the performance of BCI. The EEGNet-MDFTL uses bagging ensemble learning to learn domain-invariant features from the multi-source domain and utilizes model loss value to filter the multi-source domain. Compared with baseline methods, the accuracy of the EEGNet-MDFTL reaches 91.96%, higher than two state-of-the-art methods, which demonstrates source domain filter can select similar source domains to improve the accuracy of the model, and remains a high level even when the data amount is reduced to 1/8, proving that ensemble learning learns enough domain invariant features from the multi-source domain to make the model insensitive to data amount. The proposed EEGNet-MDFTL is effective in improving the decoding performance with a small amount of data, which is helpful to save the BCI training cost., (© 2023. International Federation for Medical and Biological Engineering.)

Objective . Motor imagery-based brain-computer interaction (MI-BCI) is a novel method of achieving human and external environment interaction that can assist individuals with motor disorders to rehabilitate. However, individual differences limit the utility of the MI-BCI. In this study, a personalized MI prediction model based on the individual difference of event-related potential (ERP) is proposed to solve the MI individual difference. Approach. A novel paradigm named action observation-based multi-delayed matching posture task evokes ERP during a delayed matching posture task phase by retrieving picture stimuli and videos, and generates MI electroencephalogram through action observation and autonomous imagery in an action observation-based motor imagery phase. Based on the correlation between the ERP and MI, a logistic regression-based personalized MI prediction model is built to predict each individual's suitable MI action. 32 subjects conducted the MI task with or without the help of the prediction model to select the MI action. Then classification accuracy of the MI task is used to evaluate the proposed model and three traditional MI methods. Main results. The personalized MI prediction model successfully predicts suitable action among 3 sets of daily actions. Under suitable MI action, the individual's ERP amplitude and event-related desynchronization (ERD) intensity are the largest, which helps to improve the accuracy by 14.25%. Significance. The personalized MI prediction model that uses the temporal ERP features to predict the classification accuracy of MI is feasible for improving the individual's MI-BCI performance, providing a new personalized solution for the individual difference and practical BCI application., (© 2024 IOP Publishing Ltd.)

Objective . Motor imagery (MI)-based brain-computer interfaces (BCIs) provide an additional control pathway for people by decoding the intention of action imagination. The way people imagine greatly affects MI-BCI performance. Action itself is one of the factors that influence the way people imagine. Whether the different actions cause a difference in the MI performance is unknown. What is more important is how to manifest this action difference in the process of imagery, which has the potential to guide people to use their individualized actions to imagine more effectively. Approach. To explore action differences, this study proposes a novel paradigm named as action observation based delayed matching posture task. Ten subjects are required to observe, memorize, match, and imagine three types of actions (cutting, grasping and writing) given by visual images or videos, to accomplish the phases of encoding, retrieval and reinforcement of MI. Event-related potential (ERP), MI features, and classification accuracy of the left or the right hand are used to evaluate the effect of the action difference on the MI difference. Main results. Action differences cause different feature distributions, resulting in that the accuracy with high event-related (de)synchronization (ERD/ERS) is 27.75% higher than the ones with low ERD/ERS ( p < 0.05), which indicates that the action difference has impact on the MI difference and the BCI performance. In addition, significant differences in the ERP amplitudes exists among the three actions: the amplitude of P300-N200 potential reaches 9.28 μ V of grasping, 5.64 μ V and 5.25 μ V higher than the cutting and the writing, respectively ( p < 0.05). Significance. The ERP amplitudes derived from the supplementary motor area shows positive correlation to the MI classification accuracy, implying that the ERP might be an index of the MI performance when the people is faced with action selection. This study demonstrates that the MI difference is related to the action difference, and can be manifested by the ERP, which is important for improving MI training by selecting suitable action; the relationship between the ERP and the MI provides a novel index to find the suitable action to set up an individualized BCI and improve the performance further., (© 2023 IOP Publishing Ltd.)

The convolutional neural network (CNN) has emerged as a powerful tool for decoding electroencephalogram (EEG), which owns the potential use in the event-related potential-based brain-computer interface (ERP-BCI). However, the intra-individual difference of ERP makes the traditional learning models trained on static EEG data hard to decode when the EEG features vary along the time, which limits the long-time performance of the model. Addressing this problem, this study proposes a three-dimension CNN (3D-CNN)-based model to decode the ERPs dynamically. As input, the EEG is transformed into a brain topographic map stream along time. Then the 3D-CNN applies three-dimension kernels to capture the dynamical characteristic of spatial feature at several time points. Ten subjects participated in a cross-time task for 6 or 12[Formula: see text]h. The 3D-CNN shows higher accuracies and shorter computational cost than the baseline models of the 2D-CNN, the long short term memory (LSTM), the back propagation (BP), and the fisher linear discriminant analysis (FLDA) when detecting the ERPs. In addition, four schemes of the 3D-CNN are compared to explore the influence of the structure on the performance. This result demonstrates advanced robustness of the 3D-CNN kernel to the intra-individual EEG difference, helping to launch a more practical EEG decoding model for a long-time use.

Background: An asynchronous event-related potential-based brain computer interface (ERP-BCI) permits the subjects to output intentions at their own pace, which provides a more free and practical communication pathways without the need for muscle activity. The core of constructing this type of system is to discriminate both the intentions and brain states., New Methods: This study proposes a fisher linear discriminant analysis classification algorithm fused with naïve Bayes (B-FLDA) for the ERP-BCI to simultaneous recognize the subjects' intentions, working and idle states. This method uses the spectral characteristics of visual-evoked potential and the time-domain characteristics of ERP to simultaneously detect brain states and target stimulus, and obtain the final discrimination result through probability fusion., Results: The accuracy and the information transfer rate increase to 98.61% and 62.80 bits/min under 10 repetitions and 1 repetition, respectively. The three parameters of receiver operator characteristic curve have achieved better performance., Comparison With Existing Methods: Ten subjects participate in this study with the proposed algorithms and two other control methods. The accuracy and information transfer rate of this algorithm are better than the other methods., Conclusions: It indicates that the naïve Bayes-FLDA algorithm is able to improve the performance of an asynchronous BCI system by detecting the intentions and states simultaneously., (Copyright © 2022 Elsevier B.V. All rights reserved.)

Brain-computer interface (BCI) is a useful device for people without relying on peripheral nerves and muscles. However, the performance of the event-related potential (ERP)-based BCI declines when applying it to real environments, especially in cross-state and cross-subject conditions. Here we employ temporal modeling and adversarial training to improve the visual ERP-based BCI under different mental workload states and to alleviate the problems above. The rationality of our method is that the ERP-based BCI is based on electroencephalography (EEG) signals recorded from the scalp's surface, continuously changing with time and somewhat stochastic. In this paper, we propose a hierarchical recurrent network to encode all ERP signals in each repetition at the same time and model them with a temporal manner to predict which visual event elicited an ERP. The hierarchical architecture is a simple yet effective method for organizing recurrent layers in a deep structure to model long sequence signals. Taking a cue from recent advances in adversarial training, we further applied dynamic adversarial perturbations to create adversarial examples to enhance the model performance. We conduct our experiments on one published visual ERP-based BCI task with 15 subjects and 3 different auditory workload states. The results indicate that our hierarchical method can effectively model the long sequence EEG raw data, outperform the baselines on most conditions, including cross-state and cross-subject conditions. Finally, we show how deep learning-based methods with limited EEG data can improve ERP-based BCI with adversarial training. Our code is available at https://github.com/aispeech-lab/VisBCI.

Humanoid robots are widely used in brain computer interface (BCI). Using a humanoid robot stimulus could increase the amplitude of event-related potentials (ERPs), which improves BCI performance. Since a humanoid robot contains many human elements, the element that increases the ERPs amplitude is unclear, and how to test the effect of it on the brain is a problem. This study used different graphic structures of an NAO humanoid robot to design three types of robot stimuli: a global robot, its local information, and its topological action. Ten subjects first conducted an odd-ball-based BCI (OD-BCI) by applying these stimuli. Then, they accomplished a delayed matching-to-sample task (DMST) that was used to specialize the encoding and retrieval phases of the OD-BCI task. In the retrieval phase of the DMST, the global stimulus induces the largest N200 and P300 potentials with the shortest latencies in the frontal, central, and occipital areas. This finding is in accordance with the P300 and classification performance of the OD-BCI task. When induced by the local stimulus, the subjects responded faster and more accurately in the retrieval phase of the DMST than in the other two conditions, indicating that the local stimulus improved the subject's responses. These results indicate that the OD-BCI task causes subject's retrieval work when the subject recognizes and outputs the stimulus. The global stimulus that contains topological and local elements could make brain react faster and induce larger ERPs, this finding could be used during the development of visual stimuli to improve BCI performance.

Increasing the amplitude of event-related potential is one of the key methods to improve the accuracy of the potential-based brain-computer interface, e.g., P300-based brain-computer interface. The brain-computer interface systems often use symbols or controlled objects as vision stimuli, but what visual stimuli can induce more obvious event-related potential is still unknown. This paper designed three kinds of visual stimuli, i.e., a square, an arrow, and a robot attached with an arrow, to analyze the influence of concreteness degree of the graph on the N200 and P300 potentials, and applied a support vector machine to compare the performance of the brain-computer interface under different stimuli. The results showed that, compared with the square, the robot attached with arrow and the arrow both induced larger N200 potential ( P = 1.6 × 10 -3 , P = 4.2 × 10 -2 ) and longer P300 potential ( P = 2.2 × 10 -3 , P = 1.9 × 10 -2 ) in the frontal area, but the amplitude under the arrow condition is smaller than the one under the robot attached with arrow condition. The robot attached with arrow increased the N200 potential amplitude of the square and arrow from 3.12 μV and 5.19 μV to 7.21 μV ( P = 1.6 × 10 -3 , P = 8.9 × 10 -2 ), and improved the accuracy rate from 59.95%, 61.67% to 74.45% ( P = 2.1 × 10 -2 , P = 1.6 × 10 -2 ), and the information transfer rate from 35.00 bits/min, 35.98 bits/min to 56.71 bits/min ( P = 2.6 × 10 -2 , P = 1.6 × 10 -2 ). This study shows that the concreteness of graphics could affect the N200 potential and the P300 potential. The abstract symbol could represent the meaning and evoke potentials, but the information contained in the concrete robot attached with an arrow is more correlated with the human experience, which is helpful to improve the amplitude. The results may provide new sight in modifying the stimulus interface of the brain-computer interface.

Background: An asynchronous brain-computer interface (BCI) allows subject to freely switch between the working state and the idle state, improving the subject's comfort. However, using only the event-related potential (ERP) to detect these two states is difficult because of the small amplitude of the ERP., Method: Our previous study finds that an odd-ball paradigm could evoke transient visual evoked potentials (TSVEPs) simultaneously with ERPs. This study adopts the TSVEP and the ERP to detect the idle state in the design of an asynchronous TSVEP-ERP-based BCI (T-E BCI). The T-E BCI extracts time and frequency features from brain signals and uses a novel probability-based fisher linear discriminant analysis (P-FLDA) to combine the classification results of the ERP and the TSVEP., Result: Ten subjects perform visual speller and video watching experiments, and their brain signals are measured under the working and idle states. The main results show that the T-E BCI achieves a higher accuracy than the ERP-based BCI when judging the subject's intentions and the two states. The P-FLDA performs better than the FLDA in combining the classification results., Conclusions: The study demonstrates that adding the TSVEP can substantially reduce the number of wrongly detected trials. The T-E BCI provides a new way of designing an asynchronous BCI without adding any additional visual stimuli, which makes the BCI more practical., (Copyright © 2020. Published by Elsevier B.V.)

Traditional training methods need to collect a large amount of data for every subject to train a subject-specific classifier, which causes subjects fatigue and training burden. This study proposes a novel training method, TrAdaBoost based on cross-validation and an adaptive threshold (CV-T-TAB), to reduce the amount of data required for training by selecting and combining multiple subjects' classifiers that perform well on a new subject to train a classifier. This method adopts cross-validation to extend the amount of the new subject's training data and sets an adaptive threshold to select the optimal combination of the classifiers. Twenty-five subjects participated in the N200- and P300-based brain-computer interface. The study compares CV-T-TAB to five traditional training methods by testing them on the training of a support vector machine. The accuracy, information transfer rate, area under the curve, recall and precision are used to evaluate the performances under nine conditions with different amounts of data. CV-T-TAB outperforms the other methods and retains a high accuracy even when the amount of data is reduced to one-third of the original amount. The results imply that CV-T-TAB is effective in improving the performance of a subject-specific classifier with a small amount of data by adopting multiple subjects' classifiers, which reduces the training cost.

The performance of the event-related potential (ERP)-based brain-computer interface (BCI) declines when applying it into the real environment, which limits the generality of the BCI. The sound is a common noise in daily life, and whether it has influence on this decline is unknown. This study designs a visual-auditory BCI task that requires the subject to focus on the visual interface to output commands and simultaneously count number according to an auditory story. The story is played at three speeds to cause different workloads. Data collected under the same or different workloads are used to train and test classifiers. The results show that when the speed of playing the story increases, the amplitudes of P300 and N200 potentials decrease by 0.86 μV ( p = 0.0239) and 0.69 μV ( p = 0.0158) in occipital-parietal area, leading to a 5.95% decline ( p = 0.0101) of accuracy and 9.53 bits/min decline ( p = 0.0416) of information transfer rate. The classifier that is trained by the high workload data achieves higher accuracy than the one trained by the low workload if using the high workload data to test the performance. The result indicates that the sound could affect the visual ERP-BCI by increasing the workload. The large similarity of the training data and testing data is as important as the amplitudes of the ERP on obtaining high performance, which gives us an insight on how make to the ERP-BCI generalized.

Event-related potential (ERP)-based brain-computer interface (BCI) has been widely used in robot control. Increasing the amplitude of the ERPs is key for improving the performance of ERP-based BCI. However, using images of robot motion as visual stimuli has not been studied widely. The aim of this study is to explore the concreteness of robot motion images on ERPs. Fifteen subjects used five kinds of visual spellers employing different images as visual stimuli: squares, arrows, a single kind of robot motion, multiple kinds of robot motions, and multiple kinds of robot motions with arrows. The three robot motion stimuli induced larger N200 and P300 potentials than non-robot motion stimuli. The topography shows that robot motion stimuli also evoke stronger negativities in the anterior and occipital areas. Concrete images provide more information to the subject about the robot motion, which might help the brain extract the meaning of the image more automatically. We use a support vector machine to detect the subject's intentions. There is substantial improvement in the classification performance when using robot motion images as visual stimuli, which implies that concrete visual stimuli improve the performance of the ERP-based BCI., (Copyright © 2020 Elsevier B.V. All rights reserved.)

Individual differences of P300 potentials lead to that a large amount of training data must be collected to construct pattern recognition models in P300-based brain-computer interface system, which may cause subjects' fatigue and degrade the system performance. TrAdaBoost is a method that transfers the knowledge from source area to target area, which improves learning effect in the target area. Our research purposed a TrAdaBoost-based linear discriminant analysis and a TrAdaBoost-based support vector machine to recognize the P300 potentials across multiple subjects. This method first trains two kinds of classifiers separately by using the data deriving from a small amount of data from same subject and a large amount of data from different subjects. Then it combines all the classifiers with different weights. Compared with traditional training methods that use only a small amount of data from same subject or mixed different subjects' data to directly train, our algorithm improved the accuracies by 19.56% and 22.25% respectively, and improved the information transfer rate of 14.69 bits/min and 15.76 bits/min respectively. The results indicate that the TrAdaBoost-based method has the potential to enhance the generalization ability of brain-computer interface on the individual differences.

Increasing command generation rate of an event-related potential-based brain-robot system is challenging, because of limited information transfer rate of a brain-computer interface system. To improve the rate, we propose a dual stimuli approach that is flashing a robot image and is scanning another robot image simultaneously. Two kinds of event-related potentials, N200 and P300 potentials, evoked in this dual stimuli condition are decoded by a convolutional neural network. Compared with the traditional approaches, this proposed approach significantly improves the online information transfer rate from 23.0 or 17.8 to 39.1 bits/min at an accuracy of 91.7%. These results suggest that combining multiple types of stimuli to evoke distinguishable ERPs might be a promising direction to improve the command generation rate in the brain-computer interface.

The most popular noninvasive Brain Robot Interaction (BRI) technology uses the electroencephalogram- (EEG-) based Brain Computer Interface (BCI), to serve as an additional communication channel, for robot control via brainwaves. This technology is promising for elderly or disabled patient assistance with daily life. The key issue of a BRI system is to identify human mental activities, by decoding brainwaves, acquired with an EEG device. Compared with other BCI applications, such as word speller, the development of these applications may be more challenging since control of robot systems via brainwaves must consider surrounding environment feedback in real-time, robot mechanical kinematics, and dynamics, as well as robot control architecture and behavior. This article reviews the major techniques needed for developing BRI systems. In this review article, we first briefly introduce the background and development of mind-controlled robot technologies. Second, we discuss the EEG-based brain signal models with respect to generating principles, evoking mechanisms, and experimental paradigms. Subsequently, we review in detail commonly used methods for decoding brain signals, namely, preprocessing, feature extraction, and feature classification, and summarize several typical application examples. Next, we describe a few BRI applications, including wheelchairs, manipulators, drones, and humanoid robots with respect to synchronous and asynchronous BCI-based techniques. Finally, we address some existing problems and challenges with future BRI techniques.

In this paper, we evaluate the control performance of SSVEP (steady-state visual evoked potential)- and P300-based models using Cerebot-a mind-controlled humanoid robot platform. Seven subjects with diverse experience participated in experiments concerning the open-loop and closed-loop control of a humanoid robot via brain signals. The visual stimuli of both the SSVEP- and P300- based models were implemented on a LCD computer monitor with a refresh frequency of 60 Hz. Considering the operation safety, we set the classification accuracy of a model over 90.0% as the most important mandatory for the telepresence control of the humanoid robot. The open-loop experiments demonstrated that the SSVEP model with at most four stimulus targets achieved the average accurate rate about 90%, whereas the P300 model with the six or more stimulus targets under five repetitions per trial was able to achieve the accurate rates over 90.0%. Therefore, the four SSVEP stimuli were used to control four types of robot behavior; while the six P300 stimuli were chosen to control six types of robot behavior. Both of the 4-class SSVEP and 6-class P300 models achieved the average success rates of 90.3% and 91.3%, the average response times of 3.65 s and 6.6 s, and the average information transfer rates (ITR) of 24.7 bits/min 18.8 bits/min, respectively. The closed-loop experiments addressed the telepresence control of the robot; the objective was to cause the robot to walk along a white lane marked in an office environment using live video feedback. Comparative studies reveal that the SSVEP model yielded faster response to the subject's mental activity with less reliance on channel selection, whereas the P300 model was found to be suitable for more classifiable targets and required less training. To conclude, we discuss the existing SSVEP and P300 models for the control of humanoid robots, including the models proposed in this paper.

• The multi-modal acquisition method based on Electroencephalography and Functional near-infrared spectroscopy can obtain more comprehensive physiological information of motor imagery task. • Feature layer fusion multi-modal information has higher robustness. • A reasonable feature screening method can effectively reduce the feature dimension and improve the classification performance. • More and more researchers pay attention to multimodal motion imagination brain computer interface. The multimodal brain–computer interface (BCI) is an innovative paradigm for human–computer interaction that utilizes both electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) signals; therefore, it has received considerable interest. In this study, we concurrently collected EEG and fNIRS data from eighteen healthy participants while they engaged in the mental simulation of performing grasping movements with their left and right hands. During the feature screening session, we assessed the effectiveness of combining the Relief and minimum redundancy maximum relevance (mRMR) algorithms. This algorithm was applied individually to analyze the common spatial pattern (CSP) characteristics of EEG signals across distinct frequency bands as well as the modified CSP (MCSP) attributes of fNIRS signals. Moreover, the enhancement of classification accuracy via feature-level fusion of the two signal types was investigated. The support vector machine (SVM) algorithm was used as the classifier for both training and validation. The results show a significant decrease in the feature count and a substantial enhancement in classification accuracy. Additionally, the highest classification accuracy (88.33 % ± 5.80 % for EEG + HbO + HbR, P < 0.05) was achieved when utilizing multimodal features, which exceeds that when utilizing EEG alone (84.28 % ± 7.56 %). Furthermore, the group of participants yielding an enhanced classification accuracy under the multi-modal characteristics constituted the highest percentage among all participants in the case of combined EEG and HbR (88.89 %). The proposed multi-modal information fusion strategy can serve as an effective reference for task recognition in BCI. [ABSTRACT FROM AUTHOR]