Search

Achieving high classification performance is challenging due to non-stationarity and low signal-to-noise ratio (low SNR) characteristics of EEG signals. Spatial filtering is commonly used to improve the SNR yet the individual differences in the underlying temporal or frequency information is often ignored. This paper investigates motor imagery signals via orthogonal wavelet decomposition, by which the raw signals are decomposed into multiple unrelated sub-band components. Furthermore, channel-wise spectral filtering via weighting the sub-band components are implemented jointly with spatial filtering to improve the discriminability of EEG signals, with an l 2-norm regularization term embedded in the objective function to address the underlying over-fitting issue. Finally, sparse Bayesian learning with Gaussian prior is applied to the extracted power features, yielding an RVM classifier. The classification performance of SEOWADE is significantly better than those of several competing algorithms (CSP, FBCSP, CSSP, CSSSP, and shallow ConvNet). Moreover, scalp weight maps of the spatial filters optimized by SEOWADE are more neurophysiologically meaningful. In summary, these results demonstrate the effectiveness of SEOWADE in extracting relevant spatio-temporal information for single-trial EEG classification. [ABSTRACT FROM AUTHOR]

Fusion of multimodal features is a momentous problem for video emotion recognition. As the development of deep learning, directly fusing feature matrixes of each mode through neural networks at feature level becomes mainstream method. However, unlike unimodal issues, for multimodal analysis, finding the correlations between different modal is as important as discovering effective unimodal features. To make up the deficiency in unearthing the intrinsic relationships between multimodal, a novel modularized multimodal emotion recognition model based on deep canonical correlation analysis (MERDCCA) is proposed in this letter. In MERDCCA, four utterances are gathered as a new group and each utterance contains text, audio and visual information as multimodal input. Gated recurrent unit layers are used to extract the unimodal features. Deep canonical correlation analysis based on encoder-decoder network is designed to extract cross-modal correlations by maximizing the relevance between multimodal. The experiments on two public datasets show that MERDCCA achieves the better results. [ABSTRACT FROM AUTHOR]

A brain-computer interface (BCI) measures and analyzes brain activity and converts this activity into computer commands to control external devices. In contrast to traditional BCIs that require a subject-specific calibration process before being operated, a subject-independent BCI learns a subject-independent model and eliminates subject-specific calibration for new users. However, building subject-independent BCIs remains difficult because electroencephalography (EEG) is highly noisy and varies by subject. In this study, we propose an invariant pattern learning method based on a convolutional neural network (CNN) and big EEG data for subject-independent P300 BCIs. The CNN was trained using EEG data from a large number of subjects, allowing it to extract subject-independent features and make predictions for new users. We collected EEG data from 200 subjects in a P300-based spelling task using two different types of amplifiers. The offline analysis showed that almost all subjects obtained significant cross-subject and cross-amplifier effects, with an average accuracy of more than 80%. Furthermore, more than half of the subjects achieved accuracies above 85%. These results indicated that our method was effective for building a subject-independent P300 BCI, with which more than 50% of users could achieve high accuracies without subject-specific calibration. [ABSTRACT FROM AUTHOR]

Event-related potential (ERP) is bioelectrical activity that occurs in the brain in response to specific events or stimuli, reflecting the electrophysiological changes in the brain during cognitive processes. ERP is important in cognitive neuroscience and has been applied to brain-computer interfaces (BCIs). However, because ERP signals collected on the scalp are weak, mixed with spontaneous electroencephalogram (EEG) signals, and their temporal and spatial features are complex, accurate ERP detection is challenging. Compared to traditional neural networks, the capsule network (CapsNet) replaces scalar-output neurons with vector-output capsules, allowing the various input information to be well preserved in the capsules. In this study, we expect to utilize CapsNet to extract the discriminative spatial-temporal features of ERP and encode them in capsules to reduce the loss of valuable information, thereby improving the ERP detection performance for BCI. Therefore, we propose ERP-CapsNet to perform ERP detection in a BCI speller application. The experimental results on BCI Competition datasets and the Akimpech dataset show that ERP-CapsNet achieves better classification performances than do the state-of-the-art techniques. We also use a decoder to investigate the attributes of ERPs encoded in capsules. The results show that ERP-CapsNet relies on the P300 and P100 components to detect ERP. Therefore, ERP-CapsNet not only acts as an outstanding method for ERP detection, but also provides useful insights into the ERP detection mechanism. [ABSTRACT FROM AUTHOR]

Achieving high classification performance in electroencephalogram (EEG)-based brain–computer interfaces (BCIs) often entails a large number of channels, which impedes their use in practical applications. Despite the previous efforts, it remains a challenge to determine the optimal subset of channels in a subject-specific manner without heavily compromising the classification performance. In this article, we propose a new method, called spatiotemporal-filtering-based channel selection (STECS), to automatically identify a designated number of discriminative channels by leveraging the spatiotemporal information of the EEG data. In STECS, the channel selection problem is cast under the framework of spatiotemporal filter optimization by incorporating a group sparsity constraints, and a computationally efficient algorithm is developed to solve the optimization problem. The performance of STECS is assessed on three motor imagery EEG datasets. Compared with state-of-the-art spatiotemporal filtering algorithms using full EEG channels, STECS yields comparable classification performance with only half of the channels. Moreover, STECS significantly outperforms the existing channel selection methods. These results suggest that this algorithm holds promise for simplifying BCI setups and facilitating practical utility. [ABSTRACT FROM AUTHOR]

Motor imagery (MI) decoding is an important part of brain-computer interface (BCI) research, which translates the subject’s intentions into commands that external devices can execute. The traditional methods for discriminative feature extraction, such as common spatial pattern (CSP) and filter bank common spatial pattern (FBCSP), have only focused on the energy features of the electroencephalography (EEG) and thus ignored the further exploration of temporal information. However, the temporal information of spatially filtered EEG may be critical to the performance improvement of MI decoding. In this paper, we proposed a deep learning approach termed filter-bank spatial filtering and temporal-spatial convolutional neural network (FBSF-TSCNN) for MI decoding, where the FBSF block transforms the raw EEG signals into an appropriate intermediate EEG presentation, and then the TSCNN block decodes the intermediate EEG signals. Moreover, a novel stage-wise training strategy is proposed to mitigate the difficult optimization problem of the TSCNN block in the case of insufficient training samples. Firstly, the feature extraction layers are trained by optimization of the triplet loss. Then, the classification layers are trained by optimization of the cross-entropy loss. Finally, the entire network (TSCNN) is fine-tuned by the back-propagation (BP) algorithm. Experimental evaluations on the BCI IV 2a and SMR-BCI datasets reveal that the proposed stage-wise training strategy yields significant performance improvement compared with the conventional end-to-end training strategy, and the proposed approach is comparable with the state-of-the-art method. [ABSTRACT FROM AUTHOR]

Spectral–spatial Gabor filtering, which is based on 3-D local harmonic analysis, has been a powerful spectral–spatial feature extraction tool for hyperspectral image (HSI) classification. However, existing spectral–spatial Gabor approaches are prone to oversmoothing, neglecting the existences of edges and negatively affecting the classification. In this article, we propose a new HSI Gabor filtering concept, called spectral–spatial-range Gabor filtering, which intends to restrain edge interference from disturbing local spectral–spatial harmonic components. Contributions and novelties of our work can be identified as follows: 1) an HSI filtering framework is created, which can accommodate various Gabor filtering procedures and hence offer the potential to guide the design of new Gabor filters; 2) following such a unified filtering framework and taking into consideration both local spectral–spatial harmonic characteristics and range domain variations, we develop a new concept of spectral–spatial-range Gabor filtering; and 3) utilizing this proposed Gabor prototype and elaborating mathematical derivations, we achieve a novel discriminative spectral–spatial-range Gabor filtering method, which can deal with discriminative local harmonics and edge interference simultaneously along the spectral–spatial-range domain, obtaining highly discriminative Gabor features while yielding linear computational complexity. Our novel method is evaluated on four real HSI data sets and achieves excellent performances. [ABSTRACT FROM AUTHOR]

Segmentation of cardiac ventricle from magnetic resonance images is significant for cardiac disease diagnosis, progression assessment, and monitoring cardiac conditions. Manual segmentation is so time consuming, tedious, and subjective that automated segmentation methods are highly desired in practice. However, conventional segmentation methods performed poorly in cardiac ventricle, especially in the right ventricle. Compared with the left ventricle, whose shape is a simple thick-walled circle, the structure of the right ventricle is more complex due to ambiguous boundary, irregular cavity, and variable crescent shape. Hence, effective feature extractors and segmentation models are preferred. In this paper, we propose a dilated-inception net (DIN) to extract and aggregate multi-scale features for right ventricle segmentation. The DIN outperforms many state-of-the-art models on the benchmark database of right ventricle segmentation challenge. In addition, the experimental results indicate that the proposed model has potential to reach expert-level performance in right ventricular epicardium segmentation. More importantly, DIN behaves similarly to clinical expert with high correlation coefficients in four clinical cardiac indices. Therefore, the proposed DIN is promising for automated cardiac right ventricle segmentation in clinical applications. [ABSTRACT FROM AUTHOR]

Detecting event-related potential (ERP) is a challenging problem because of its low signal-to-noise ratio and complex spatial–temporal features. Conventional detection methods usually rely on the ensemble averaging technique, which may eliminate subtle but important information in ERP signals and lead to poor detection performance. Inspired by the good performance of discriminative restricted Boltzmann machine (DRBM) in feature extraction and classification, we propose a spatial–temporal DRBM (ST-DRBM) to extract spatial and temporal features for ERP detection. The experimental results and statistical analyses demonstrate that the proposed method is able to achieve state-of-the-art ERP detection performance. The ST-DRBM is not only an effective ERP detector, but also a practical tool for ERP analysis. Based on the proposed model, similar scalp distribution and temporal variations were found in the ERP signals of different sessions, which indicated the feasibility of cross-session ERP detection. Given its state-of-the-art performance and effective analytical technique, ST-DRBM is promising for ERP-based brain–computer interfaces and neuroscience research. [ABSTRACT FROM AUTHOR]

Detecting and Please provide the correct one analyzing the event-related potential (ERP) remains an important problem in neuroscience. Due to the low signal-to-noise ratio and complex spatio-temporal patterns of ERP signals, conventional methods usually rely on ensemble averaging technique for reliable detection, which may obliterate subtle but important information in each trial of ERP signals. Inspired by deep learning methods, we propose a novel hybrid network termed ERP-NET. With hybrid deep structure, the proposed network is able to learn complex spatial and temporal patterns from single-trial ERP signals. To verify the effectiveness of ERP-NET, we carried out a few ERP detection experiments that the proposed model achieved cutting-edge performance. The experimental results demonstrate that the patterns learned by the ERP-NET are discriminative ERP components in which the ERP signals are properly characterized. More importantly, as an effective approach to single-trial analysis, ERP-NET is able to discover new ERP patterns which are significant to neuroscience study as well as BCI applications. Therefore, the proposed ERP-NET is a promising tool for the research on ERP signals. [ABSTRACT FROM AUTHOR]

In off-line training of motor imagery-based brain-computer interfaces (BCIs), to enhance the generalization performance of the learned classifier, the local information contained in test data could be used to improve the performance of motor imagery as well. Further considering that the covariance matrices of electroencephalogram (EEG) signal lie on Riemannian manifold, in this paper, we construct a Riemannian graph to incorporate the information of training and test data into processing. The adjacency and weight in Riemannian graph are determined by the geodesic distance of Riemannian manifold. Then, a new graph embedding algorithm, called bilinear regularized locality preserving (BRLP), is derived upon the Riemannian graph for addressing the problems of high dimensionality frequently arising in BCIs. With a proposed regularization term encoding prior information of EEG channels, the BRLP could obtain more robust performance. Finally, an efficient classification algorithm based on extreme learning machine is proposed to perform on the tangent space of learned embedding. Experimental evaluations on the BCI competition and in-house data sets reveal that the proposed algorithms could obtain significantly higher performance than many competition algorithms after using same filter process. [ABSTRACT FROM AUTHOR]

Electrooculography (EOG) signals, which can be used to infer the intentions of a user based on eye movements, are widely used in human–computer interface (HCI) systems. Most existing EOG-based HCI systems incorporate a limited number of commands because they generally associate different commands with a few different types of eye movements, such as looking up, down, left, or right. This paper presents a novel single-channel EOG-based HCI that allows users to spell asynchronously by only blinking. Forty buttons corresponding to 40 characters displayed to the user via a graphical user interface are intensified in a random order. To select a button, the user must blink his/her eyes in synchrony as the target button is flashed. Two data processing procedures, specifically support vector machine (SVM) classification and waveform detection, are combined to detect eye blinks. During detection, we simultaneously feed the feature vectors extracted from the ongoing EOG signal into the SVM classification and waveform detection modules. Decisions are made based on the results of the SVM classification and waveform detection. Three online experiments were conducted with eight healthy subjects. We achieved an average accuracy of 94.4% and a response time of 4.14 s for selecting a character in synchronous mode, as well as an average accuracy of 93.43% and a false positive rate of 0.03/min in the idle state in asynchronous mode. The experimental results, therefore, demonstrated the effectiveness of this single-channel EOG-based speller. [ABSTRACT FROM AUTHOR]

The key issue of electroencephalography (EEG)-based brain switches is to detect the control and idle states in an asynchronous manner. Most existing methods rely on a threshold. However, it is often time consuming to select a satisfactory threshold, and the chosen threshold might be inappropriate over a long period of time due to the variability of the EEG signals. This paper presents a new P300-based threshold-free brain switch. Specifically, one target button and three pseudo buttons, which are intensified in a random order to produce P300 potential, are set in the graphical user interface. The user can issue a switch command by focusing on the target button. Two support vector machine (SVM) classifiers, namely, SVM1 and SVM2, are used in the detection algorithm. During detection, we first obtained four SVM scores, corresponding to the four flashing buttons, by applying SVM1 to the ongoing EEG. If the SVM score corresponding to the target button was negative or not at the maximum, then an idle state was determined. Moreover, if the target button had a maximum and positive score, then we fed the four SVM scores as features into SVM2 to further discriminate the control and idle states. As an application, this brain switch was used to produce a start/stop command for an intelligent wheelchair, of which the left, right, forward, backward functions were carried out by an autonomous navigation system. Several experiments were conducted with eight healthy subjects and five patients with spinal cord injuries (SCIs). The experimental results not only demonstrated the effectiveness of our approach but also illustrated the potential application for patients with SCIs. [ABSTRACT FROM AUTHOR]

Spectral–spatial classification of remotely sensed hyperspectral images has attracted a lot of attention in recent years. Although Gabor filtering has been used for feature extraction from hyperspectral images, its capacity to extract relevant information from both the spectral and the spatial domains of the image has not been fully explored yet. In this paper, we present a new discriminative low-rank Gabor filtering (DLRGF) method for spectral–spatial hyperspectral image classification. A main innovation of the proposed approach is that our implementation is accomplished by decomposing the standard 3-D spectral–spatial Gabor filter into eight subfilters, which correspond to different combinations of low-pass and bandpass single-rank filters. Then, we show that only one of the subfilters (i.e., the one that performs low-pass spatial filtering and bandpass spectral filtering) is actually appropriate to extract suitable features based on the characteristics of hyperspectral images. This allows us to perform spectral–spatial classification in a highly discriminative and computationally efficient way, by significantly decreasing the computational complexity (from cubic to linear order) compared with the 3-D spectral–spatial Gabor filter. In order to theoretically prove the discriminative ability of the selected subfilter, we derive an overall classification risk bound to evaluate the discriminating abilities of the features provided by the different subfilters. Our experimental results, conducted using different hyperspectral images, indicate that the proposed DLRGF method exhibits significant improvements in terms of classification accuracy and computational performance when compared with the 3-D spectral–spatial Gabor filter and other state-of-the-art spectral–spatial classification methods. [ABSTRACT FROM PUBLISHER]

During the development of a brain-computer interface, it is beneficial to exploit information in multiple electrode signals. However, a small channel subset is favored for not only machine learning feasibility, but also practicality in commercial and clinical BCI applications. An embedded channel selection approach based on grouped automatic relevance determination is proposed. The proposed Gaussian conjugate group-sparse prior and the embedded nature of the concerned Bayesian linear model enable simultaneous channel selection and feature classification. Moreover, with the marginal likelihood (evidence) maximization technique, hyper-parameters that determine the sparsity of the model are directly estimated from the training set, avoiding time-consuming cross-validation. Experiments have been conducted on P300 speller BCIs. The results for both public and in-house datasets show that the channels selected by our techniques yield competitive classification performance with the state-of-the-art and are biologically relevant to P300. [ABSTRACT FROM AUTHOR]

Goal: Motor imagery-related mu/beta rhythms, which can be voluntarily modulated by subjects, have been widely used in EEG-based brain computer interfaces (BCIs). Moreover, it has been suggested that motor imagery-specific EEG differences can be enhanced by feedback training. However, the differences observed in the EEGs of naive subjects are typically not sufficient to provide reliable EEG control and thus result in unintended feedback. Such feedback can frustrate subjects and impede training. In this study, a hybrid BCI paradigm combining motor imagery and steady-state visually evoked potentials (SSVEPs) has been proposed to provide effective continuous feedback for motor imagery training. Methods: During the initial training sessions, subjects must focus on flickering buttons to evoke SSVEPs as they perform motor imagery tasks. The output/feedback of the hybrid BCI is based on hybrid features consisting of motor imagery- and SSVEP-related brain signals. In this context, the SSVEP plays a more important role than motor imagery in generating feedback. As the training progresses, the subjects can gradually decrease their visual attention to the flickering buttons, provided that the feedback is still effective. In this case, the feedback is mainly based on motor imagery. Results: Our experimental results demonstrate that subjects generate distinguishable brain patterns of hand motor imagery after only five training sessions lasting approximately 1.5 h each. Conclusion: The proposed hybrid feedback paradigm can be used to enhance motor imagery training. Significance: This hybrid BCI system with feedback can effectively identify the intentions of the subjects. [ABSTRACT FROM PUBLISHER]

Brain–computer interfaces (BCIs) are used to translate brain activity signals into control signals for external devices. Currently, it is difficult for BCI systems to provide the multiple independent control signals necessary for the multi-degree continuous control of a wheelchair. In this paper, we address this challenge by introducing a hybrid BCI that uses the motor imagery-based mu rhythm and the P300 potential to control a brain-actuated simulated or real wheelchair. The objective of the hybrid BCI is to provide a greater number of commands with increased accuracy to the BCI user. Our paradigm allows the user to control the direction (left or right turn) of the simulated or real wheelchair using left- or right-hand imagery. Furthermore, a hybrid manner can be used to control speed. To decelerate, the user imagines foot movement while ignoring the flashing buttons on the graphical user interface (GUI). If the user wishes to accelerate, then he/she pays attention to a specific flashing button without performing any motor imagery. Two experiments were conducted to assess the BCI control; both a simulated wheelchair in a virtual environment and a real wheelchair were tested. Subjects steered both the simulated and real wheelchairs effectively by controlling the direction and speed with our hybrid BCI system. Data analysis validated the use of our hybrid BCI system to control the direction and speed of a wheelchair. [ABSTRACT FROM AUTHOR]

To control a cursor on a monitor screen, a user generally needs to perform two tasks sequentially. The first task is to move the cursor to a target on the monitor screen (termed a 2-D cursor movement), and the second task is either to select a target of interest by clicking on it or to reject a target that is not of interest by not clicking on it. In a previous study, we implemented the former function in an EEG-based brain–computer interface system using motor imagery and the P300 potential to control the horizontal and vertical cursor movements, respectively. In this study, the target selection or rejection functionality is implemented using a hybrid feature from motor imagery and the P300 potential. Specifically, to select the target of interest, the user must focus his or her attention on a flashing button to evoke the P300 potential, while simultaneously maintaining an idle state of motor imagery. Otherwise, the user performs left-/right-hand motor imagery without paying attention to any buttons to reject the target. Our data analysis and online experimental results validate the effectiveness of our approach. The proposed hybrid feature is shown to be more effective than the use of either the motor imagery feature or the P300 feature alone. Eleven subjects attended our online experiment, in which a trial involved sequential 2-D cursor movement and target selection. The average duration of each trial and average accuracy of target selection were 18.19 s and 93.99\%, respectively, and each target selection or rejection event was performed within 2 s. [ABSTRACT FROM PUBLISHER]

Abstract: We present a new linear discriminant analysis method based on information theory, where the mutual information between linearly transformed input data and the class labels is maximized. First, we introduce a kernel-based estimate of mutual information with a variable kernel size. Furthermore, we devise a learning algorithm that maximizes the mutual information w.r.t. the linear transformation. Two experiments are conducted: the first one uses a toy problem to visualize and compare the transformation vectors in the original input space; the second one evaluates the performance of the method for classification by employing cross-validation tests on four datasets from the UCI repository. Various classifiers are investigated. Our results show that this method can significantly boost class separability over conventional methods, especially for nonlinear classification. [Copyright &y& Elsevier]