Search

Speech imagery is emerging as a significant neuro‐paradigm for designing an electroencephalography (EEG)‐based brain–computer interface (BCI) system for the purpose of rehabilitation, medical neurology, and to aid people with disabilities in interacting with their surroundings. Neural correlates of speech imagery EEG signals are variable and weak as compared to the vocal state; hence, it is challenging to interpret them using machine learning (ML)–based classifiers. The applicability of modern deep learning methods such as convolutional neural networks (CNNs) and bidirectional long short‐term memory (Bi‐LSTM) networks has seen substantial advances in complex EEG signal analysis as compared to ML‐based methods. The objective of this article is to design a firefly‐optimized discrete wavelet transform (DWT) and CNN‐Bi‐LSTM–based imagined speech recognition (ISR) system to interpret imagined speech EEG signals. This study utilizes two publicly available datasets. EEG signal is enhanced using firefly optimization algorithm (FOA)–based optimized soft thresholding of high‐frequency detail components obtained by DWT decomposition. The enhanced EEG signal is augmented using sliding window data augmentation to increase the training data. Frequency‐domain features like power spectral density (PSD), frequency band power (FBP), band ratios, peak frequency, mean frequency, median frequency, spectral entropy, and relative power are extracted from augmented EEG segments. The extracted feature vector is fed to the designed CNN‐Bi‐LSTM classifier such that the EEG data are classified into two‐class, three‐class, and four‐class categories. To achieve optimal performance, the CNN‐Bi‐LSTM model was optimized using the Keras tuner library. The designed CNN consists of one‐dimensional (1‐D) convolutional layers and max pooling layers for familiarizing local associations along with mining hierarchical connections, and the Bi‐LSTM network acquires long‐term dependencies from the features learned by the former CNN. Bi‐LSTM network improves the performance and acquires potentially more affluent representations by looking at the sequence in both forward and reverse ways to capture representations that might be left unexploited by the sequential‐order kind alone. The performance of the designed FOA‐DWT‐CNN‐Bi‐LSTM–based ISR system is assessed using four evaluation measures: accuracy, F1 score, recall, and precision. It is found that the proposed system achieves the highest classification accuracy of 99.43 ± 2.5%, 94.41 ± 3.31%, and 89.57 ± 4.3% for two‐class, three‐class, and four‐class categories, respectively. [ABSTRACT FROM AUTHOR]

The classification performance of endogenous electroencephalogram (EEG) brain-computer interfaces (BCIs) can be improved by hybridizing the paradigm through the use of commands from multiple paradigms. Hybrid paradigms using motor imagery (MI) and speech imagery (SI) have shown promise, but there is a lack of research into: i) their effectiveness when compared to pure MI and SI for multiclass problems, and ii) automated command selection. This study investigates multiclass MI and SI hybrid paradigms and compares the results to those obtained using pure MI and SI. Performance was assessed using F1 score and accuracy. The performances of all possible hybrid paradigm designs were assessed. The analysis indicated that hybridization does not always guarantee improved performance when compared to the pure paradigms, and there is inter-subject variation in the best paradigm. This confirmed the need for automated subject-specific hybrid paradigm designs. An automated hybrid paradigm selection technique using successive halving (SH) for expedited computational times was developed and results were compared to those obtained using a standard grid search. The SH approach resulted in an improvement in F1 score of 21.09% and 36.86% compared to MI and SI and led to a reduction in computational times of 82.80% compared to grid search. [ABSTRACT FROM AUTHOR]

Brain-Computer Interfaces (BCIs) aim to establish a pathway between the brain and an external device without the involvement of the motor system, relying exclusively on neural signals. Such systems have the potential to provide a means of communication for patients who have lost the ability to speak due to a neurological disorder. Traditional methodologies for decoding imagined speech directly from brain signals often deploy static classifiers, that is, decoders that are computed once at the beginning of the experiment and remain unchanged throughout the BCI use. However, this approach might be inadequate to effectively handle the non-stationary nature of electroencephalography (EEG) signals and the learning that accompanies BCI use, as parameters are expected to change, and all the more in a real-time setting. To address this limitation, we developed an adaptive classifier that updates its parameters based on the incoming data in real time. We first identified optimal parameters (the update coefficient, UC) to be used in an adaptive Linear Discriminant Analysis (LDA) classifier, using a previously recorded EEG dataset, acquired while healthy participants controlled a binary BCI based on imagined syllable decoding. We subsequently tested the effectiveness of this optimization in a real-time BCI control setting. Twenty healthy participants performed two BCI control sessions based on the imagery of two syllables, using a static LDA and an adaptive LDA classifier, in randomized order. As hypothesized, the adaptive classifier led to better performances than the static one in this real-time BCI control task. Furthermore, the optimal parameters for the adaptive classifier were closely aligned in both datasets, acquired using the same syllable imagery task. These findings highlight the effectiveness and reliability of adaptive LDA classifiers for real-time imagined speech decoding. Such an improvement can shorten the training time and favor the development of multi-class BCIs, representing a clear interest for non-invasive systems notably characterized by low decoding accuracies. [ABSTRACT FROM AUTHOR]

We propose a Transfer learning-enabled electroencephalography-based intuitive brain-computer interface system by utilizing phase-based brain functional connectivity methods such as phase lag index (PLI) and Intersite phase clustering (ISPC) along with power features to explore both phase and power-based information from electroencephalography (EEG) signals. Time-frequency decomposition using a complex morlet wavelet is applied to analyze the signal components in both the time and frequency domains and extract phase connectivity and power features. Functional connectivity methods aim to recognize functional interactions and statistical mutuality among signals acquired across various brain areas. The phase-based connectivity features are extracted simultaneously for multiple channels to investigate the phase synchronization among EEG signals across the entire brain. Next, Graph theory is adopted to trace connectivity between brain regions by calculating the connectivity degree of extracted PLI and ISPC features with other electrodes. In Parallel, Discrete wavelet convolution is performed to calculate the time variable frequency band’s specific power from the imagined speech EEG data. Finally, Time-frequency images of the above-mentioned PLI, ISPC, and EEG power features are fed as input to DenseNet-121 architecture for classification. Dense Net architecture overcomes the problem of ‘vanishing gradient’ by connecting each layer directly with other layers, making the network densely connected. The maximum classification accuracy achieved is 100%, 99.14%, and 98.72% for binary, three-class, and four-class classifications, respectively. The experimental results indicate that the proposed phase-based connectivity features, EEG power, and the DenseNet-121 model have achieved excellent accuracy for two public datasets, outperforming the state-of-the-art methods. The outstanding results strengthen the possibility of real-time EEG-based intuitive brain-computer interface communication.

Speech imagery is one type of mental imagery specific to processing verbal sequences and plays a vital role in human thought processes. Speech imagery has become an interesting paradigm for researchers as speech imagery has a high similarity to real voice communication. Electroencephalography (EEG) is a noninvasive electrophysiological technique that measures the mental state of the brain directly from the scalp. The nature of the acquired EEG signals is nonlinear and nonstationary. As EEG signals have a low signal-to-noise ratio (SNR), artifacts occur during acquisition. Hence, an efficient framework of pre-processing is required to obtain artifact-free EEG for further applications. Selection of the optimal pre-processing techniques for EEG still remains a challenging task. This work focuses on employing and comparing the different pre-processing techniques and lists out the optimal solutions for pre-processing Speech imagery-based EEG signals. The techniques are compared based on the Mean Square Error and Peak Signal-to-Noise Ratio values. [ABSTRACT FROM AUTHOR]

Speech imagery recognition from electroencephalograms (EEGs) could potentially become a strong contender among non-invasive brain-computer interfaces (BCIs). In this report, first we extract language representations as the difference of line-spectra of phones by statistically analyzing many EEG signals from the Broca area. Then we extract vowels by using iterative search from hand-labeled short-syllable data. The iterative search process consists of principal component analysis (PCA) that visualizes linguistic representation of vowels through eigen-vectors 0(m), and subspace method (SM) that searches an optimum line-spectrum for redesigning 0(m). The extracted linguistic representation of Japanese vowels /i/ /e/ /a/ /o/ /u/ shows 2 distinguished spectral peaks (P1, P2) in the upper frequency range. The 5 vowels are aligned on the P1-P2 chart. A 5-vowel recognition experiment using a data set of 5 subjects and a convolutional neural network (CNN) classifier gave a mean accuracy rate of 72.6%. [ABSTRACT FROM AUTHOR]

Brain-Computer Interfaces (BCIs) aim to establish a pathway between the brain and an external device without the involvement of the motor system, relying exclusively on neural signals. Such systems have the potential to provide a means of communication for patients who have lost the ability to speak due to a neurological disorder. Traditional methodologies for decoding imagined speech directly from brain signals often deploy static classifiers, that is, decoders that are computed once at the beginning of the experiment and remain unchanged throughout the BCI use. However, this approach might be inadequate to effectively handle the non-stationary nature of electroencephalography (EEG) signals and the learning that accompanies BCI use, as parameters are expected to change, and all the more in a real-time setting. To address this limitation, we developed an adaptive classifier that updates its parameters based on the incoming data in real time. We first identified optimal parameters (the update coefficient, UC) to be used in an adaptive Linear Discriminant Analysis (LDA) classifier, using a previously recorded EEG dataset, acquired while healthy participants controlled a binary BCI based on imagined syllable decoding. We subsequently tested the effectiveness of this optimization in a real-time BCI control setting. Twenty healthy participants performed two BCI control sessions based on the imagery of two syllables, using a static LDA and an adaptive LDA classifier, in randomized order. As hypothesized, the adaptive classifier led to better performances than the static one in this real-time BCI control task. Furthermore, the optimal parameters for the adaptive classifier were closely aligned in both datasets, acquired using the same syllable imagery task. These findings highlight the effectiveness and reliability of adaptive LDA classifiers for real-time imagined speech decoding. Such an improvement can shorten the training time and favor the development of multi-class BCIs, representing a clear interest for non-invasive systems notably characterized by low decoding accuracies.

Brain computer interface (BCI) based on speech imagery can help people with motor disorders communicate their thoughts to the outside world in a natural way. Due to being portable, non-invasive, and safe, functional near-infrared spectroscopy (fNIRS) is preferred for developing BCIs. Previous BCIs based on fNIRS mainly relied on activation information, which ignored the functional connectivity between neural areas. In this study, a 4-class speech imagery BCI based on fNIRS is presented to decode simplified articulation motor imagery (only the movements of jaw and lip were retained) of different vowels. Synchronization information in the motor cortex was extracted as features. In multiclass (four classes) settings, the mean subject-dependent classification accuracies approximated or exceeded 40% in the 0-2.5 s and 0-10 s time windows, respectively. In binary class settings (the average classification accuracies of all pairwise comparisons between two vowels), the mean subject-dependent classification accuracies exceeded 70% in the 0-2.5 s and 0-10 s time windows. These results demonstrate that connectivity features can effectively differentiate different vowels even if the time window size was reduced from 10 s to 2.5 s and the decoding performance in both the time windows was almost the same. This finding suggests that speech imagery BCI based on fNIRS can be further optimized in terms of feature extraction and command generation time reduction. In addition, simplified articulation motor imagery of vowels can be distinguished, and therefore, the potential contribution of articulation motor imagery information extracted from the motor cortex should be emphasized in speech imagery BCI based on fNIRS to improve decoding performance. [ABSTRACT FROM AUTHOR]

Brain-computer interfaces (BCIs) based on electroencephalography (EEG) have been attracted enough attention by researchers. In order to determine whether silent reading can improve mental tasks for BCI systems, this paper proposed a two-step experiment: mental tasks with speech imagery and mental tasks without speech imagery. Reading Chinese characters in mind is set as speech imagery. Since Chinese characters are monosyllabic, it is very convenient to read them in mind with related mental tasks simultaneously. Ten Chinese subjects are trained by two steps in this experiment. Feature vectors of EEG signals are extracted and classified by common spatial patterns (CSP) and support vector machine (SVM), respectively. Compared with just mental tasks, the accuracies between two tasks have been significantly improved by appending speech imagery, and the average of accuracies of ten subjects is increased from 76.3% to 82.3%. During the imagery period, the temporal stability of EEG signals is evaluated by Cronbach’s alpha coefficients. The steadiness of signals is different between mental tasks, and EEG signals are more stabilization with speech imagery. The stability of brain activity is conducive to the operation of BCIs. [ABSTRACT FROM AUTHOR]

Highlights: [•] In order to complement existing motor-based control paradigms, speech imagery was proposed. [•] Feature vectors of EEG signal were extracted by common spatial patterns (CSP). [•] Feature vectors were classified by support vector machine (SVM). [ABSTRACT FROM AUTHOR]