Search

In this article, we study the challenging cross-modal image retrieval task, Composed Query-Based Image Retrieval (CQBIR), in which the query is not a single text query but a composed query, i.e., a reference image, and a modification text. Compared with the conventional cross-modal image-text retrieval task, the CQBIR is more challenging as it requires properly preserving and modifying the specific image region according to the multi-level semantic information learned from the multi-modal query. Most recent works focus on extracting preserved and modified information and compositing it into a unified representation. However, we observe that the preserved regions learned by the existing methods contain redundant modified information, inevitably degrading the overall retrieval performance. To this end, we propose a novel method termed Cross-Modal Attention Preservation (CMAP). Specifically, we first leverage the cross-level interaction to fully account for multi-granular semantic information, which aims to supplement the high-level semantics for effective image retrieval. Furthermore, different from conventional contrastive learning, our method introduces self-contrastive learning into learning preserved information, to prevent the model from confusing the attention for the preserved part with the modified part. Extensive experiments on three widely used CQBIR datasets, i.e., FashionIQ, Shoes, and Fashion200k, demonstrate that our proposed CMAP method significantly outperforms the current state-of-the-art methods on all the datasets. The anonymous implementation code of our CMAP method is available at https://github.com/CFM-MSG/Code_CMAP. [ABSTRACT FROM AUTHOR]

In this paper, we extend the Discrete Empirical Interpolation Method (DEIM) to the third-order tensor case based on the t-product and use it to select important/significant lateral and horizontal slices/features. The proposed Tubal DEIM (TDEIM) is investigated both theoretically and numerically. In particular, the details of the error bounds of the proposed TDEIM method are derived. The experimental results show that the TDEIM can provide more accurate approximations than the existing methods. An application of the proposed method to the supervised classification task is also presented. [ABSTRACT FROM AUTHOR]

Efficient and fast computation of a tensor singular value decomposition (t-SVD) with a few passes over the underlying data tensor is crucial because of its many potential applications. The current/existing subspace randomized algorithms need (2 q + 2) passes over the data tensor to compute a t-SVD, where q is a non-negative integer number (power iteration parameter). In this paper, we propose an efficient and flexible randomized algorithm that can handle any number of passes q, which not necessary need be even. The flexibility of the proposed algorithm in using fewer passes naturally leads to lower computational and communication costs. This advantage makes it particularly appropriate when our task calls for several tensor decompositions or when the data tensors are huge. The proposed algorithm is a generalization of the methods developed for matrices to tensors. The expected/average error bound of the proposed algorithm is derived. Extensive numerical experiments on random and real-world data sets are conducted, and the proposed algorithm is compared with some baseline algorithms. The extensive computer simulation experiments demonstrate that the proposed algorithm is practical, efficient, and in general outperforms the state of the arts algorithms. We also demonstrate how to use the proposed method to develop a fast algorithm for the tensor completion problem. [ABSTRACT FROM AUTHOR]

Introduction: The common spatial patterns (CSP) algorithm is the most popular technique for extracting electroencephalogram (EEG) features in motor imagery based brain-computer interface (BCI) systems. CSP algorithm embeds the dimensionality of multichannel EEG data to extract features of motor imagery tasks. Most previous studies focused on the optimization of the time domain and the spectrum domain of EEG signal to improve the effectiveness of CSP, whereas ignoring the constraint on the projected feature space. Methods: This study proposed a variance characteristic preserving CSP (VPCSP) that is modified by a regularization item based on graph theory. Specifically, we calculated the loss of abnormalities of the projected data while preserving the variance characteristic locally. Then the loss could be rewritten as a matrix with the introduction of the Laplace matrix, which turned it into a generalized eigenvalue problem equivalent to CSP. This study evaluated the proposed method on two public EEG datasets from the BCI competition. The modified method could extract robust and distinguishable features that provided higher classification performance. Experimental results showed that the proposed regularization improved the effectiveness of CSP significantly and achieved superior performance compared with reported modified CSP algorithms significantly. Results: The classification accuracy of the proposed method achieved 87.88 %, 90.07 %, and 76.06 % on public dataset IV part I, III part IVa and the self-collected dataset, respectively. Comparative experiments are conducted on two public datasets and one self-collected dataset. Results showed that the proposed method outperformed the reported algorithm. Discussion: The proposed method can extract robust features to increase the performance of BCI systems. And the proposal still has expandability. These results show that our proposal is a promising candidate for the performance improvement of MI-BCI. [ABSTRACT FROM AUTHOR]

Multi-channel electroencephalography (EEG) is used to capture features associated with motor imagery (MI) based brain-computer interface (BCI) with a wide spatial coverage across the scalp. However, redundant EEG channels are not conducive to improving BCI performance. Therefore, removing irrelevant channels can help improve the classification performance of BCI systems. We present a new method for identifying relevant EEG channels. Our method is based on the assumption that useful channels share related information and that this can be measured by inter-channel connectivity. Specifically, we treat all candidate EEG channels as a graph and define channel selection as the problem of node classification on a graph. Then we design a graph convolutional neural network (GCN) model for channels classification. Channels are selected based on the outputs of our GCN model. We evaluate our proposed GCN-based channel selection (GCN-CS) method on three MI datasets. On three datasets, GCN-CS achieves performance improvements by reducing the number of channels. Specifically, we achieve classification accuracies of 79.76% on Dataset 1, 89.14% on Dataset 2 and 87.96% on Dataset 3, which outperform competing methods significantly. [ABSTRACT FROM AUTHOR]

"Faster, higher, stronger" is the motto of any professional athlete. Does that apply to brain dynamics as well? In our paper, we performed a series of EEG experiments on Visually Evoked Potentials and a series of cognitive tests—reaction time and visual search, with professional eSport players in Counter-Strike: Global Offensive (CS:GO) and novices (control group) in order to find important differences between them. EEG data were studied in a temporal domain by Event-Related Potentials (ERPs) and in a frequency domain by Variational Mode Decomposition. The EEG analysis showed that the brain reaction of eSport players is faster (P300 latency is earlier on average by 20-70 ms, p < 0.005) and stronger (P300 peak amplitude is higher on average by 7-9 mkV, p < 0.01). Professional eSport players also exhibit stronger stimulus-locked alpha-band power. Besides, the Spearman correlation analysis showed a significant correlation between hours spend in CS:GO and mean amplitude of P200 and N200 for the professional players. The comparison of cognitive test results showed the superiority of the professional players to the novices in reaction time (faster) and choice reaction time—faster reaction, but similar correctness, while a significant difference in visual search skills was not detected. Thus, significant differences in EEG signals (in spectrograms and ERPs) and cognitive test results (reaction time) were detected between the professional players and the control group. Cognitive tests could be used to separate skilled players from novices, while EEG testing can help to understand the skilled player's level. The results can contribute to understanding the impact of eSport on a player's cognitive state and associating eSport with a real sport. Moreover, the presented results can be useful for evaluating eSport team members and making training plans. [ABSTRACT FROM AUTHOR]

Purpose: Adrenal gland is a common site of metastasis and on the other hand, metastases are the most frequent malignant adrenal tumors. The aim of this study was to estimate the risk of malignancy in suspicious adrenal mass in patients with a history of cancer. Methods: This is a single-center retrospective analysis of patients with adrenal tumors treated previously for different types of cancers. Between 2004 and 2021 a hundred and six such patients were identified. Mean age of patients was 62.6 years (30-78), and mean time from oncologic treatment was 55.8 months (0-274). The most common primary cancer was kidney (RCC): 29 (27.4%), colon/rectum (CRC): 20 (18.9%) and lung (NSCLC): 20 (18.9%). Results: Of 106 patients, 12 had hormonally active (HA) (11,3%) and 94 (88,7%) non active (HNA) tumors In group of patients with HA tumours 4 had hypercortisolaemia and 8 had elevation of urinary metanephrines. In the first group of HA patients pathology confirmed preoperative diagnosis of adrenocortical cancer and no metastasis was found. In all patients from the second group pheochromocytomas were confirmed. Primary (PM) and secondary (SM) malignancies were found in 50 patients (47.2%). In hormone inactive group only SM - 46/94 (48.9%) were diagnosed. The odds that adrenal lesion was a metastasis were higher if primary cancer was RCC (OR 4.29) and NSCLC (OR 12.3). Metastases were also more likely with high native tumor density, and bigger size in CT. The cut-off values for tumor size and native density calculated from receiver operating characteristic (ROC) curves were 37mm and 24, respectively. Conclusion: Risk of malignancy of adrenal mass in a patient with a history of cancer is high (47,2%), regardless of hormonal status. 47,2% risk of malignancy. In preoperative assessment type of primary cancer, adrenal tumour size and native density on CT should be taken into consideration as predictive factors of malignancy. Native density exceeding 24 HU was the strongest risk factor of adrenal malignancy (RR 3.23), followed by history of lung or renal cancer (RR 2.82) and maximum tumor diameter over 37 mm (RR 2.14). [ABSTRACT FROM AUTHOR]

Neuroendocrine neoplasms of the small intestine (SI-NENs) are one of the most commonly recognized gastroenteropancreatic neuroendocrine neoplasms (GEP-NENs). Carcinoid heart disease (CHD) is the primary cause of death in patients with the carcinoid syndrome (CS). The aim of this retrospective study was to evaluate possible factors impacting upon overall survival (OS) in subjects with both neuroendocrine tumors (NETs) G1/G2 of the small intestine (SI-NET) and CHD. Enrolled in our study of 275 patients with confirmed G1/G2 SI-NET, were 28 (10%) individuals with CHD. Overall survival was assessed using the Kaplan–Meier method. The Cox–Mantel test was used to determine how OS varied between groups. A Cox proportional hazards model was used to conduct univariate analyses of predictive factors for OS and estimate hazard ratios (HRs). Of the 28 individuals with confirmed carcinoid heart disease, 12 (43%) were found to have NET G1 and 16 (57%) were found to have NET G2. Univariate analysis revealed that subjects with CHD and without resection of the primary tumor had a lower OS. Our retrospective study observed that patients who presented with CHD and without resection of primary tumor had worse prognosis of survival. These results suggest that primary tumors may need to be removed when feasible, but further research is needed. However, no solid recommendations can be issued on the basis of our single retrospective study. [ABSTRACT FROM AUTHOR]

Motor imagery-based brain-computer interfaces (MI-BCIs) features are generally extracted from a wide fixed frequency band and time window of EEG signal. The performance suffers from individual differences in corresponding time to MI tasks. In order to solve the problem, in this study, we propose a novel method named Riemannian sparse optimization and Dempster-Shafer fusion of multi-time-frequency patterns (RSODSF) to enhance the decoding efficiency. First, we effectively combine the Riemannian geometry of the spatial covariance matrix with sparse optimization to extract more robust and distinct features. Second, the Dempster-Shafer theory is introduced and used to fuse each time window after sparse optimization of Riemannian features. Besides, the probabilistic values of the support vector machine (SVM) are obtained and transformed to effectively fuse multiple classifiers to leverage potential soft information of multiple trained SVM. The open-access BCI Competition IV dataset IIa and Competition III dataset IIIa are employed to evaluate the performance of the proposed RSODSF. It achieves higher average accuracy (89.7% and 96.8%) than state-of-the-art methods. The improvement over the common spatial patterns (SFBCSP) are respectively 9.9% and 12.4% (p < 0.01, paired t-test). These results show that our proposed RSODSF method is a promising candidate for the performance improvement of MI-BCI. [ABSTRACT FROM AUTHOR]

Low-rank matrix/tensor decompositions are promising methods for reducing the inference time, computation, and memory consumption of deep neural networks (DNNs). This group of methods decomposes the pre-trained neural network weights through low-rank matrix/tensor decomposition and replaces the original layers with lightweight factorized layers. A main drawback of the technique is that it demands a great amount of time and effort to select the best ranks of tensor decomposition for each layer in a DNN. This paper proposes a Proxy-based Automatic tensor Rank Selection method (PARS) that utilizes a Bayesian optimization approach to find the best combination of ranks for neural network (NN) compression. We observe that the decomposition of weight tensors adversely influences the feature distribution inside the neural network and impairs the predictability of the post-compression DNN performance. Based on this finding, a novel proxy metric is proposed to deal with the abovementioned issue and to increase the quality of the rank search procedure. Experimental results show that PARS improves the results of existing decomposition methods on several representative NNs, including ResNet-18, ResNet-56, VGG-16, and AlexNet. We obtain a 3 × FLOP reduction with almost no loss of accuracy for ILSVRC-2012ResNet-18 and a 5.5 × FLOP reduction with an accuracy improvement for ILSVRC-2012 VGG-16. [ABSTRACT FROM AUTHOR]

Though Electroencephalogram (EEG) could objectively reflect emotional states of our human beings, its weak, non-stationary, and low signal-to-noise properties easily cause the individual differences. To enhance the universality of affective brain-computer interface systems, transfer learning has been widely used to alleviate the data distribution discrepancies among subjects. However, most of existing approaches focused mainly on the domain-invariant feature learning, which is not unified together with the recognition process. In this paper, we propose a joint feature adaptation and graph adaptive label propagation model (JAGP) for cross-subject emotion recognition from EEG signals, which seamlessly unifies the three components of domain-invariant feature learning, emotional state estimation and optimal graph learning together into a single objective. We conduct extensive experiments on two benchmark SEED_IV and SEED_V data sets and the results reveal that 1) the recognition performance is greatly improved, indicating the effectiveness of the triple unification mode; 2) the emotion metric of EEG samples are gradually optimized during model training, showing the necessity of optimal graph learning, and 3) the projection matrix-induced feature importance is obtained based on which the critical frequency bands and brain regions corresponding to subject-invariant features can be automatically identified, demonstrating the superiority of the learned shared subspace. [ABSTRACT FROM AUTHOR]

With the popularity of smartphones and the pervasion of mobile apps, people spend more and more time to interact with a diversity of apps on their smartphones, especially for young population. This raises a question: how people allocate attention to interfaces of apps during using them. To address this question, we, in this study, designed an experiment with two sessions (i.e., Session1: browsing original interfaces; Session 2: browsing interfaces after removal of colors and background) integrating with an eyetracking system. Attention fixation durations were recorded by an eye-tracker while participants browsed app interfaces. The whole screen of smartphone was divided into four even regions to explore fixation durations. The results revealed that participants gave significantly longer total fixation duration on the bottom left region compared to other regions in the session (1) Longer total fixation duration on the bottom was preserved, but there is no significant difference between left side and right side in the session2. Similar to the finding of total fixation duration, first fixation duration is also predominantly paid on the bottom area of the interface. Moreover, the skill in the use of mobile phone was quantified by assessing familiarity and accuracy of phone operation and was investigated in the association with the fixation durations. We found that first fixation duration of the bottom left region is significantly negatively correlated with the smartphone operation level in the session 1, but there is no significant correlation between them in the session (2) According to the results of ratio exploration, the ratio of the first fixation duration to the total fixation duration is not significantly different between areas of interest for both sessions. The findings of this study provide insights into the attention allocation during browsing app interfaces and are of implications on the design of app interfaces and advertisements as layout can be optimized according to the attention allocation to maximally deliver information. [ABSTRACT FROM AUTHOR]

Electroencephalogram(EEG) signals are generated from central nervous system which are difficult to disguise, leading to its popularity in emotion recognition. Recently,semi-supervisedlearning exhibits promisingemotion recognition performance by involving unlabeled EEG data into model training. However, if we first build a graph to characterize the sample similarities and then perform label propagation on this graph, these two steps cannotwell collaborate with each other. In this paper, we propose an OptimalGraph coupledSemi-Supervised Learning (OGSSL) model for EEG emotion recognition by unifying the adaptive graph learning and emotion recognition into a single objective. Besides, we improve the label indicator matrix of unlabeledsamples in order to directly obtain theiremotional states. Moreover, the key EEG frequency bands and brain regions in emotion expression are automatically recognized by the projectionmatrix of OGSSL. Experimental results on the SEED-IV data set demonstrate that 1) OGSSL achieves excellent average accuracies of 76.51%, 77.08% and 81.29% in three cross-sessionemotion recognition tasks, 2) OGSSL is competent for discriminative EEG feature selection in emotion recognition, and 3) the Gamma frequency band, the left/righttemporal, prefrontal,and (central) parietal lobes are identified to be more correlated with the occurrence of emotions. [ABSTRACT FROM AUTHOR]

It is difficult to identify optimal cut-off frequencies for filters used with the common spatial pattern (CSP) method in motor imagery (MI)-based brain-computer interfaces (BCIs). Most current studies choose filter cut-frequencies based on experience or intuition, resulting in sub-optimal use of MI-related spectral information in the electroencephalography (EEG). To improve information utilization, we propose a SincNet-based hybrid neural network (SHNN) for MI-based BCIs. First, raw EEG is segmented into different time windows and mapped into the CSP feature space. Then, SincNets are used as filter bank band-pass filters to automatically filter the data. Next, we used squeeze-and-excitation modules to learn a sparse representation of the filtered data. The resulting sparse data were fed into convolutional neural networks to learn deep feature representations. Finally, these deep features were fed into a gated recurrent unit module to seek sequential relations, and a fully connected layer was used for classification. We used the BCI competition IV datasets 2a and 2b to verify the effectiveness of our SHNN method. The mean classification accuracies (kappa values) of our SHNN method are 0.7426 (0.6648) on dataset 2a and 0.8349 (0.6697) on dataset 2b, respectively. The statistical test results demonstrate that our SHNN can significantly outperform other state-of-the-art methods on these datasets. [ABSTRACT FROM AUTHOR]

The P300-based brain–computer interfaces (BCIs) enable participants to communicate by decoding the electroencephalography (EEG) signal. Different regions of the brain correspond to various mental activities. Therefore, removing weak task-relevant and noisy channels through channel selection is necessary when decoding a specific type of activity from EEG. It can improve the recognition accuracy and reduce the training time of the subsequent models. This study proposes a novel block sparse Bayesian-based channel selection method for the P300 speller. In this method, we introduce block sparse Bayesian learning (BSBL) into the channel selection of P300 BCI for the first time and propose a regional smoothing BSBL (RSBSBL) by combining the spatial distribution properties of EEG. The RSBSBL can determine the number of channels adaptively. To ensure practicality, we design an automatic selection iteration strategy model to reduce the time cost caused by the inverse operation of the large-size matrix. We verified the proposed method on two public P300 datasets and on our collected datasets. The experimental results show that the proposed method can remove the inferior channels and work with the classifier to obtain high-classification accuracy. Hence, RSBSBL has tremendous potential for channel selection in P300 tasks. [ABSTRACT FROM AUTHOR]

Schizophrenia seriously affects the quality of life. To date, both simple (e.g., linear discriminant analysis) and complex (e.g., deep neural network) machine-learning methods have been utilized to identify schizophrenia based on functional connectivity features. The existing simple methods need two separate steps (i.e., feature extraction and classification) to achieve the identification, which disables simultaneous tuning for the best feature extraction and classifier training. The complex methods integrate two steps and can be simultaneously tuned to achieve optimal performance, but these methods require a much larger amount of data for model training. To overcome the aforementioned drawbacks, we proposed a multikernel capsule network (MKCapsnet), which was developed by considering the brain anatomical structure. Kernels were set to match partition sizes of the brain anatomical structure in order to capture interregional connectivities at the varying scales. With the inspiration of the widely used dropout strategy in deep learning, we developed capsule dropout in the capsule layer to prevent overfitting of the model. The comparison results showed that the proposed method outperformed the state-of-the-art methods. Besides, we compared performances using different parameters and illustrated the routing process to reveal characteristics of the proposed method. MKCapsnet is promising for schizophrenia identification. Our study first utilized the capsule neural network for analyzing functional connectivity of magnetic resonance imaging (MRI) and proposed a novel multikernel capsule structure with the consideration of brain anatomical parcellation, which could be a new way to reveal brain mechanisms. In addition, we provided useful information in the parameter setting, which is informative for further studies using a capsule network for other neurophysiological signal classification. [ABSTRACT FROM AUTHOR]

The motor imagery (MI) based brain-computer interfaces (BCIs) have been proposed as a potential physical rehabilitation technology. However, the low classification accuracy achievable with MI tasks is still a challenge when building effective BCI systems. We propose a novel MI classification model based on measurement of functional connectivity between brain regions and graph theory. Specifically, motifs describing local network structures in the brain are extracted from functional connectivity graphs. A graph embedding model called Ego-CNNs is then used to build a classifier, which can convert the graph from a structural representation to a fixed-dimensional vector for detecting critical structure in the graph. We validate our proposed method on four datasets, and the results show that our proposed method produces high classification accuracies in two-class classification tasks (92.8% for dataset 1, 93.4% for dataset 2, 96.5% for dataset 3, and 80.2% for dataset 4) and multiclass classification tasks (90.33% for dataset 1). Our proposed method achieves a mean Kappa value of 0.88 across nine participants, which is superior to other methods we compared it to. These results indicate that there is a local structural difference in functional connectivity graphs extracted under different motor imagery tasks. Our proposed method has great potential for motor imagery classification in future studies. [ABSTRACT FROM AUTHOR]

Objective The stimulus color of P300-BCI systems has been successfully modified. However, the effects of different color combinations have not been widely investigated. In this study, we designed new stimulus patterns to evaluate the influence of color modulation on the BCI performance and waveforms of the evoked related potential (ERP).Methods Comparison was performed for three new stimulus patterns consisting of red face and colored block-shape, namely, red face with a white rectangle (RFW), red face with a blue rectangle (RFB), and red face with a red rectangle (RFR). Bayesian linear discriminant analysis (BLDA) was used to construct the individual classifier model. Repeated-measures ANOVA and Bonferroni correction were applied for statistical analysis. Results The RFW pattern obtained the highest average online accuracy with 96.94%, and those of RFR and RFB patterns were 93.61% and of 92.22% respectively. Significant differences in online accuracy and information transfer rate (ITR) were found between RFW and RFR patterns (p < 0.05). Conclusion Compared with RFR and RFB patterns, RFW yielded the best performance in P300-BCI. These new stimulus patterns with different color combinations have considerable importance to BCI applications and user-friendliness. [ABSTRACT FROM AUTHOR]

During the acquisition of electroencephalographic (EEG) signals, data may be missing or corrupted by noise and artifacts. To reconstruct the incomplete data, EEG signals are firstly converted into a three-order tensor (multi-dimensional data) of shape time × channel × trial. Then, the missing data can be efficiently recovered by applying a tensor completion method (TCM). However, there is not a unique way to organize channels and trials in a tensor, and different numbers of channels are available depending on the EEG setting used, which may affect the quality of the tensor completion results. The main goal of this paper is to evaluate the robustness of EEG completion methods with several designed parameters such as the ordering of channels and trials, the number of channels, and the amount of missing data. In this work, the results of completing missing data by several TCMs were compared. To emulate different scenarios of missing data, three different patterns of missing data were designed. Firstly, the amount of missing data on completion effects was analyzed, including the time lengths of missing data and the number of channels or trials affected by missing data. Secondly, the numerical stability of the completion methods was analyzed by shuffling the indices along channels or trials in the EEG data tensor. Finally, the way that the number of electrodes of EEG tensors influences completion effects was assessed by changing the number of channels. Among all the applied TCMs, the simultaneous tensor decomposition and completion (STDC) method achieves the best performance in providing stable results when the amount of missing data or the electrode number of EEG tensors is changed. In other words, STDC proves to be an excellent choice of TCM, since permutations of trials or channels have almost no influence on the complete results. The STDC method can efficiently complete the missing EEG signals. The designed simulations can be regarded as a procedure to validate whether or not a completion method is useful enough to complete EEG signals. [ABSTRACT FROM AUTHOR]

Motor imagery (MI) based brain–computer interfaces help patients with movement disorders to regain the ability to control external devices. Common spatial pattern (CSP) is a popular algorithm for feature extraction in decoding MI tasks. However, due to noise and nonstationarity in electroencephalography (EEG), it is not optimal to combine the corresponding features obtained from the traditional CSP algorithm. In this paper, we designed a novel CSP feature selection framework that combines the filter method and the wrapper method. We first evaluated the importance of every CSP feature by the infinite latent feature selection method. Meanwhile, we calculated Wasserstein distance between feature distributions of the same feature under different tasks. Then, we redefined the importance of every CSP feature based on two indicators mentioned above, which eliminates half of CSP features to create a new CSP feature subspace according to the new importance indicator. At last, we designed the improved binary gravitational search algorithm (IBGSA) by rebuilding its transfer function and applied IBGSA on the new CSP feature subspace to find the optimal feature set. To validate the proposed method, we conducted experiments on three public BCI datasets and performed a numerical analysis of the proposed algorithm for MI classification. The accuracies were comparable to those reported in related studies and the presented model outperformed other methods in literature on the same underlying data. [ABSTRACT FROM AUTHOR]

The common spatial patterns (CSP) algorithm is the most popular spatial filtering method applied to extract electroencephalogram (EEG) features for motor imagery (MI) based brain-computer interface (BCI) systems. The effectiveness of the CSP algorithm depends on optimal selection of the frequency band and time window from the EEG. Many algorithms have been designed to optimize frequency band selection for CSP, while few algorithms seek to optimize the time window. This study proposes a novel framework, termed common time-frequency-spatial patterns (CTFSP), to extract sparse CSP features from multi-band filtered EEG data in multiple time windows. Specifically, the whole MI period is first segmented into multiple subseries using a sliding time window approach. Then, sparse CSP features are extracted from multiple frequency bands in each time window. Finally, multiple support vector machine (SVM) classifiers with the Radial Basis Function (RBF) kernel are trained to identify the MI tasks and the voting result of these classifiers determines the final output of the BCI. This study applies the proposed CTFSP algorithm to three public EEG datasets (BCI competition III dataset IVa, BCI competition III dataset IIIa, and BCI competition IV dataset 1) to validate its effectiveness, compared against several other state-of-the-art methods. The experimental results demonstrate that the proposed algorithm is a promising candidate for improving the performance of MI-BCI systems. [ABSTRACT FROM AUTHOR]

In electroencephalography (EEG)-based affective brain–computer interfaces (aBCIs), there is a consensus that EEG features extracted from different frequency bands and channels have different abilities in emotion expression. Besides, EEG is so weak and non-stationary that easily causes distribution discrepancies for EEG data collected at different times; therefore, it is necessary to explore the affective activation patterns in cross-session emotion recognition. To address these two problems, we propose a self-weighted semi-supervised classification (SWSC) model in this article for joint EEG-based cross-session emotion recognition and affective activation patterns mining, whose merits include: 1) using both the labeled and unlabeled samples from different sessions for better capturing data characteristics; 2) introducing a self-weighted variable to learn the importance of EEG features adaptively and quantitatively; and 3) mining the activation patterns including the critical EEG frequency bands and channels automatically based on the learned self-weighted variable. Extensive experiments are conducted on the benchmark SEED_IV emotional dataset and SWSC obtained excellent average accuracies of 77.40%, 79.55%, and 81.52% in three cross-session emotion recognition tasks. Moreover, SWSC identifies that the Gamma frequency band contributes the most and the EEG channels in prefrontal, left/right temporal, and (central) parietal lobes are more important for cross-session emotion recognition. [ABSTRACT FROM AUTHOR]

The common spatial patterns (CSP) algorithm is one of the most frequently used and effective spatial filtering methods for extracting relevant features for use in motor imagery brain–computer interfaces (MI-BCIs). However, the inherent defect of the traditional CSP algorithm is that it is highly sensitive to potential outliers, which adversely affects its performance in practical applications. In this work, we propose a novel feature optimization and outlier detection method for the CSP algorithm. Specifically, we use the minimum covariance determinant (MCD) to detect and remove outliers in the dataset, then we use the Fisher score to evaluate and select features. In addition, in order to prevent the emergence of new outliers, we propose an iterative minimum covariance determinant (IMCD) algorithm. We evaluate our proposed algorithm in terms of iteration times, classification accuracy and feature distribution using two BCI competition datasets. The experimental results show that the average classification performance of our proposed method is 12% and 22.9% higher than that of the traditional CSP method in two datasets (p < 0. 0 5), and our proposed method obtains better performance in comparison with other competing methods. The results show that our method improves the performance of MI-BCI systems. [ABSTRACT FROM AUTHOR]

Tensor decomposition methods have been widely applied to big data analysis as they bring multiple modes and aspects of data to a unified framework, which allows us to discover complex internal structures and correlations of data. Unfortunately most existing approaches are not designed to meet the challenges posed by big data dilemma. This paper attempts to improve the scalability of tensor decompositions and makes two contributions: A flexible and fast algorithm for the CP decomposition (FFCP) of tensors based on their Tucker compression; A distributed randomized Tucker decomposition approach for arbitrarily big tensors but with relatively low multilinear rank. These two algorithms can deal with huge tensors, even if they are dense. Extensive simulations provide empirical evidence of the validity and efficiency of the proposed algorithms. [ABSTRACT FROM AUTHOR]

Structured Tucker tensor decomposition models complete or incomplete multiway data sets (tensors), where the core tensor and the factor matrices can obey different constraints. The model includes block-term decomposition or canonical polyadic decomposition as special cases. We propose a very flexible optimization method for the structured Tucker decomposition problem, based on the second-order Levenberg-Marquardt optimization, using an approximation of the Hessian matrix by the Krylov subspace method. An algorithm with limited sensitivity of the decomposition is included. The proposed algorithm is shown to perform well in comparison to existing tensor decomposition methods. [ABSTRACT FROM AUTHOR]

Optimal rank selection is an important issue in tensor decomposition problems, especially for Tensor Train (TT) and Tensor Ring (TR) (also known as Tensor Chain) decompositions. In this paper, a new rank selection method for TR decomposition has been proposed for automatically finding near-optimal TR ranks, which result in a lower storage cost, especially for tensors with inexact TT or TR structures. In many of the existing approaches, TR ranks are determined in advance or by using truncated Singular Value Decomposition (t-SVD). There are also other approaches for selecting TR ranks adaptively. In our approach, the TR ranks are not determined in advance, but are increased gradually in each iteration until the model achieves a desired approximation accuracy. For this purpose, in each iteration, the sensitivity of the approximation error to each of the core tensors is measured and the core tensors with the highest sensitivity measures are selected and their sizes are increased. Simulation results confirmed that the proposed approach reduces the storage cost considerably and allows us to find optimal model in TR format, while preserving the desired accuracy of the approximation. [ABSTRACT FROM AUTHOR]

Tactile P300 brain-computer interface (BCI) generally has a worse accuracy and information transfer rate (ITR) than the visual-based BCI. It may be due to the fact that human beings have a relatively poor tactile perception. This study investigated the influence of visual attention on the performance of a tactile P300 BCI. We designed our paradigms based on a novel cheeks-stim paradigm which attached the stimulators on the subject's cheeks. Two paradigms were designed as follows: a paradigm with no visual attention and another paradigm with visual attention to the target position. Eleven subjects were invited to perform the two paradigms. We also recorded and analyzed the eyeball movement data during the paradigm with visual attention to explore whether the eyeball movement would have an effect on the BCI classification. The average online accuracy was 89.09% for the paradigm with visual attention, which was significantly higher than that of the paradigm with no visual attention (70.45%). Significant difference in ITR was also found between the two paradigms (p < 0. 0 5). The results demonstrated that visual attention was an effective method to improve the performance of tactile P300 BCI. Our findings suggested that it may be feasible to complete an efficient tactile BCI system by adding visual attention. [ABSTRACT FROM AUTHOR]

Tactile perception, the primary sensing channel of the tactile brain-computer interface (BCI), is a complicated process. Skin friction plays a vital role in tactile perception. This study aimed to examine the effects of skin friction on tactile P300 BCI performance. Two kinds of oddball paradigms were designed, silk-stim paradigm (SSP) and linen-stim paradigm (LSP), in which silk and linen were wrapped on target vibration motors, respectively. In both paradigms, the disturbance vibrators were wrapped in cotton. The experimental results showed that LSP could induce stronger event-related potentials (ERPs) and achieved a higher classification accuracy and information transfer rate (ITR) compared with SSP. The findings indicate that high skin friction can achieve high performance in tactile BCI. This work provides a novel research direction and constitutes a viable basis for the future tactile P300 BCI, which may benefit patients with visual impairments. [ABSTRACT FROM AUTHOR]

Background. Stroke is the leading cause of serious and long-term disability worldwide. Survivors may recover some motor functions after rehabilitation therapy. However, many stroke patients missed the best time period for recovery and entered into the sequela stage of chronic stroke. Method. Studies have shown that motor imagery- (MI-) based brain-computer interface (BCI) has a positive effect on poststroke rehabilitation. This study used both virtual limbs and functional electrical stimulation (FES) as feedback to provide patients with a closed-loop sensorimotor integration for motor rehabilitation. An MI-based BCI system acquired, analyzed, and classified motor attempts from electroencephalogram (EEG) signals. The FES system would be activated if the BCI detected that the user was imagining wrist dorsiflexion on the instructed side of the body. Sixteen stroke patients in the sequela stage were randomly assigned to a BCI group and a control group. All of them participated in rehabilitation training for four weeks and were assessed by the Fugl-Meyer Assessment (FMA) of motor function. Results. The average improvement score of the BCI group was 3.5, which was higher than that of the control group (0.9). The active EEG patterns of the four patients in the BCI group whose FMA scores increased gradually became centralized and shifted to sensorimotor areas and premotor areas throughout the study. Conclusions. Study results showed evidence that patients in the BCI group achieved larger functional improvements than those in the control group and that the BCI-FES system is effective in restoring motor function to upper extremities in stroke patients. This study provides a more autonomous approach than traditional treatments used in stroke rehabilitation. [ABSTRACT FROM AUTHOR]

Background. Stroke is the leading cause of serious and long-term disability worldwide. Survivors may recover some motor functions after rehabilitation therapy. However, many stroke patients missed the best time period for recovery and entered into the sequela stage of chronic stroke. Method. Studies have shown that motor imagery- (MI-) based brain-computer interface (BCI) has a positive effect on poststroke rehabilitation. This study used both virtual limbs and functional electrical stimulation (FES) as feedback to provide patients with a closed-loop sensorimotor integration for motor rehabilitation. An MI-based BCI system acquired, analyzed, and classified motor attempts from electroencephalogram (EEG) signals. The FES system would be activated if the BCI detected that the user was imagining wrist dorsiflexion on the instructed side of the body. Sixteen stroke patients in the sequela stage were randomly assigned to a BCI group and a control group. All of them participated in rehabilitation training for four weeks and were assessed by the Fugl-Meyer Assessment (FMA) of motor function. Results. The average improvement score of the BCI group was 3.5, which was higher than that of the control group (0.9). The active EEG patterns of the four patients in the BCI group whose FMA scores increased gradually became centralized and shifted to sensorimotor areas and premotor areas throughout the study. Conclusions. Study results showed evidence that patients in the BCI group achieved larger functional improvements than those in the control group and that the BCI-FES system is effective in restoring motor function to upper extremities in stroke patients. This study provides a more autonomous approach than traditional treatments used in stroke rehabilitation. [ABSTRACT FROM AUTHOR]

The performance of motor imagery (MI) based Brain-computer interfacing (BCI) is easily affected by noise and redundant information that exists in the multi-channel electroencephalogram (EEG). To solve this problem, many temporal and spatial feature based channel selection methods have been proposed. However, temporal and spatial features do not accurately reflect changes in the power of the oscillatory EEG. Thus, spectral features of MI-related EEG signals may be useful for channel selection. Bispectrum analysis is a technique developed for extracting non-linear and non-Gaussian information from non-linear and non-Gaussian signals. The features extracted from bispectrum analysis can provide frequency domain information about the EEG. Therefore, in this study, we propose a bispectrum-based channel selection (BCS) method for MI-based BCI. The proposed method uses the sum of logarithmic amplitudes (SLA) and the first order spectral moment (FOSM) features extracted from bispectrum analysis to select EEG channels without redundant information. Three public BCI competition datasets (BCI competition IV dataset 1, BCI competition III dataset IVa and BCI competition III dataset IIIa) were used to validate the effectiveness of our proposed method. The results indicate that our BCS method outperforms use of all channels (83.8% vs 69.4%, 86.3% vs 82.9% and 77.8% vs 68.2%, respectively). Furthermore, compared to the other state-of-the-art methods, our BCS method also can achieve significantly better classification accuracies for MI-based BCI (Wilcoxon signed test, p < 0.05). [ABSTRACT FROM AUTHOR]

Previous studies made progress in the early diagnosis of Alzheimer’s disease (AD) using electroencephalography (EEG) without considering EEG connectivity. To fill this gap, we explored significant differences between early AD patients and controls based on frequency domain and spatial properties using functional connectivity in mild cognitive impairment (MCI) and mild AD datasets. Four global metrics, network resilience, connection-level metrics and node versatility were used to distinguish between controls and patients. The results show that the main frequency bands that are different between MCI patients and controls are the $\theta $ and low $\alpha $ bands, and the differently affected brain areas are the frontal, left temporal and parietal areas. Compared to MCI patients, in patients with mild AD, the main frequency bands that are different are the low and high $\alpha $ bands, and the main differently affected brain region is a larger right temporal area. Four LOFC bands were used as input to train the ResNet-18 model. For the MCI dataset, the average accuracy of 20 runs was 93.42% and the best accuracy was 98.33%, while for the mild AD dataset, the average accuracy was 98.54% and the best accuracy was 100%. To determine the timing of early treatment and discovering the susceptible patients, and to slow the progression of the disease, we assume that the occurrence of MCI and mild AD and their progression to more serious AD and dementia could be inferred by analyzing the topological structure of the brain network generated by EEG. Our findings provide a novel solution for connectome-based biomarker analysis to improve personalized medicine. [ABSTRACT FROM AUTHOR]

A novel algorithm to solve the quadratic programming (QP) problem over ellipsoids is proposed. This is achieved by splitting the QP problem into two optimisation sub-problems, (1) quadratic programming over a sphere and (2) orthogonal projection. Next, an augmented-Lagrangian algorithm is developed for this multiple constraint optimisation. Benefitting from the fact that the QP over a single sphere can be solved in a closed form by solving a secular equation, we derive a tighter bound of the minimiser of the secular equation. We also propose to generate a new positive semidefinite matrix with a low condition number from the matrices in the quadratic constraint, which is shown to improve convergence of the proposed augmented-Lagrangian algorithm. Finally, applications of the quadratically constrained QP to bounded linear regression and tensor decomposition paradigms are presented. [ABSTRACT FROM AUTHOR]

Presurgical investigations for categorizing focal patterns are crucial, leading to localization and surgical removal of the epileptic focus. This paper presents a machine learning approach using information theoretic features extracted from high-frequency subbands to detect the epileptic focus from interictal intracranial electroencephalogram (iEEG). It is known that high-frequency subbands (>80 Hz) include important biomarkers such as high-frequency oscillations (HFOs) for identifying epileptic focus commonly referred to as the seizure onset zone (SOZ). In this analysis, the multi-channel interictal iEEG signals were splitted into segments and each segment was decomposed into multiple high-frequency subbands. The different types of entropy were calculated for each of the subbands and the sparse linear discriminant analysis (sLDA) was applied to select the prominent entropy features. Due to the imbalance of SOZ and non-SOZ channels in iEEG data, the use of machine learning techniques is always tricky. To deal with the imbalanced learning problem, an adaptive synthetic oversampling approach (ADASYN) with radial basis function kernel-based SVM was used to detect the focal segments. Finally, the epileptic focus was identified based on detection of focal segments on SOZ and non-SOZ channels. Eight patients were examined to observe the efficiency of the automatic detector. The experimental results and statistical tests indicate that the proposed automatic detector can identify the epileptic focus accurately and efficiently. [ABSTRACT FROM AUTHOR]

Motor imagery (MI) is a mental representation of motor behavior and has been widely used in electroencephalogram based brain–computer interfaces (BCIs). Several studies have demonstrated the efficacy of MI-based BCI-feedback training in post-stroke rehabilitation. However, in the earliest stage of the training, calibration data typically contain insufficient discriminability, resulting in unreliable feedback, which may decrease subjects' motivation and even hinder their training. To improve the performance in the early stages of MI training, a novel hybrid BCI paradigm based on MI and P300 is proposed in this study. In this paradigm, subjects are instructed to imagine writing the Chinese character following the flash order of the desired Chinese character displayed on the screen. The event-related desynchronization/synchronization (ERD/ERS) phenomenon is produced with writing based on one's imagination. Simultaneously, the P300 potential is evoked by the flash of each stroke. Moreover, a fusion method of P300 and MI classification is proposed, in which unreliable P300 classifications are corrected by reliable MI classifications. Twelve healthy naïve MI subjects participated in this study. Results demonstrated that the proposed hybrid BCI paradigm yielded significantly better performance than the single-modality BCI paradigm. The recognition accuracy of the fusion method is significantly higher than that of P300 (p < 0.05) and MI (p < 0.01). Moreover, the training data size can be reduced through fusion of these two modalities. [ABSTRACT FROM AUTHOR]

Cognitive motor dissociation describes a subset of patients with disorders of consciousness who show neuroimaging evidence of consciousness but no detectable command-following behaviours. Although essential for family counselling, decision-making, and the design of rehabilitation programmes, the prognosis for patients with cognitive motor dissociation remains under-investigated. The current study included 78 patients with disorders of consciousness who showed no detectable command-following behaviours. These patients included 45 patients with unresponsive wakefulness syndrome and 33 patients in a minimally conscious state, as diagnosed using the Coma Recovery Scale-Revised. Each patient underwent an EEG-based brain-computer interface experiment, in which he or she was instructed to perform an item-selection task (i.e. select a photograph or a number from two candidates). Patients who achieved statistically significant brain-computer interface accuracies were identified as cognitive motor dissociation. Two evaluations using the Coma Recovery Scale-Revised, one before the experiment and the other 3 months later, were carried out to measure the patients' behavioural improvements. Among the 78 patients with disorders of consciousness, our results showed that within the unresponsive wakefulness syndrome patient group, 15 of 18 patients with cognitive motor dissociation (83.33%) regained consciousness, while only five of the other 27 unresponsive wakefulness syndrome patients without significant brain-computer interface accuracies (18.52%) regained consciousness. Furthermore, within the minimally conscious state patient group, 14 of 16 patients with cognitive motor dissociation (87.5%) showed improvements in their Coma Recovery Scale-Revised scores, whereas only four of the other 17 minimally conscious state patients without significant brain-computer interface accuracies (23.53%) had improved Coma Recovery Scale-Revised scores. Our results suggest that patients with cognitive motor dissociation have a better outcome than other patients. Our findings extend current knowledge of the prognosis for patients with cognitive motor dissociation and have important implications for brain-computer interface-based clinical diagnosis and prognosis for patients with disorders of consciousness. [ABSTRACT FROM AUTHOR]

Objective. Tactile P300 brain-computer interfaces (BCIs) can be manipulated by users who only need to focus their attention on a single-target stimulus within a stream of tactile stimuli. To date, a multitude of tactile P300 BCIs have been proposed. In this study, our main purpose is to explore and investigate the effects of visual attention on a tactile P300 BCI. Approach. We designed a conventional tactile P300 BCI where vibration stimuli were provided by five stimulators and two of them were fixed on target locations on the participant's left and right wrists. Two conditions (one condition with visual attention and the other condition without visual attention) were tested by eleven healthy participants. Main Results. Our results showed that, when participants visually attended to the location of target stimulus, significantly higher classification accuracies and information transfer rates were obtained (both for p < 0.05). Furthermore, participants reported that visually attending to the stimulus made it easier to identify the target stimulus in random sequences of vibration stimuli. Significance. These findings suggest that visual attention has positive effects on both tactile P300 BCI performance and user-evaluation. [ABSTRACT FROM AUTHOR]

Neurons selective for faces exist in humans and monkeys. However, characteristics of face cell receptive fields are poorly understood. In this theoretical study, we explore the effects of complexity, defined as algorithmic information (Kolmogorov complexity) and logical depth, on possible ways that face cells may be organized. We use tensor decompositions to decompose faces into a set of components, called tensorfaces, and their associated weights, which can be interpreted as model face cells and their firing rates. These tensorfaces form a high-dimensional representation space in which each tensorface forms an axis of the space. A distinctive feature of the decomposition algorithm is the ability to specify tensorface complexity. We found that low-complexity tensorfaces have blob-like appearances crudely approximating faces, while high-complexity tensorfaces appear clearly face-like. Low-complexity tensorfaces require a larger population to reach a criterion face reconstruction error than medium- or high-complexity tensorfaces, and thus are inefficient by that criterion. Low-complexity tensorfaces, however, generalize better when representing statistically novel faces, which are faces falling beyond the distribution of face description parameters found in the tensorface training set. The degree to which face representations are parts based or global forms a continuum as a function of tensorface complexity, with low and medium tensorfaces being more parts based. Given the computational load imposed in creating high-complexity face cells (in the form of algorithmic information and logical depth) and in the absence of a compelling advantage to using high-complexity cells, we suggest face representations consist of a mixture of low- and medium-complexity face cells. [ABSTRACT FROM AUTHOR]

Objective: Previous studies have shown that combing with color properties may be used as part of the display presented to BCI users in order to improve performance. Build on this, we explored the effects of combinations of face stimuli with three primary colors (RGB) on BCI performance which is assessed by classification accuracy and information transfer rate (ITR). Furthermore, we analyzed the waveforms of three patterns. Methods: We compared three patterns in which semitransparent face is overlaid three primary colors as stimuli: red semitransparent face (RSF), green semitransparent face (GSF), and blue semitransparent face (BSF). Bayesian linear discriminant analysis (BLDA) was used to construct the individual classifier model. In addition, a Repeated-measures ANOVA (RM-ANOVA) and Bonferroni correction were chosen for statistical analysis. Results: The results indicated that the RSF pattern achieved the highest online averaged accuracy with 93.89%, followed by the GSF pattern with 87.78%, while the lowest performance was caused by the BSF pattern with an accuracy of 81.39%. Furthermore, significant differences in classification accuracy and ITR were found between RSF and GSF (p < 0.05) and between RSF and BSF patterns (p < 0.05). Conclusion: The semitransparent faces colored red (RSF) pattern yielded the best performance of the three patterns. The proposed patterns based on ERP-BCI system have a clinically significant impact by increasing communication speed and accuracy of the P300-speller for patients with severe motor impairment. [ABSTRACT FROM AUTHOR]

P300-based brain-computer interfaces (BCIs) provide an additional communication channel for individuals with communication disabilities. In general, P300-based BCIs need to be trained, offline, for a considerable period of time, which causes users to become fatigued. This reduces the efficiency and performance of the system. In order to shorten calibration time and improve system performance, we introduce the concept of a generic model set. We used ERP data from 116 participants to train the generic model set. The resulting set consists of ten models, which are trained by weighted linear discriminant analysis (WLDA). Twelve new participants were then invited to test the validity of the generic model set. The results demonstrated that all new participants matched the best generic model. The resulting mean classification accuracy equaled 80% after online training, an accuracy that was broadly equivalent to the typical training model method. Moreover, the calibration time was shortened by 70.7% of the calibration time of the typical model method. In other words, the best matching model method only took 81s to calibrate, while the typical model method took 276s. There were also significant differences in both accuracy and raw bit rate between the best and the worst matching model methods. We conclude that the strategy of combining the generic models with online training is easily accepted and achieves higher levels of user satisfaction (as measured by subjective reports). Thus, we provide a valuable new strategy for improving the performance of P300-based BCI. [ABSTRACT FROM AUTHOR]