Search

A brain–computer interface (BCI) enables a user to communicate directly with a computer using only the central nervous system. An affective BCI (aBCI) monitors and/or regulates the emotional state of the brain, which could facilitate human cognition, communication, decision-making, and health. The last decade has witnessed rapid progress in aBCI research and applications, but there does not exist a comprehensive and up-to-date tutorial on aBCIs. This tutorial fills the gap. It introduces first the basic concepts of BCIs and then, in detail, the individual components in a closed-loop aBCI system, including signal acquisition, signal processing, feature extraction, emotion recognition, and brain stimulation. Next, it describes three representative applications of aBCIs, i.e., cognitive workload recognition, fatigue estimation, and depression diagnosis and treatment. Several challenges and opportunities in aBCI research and applications, including brain signal acquisition, emotion labeling, diversity and size of aBCI datasets, algorithm comparison, negative transfer in emotion recognition, and privacy protection and security of aBCIs, are also explained.

Multimodal signals are powerful for emotion recognition since they can represent emotions comprehensively. In this article, we compare the recognition performance and robustness of two multimodal emotion recognition models: 1) deep canonical correlation analysis (DCCA) and 2) bimodal deep autoencoder (BDAE). The contributions of this article are threefold: 1) we propose two methods for extending the original DCCA model for multimodal fusion: a) weighted sum fusion and b) attention-based fusion; 2) we systemically compare the performance of DCCA, BDAE, and traditional approaches on five multimodal data sets; and 3) we investigate the robustness of DCCA, BDAE, and traditional approaches on SEED-V and DREAMER data sets under two conditions: 1) adding noises to multimodal features and 2) replacing electroencephalography features with noises. Our experimental results demonstrate that DCCA achieves state-of-the-art recognition results on all five data sets: 1) 94.6% on the SEED data set; 2) 87.5% on the SEED-IV data set; 3) 84.3% and 85.6% on the DEAP data set; 4) 85.3% on the SEED-V data set; and 5) 89.0%, 90.6%, and 90.7% on the DREAMER data set. Meanwhile, DCCA has greater robustness when adding various amounts of noises to the SEED-V and DREAMER data sets. By visualizing features before and after DCCA transformation on the SEED-V data set, we find that the transformed features are more homogeneous and discriminative across emotions.

Traditional electroencephalograph (EEG)-based emotion recognition requires a large number of calibration samples to build a model for a specific subject, which restricts the application of the affective brain computer interface (BCI) in practice. We attempt to use the multi-modal data from the past session to realize emotion recognition in the case of a small amount of calibration samples. To solve this problem, we propose a multi-modal domain adaptive variational autoencoder (MMDA-VAE) method, which learns shared cross-domain latent representations of the multi-modal data. Our method builds a multi-modal variational autoencoder (MVAE) to project the data of multiple modalities into a common space. Through adversarial learning and cycle-consistency regularization, our method can reduce the distribution difference of each domain on the shared latent representation layer and realize the transfer of knowledge. Extensive experiments are conducted on two public datasets, SEED and SEED-IV, and the results show the superiority of our proposed method. Our work can effectively improve the performance of emotion recognition with a small amount of labelled multi-modal data.

A brain–computer interface (BCI) enables a user to communicate with a computer directly using brain signals. The most common noninvasive BCI modality, electroencephalogram (EEG), is sensitive to noise/artifact and suffers between-subject/within-subject nonstationarity. Therefore, it is difficult to build a generic pattern recognition model in an EEG-based BCI system that is optimal for different subjects, during different sessions, for different devices and tasks. Usually, a calibration session is needed to collect some training data for a new subject, which is time consuming and user unfriendly. Transfer learning (TL), which utilizes data or knowledge from similar or relevant subjects/sessions/devices/tasks to facilitate learning for a new subject/session/device/task, is frequently used to reduce the amount of calibration effort. This article reviews journal publications on TL approaches in EEG-based BCIs in the last few years, i.e., since 2016. Six paradigms and applications—motor imagery, event-related potentials, steady-state visual evoked potentials, affective BCIs, regression problems, and adversarial attacks—are considered. For each paradigm/application, we group the TL approaches into cross-subject/session, cross-device, and cross-task settings and review them separately. Observations and conclusions are made at the end of the article, which may point to future research directions.

Various reports have shown that the rate of road traffic accidents has increased due to reduced driver vigilance. Therefore, an accurate estimation of the driver’s alertness status plays an important part. To estimate vigilance, we adopt a novel strategy that is a deep autoencoder with subnetwork nodes (DAESN). The proposed network model is designed not only for sparse representation but also for dimension reduction. Some hidden layers are not calculated by randomly acquired, but by replacement technologies. Unlike the traditional electrooculogram (EOG) signals, the forehead EOG (EOGF) signals are collected through forehead electrodes that do not have to surround the eyes, which has a convenient and effective practical application. The root-mean-square error (RMSE) and correlation coefficient (COR) while separately using three EOGF features improved to 0.11/0.79, 0.10/0.83, and 0.11/0.80, respectively. Implemented in an experimental environment, percentage of eye closure over time is calculated in real time through SMI eye-tracking-glasses, up to 120 frames/s. In addition, the time to extract features from the raw signal and display the prediction is only 34 ms, that is the level of the driver’s fatigue can be detected quickly. The experimental study shows that the proposed model for vigilance analysis has better robustness and learning capability.

In recent years, it has been observed that there is an increasing rate of road accidents due to the low vigilance of drivers. Thus, the estimation of drivers’ vigilance state plays a significant role in public transportation safety. We have adopted a feature fusion strategy that combines the electroencephalogram (EEG) signals collected from various sites of the human brain, including forehead, temporal, and posterior and forehead electrooculogram (forehead-EOG) signals, to address this factor. The level of vigilance is predicted through a new learning model known as double-layered neural network with subnetwork nodes (DNNSNs), which comprises several subnetwork nodes, and each node in turn is composed of many hidden nodes that have various capabilities of feature selection (dimension reduced), feature learning, etc. The proposed single modality that uses only forehead-EOG signal exhibits a mean root-mean-square error (RMSE) of 0.12 and a mean Pearson product-moment correlation coefficient (COR) of 0.78. On one hand, an EEG signal achieved a mean RMSE of 0.13 and a mean COR of 0.72. Whereas, on the other, the proposed multimodality achieved values of 0.09 and 0.85 for the mean RMSE and the mean COR, respectively. Experimental results show that the proposed DNNSN with multimodality fusion outperforms the model with single modality for vigilance estimation due to the complementary information between forehead-EOG and EEG. After a favorable learning rate was applied to the input layer, the mean RMSE/COR improved to 0.11/0.79, 0.12/0.74, and 0.08/0.86, respectively. Hence, this quantitative analysis proves that the proposed method provides better feasibility and efficiency learning capability and surmounts other state-of-the-art techniques.

Bilingual word embedding has been shown to be helpful for Statistical Machine Translation (SMT). However, most existing methods suffer from two obvious drawbacks. First, they only focus on simple contexts such as an entire document or a fixed-sized sliding window to build word embedding and ignore latent useful information from the selected context. Second, the word sense but not the word should be the minimal semantic unit; however, most existing methods still use word representation.To overcome these drawbacks, this article presents a novel Graph-Based Bilingual Word Embedding (GBWE) method that projects bilingual word senses into a multidimensional semantic space. First, a bilingual word co-occurrence graph is constructed using the co-occurrence and pointwise mutual information between the words. Then, maximum complete subgraphs (cliques), which play the role of a minimal unit for bilingual sense representation, are dynamically extracted according to the contextual information. Consequently, correspondence analysis, principal component analyses, and neural networks are used to summarize the clique-word matrix into lower dimensions to build the embedding model.Without contextual information, the proposed GBWE can be applied to lexical translation. In addition, given contextual information, GBWE is able to give a dynamic solution for bilingual word representations, which can be applied to phrase translation and generation. Empirical results show that GBWE can enhance the performance of lexical translation, as well as Chinese/French-to-English and Chinese-to-Japanese phrase-based SMT tasks (IWSLT, NTCIR, NIST, and WAT).

Emotions play a crucial role in decision-making, brain activity, human cognition, and social intercourse. This paper proposes a hierarchical network structure with subnetwork nodes to discriminate three human emotions: 1) positive; 2) neutral; and 3) negative. Each subnetwork node embedded in the network that are formed by hundreds of hidden nodes, could be functional as an independent hidden layer for feature representation. The top layer of the hierarchical network, like the mammal cortex in the brain, combine such features generated from subnetwork nodes, but simultaneously, recast these features into a mapping space so that the network can be performed to produce more reliable cognition. The proposed method is compared with other state-of-the-art methods. The experimental results from two different EEG datasets show that a promising result is obtained when using the proposed method with both single and multiple modality.

The Language Model (LM) is an essential component of Statistical Machine Translation (SMT). In this article, we focus on developing efficient methods for LM construction. Our main contribution is that we propose a Natural N-grams based Converting (NNGC) method for transforming a Continuous-Space Language Model (CSLM) to a Back-off N-gram Language Model (BNLM). Furthermore, a Bilingual LM Pruning (BLMP) approach is developed for enhancing LMs in SMT decoding and speeding up CSLM converting. The proposed pruning and converting methods can convert a large LM efficiently by working jointly. That is, a LM can be effectively pruned before it is converted from CSLM without sacrificing performance, and further improved if an additional corpus contains out-of-domain information. For different SMT tasks, our experimental results indicate that the proposed NNGC and BLMP methods outperform the existing counterpart approaches significantly in BLEU and computational cost.

Artificial immune system is a class of computational intelligence methods drawing inspiration from human immune system. As one type of popular artificial immune computing model, clonal selection algorithm (CSA) has been widely used for many optimization problems. CSA mainly generates new schemes by hyper-mutation operators which simulate the immune response process. However, these hyper-mutation operators, which usually perturb the antibodies in population, are semi-blind and not effective enough for complex optimization problems. In this paper, we propose a hybrid learning clonal selection algorithm (HLCSA) by incorporating two learning mechanisms, Baldwinian learning and orthogonal learning, into CSA to guide the immune response process. Specifically, (1) Baldwinian learning is used to direct the genotypic changes based on the Baldwin effect, and this operator can enhance the antibody information by employing other antibodies’ information to alter the search space; (2) Orthogonal learning operator is used to search the space defined by one antibody and its best Baldwinian learning vector. In HLCSA, the Baldwinian learning works for exploration (global search) while the orthogonal learning for exploitation (local refinement). Therefore, orthogonal learning can be viewed as the compensation for the search ability of Baldwinian learning. In order to validate the effectiveness of the proposed algorithm, a suite of sixteen benchmark test problems are fed into HLCSA. Experimental results show that HLCSA performs very well in solving most of the optimization problems. Therefore, HLCSA is an effective and robust algorithm for optimization. [ABSTRACT FROM AUTHOR]

Larger n-gram language models (LMs) perform better in statistical machine translation (SMT). However, the existing approaches have two main drawbacks for constructing larger LMs: 1) it is not convenient to obtain larger corpora in the same domain as the bilingual parallel corpora in SMT; 2) most of the previous studies focus on monolingual information from the target corpora only, and redundant n-grams have not been fully utilized in SMT. Nowadays, continuous-space language model (CSLM), especially neural network language model (NNLM), has been shown great improvement in the estimation accuracies of the probabilities for predicting the target words. However, most of these CSLM and NNLM approaches still consider monolingual information only or require additional corpus. In this paper, we propose a novel neural network based bilingual LM growing method. Compared to the existing approaches, the proposed method enables us to use bilingual parallel corpus for LM growing in SMT. The results show that our new method outperforms the existing approaches on both SMT performance and computational efficiency significantly.

The Nyström method is an efficient technique for the eigenvalue decomposition of large kernel matrices. However, to ensure an accurate approximation, a sufficient number of columns have to be sampled. On very large data sets, the singular value decomposition (SVD) step on the resultant data submatrix can quickly dominate the computations and become prohibitive. In this paper, we propose an accurate and scalable Nyström scheme that first samples a large column subset from the input matrix, but then only performs an approximate SVD on the inner submatrix using the recent randomized low-rank matrix approximation algorithms. Theoretical analysis shows that the proposed algorithm is as accurate as the standard Nyström method that directly performs a large SVD on the inner submatrix. On the other hand, its time complexity is only as low as performing a small SVD. Encouraging results are obtained on a number of large-scale data sets for low-rank approximation. Moreover, as the most computational expensive steps can be easily distributed and there is minimal data transfer among the processors, significant speedup can be further obtained with the use of multiprocessor and multi-GPU systems. [ABSTRACT FROM AUTHOR]

To deal with the problem of detecting driver fatigue, we propose a vigilance estimation method using electrooculogram (EOG) and investigate the correlation between multiple EOG features and vigilance. We examine four kinds of features extracted from EOG: slow eye movement (SEM), saccade, blink, and EOG energy. First, we introduce three eye movement detection algorithms to identify SEMs, saccades, and blinks from EOGs. Then we define and extract twenty different features from those eye movements. Second, the features are processed by a linear dynamic system (LDS) approach, which can effectively remove noises and some EOG components that are irrelevant to vigilance. Finally, we analyze the de-noised features with the vigilance reference obtained from a monotonous visual task. Our experimental results on an EOG data set of twentytwo subjects indicate that combination of SEMs, saccades, blinks, and energy features of EOGs has a high correlation coefficient with the vigilance references, up to the average 0.75. This work provides a potent support to developing vigilance estimation system based on multiple EOG features for detecting driver fatigue.

This article explores the interdependences between subcellular locations and incorporates them with support vector machines for prediction of protein subcellular localisation. Traditional prediction systems utilise a 'flat' structure of classifiers, such as the one-versus-all and one-versus-one schemes, with amino acid compositions to perform the prediction. Apart from those existing studies that ignore the interdependences between subcellular locations, we take advantage of a hierarchical structure to organise the subcellular locations and model their relationships. Here, we propose to use four kinds of hierarchical prediction methods and make comparative studies on three datasets. Experimental results show that three of the hierarchical models outperform the traditional 'flat' model in terms of tree loss values. In particular, one hierarchical model outperforms the traditional 'flat' model for all evaluation measures. Moreover, we gained some valuable insights into the sorting process by using hierarchical structures.

The solid solution crystal Pb(Zn1/3Nb2/3)O3-PbTiO3 [PZNT] has been grown from the high temperature solution. The crystallization behavior of PZNT was investigated. The evaporation of PbO was controlled by sealing crucible and the erosion of the melt or solution on Pt crucible was discussed. By optimizing growth conditions, high quality PZNT crystal was obtained. The ferroelectric properties were measured and the results showed as–grown crystal has high performance.

LiNbO3 (Mg:LN) crystal is an important electro-optical material and MgO-doped LiNbO3 was expected to improve its optical damage resistance. In the present work, we reported the Bridgman growth of MgO-doped LiNbO3 crystal. The growth parameters were discussed and the defects were investigated by the chemical etching method. The etch hillocks and etch pits were observed on the negative Z surface while only etch pits on the positive Z surface by the optical microscope and EPMA. The etching morphology was discussed considering the domain structure.

Langasite (La3Ga5SiO14, LGS) and its isomorphs, were regarded as potential substrate materials for surface acoustic wave (SAW) and bulk acoustic wave (BAW) applications. Among these homologous compounds, the ordered structure crystals have attracted much attention for their low Ga2O3 content, high acoustic velocity and low acoustic loss. In this work, the ordered langasite structure crystals Ca3NbGa3Si2O14 (CNGS) were grown with vertical Bridgman method for the first time. The CNGS crystals with 25mm in diameter and 40mm in length were grown successfully from its stoichiometric melt through LGS crystal of Ø10mm×40mm being chosen as seed. X-ray powder diffraction experiments showed that the CNGS single crystals had a single phase and had the same structure as the LGS crystal. The transmittance spectra with the wavelength from 200 to 800nm were recorded by means of Spectrophotometer at room temperature.