Search

The aim of this study is to develop a steady-state visual evoked potential (SSVEP)-based brain-computer interface (BCI) system with enhanced performance in an augmented reality (AR) environment by dynamically adjusting colors of visual stimuli to contrast with the background seen through the transparent display. Our proposed method extracts the average color value from the area surrounding the visual stimulus location. It then calculates the contrast value using the HSV color model and applies this to the stimulus color. In an offline experiment, we determined the optimal visual stimulus presentation strategy by comparing the performances of three different methods for determining the colors of visual stimuli in an AR environment. We then evaluated the feasibility of the proposed strategy through online experiments conducted in both indoor and outdoor conditions. The classification performance of the SSVEP-BCI system in an AR environment based on our proposed stimulus presentation strategy was 95.0% for a window size of 3.5 s in offline experiments performed with 17 participants. This was significantly higher than the performance of the conventional black-and-white color strategy. Additionally, it was confirmed by the online experiments that there was no large performance degradation between indoor and outdoor uses.

Abstract Background Brain-computer interface (BCI) technology can enhance neural plasticity and motor recovery in persons with stroke. However, the effects of BCI training with motor imagery (MI)-contingent feedback versus MI-independent feedback remain unclear. This study aimed to investigate whether the contingent connection between MI-induced brain activity and feedback influences functional and neural plasticity outcomes. We hypothesized that BCI training, with MI-contingent feedback, would result in greater improvements in upper limb function and neural plasticity compared to BCI training, with MI-independent feedback. Methods This randomized controlled trial included persons with chronic stroke who underwent BCI training involving functional electrical stimulation feedback on the affected wrist extensor. Primary outcomes included the Medical Research Council (MRC) scale score for muscle strength in the wrist extensor (MRC-WE) and active range of motion in wrist extension (AROM-WE). Resting-state electroencephalogram recordings were used to assess neural plasticity. Results Compared to the MI-independent feedback BCI group, the MI-contingent feedback BCI group showed significantly greater improvements in MRC-WE scores (mean difference = 0.52, 95% CI = 0.03–1.00, p = 0.036) and demonstrated increased AROM-WE at 4 weeks post-intervention (p = 0.019). Enhanced functional connectivity in the affected hemisphere was observed in the MI-contingent feedback BCI group, correlating with MRC-WE and Fugl-Meyer assessment-distal scores. Improvements were also observed in the unaffected hemisphere’s functional connectivity. Conclusions BCI training with MI-contingent feedback is more effective than MI-independent feedback in improving AROM-WE, MRC, and neural plasticity in individuals with chronic stroke. BCI technology could be a valuable addition to conventional rehabilitation for stroke survivors, enhancing recovery outcomes. Trial registration CRIS (KCT0009013).

IntroductionRecent resting-state electroencephalogram (EEG) studies have consistently reported an association between aberrant functional brain networks (FBNs) and treatment-resistant traits in patients with major depressive disorder (MDD). However, little is known about the changes in FBNs in response to external stimuli in these patients. This study investigates whether changes in the salience network (SN) could predict responsiveness to pharmacological treatment in resting-state and external stimuli conditions.MethodsThirty-one drug-naïve patients with MDD (aged 46.61 ± 10.05, female 28) and twenty-one healthy controls (aged 43.86 ± 14.14, female 19) participated in the study. After 8 weeks of pharmacological treatment, the patients were divided into non-remitted MDD (nrMDD, n = 14) and remitted-MDD (rMDD, n = 17) groups. EEG data under three conditions (resting-state, standard, and deviant) were analyzed. The SN was constructed with three cortical regions as nodes and weighted phase-lag index as edges, across alpha, low-beta, high-beta, and gamma bands. A repeated measures analysis of the variance model was used to examine the group-by-condition interaction. Machine learning-based classification analyses were also conducted between the nrMDD and rMDD groups.ResultsA notable group-by-condition interaction was observed in the high-beta band between nrMDD and rMDD. Specifically, patients with nrMDD exhibited hypoconnectivity between the dorsal anterior cingulate cortex and right insula (p = 0.030). The classification analysis yielded a maximum classification accuracy of 80.65%.ConclusionOur study suggests that abnormal condition-dependent changes in the SN could serve as potential predictors of pharmacological treatment efficacy in patients with MDD.

Transcranial photobiomodulation (tPBM) has been widely studied for its potential to enhance cognitive functions of the elderly. However, its efficacy varies, with some individuals exhibiting no significant response to the treatment. Considering these inconsistencies, we introduce a machine learning approach aimed at distinguishing between individuals that respond and do not respond to tPBM treatment based on functional near-infrared spectroscopy (fNIRS) acquired before the treatment. We measured nine cognitive scores and recorded fNIRS data from 62 older adults with cognitive decline (43 experimental and 19 control subjects). The experimental group underwent tPBM intervention over a span of 12 weeks. Based on the comparison of the global cognitive score (GCS), merging the nine cognitive scores into a single representation, acquired before and after tPBM treatment, we classified all participants as responders or non-responders to tPBM with a threshold for the GCS change. The fNIRS data were recorded during the resting state, recognition memory task (RMT), Stroop task, and verbal fluency task. A regularized support vector machine was utilized to classify the responders and non-responders to tPBM. The most promising performance of our machine learning model was observed when using the fNIRS data collected during the RMT, which yielded an accuracy of 0.8537, an F1-score of 0.8421, sensitivity of 0.7619, and specificity of 0.95. To the best of our knowledge, this is the first study to demonstrate the feasibility of predicting the tPBM efficacy. Our approach is expected to contribute to more efficient treatment planning by excluding ineffective treatment options.

Background: Temporal interference stimulation (TIS) is a neuromodulation technique that could stimulate deep brain regions by inducing interfering electrical signals based on high-frequency electrical stimulations of multiple electrode pairs from outside the brain. Despite numerous TIS studies, however, there has been limited investigation into the neurochemical effects of TIS. Objective: We performed two experiments to investigate the effect of TIS on the medial forebrain bundle (MFB)-evoked phasic dopamine (DA) response. Methods: In the first experiment, we applied TIS next to a carbon fiber microelectrode (CFM) to examine the modulation of the MFB-evoked phasic DA response in the striatum (STr). Beat frequencies and intensities of TIS were 0, 2, 6, 10, 20, 60, 130 Hz and 0, 100, 200, 300, 400, 500 μA. In the second experiment, we examined the effect of TIS with a 2 Hz beat frequency (based on the first experiment) on MFB-evoked phasic DA release when applied above the cortex (with a simulation-based stimulation site targeting the striatum). We employed 0 Hz and 2 Hz beat frequencies and a control condition without stimulation. Results: In the first experiment, TIS with a beat frequency of 2 Hz and an intensity of 400 μA or greater decreased MFB-evoked phasic DA release by roughly 40%, which continued until the experiment's end. In contrast, TIS at beat frequencies other than 2 Hz and intensities less than 400 μA did not affect MFB-evoked phasic DA release. In the second experiment, TIS with a 2 Hz beat frequency decreased only the MFB-evoked phasic DA response, but the reduction in DA release was not sustained. Conclusions: STr-applied and cortex-applied TIS with delta frequency dampens evoked phasic DA release in the STr. These findings demonstrate that TIS could influence the neurochemical modulation of the brain.

Abstract Background Cross-frequency phase-amplitude coupling (PAC) of cortical oscillations is observed within and across cortical regions during higher-order cognitive processes. Particularly, the PAC of alpha and gamma waves in the occipital cortex is closely associated with visual perception. In theory, gamma oscillation is a neuronal representation of visual stimuli, which drives the duty cycle of visual perception together with alpha oscillation. Therefore, it is believed that the timing of entrainment in alpha-gamma PAC may play a critical role in the performance of visual perception. We hypothesized that transcranial alternating current stimulation (tACS) with gamma waves entrained at the troughs of alpha waves would enhance the dynamic visual acuity (DVA). Method We attempted to modulate the performance of DVA by using tACS. The waveforms of the tACS were tailored to target PAC over the occipital cortex. The waveforms contained gamma (80 Hz) waves oscillating at either the peaks or troughs of alpha (10 Hz) waves. Participants performed computerized DVA task before, immediately after, and 10 min after each stimulation sessions. EEG and EOG were recorded during the DVA task to assess inter-trial phase coherence (ITPC), the alpha-gamma PAC at occipital site and the eye movements. Results tACS with gamma waves entrained at alpha troughs effectively enhanced DVA, while the tACS with gamma waves entrained at alpha peaks did not affect DVA performance. Importantly, analyses of EEG and EOG showed that the enhancement of DVA performance originated solely from the neuromodulatory effects, and was not related to the modulation of saccadic eye movements. Consequently, DVA, one of the higher-order cognitive abilities, was successfully modulated using tACS with a tailored waveform. Conclusions Our experimental results demonstrated that DVA performances were enhanced when tACS with gamma bursts entrained on alpha wave troughs were applied over the occipital cortex. Our findings suggest that using tACS with tailored waveforms, modulation of complex neuronal features could effectively enhance higher-order cognitive abilities such as DVA, which has never been modulated with conventional noninvasive brain stimulation methods.

Abstract While zero-phase lag synchronization between multiple brain regions has been widely observed, relatively recent reports indicate that systematic phase delays between cortical regions reflect the direction of communications between cortical regions. For example, it has been suggested that a non-zero phase delay of electroencephalography (EEG) signals at the gamma frequency band between the bilateral parietal areas may reflect the direction of communication between these areas. We hypothesized that the direction of communication between distant brain areas might be modulated by multi-site transcranial alternating current stimulation (tACS) with specific phase delays other than 0° and 180°. In this study, a new noninvasive brain stimulation (NIBS) method called multi-site multi-phase tACS (msmp-tACS) was proposed. The efficacy of the proposed method was tested in a case study using a visuospatial working memory (VWM) paradigm in which the optimal stimulation conditions including amplitudes and phases of multiple scalp electrodes were determined using finite element analysis adopting phasor representation. msmp-tACS was applied over the bilateral intraparietal sulci (IPS) and showed that 80 Hz tACS with the phase for the right IPS leading that for the left IPS by 90° (= 3.125 ms) partialized VWM performance toward the right visual hemifield. The three stimulation conditions were synchronized, RL, and LR, which refers to stimulation condition with no phase lag, stimulation phase of right IPS (rIPS) leading left IPS (lIPS) by 90° and the stimulation of lIPS leading rIPS by 90°, respectively. The lateralization of VWM significantly shifted towards right visual hemifield under the RL condition compared to the synchronized and LR conditions. The shift in VWM was the result of the stimulation affecting both left and right visual hemifield trials to certain degrees, rather than significantly increasing or decreasing VWM capacity of a specific visual hemifield. Altered brain dynamics caused by msmp-tACS partialized VWM performance, likely due to modulation of effective connectivity between the rIPS and lIPS. Our results suggest that msmp-tACS is a promising NBS method that can effectively modulate cortical networks that cannot be readily modulated with conventional multi-site stimulation methods.

Abstract Augmented reality (AR) is a computer graphics technique that creates a seamless interface between the real and virtual worlds. AR usage rapidly spreads across diverse areas, such as healthcare, education, and entertainment. Despite its immense potential, AR interface controls rely on an external joystick, a smartphone, or a fixed camera system susceptible to lighting. Here, an AR‐integrated soft wearable electronic system that detects the gestures of a subject for more intuitive, accurate, and direct control of external systems is introduced. Specifically, a soft, all‐in‐one wearable device includes a scalable electrode array and integrated wireless system to measure electromyograms for real‐time continuous recognition of hand gestures. An advanced machine learning algorithm embedded in the system enables the classification of ten different classes with an accuracy of 96.08%. Compared to the conventional rigid wearables, the multi‐channel soft wearable system offers an enhanced signal‐to‐noise ratio and consistency over multiple uses due to skin conformality. The demonstration of the AR‐integrated soft wearable system for drone control captures the potential of the platform technology to offer numerous human–machine interface opportunities for users to interact remotely with external hardware and software.

Over the past decades, brain-computer interfaces (BCIs) have been developed to provide individuals with an alternative communication channel toward external environment. Although the primary target users of BCI technologies include the disabled or the elderly, most newly developed BCI applications have been tested with young, healthy people. In the present study, we developed an online home appliance control system using a steady-state visual evoked potential (SSVEP)-based BCI with visual stimulation presented in an augmented reality (AR) environment and electrooculogram (EOG)-based eye tracker. The performance and usability of the system were evaluated for individuals aged over 65. The participants turned on the AR-based home automation system using an eye-blink-based switch, and selected devices to control with three different methods depending on the user’s preference. In the online experiment, all 13 participants successfully completed the designated tasks to control five home appliances using the proposed system, and the system usability scale exceeded 70. Furthermore, the BCI performance of the proposed online home appliance control system surpassed the best results of previously reported BCI systems for the elderly.

Abstract Background Early diagnosis of mild cognitive impairment (MCI) is essential for timely treatment planning. With recent advances in the wearable technology, interest has increasingly shifted toward computer-aided self-diagnosis of MCI using wearable electroencephalography (EEG) devices in daily life. However, no study so far has investigated the optimal electrode configurations for the efficient diagnosis of MCI while considering the design factors of wearable EEG devices. In this study, we aimed to determine the optimal channel configurations of wearable EEG devices for the computer-aided diagnosis of MCI. Method We employed an EEG dataset collected from 21 patients with MCI and 21 healthy control subjects. After evaluating the classification accuracies for all possible electrode configurations for the two-, four-, six-, and eight-electrode conditions using a support vector machine, the optimal electrode configurations that provide the highest diagnostic accuracy were suggested for each electrode condition. Results The highest classification accuracies of 74.04% ± 4.82, 82.43% ± 6.14, 86.28% ± 2.81, and 86.85% ± 4.97 were achieved for the optimal two-, four-, six-, and eight-electrode configurations, respectively, which demonstrated the possibility of precise machine-learning-based diagnosis of MCI with a limited number of EEG electrodes. Additionally, further simulations with the EEG dataset revealed that the optimal electrode configurations had significantly higher classification accuracies than commercial EEG devices with the same number of electrodes, which suggested the importance of electrode configuration optimization for wearable EEG devices based on clinical EEG datasets. Conclusions This study highlighted that the optimization of the electrode configuration, assuming the wearable EEG devices can potentially be utilized for daily life monitoring of MCI, is necessary to enhance the performance and portability.

Abstract Working memory (WM) is a complex cognitive function involved in the temporary storage and manipulation of information, which has been one of the target cognitive functions to be restored in neurorehabilitation. WM capacity is known to be proportional to the number of gamma cycles nested in a single theta cycle. Therefore, gamma-band transcranial alternating current stimulation (tACS) should be dependent of the stimulation frequency; however, the results of previous studies that employed 40 Hz tACS have not been consistent. The optimal locations and injection currents of multiple scalp electrodes were determined based on numerical simulations of electric field. Experiments were conducted with 20 healthy participants. The order of three stimulation conditions (40 Hz tACS, 80 Hz tACS, and sham stimulation) were randomized but counterbalanced. Visual hemifield-specific visual WM capacity was assessed using a delayed visual match to the sample task. High gamma tACS significantly increased WM capacity, while low gamma tACS had no significant effect. Notably, 80 Hz tACS increased WM capacity on both the left and right visual hemifields, while previous tACS studies only reported the effects of tACS on contralateral hemifields. This is the first study to investigate the frequency-dependent effect of gamma-band tACS on WM capacity. Our findings also suggest that high gamma tACS might influence not only WM capacity but also communication between interhemispheric cortical regions. It is expected that high gamma tACS could be a promising neurorehabilitation method to enhance higher-order cognitive functions with similar mechanisms.

Controlling the in-car environment, including temperature and ventilation, is necessary for a comfortable driving experience. However, it often distracts the driver’s attention, potentially causing critical car accidents. In the present study, we implemented an in-car environment control system utilizing a brain-computer interface (BCI) based on steady-state visual evoked potential (SSVEP). In the experiment, four visual stimuli were displayed on a laboratory-made head-up display (HUD). This allowed the participants to control the in-car environment by simply staring at a target visual stimulus, i.e., without pressing a button or averting their eyes from the front. The driving performances in two realistic driving tests—obstacle avoidance and car-following tests—were then compared between the manual control condition and SSVEP-BCI control condition using a driving simulator. In the obstacle avoidance driving test, where participants needed to stop the car when obstacles suddenly appeared, the participants showed significantly shorter response time (1.42 ± 0.26 s) in the SSVEP-BCI control condition than in the manual control condition (1.79 ± 0.27 s). No-response rate, defined as the ratio of obstacles that the participants did not react to, was also significantly lower in the SSVEP-BCI control condition (4.6 ± 14.7%) than in the manual control condition (20.5 ± 25.2%). In the car-following driving test, where the participants were instructed to follow a preceding car that runs at a sinusoidally changing speed, the participants showed significantly lower speed difference with the preceding car in the SSVEP-BCI control condition (15.65 ± 7.04 km/h) than in the manual control condition (19.54 ± 11.51 km/h). The in-car environment control system using SSVEP-based BCI showed a possibility that might contribute to safer driving by keeping the driver’s focus on the front and thereby enhancing the overall driving performance.

Abstract Default mode network (DMN) is a set of functional brain structures coherently activated when individuals are in resting-state. In this study, we constructed multi-frequency band resting-state EEG-based DMN functional network models for major psychiatric disorders to easily compare their pathophysiological characteristics. Phase-locking values (PLVs) were evaluated to quantify functional connectivity; global and nodal clustering coefficients (CCs) were evaluated to quantify global and local connectivity patterns of DMN nodes, respectively. DMNs of patients with post-traumatic stress disorder (PTSD), obsessive compulsive disorder (OCD), panic disorder, major depressive disorder (MDD), bipolar disorder, schizophrenia (SZ), mild cognitive impairment (MCI), and Alzheimer’s disease (AD) were constructed relative to their demographically-matched healthy control groups. Overall DMN patterns were then visualized and compared with each other. In global CCs, SZ and AD showed hyper-clustering in the theta band; OCD, MCI, and AD showed hypo-clustering in the low-alpha band; OCD and MDD showed hypo-clustering and hyper-clustering in low-beta, and high-beta bands, respectively. In local CCs, disease-specific patterns were observed. In the PLVs, lowered theta-band functional connectivity between the left lingual gyrus and the left hippocampus was frequently observed. Our comprehensive comparisons suggest EEG-based DMN as a useful vehicle for understanding altered brain networks of major psychiatric disorders.

Abstract Transorbital electrical stimulation (tES) has been studied as a new noninvasive method for treating intractable eye diseases by delivering weak electrical current to the eye through a pair of electrodes attached to the skin around the eye. Studies have reported that the therapeutic effect of tES is determined by the effective stimulation of retinal cells that are densely distributed in the posterior part of the retina. However, in conventional tES with a pair of electrodes, a greater portion of the electric field is delivered to the anterior part of the retina. In this study, to address this issue, a new electrode montage with multiple electrodes was proposed for the effective delivery of electric fields to the posterior retina. Electric field analysis based on the finite element method was performed with a realistic human head model, and optimal injection currents were determined using constrained convex optimization. The resultant electric field distributions showed that the proposed multi-channel tES enables a more effective stimulation of the posterior retina than the conventional tES with a pair of electrodes.

In this study, we proposed a new type of hybrid visual stimuli for steady-state visual evoked potential (SSVEP)-based brain-computer interfaces (BCIs), which incorporate various periodic motions into conventional flickering stimuli (FS) or pattern reversal stimuli (PRS). Furthermore, we investigated optimal periodic motions for each FS and PRS to enhance the performance of SSVEP-based BCIs. Periodic motions were implemented by changing the size of the stimulus according to four different temporal functions denoted by none, square, triangular, and sine, yielding a total of eight hybrid visual stimuli. Additionally, we developed the extended version of filter bank canonical correlation analysis (FBCCA), which is a state-of-the-art training-free classification algorithm for SSVEP-based BCIs, to enhance the classification accuracy for PRS-based hybrid visual stimuli. Twenty healthy individuals participated in the SSVEP-based BCI experiment to discriminate four visual stimuli with different frequencies. An average classification accuracy and information transfer rate (ITR) were evaluated to compare the performances of SSVEP-based BCIs for different hybrid visual stimuli. Additionally, the user's visual fatigue for each of the hybrid visual stimuli was also evaluated. As the result, for FS, the highest performances were reported when the periodic motion of the sine waveform was incorporated for all window sizes except for 3 s. For PRS, the periodic motion of the square waveform showed the highest classification accuracies for all tested window sizes. A significant statistical difference in the performance between the two best stimuli was not observed. The averaged fatigue scores were reported to be 5.3 ± 2.05 and 4.05 ± 1.28 for FS with sine-wave periodic motion and PRS with square-wave periodic motion, respectively. Consequently, our results demonstrated that FS with sine-wave periodic motion and PRS with square-wave periodic motion could effectively improve the BCI performances compared to conventional FS and PRS. In addition, thanks to its low visual fatigue, PRS with square-wave periodic motion can be regarded as the most appropriate visual stimulus for the long-term use of SSVEP-based BCIs, particularly for window sizes equal to or larger than 2 s.

Electroencephalography (EEG)-based diagnosis of psychiatric diseases using machine-learning approaches has made possible the objective diagnosis of various psychiatric diseases. The objective of this study was to improve the performance of a resting-state EEG-based computer-aided diagnosis (CAD) system to diagnose post-traumatic stress disorder (PTSD), by optimizing the frequency bands used to extract EEG features. We used eyes-closed resting-state EEG data recorded from 77 PTSD patients and 58 healthy controls (HC). Source-level power spectrum densities (PSDs) of the resting-state EEG data were extracted from 6 frequency bands (delta, theta, alpha, low-beta, high-beta, and gamma), and the PSD features of each frequency band and their combinations were independently used to discriminate PTSD and HC. The classification performance was evaluated using support vector machine with leave-one-out cross validation. The PSD features extracted from slower-frequency bands (delta and theta) showed significantly higher classification performance than those of relatively higher-frequency bands. The best classification performance was achieved when using delta PSD features (86.61%), which was significantly higher than that reported in a recent study by about 13%. The PSD features selected to obtain better classification performances could be explained from a neurophysiological point of view, demonstrating the promising potential to develop a clinically reliable EEG-based CAD system for PTSD diagnosis.

Brain–computer interfaces (BCIs) based on electroencephalogram (EEG) have recently attracted increasing attention in virtual reality (VR) applications as a promising tool for controlling virtual objects or generating commands in a “hands-free” manner. Video-oculography (VOG) has been frequently used as a tool to improve BCI performance by identifying the gaze location on the screen, however, current VOG devices are generally too expensive to be embedded in practical low-cost VR head-mounted display (HMD) systems. In this study, we proposed a novel calibration-free hybrid BCI system combining steady-state visual-evoked potential (SSVEP)-based BCI and electrooculogram (EOG)-based eye tracking to increase the information transfer rate (ITR) of a nine-target SSVEP-based BCI in VR environment. Experiments were repeated on three different frequency configurations of pattern-reversal checkerboard stimuli arranged in a 3 × 3 matrix. When a user was staring at one of the nine visual stimuli, the column containing the target stimulus was first identified based on the user’s horizontal eye movement direction (left, middle, or right) classified using horizontal EOG recorded from a pair of electrodes that can be readily incorporated with any existing VR-HMD systems. Note that the EOG can be recorded using the same amplifier for recording SSVEP, unlike the VOG system. Then, the target visual stimulus was identified among the three visual stimuli vertically arranged in the selected column using the extension of multivariate synchronization index (EMSI) algorithm, one of the widely used SSVEP detection algorithms. In our experiments with 20 participants wearing a commercial VR-HMD system, it was shown that both the accuracy and ITR of the proposed hybrid BCI were significantly increased compared to those of the traditional SSVEP-based BCI in VR environment.

With the rapid development of virtual reality (VR) technology and the market growth of social network services (SNS), VR-based SNS have been actively developed, in which 3D avatars interact with each other on behalf of the users. To provide the users with more immersive experiences in a metaverse, facial recognition technologies that can reproduce the user’s facial gestures on their personal avatar are required. However, it is generally difficult to employ traditional camera-based facial tracking technology to recognize the facial expressions of VR users because a large portion of the user’s face is occluded by a VR head-mounted display (HMD). To address this issue, attempts have been made to recognize users’ facial expressions based on facial electromyogram (fEMG) recorded around the eyes. fEMG-based facial expression recognition (FER) technology requires only tiny electrodes that can be readily embedded in the HMD pad that is in contact with the user’s facial skin. Additionally, electrodes recording fEMG signals can simultaneously acquire electrooculogram (EOG) signals, which can be used to track the user’s eyeball movements and detect eye blinks. In this study, we implemented an fEMG- and EOG-based FER system using ten electrodes arranged around the eyes, assuming a commercial VR HMD device. Our FER system could continuously capture various facial motions, including five different lip motions and two different eyebrow motions, from fEMG signals. Unlike previous fEMG-based FER systems that simply classified discrete expressions, with the proposed FER system, natural facial expressions could be continuously projected on the 3D avatar face using machine-learning-based regression with a new concept named the virtual blend shape weight, making it unnecessary to simultaneously record fEMG and camera images for each user. An EOG-based eye tracking system was also implemented for the detection of eye blinks and eye gaze directions using the same electrodes. These two technologies were simultaneously employed to implement a real-time facial motion capture system, which could successfully replicate the user’s facial expressions on a realistic avatar face in real time. To the best of our knowledge, the concurrent use of fEMG and EOG for facial motion capture has not been reported before.

There remains an active investigation on elevating the classification accuracy and information transfer rate of brain-computer interfaces based on steady-state visual evoked potential. However, it has often been ignored that the performance of steady-state visual evoked potential (SSVEP)-based brain-computer interfaces (BCIs) can be affected through the minor displacement of the electrodes from their optimal locations in practical applications because of the mislocation of electrodes and/or concurrent use of electroencephalography (EEG) devices with external devices, such as virtual reality headsets. In this study, we evaluated the performance robustness of SSVEP-based BCIs with respect to the changes in electrode locations for various channel configurations and classification algorithms. Our experiments involved 21 participants, where EEG signals were recorded from the scalp electrodes densely attached to the occipital area of the participants. The classification accuracies for all the possible cases of electrode location shifts for various channel configurations (1–3 channels) were calculated using five training-free SSVEP classification algorithms, i.e., the canonical correlation analysis (CCA), extended CCA, filter bank CCA, multivariate synchronization index (MSI), and extended MSI (EMSI). Then, the performances of the BCIs were evaluated using two measures, i.e., the average classification accuracy (ACA) across the electrode shifts and robustness to the electrode shift (RES). Our results showed that the ACA increased with an increase in the number of channels regardless of the algorithm. However, the RES was enhanced with an increase in the number of channels only when MSI and EMSI were employed. While both ACA and RES values for the five algorithms were similar under the single-channel condition, both ACA and RES values for MSI and EMSI were higher than those of the other algorithms under the multichannel (i.e., two or three electrodes) conditions. In addition, EMSI outperformed MSI when comparing the ACA and RES values under the multichannel conditions. In conclusion, our results suggested that the use of multichannel configuration and employment of EMSI could make the performance of SSVEP-based BCIs more robust to the electrode shift from the optimal locations.

Recent developments of social virtual reality (VR) services using avatars have increased the need for facial expression recognition (FER) technology. FER systems are generally implemented using optical cameras; however, the performance of these systems can be limited when users are wearing head-mounted displays (HMDs) as users' faces are largely covered by the HMDs. Facial electromyograms (fEMGs) that can be recorded around users' eyes can be potentially used for implementing FER systems for VR applications. However, this technology lacks practicality owing to the need for large-scale training datasets; furthermore, it is hampered by a relatively low performance. In this study, we proposed an fEMG-based FER system based on the Riemannian manifold-based approach to reduce the number of training datasets needed and enhance FER performance. Our experiments with 42 participants showed an average classification accuracy as high as 85.01% for recognizing 11 facial expressions with only a single training dataset for each expression. We further developed an online FER system that could animate a virtual avatar's expression reflecting a user's facial expression in real time, thus demonstrating that our FER system can be potentially used for practical interactive VR applications, such as social VR networks, smart education, and virtual training.

The proliferating and excessive use of internet games has caused various comorbid diseases, such as game addiction, which is now a major social problem. Recently, the American Psychiatry Association classified “Internet gaming disorder (IGD)” as an addiction/mental disorder. Although many studies have been conducted on the diagnosis, treatment, and prevention of IGD, screening studies for IGD are still scarce. In this study, we classified gamers using multiple physiological signals to contribute to the treatment and prevention of IGD. Participating gamers were divided into three groups based on Young’s Internet Addiction Test score and average game time as follows: Group A, those who rarely play games; Group B, those who enjoy and play games regularly; and Group C, those classified as having IGD. In our game-related cue-based experiment, we obtained self-reported craving scores and multiple physiological data such as electrooculogram (EOG), photoplethysmogram (PPG), and electroencephalogram (EEG) from the users while they watched neutral (natural scenery) or stimulating (gameplay) videos. By analysis of covariance (ANCOVA), 13 physiological features (vertical saccadic movement from EOG, standard deviation of N-N intervals, and PNN50 from PPG, and many EEG spectral power indicators) were determined to be significant to classify the three groups. The classification was performed using a 2-layers feedforward neural network. The fusion of three physiological signals showed the best result compared to other cases (combination of EOG and PPG or EEG only). The accuracy was 0.90 and F-1 scores were 0.93 (Group A), 0.89 (Group B), and 0.88 (Group C). However, the subjective self-reported scores did not show a significant difference among the three groups by ANCOVA analysis. The results indicate that the fusion of physiological signals can be an effective method to objectively classify gamers.

Neurocinematics is an emerging discipline in neuroscience, which aims to provide new filmmaking techniques by analyzing the brain activities of a group of audiences. Several neurocinematics studies attempted to track temporal changes in mental states during movie screening; however, it is still needed to develop efficient and robust electroencephalography (EEG) features for tracking brain states precisely over a long period. This study proposes a novel method for estimating emotional arousal changes in a group of individuals during movie screening by employing steady-state visual evoked potential (SSVEP), which is a widely used EEG response elicited by the presentation of periodic visual stimuli. Previous studies have reported that the emotional arousal of each individual modulates the strength of SSVEP responses. Based on this phenomenon, movie clips were superimposed on a background, eliciting an SSVEP response with a specific frequency. Two emotionally arousing movie clips were presented to six healthy male participants, while EEG signals were recorded from the occipital channels. We then investigated whether the movie scenes that elicited higher SSVEP responses coincided well with those rated as the most impressive scenes by 37 viewers in a separate experimental session. Our results showed that the SSVEP response averaged across six participants could accurately predict the overall impressiveness of each movie, evaluated with a much larger group of individuals.

Functional near-infrared spectroscopy (fNIRS) has attracted increasing attention in the field of brain–computer interfaces (BCIs) owing to their advantages such as non-invasiveness, user safety, affordability, and portability. However, fNIRS signals are highly subject-specific and have low test-retest reliability. Therefore, individual calibration sessions need to be employed before each use of fNIRS-based BCI to achieve a sufficiently high performance for practical BCI applications. In this study, we propose a novel deep convolutional neural network (CNN)-based approach for implementing a subject-independent fNIRS-based BCI. A total of 18 participants performed the fNIRS-based BCI experiments, where the main goal of the experiments was to distinguish a mental arithmetic task from an idle state task. Leave-one-subject-out cross-validation was employed to evaluate the average classification accuracy of the proposed subject-independent fNIRS-based BCI. As a result, the average classification accuracy of the proposed method was reported to be 71.20 ± 8.74%, which was higher than the threshold accuracy for effective BCI communication (70%) as well as that obtained using conventional shrinkage linear discriminant analysis (65.74 ± 7.68%). To achieve a classification accuracy comparable to that of the proposed subject-independent fNIRS-based BCI, 24 training trials (of approximately 12 min) were necessary for the traditional subject-dependent fNIRS-based BCI. It is expected that our CNN-based approach would reduce the necessity of long-term individual calibration sessions, thereby enhancing the practicality of fNIRS-based BCIs significantly.

BackgroundPathologies of schizophrenia and bipolar disorder have been poorly understood. Brain network analysis could help understand brain mechanisms of schizophrenia and bipolar disorder. This study investigates the source-level brain cortical networks using resting-state electroencephalography (EEG) in patients with schizophrenia and bipolar disorder.MethodsResting-state EEG was measured in 38 patients with schizophrenia, 34 patients with bipolar disorder type I, and 30 healthy controls. Graph theory based source-level weighted functional networks were evaluated: strength, clustering coefficient (CC), path length (PL), and efficiency in six frequency bands.ResultsAt the global level, patients with schizophrenia or bipolar disorder showed higher strength, CC, and efficiency, and lower PL in the theta band, compared to healthy controls. At the nodal level, patients with schizophrenia or bipolar disorder showed higher CCs, mostly in the frontal lobe for the theta band. Particularly, patients with schizophrenia showed higher nodal CCs in the left inferior frontal cortex and the left ascending ramus of the lateral sulcus compared to patients with bipolar disorder. In addition, the nodal-level theta band CC of the superior frontal gyrus and sulcus (cognition-related region) correlated with positive symptoms and social and occupational functioning scale (SOFAS) scores in the schizophrenia group, while that of the middle frontal gyrus (emotion-related region) correlated with SOFAS scores in the bipolar disorder group.ConclusionsAltered cortical networks were revealed and these alterations were significantly correlated with core pathological symptoms of schizophrenia and bipolar disorder. These source-level cortical network indices could be promising biomarkers to evaluate patients with schizophrenia and bipolar disorder.

In this study, we implemented a new home appliance control system by combining electroencephalography (EEG)-based brain-computer interface (BCI), augmented reality (AR), and internet of things (IoT) technologies. We adopted a steady-state visual evoked potential (SSVEP)-based BCI paradigm for the implementation of a fast and robust BCI system. In the offline experiment, we compared the performances of three BCIs adopting different types of visual stimuli in an AR environment to determine the optimal visual stimulus. In the online experiment, we evaluated the feasibility of the proposed smart home system using the optimal stimulus by controlling three home appliances in real time. The visual stimuli were presented on a see-through head-mounted display (HMD), while the recorded brain activity was analyzed to classify the control command, and the home appliances were controlled through IoT. In the offline experiment, a grow/shrink stimulus (GSS) consisting of a star-shaped flickering object of varying size was selected as the optimal stimulus, eliciting SSVEP responses more effectively than the other options. In the online experiment, all users could turn the BCI-based control system on/off whenever they wanted using the eye-blinking-based electrooculogram (EOG) switch, and could successfully perform all the designated control tasks without difficulty. The average classification accuracy of the SSVEP-BCI-based control system was 92.8%, with an information transfer rate (ITR) of 37.4 bits/min. The proposed system exhibited an excellent performance, surpassing the best results reported in previous studies regarding external device control based on BCI using an HMD as rendering device.

Abstract Background Brain–computer interfaces (BCIs) have demonstrated the potential to provide paralyzed individuals with new means of communication, but an electroencephalography (EEG)-based endogenous BCI has never been successfully used for communication with a patient in a completely locked-in state (CLIS). Methods In this study, we investigated the possibility of using an EEG-based endogenous BCI paradigm for online binary communication by a patient in CLIS. A female patient in CLIS participated in this study. She had not communicated even with her family for more than one year with complete loss of motor function. Offline and online experiments were conducted to validate the feasibility of the proposed BCI system. In the offline experiment, we determined the best combination of mental tasks and the optimal classification strategy leading to the best performance. In the online experiment, we investigated whether our BCI system could be potentially used for real-time communication with the patient. Results An online classification accuracy of 87.5% was achieved when Riemannian geometry-based classification was applied to real-time EEG data recorded while the patient was performing one of two mental-imagery tasks for 5 s. Conclusions Our results suggest that an EEG-based endogenous BCI has the potential to be used for online communication with a patient in CLIS.

Abstract Individuals who have lost normal pathways for communication need augmentative and alternative communication (AAC) devices. In this study, we propose a new electrooculogram (EOG)-based human-computer interface (HCI) paradigm for AAC that does not require a user’s voluntary eye movement for binary yes/no communication by patients in locked-in state (LIS). The proposed HCI uses a horizontal EOG elicited by involuntary auditory oculogyric reflex, in response to a rotating sound source. In the proposed HCI paradigm, a user was asked to selectively attend to one of two sound sources rotating in directions opposite to each other, based on the user’s intention. The user’s intentions could then be recognised by quantifying EOGs. To validate its performance, a series of experiments was conducted with ten healthy subjects, and two patients with amyotrophic lateral sclerosis (ALS). The online experimental results exhibited high-classification accuracies of 94% in both healthy subjects and ALS patients in cases where decisions were made every six seconds. The ALS patients also participated in a practical yes/no communication experiment with 26 or 30 questions with known answers. The accuracy of the experiments with questionnaires was 94%, demonstrating that our paradigm could constitute an auxiliary AAC system for some LIS patients.

Abstract Background Functional near infrared spectroscopy (fNIRS) finds extended applications in a variety of neuroscience fields. We investigated the potential of fNIRS to monitor voluntary engagement of users during neurorehabilitation, especially during combinatory exercise (CE) that simultaneously uses both, passive and active exercises. Although the CE approach can enhance neurorehabilitation outcome, compared to the conventional passive or active exercise strategies, the active engagement of patients in active motor movements during CE is not known. Methods We determined hemodynamic responses induced by passive exercise and CE to evaluate the active involvement of users during CEs using fNIRS. In this preliminary study, hemodynamic responses of eight healthy subjects during three different tasks (passive exercise alone, passive exercise with motor imagery, and passive exercise with active motor execution) were recorded. On obtaining statistically significant differences, we classified the hemodynamic responses induced by passive exercise and CEs to determine the identification accuracy of the voluntary engagement of users using fNIRS. Results Stronger and broader activation around the sensorimotor cortex was observed during CEs, compared to that during passive exercise. Moreover, pattern classification results revealed more than 80% accuracy. Conclusions Our preliminary study demonstrated that fNIRS can be potentially used to assess the engagement of users of the combinatory neurorehabilitation strategy.

The implementation of a multi-class brain-computer interface (BCI) is an efficient way to increase the information transfer rate (ITR) generally expressed in a unit of bits/trial. The improvement of ITR is of specific importance for near-infrared spectroscopy (NIRS)-BCI, because brain hemodynamic responses recorded by NIRS are much slower than the electrophysiological responses of the brain. In this paper, to implement a ternary NIRS-BCI with increased ITR, we used prefrontal cortex (PFC) hemodynamic changes induced by breath-holding (i.e., the voluntary suppression of breathing movements), which have never been used in the field of BCIs. Additionally, we used traditional BCI tasks such as mental arithmetic and idle state to implement ternary NIRS-BCI. As a result, an average offline ternary classification accuracy of 72.6 ± 10.7% could be achieved, which is the best performance of NIRS-BCI based on PFC hemodynamic changes reported to date. Because the number of available input commands was increased and the classification accuracy remained at an acceptable level, the ITR of the ternary BCI (0.51 ± 0.29 bits/trial) was 1.6 times higher than that of the traditional binary BCI (0.31 ± 0.21 bits/trial). Although the hemodynamic changes induced by the breath-holding were not caused by mental state changes, breath-holding proved to be a promising hybrid BCI task for implementing more efficient ternary NIRS-BCI.

Background: Individuals who experience childhood trauma are vulnerable to various psychological and behavioral problems throughout their lifetime. This study aimed to investigate whether individuals with childhood trauma show altered frontal lobe activity during response inhibition tasks. Methods: In total, 157 healthy individuals were recruited and instructed to perform a Go/Nogo task during electroencephalography recording. Source activities of N2 and P3 of Nogo event-related potentials (ERP) were analyzed. The Childhood Trauma Questionnaire (CTQ) and Barratt Impulsivity Scale (BIS) were applied. Individuals were divided into three groups based on their total CTQ score: low CTQ, middle CTQ, and high CTQ groups. Results: The high CTQ group exhibited significantly higher BIS scores than the low CTQ group. P3 amplitudes of the differences between Nogo and Go ERP waves exhibited higher mean values in the low CTQ than the high CTQ group, with trending effects. In Nogo-P3, the source activities of the right anterior cingulate cortex, bilateral medial frontal cortex (MFC), bilateral superior frontal gyrus (SFG), and right precentral gyrus were significantly lower in the high CTQ than the low CTQ group. Motor impulsivity showed a significant negative correlation with activities of the bilateral MFC and SFG in Nogo-P3 conditions. Conclusions: Our study revealed that individuals with childhood trauma have inhibitory failure and frontal lobe dysfunction in regions related to Nogo-P3. Keywords: Childhood trauma, Frontal lobe dysfunction, Go/Nogo task, Inhibitory function, Source activity

Background: Electroencephalogram (EEG)-based brain network analysis is a useful biological correlate reflecting brain function. Sensor-level network analysis might be contaminated by volume conduction and does not explain regional brain characteristics. Source-level network analysis could be a useful alternative. We analyzed EEG-based source-level network in major depressive disorder (MDD). Method: Resting-state EEG was recorded in 87 MDD and 58 healthy controls, and cortical source signals were estimated. Network measures were calculated: global indices (strength, clustering coefficient (CC), path length (PL), and efficiency) and nodal indices (eigenvector centrality and nodal CC) in six frequency. Correlation analyses were performed between network indices and symptom scales. Results: At the global level, MDD showed decreased strength, CC in theta and alpha bands, and efficiency in alpha band, while enhanced PL in alpha band. At nodal level, eigenvector centrality of alpha band showed region dependent changes in MDD. Nodal CCs of alpha band were reduced in MDD and were negatively correlated with depression and anxiety scales. Conclusion: Disturbances in EEG-based brain network indices might reflect altered emotional processing in MDD. These source-level network indices might provide useful biomarkers to understand regional brain pathology in MDD. Keywords: Major depressive disorder, Electroencephalogram, Brain electrical activity mapping, Source-level brain network

Abstract This study investigated the effectiveness of using a high-density multi-distance source-detector (SD) separations in near-infrared spectroscopy (NIRS), for enhancing the performance of a functional NIRS (fNIRS)-based brain-computer interface (BCI). The NIRS system that was used for the experiment was capable of measuring signals from four SD separations: 15, 21.2, 30, and 33.5 mm, and this allowed the measurement of hemodynamic response alterations at various depths. Fifteen participants were asked to perform mental arithmetic and word chain tasks, to induce task-related hemodynamic response variations, or they were asked to stay relaxed to acquire a baseline signal. To evaluate the degree of BCI performance enhancement by high-density channel configuration, the classification accuracy obtained using a typical low-density lattice SD arrangement, was compared to that obtained using the high-density SD arrangement, while maintaining the SD separation at 30 mm. The analysis results demonstrated that the use of a high-density channel configuration did not result in a noticeable enhancement of classification accuracy. However, the combination of hemodynamic variations, measured by two multi-distance SD separations, resulted in the significant enhancement of overall classification accuracy. The results of this study indicated that the use of high-density multi-distance SD separations can likely provide a new method for enhancing the performance of an fNIRS-BCI.

Abstract Background Electrooculogram (EOG) can be used to continuously track eye movements and can thus be considered as an alternative to conventional camera-based eye trackers. Although many EOG-based eye tracking systems have been studied with the ultimate goal of providing a new way of communication for individuals with amyotrophic lateral sclerosis (ALS), most of them were tested with healthy people only. In this paper, we investigated the feasibility of EOG-based eye-writing as a new mode of communication for individuals with ALS. Methods We developed an EOG-based eye-writing system and tested this system with 18 healthy participants and three participants with ALS. We also applied a new method for removing crosstalk between horizontal and vertical EOG components. All study participants were asked to eye-write specially designed patterns of 10 Arabic numbers three times after a short practice session. Results Our system achieved a mean recognition rates of 95.93% for healthy participants and showed recognition rates of 95.00%, 66.67%, and 93.33% for the three participants with ALS. The low recognition rates in one of the participants with ALS was mainly due to miswritten letters, the number of which decreased as the experiment proceeded. Conclusion Our proposed eye-writing system is a feasible human-computer interface (HCI) tool for enabling practical communication of individuals with ALS.

Ensemble classifiers have been proven to result in better classification accuracy than that of a single strong learner in many machine learning studies. Although many studies on electroencephalography-brain-computer interface (BCI) used ensemble classifiers to enhance the BCI performance, ensemble classifiers have hardly been employed for near-infrared spectroscopy (NIRS)-BCIs. In addition, since there has not been any systematic and comparative study, the efficacy of ensemble classifiers for NIRS-BCIs remains unknown. In this study, four NIRS-BCI datasets were employed to evaluate the efficacy of linear discriminant analysis ensemble classifiers based on the bootstrap aggregating. From the analysis results, significant (or marginally significant) increases in the bitrate as well as the classification accuracy were found for all four NIRS-BCI datasets employed in this study. Moreover, significant bitrate improvements were found in two of the four datasets.

Neural activity related to language can be modulated within widespread networks following learning or in response to disruption—including the experimental application of noninvasive brain stimulation. However, the spatiotemporal characteristics of such modulation remain insufficiently explored. The present study combined transcranial magnetic stimulation (TMS) and electroencephalography (EEG) to explore the modulation of activity across the language network following continuous theta-burst stimulation (cTBS) of the left pars opercularis. In 10 healthy subjects (21 ± 2 years old, four females), neuronavigated cTBS was delivered over the left pars opercularis of the frontal operculum (part of the traditional Broca’s area) at 80% of active motor threshold (AMT) stimulation intensity. Real cTBS and sham cTBS were performed in two different visits separated by at least 48 h. Before, immediately, and 10 min after cTBS, 30 single pulses of TMS were delivered to the left pars opercularis at 80% of the resting motor threshold (RMT), whereas EEG was simultaneously recorded. We examined the cTBS-induced modulation of phase locking values (PLVs) between the TMS-evoked potentials (TEPs) recorded over the pars opercularis and those recorded over its right-hemispheric homolog area, the left supramarginal area, and the left superior temporal area in different EEG frequency bands and different time windows following cTBS. cTBS to the left pars opercularis induced within the gamma band: (1) a significant increase in TEP phase synchronization between the left and right pars opercularis at an early time window (250–350 ms) following cTBS; and (2) significantly increased PLV with the left supramarginal area and the left superior temporal area at a later time window (600–700 ms). In the theta and delta band, cTBS to the left pars opercularis induced significantly increased phase synchronization of TEPs between the left pars opercularis and the posterior left hemispheric language areas at a late time window. In sham condition, there was a significant decrease in TEP phase synchronization of the high beta band between left pars opercularis and left superior temporal area at 200–300 ms. These results contribute to characterize the dynamics of the language network and may have implications in the development of noninvasive stimulation protocols to promote the language rehabilitation in aphasia patients.

It has been demonstrated that the performance of typical unimodal brain-computer interfaces (BCIs) can be noticeably improved by combining two different BCI modalities. This so-called "hybrid BCI" technology has been studied for decades; however, hybrid BCIs that particularly combine electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) (hereafter referred to as hBCIs) have not been widely used in practical settings. One of the main reasons why hBCI systems are so unpopular is that their hardware is generally too bulky and complex. Therefore, to make hBCIs more appealing, it is necessary to implement a lightweight and compact hBCI system with minimal performance degradation. In this study, we investigated the feasibility of implementing a compact hBCI system with significantly less EEG channels and fNIRS source-detector (SD) pairs, but that can achieve a classification accuracy high enough to be used in practical BCI applications. EEG and fNIRS data were acquired while participants performed three different mental tasks consisting of mental arithmetic, right-hand motor imagery, and an idle state. Our analysis results showed that the three mental states could be classified with a fairly high classification accuracy of 77.6 ± 12.1% using an hBCI system with only two EEG channels and two fNIRS SD pairs.

The behavioral differentiation of positive emotions has recently been studied in terms of their discrete adaptive functions or appraising profiles. Some preliminary neurophysiological evidences have been found with electroencephalography or autonomic nervous system measurements such as heart rate, skin conductance, etc. However, the brain’s hemodynamic responses to different positive emotions remain largely unknown. In the present study, the functional near-infrared spectroscopy (fNIRS) technique was employed. With this tool, we for the first time reported recognizable discrete positive emotions using fNIRS signals. Thirteen participants watched 30 emotional video clips to elicit 10 typical kinds of positive emotions (joy, gratitude, serenity, interest, hope, pride, amusement, inspiration, awe, and love), and their frontal neural activities were simultaneously recorded with a 24-channel fNIRS system. The multidimensional scaling analysis of participants’ subjective ratings on these 10 positive emotions revealed three distinct clusters, which could be interpreted as “playfulness” for amusement, joy, interest, “encouragement” for awe, gratitude, hope, inspiration, pride, and “harmony” for love, serenity. Hemodynamic responses to these three positive emotion clusters showed distinct patterns, and HbO-based individual-level binary classifications between them achieved an averaged accuracy of 73.79 ± 11.49% (77.56 ± 7.39% for encouragement vs. harmony, 73.29 ± 11.87% for playfulness vs. harmony, 70.51 ± 13.96% for encouragement vs. harmony). Benefited from fNIRS’s high portability, low running cost and the relative robustness against motion and electrical artifacts, our findings provided support for implementing a more fine-grained emotion recognition system with subdivided positive emotion categories.

Assistant devices such as meal-assist robots aid individuals with disabilities and support the elderly in performing daily activities. However, existing meal-assist robots are inconvenient to operate due to non-intuitive user interfaces, requiring additional time and effort. Thus, we developed a hybrid brain–computer interface-based meal-assist robot system following three features that can be measured using scalp electrodes for electroencephalography. The following three procedures comprise a single meal cycle. (1) Triple eye-blinks (EBs) from the prefrontal channel were treated as activation for initiating the cycle. (2) Steady-state visual evoked potentials (SSVEPs) from occipital channels were used to select the food per the user’s intention. (3) Electromyograms (EMGs) were recorded from temporal channels as the users chewed the food to mark the end of a cycle and indicate readiness for starting the following meal. The accuracy, information transfer rate, and false positive rate during experiments on five subjects were as follows: accuracy (EBs/SSVEPs/EMGs) (%): (94.67/83.33/97.33); FPR (EBs/EMGs) (times/min): (0.11/0.08); ITR (SSVEPs) (bit/min): 20.41. These results revealed the feasibility of this assistive system. The proposed system allows users to eat on their own more naturally. Furthermore, it can increase the self-esteem of disabled and elderly peeople and enhance their quality of life.

Owing to the increased public interest in passive brain–computer interface (pBCI) applications, many wearable devices for capturing electroencephalogram (EEG) signals in daily life have recently been released on the market. However, there exists no well-established criterion to determine the electrode configuration for such devices. Herein, an overall procedure is proposed to determine the optimal electrode configurations of wearable EEG devices that yield the optimal performance for intended pBCI applications. We utilized two EEG datasets recorded in different experiments designed to modulate emotional or attentional states. Emotion-specialized EEG headsets were designed to maximize the accuracy of classification of different emotional states using the emotion-associated EEG dataset, and attention-specialized EEG headsets were designed to maximize the temporal correlation between the EEG index and the behavioral attention index. General purpose electrode configurations were designed to maximize the overall performance in both applications for different numbers of electrodes (2, 4, 6, and 8). The performance was then compared with that of existing wearable EEG devices. Simulations indicated that the proposed electrode configurations allowed for more accurate estimation of the users’ emotional and attentional states than the conventional electrode configurations, suggesting that wearable EEG devices should be designed according to the well-established EEG datasets associated with the target pBCI applications.

Background: The development of optimal classification criteria for specific mental disorders which share similar symptoms is an important issue for precise diagnosis. We investigated whether P300 features in both sensor-level and source-level could be effectively used to classify post-traumatic stress disorder (PTSD) and major depressive disorder (MDD). Method: EEG signals were recorded from fifty-one PTSD patients, 67 MDD patients, and 39 healthy controls (HCs) while performing an auditory oddball task. Amplitude and latency of P300 were evaluated, and the current source analysis of P300 components was conducted using sLORETA. Finally, we classified two groups using machine-learning methods with both sensor- and source-level features. Moreover, we checked the comorbidity effects using the same approaches (PTSD-mono diagnosis (PTSDm, n = 28) and PTSD-comorbid diagnosis (PTSDc, n = 23)). Results: PTSD showed significantly reduced P300 amplitudes and prolonged latency compared to HCs and MDD. Moreover, PTSD showed significantly reduced source activities, and the source activities were significantly correlated with symptoms of depression and anxiety. Also, the best classification accuracy at each pair was as follows: 80.00% (PTSD-HCs), 67.92% (MDD-HCs), 70.34% (PTSD-MDD), 82.09% (PTSDm-HCs), 71.58% (PTSDm-MDD), 82.56% (PTSDc-HCs), and 76.67% (PTSDc- MDD). Conclusion: Since abnormal P300 reflects pathophysiological characteristics of PTSD, PTSD patients were well-discriminated from MDD and HCs when using P300 features. Thus, altered P300 characteristics in both sensor- and source-level may be useful biomarkers to diagnosis PTSD. Keywords: Post-traumatic stress disorder, Major depressive disorder, Cognitive function, EEG, Source imaging, Classification

Background: Abnormalities in the 40-Hz auditory steady-state response (ASSR) of the gamma range have been reported in schizophrenia (SZ) and are regarded as important pathophysiological features. Many of the previous studies reported diminished gamma oscillations in SZ, although some studies reported increased spontaneous gamma oscillations. Furthermore, brain morphological correlates of the gamma band ASSR deficits have rarely examined. We investigated different measures of the 40-Hz ASSR and their association with brain volumes and psychological measures of SZ. Methods: The 40-Hz ASSR was measured for 80 dB click sounds (1 ms, 500-ms trains at 40-Hz, with 3050 to 3500 inter-train interval) using electroencephalography with 64 electrodes in 33 patients with SZ (male: 16, female: 17 (age range: 21–60)) and 30 healthy controls (HCs) (male: 13, female: 17 (age range: 23–64)). Four gamma oscillation measures (evoked power, spontaneous oscillations (baseline and total power), and inter-trial phase coherence (ITC)) were assessed. The source activities of the ASSR were also analyzed. Brain volumes were assessed using high-resolution magnetic resonance imaging and voxel-based morphometry and superior temporal gyrus (STG) volume measures were obtained. Results: Patients with SZ had larger total and evoked powers and higher ITC than HCs. Both groups showed significantly different association between mean evoked power and right STG volume. In HCs but not SZ, mean evoked power showed significant positive correlation with right STG volume. In addition, the two groups showed significantly different association between verbal fluency and mean evoked power. High evoked power was significantly correlated with poor verbal fluency in SZ. Conclusions: The current study found increased gamma oscillation in SZ and suggests significant involvement of the STG in gamma oscillations. Keywords: 40-Hz ASSR, Gamma oscillation, STG volume, Schizophrenia, Voxel-based morphometry

The performance of a brain-computer interface (BCI) can be enhanced by simultaneously using two or more modalities to record brain activity, which is generally referred to as a hybrid BCI. To date, many BCI researchers have tried to implement a hybrid BCI system by combining electroencephalography (EEG) and functional near-infrared spectroscopy (NIRS) to improve the overall accuracy of binary classification. However, since hybrid EEG-NIRS BCI, which will be denoted by hBCI in this paper, has not been applied to ternary classification problems, paradigms and classification strategies appropriate for ternary classification using hBCI are not well investigated. Here we propose the use of an hBCI for the classification of three brain activation patterns elicited by mental arithmetic, motor imagery, and idle state, with the aim to elevate the information transfer rate (ITR) of hBCI by increasing the number of classes while minimizing the loss of accuracy. EEG electrodes were placed over the prefrontal cortex and the central cortex, and NIRS optodes were placed only on the forehead. The ternary classification problem was decomposed into three binary classification problems using the “one-versus-one” (OVO) classification strategy to apply the filter-bank common spatial patterns filter to EEG data. A 10 × 10-fold cross validation was performed using shrinkage linear discriminant analysis (sLDA) to evaluate the average classification accuracies for EEG-BCI, NIRS-BCI, and hBCI when the meta-classification method was adopted to enhance classification accuracy. The ternary classification accuracies for EEG-BCI, NIRS-BCI, and hBCI were 76.1 ± 12.8, 64.1 ± 9.7, and 82.2 ± 10.2%, respectively. The classification accuracy of the proposed hBCI was thus significantly higher than those of the other BCIs (p < 0.005). The average ITR for the proposed hBCI was calculated to be 4.70 ± 1.92 bits/minute, which was 34.3% higher than that reported for a previous binary hBCI study.

Recent studies of functional or effective connectivity in the brain have reported that motor-related brain regions were activated during motor execution and motor imagery, but the relationship between motor and cognitive areas has not yet been completely understood. The objectives of our study were to analyze the effective connectivity between motor and cognitive networks in order to define network dynamics during motor execution and motor imagery in healthy individuals. Second, we analyzed the differences in effective connectivity between correct and incorrect responses during motor execution and imagery using dynamic causal modeling (DCM) of electroencephalography (EEG) data.Twenty healthy subjects performed a sequence of finger tapping trials using either motor execution or motor imagery, and the performances were recorded. Changes in effective connectivity between the primary motor cortex (M1), supplementary motor area (SMA), premotor cortex (PMC), and dorsolateral prefrontal cortex (DLPFC) were estimated using dynamic causal modeling. Bayesian model averaging with family-level inference and fixed-effects analysis was applied to determine the most likely connectivity model for these regions.Motor execution and imagery showed inputs to distinct brain regions, the premotor cortex and the supplementary motor area, respectively. During motor execution, the coupling strength of a feedforward network from the DLPFC to the PMC was greater than that during motor imagery. During motor imagery, the coupling strengths of a feedforward network from the PMC to the SMA and of a feedback network from M1 to the PMC were higher than that during motor execution. In imagined movement, although there were connectivity differences between correct and incorrect task responses, each motor imagery task that included correct and incorrect responses showed similar network connectivity characteristics. Correct motor imagery responses showed connectivity from the PMC to the DLPFC, while the incorrect responses had characteristic connectivity from the SMA to the DLPFC.These findings provide an understanding of effective connectivity between motor and cognitive areas during motor execution and imagery as well as the basis for future connectivity studies for patients with stroke.

In the present study, we investigated whether global electroencephalography (EEG) synchronization can be a new promising index for tracking emotional arousal changes of a group of individuals during video watching. Global field synchronization (GFS), an index known to correlate with human cognitive processes, was evaluated; this index quantified the global temporal synchronization among multichannel EEG data recorded from a group of participants (n = 25) during the plays of two short video clips. The two video clips were each about 5 min long and were designed to evoke negative (fearful) or positive (happy) emotion, respectively. Another group of participants (n = 37) was asked to select the two most emotionally arousing (most touching or most fearful) scenes in each clip. The results of these questionnaire surveys were used as the ground-truth to evaluate whether the GFS could detect emotional highlights of both video clips. The emotional highlights estimated using the grand-averaged GFS waveforms of the first group were also compared with those evaluated from galvanic skin response, photoplethysmography, and multimedia content analysis, which are conventional methods used to estimate temporal changes in emotional arousal during video plays. From our results, we found that beta-band GFS values decreased during high emotional arousal, regardless of the type of emotional stimulus. Moreover, the emotional highlights estimated using the GFS waveforms coincided best with those found by the questionnaire surveys. These findings suggest that GFS might be applicable as a new index for tracking emotional arousal changes of a group of individuals during video watching, and is likely to be used to evaluate or edit movies, TV commercials, and other broadcast products.

Precise diagnosis of psychiatric diseases and a comprehensive assessment of a patient's symptom severity are important in order to establish a successful treatment strategy for each patient. Although great efforts have been devoted to searching for diagnostic biomarkers of schizophrenia over the past several decades, no study has yet investigated how accurately these biomarkers are able to estimate an individual patient's symptom severity. In this study, we applied electrophysiological biomarkers obtained from electroencephalography (EEG) analyses to an estimation of symptom severity scores of patients with schizophrenia. EEG signals were recorded from 23 patients while they performed a facial affect discrimination task. Based on the source current density analysis results, we extracted voxels that showed a strong correlation between source activity and symptom scores. We then built a prediction model to estimate the symptom severity scores of each patient using the source activations of the selected voxels. The symptom scores of the Positive and Negative Syndrome Scale (PANSS) were estimated using the linear prediction model. The results of leave-one-out cross validation (LOOCV) showed that the mean errors of the estimated symptom scores were 3.34 ± 2.40 and 3.90 ± 3.01 for the Positive and Negative PANSS scores, respectively. The current pilot study is the first attempt to estimate symptom severity scores in schizophrenia using quantitative EEG features. It is expected that the present method can be extended to other cognitive paradigms or other psychological illnesses.