Search

Brain Computer Interface (BCI) has the potential to offer a new generation of applications independent of muscular activity and controlled by the human brain. Brain imaging technologies are used to transfer the cognitive tasks into control commands for a BCI system. The electroencephalography (EEG) technology serves as the best available non-invasive solution for extracting signals from the brain. On the other hand, speech is the primary means of communication, but for patients suffering from locked-in syndrome, there is no easy way to communicate. Therefore, an ideal communication system for locked-in patients is a thought-to-speech BCI system. This research aims to investigate methods for the recognition of imagined speech from EEG signals using deep learning techniques. In order to design an optimal imagined speech recognition BCI, variety of issues have been solved. These include 1) proposing new feature extraction and classification framework for recognition of imagined speech from EEG signals, 2) grammatical class recognition of imagined words from EEG signals, 3) discriminating different cognitive tasks associated with speech in the brain such as overt speech, covert speech, and visual imagery. In this work machine learning, deep learning methods were used to analyze EEG signals. For recognition of imagined speech from EEG signals, a new EEG database was collected while the participants mentally spoke (imagined speech) the presented words. Along with imagined speech, EEG data was recorded for visual imagery (imagining a scene or an image) and overt speech (verbal speech). Spectro-temporal and spatio-temporal domain features were investigated for the classification of imagined words from EEG signals. Further, a deep learning framework using the convolutional network and attention mechanism was implemented for learning features in the spatial, temporal, and spectral domains. The method achieved a recognition rate of 76.6% for three binary word pairs. These experiments show that deep learning algorithms are ideal for imagined speech recognition from EEG signals due to their ability to interpret features from non-linear and non-stationary signals. Grammatical classes of imagined words from EEG signals were also recognized using a multi-channel convolution network framework. This method was extended to a multi-level recognition system for multi-class classification of imagined words which achieved an accuracy of 52.9% for 10 words, which is much better in comparison to previous work. In order to investigate the difference between imagined speech with verbal speech and visual imagery from EEG signals, we used multivariate pattern analysis (MVPA). MVPA provided the time segments when the neural oscillation for the different cognitive tasks was linearly separable. Further, frequencies that result in most discrimination between the different cognitive tasks were also explored. A framework was proposed to discriminate two cognitive tasks based on the spatio-temporal patterns in EEG signals. The proposed method used the K-means clustering algorithm to find the best electrode combination and convolutional-attention network for feature extraction and classification. The proposed method achieved a high recognition rate of 82.9% and 77.7%. The results in this research suggest that a communication based BCI system can be designed using deep learning methods. Further, this work add knowledge to the existing work in the field of communication based BCI system.

Gambling disorder (GD) is the most widely studied behavioural addiction (BA), however there is still an unmet need for more effective treatment strategies. With the aim to improve the understanding of transcranial direct current stimulation (tDCS) as a potential treatment intervention for GD, four experiments were conducted using different protocols and participant samples to measure neuromodulation effects during gambling-related task performance. In Experiments I and II, the effects of tDCS were investigated in low impulsive (LI) and high impulsive (HI) participants. Different high definition (HD) tDCS montages were used to target right dorsolateral prefrontal cortex (rDLPFC) and ventromedial prefrontal cortex (vmPFC), brain areas associated with decision-making and reward processing, respectively. Results revealed effects of tDCS on gambling task performance, but no difference on tDCS effects between rDLPFC and vmPFC targets, or between participant groups. In Experiment III, the potential cumulative effects of rDLPFC tDCS combined with cognitive behavioural therapy (CBT) were investigated across eight sessions, in two patients diagnosed with GD. The intervention combining tDCS and CBT resulted in reductions in gambling severity and cravings, but this effect was also seen in the sham tDCS case. In Experiment IV, physiological data, including electrodermal activity (EDA), electrocardiogram (ECG) and electroencephalogram (EEG), was used to investigate rDLPFC tDCS effects on the autonomous nervous system (ANS), in LI and HI gamblers. Results showed that real stimulation was associated with increased sympathetic activation compared with sham, which was higher during gambling-related wins compared with losses, and in HI compared with LI. There were significant correlations between gambling severity, cognitive outcomes and physiological variables, which helped to identify biological markers associated with GD. These results helped refine the knowledge of specific cognitive and physiological underpinnings of reward processing in different participant samples, and contributed to the development of novel treatment interventions for GD.

Online retailing has increased exponentially within the last decade (Mintel, 2014), and websites are becoming more interactive as technology advances. This demand of interactivity encourages retailers to understand how their consumer behaves in order to increase or maintain sales. The online shopping environment consists of many levels of interaction, this thesis looks at three in particular; browsing, videos and social media which have been modeled from Manganari et al., (2009) Online Store Environment Framework (OSEF). Consumer engagement is a fairly new construct implemented in marketing theory (Van Doorn, 2010) and has been evidenced to contain cognitive, emotional and behavioural components constituting to a Cognitive Behavioural Engagement framework (Cognitive Behavioural Engagement) (CBE). This thesis measures CBE within the OSEF through developing a revised theoretical framework. Methodological tools used to measure engagement in UK female shoppers (aged 18-30) is that of three distinct surveys (N=261, N=266, N=264) and Electroencephalogram (EEG) (N=21) to an online fashion pure-play (online only) retailer. Structural Equation Modelling (SEM) is used to confirm the revised theoretical framework. The results of survey data shapes the hypotheses of the EEG experiment. ANOVA and correlations are used to analyse engagement and emotions from brain activity data. Findings from study one (surveys) reveals that 'involvement' and 'absorption' were present in every survey. Browsing a website did not evoke 'attention' (not predicted), watching a video had a predicted relationship with 'involvement-arousal-re-visit intention' and participating in Instagram had an unexpected relationship with 'involvement-focused attention-re-visit intention'. Findings from study two (EEG experiments) demonstrated high levels of emotion overall when participating in social media and high levels of engagement were present overall with videos. Correlations were not in parrallel with survey results, the only congruent result between EEG and Surveys was the result of absorption having an assosiation with purchase intention in the website browsing task. Overall engagement is present in an online interactive shopping environment, this study sheds light on differing website elements linked to different levels of engagement for retailers, practitioners and marketers in the field of marketing and neuro-marketing.

Recent years have witnessed the widespread application of High Dynamic Range (HDR) imaging, which like the Human Visual System (HVS), has the ability to capture a wide range of luminance values. Areas of application include home-entertainment, security, scientific imaging, video processing, computer graphics, multimedia communications, and healthcare. However, in practice, HDR content cannot be displayed in full on standard or low dynamic range (LDR) displays, and this diminishes the benefits of HDR technology for many users. To address this problem, Tone-Mapping Operators (TMO) are used to convert HDR images so that they can be displayed on low-dynamic-range displays and preserve as far as possible the perception of HDR. However, this may affect the visual Quality of Experience (QoE) of the end-user. QoE is a vital issue in image and video applications. It is important to understand how humans perceive quality in response to visual stimuli as this can potentially be exploited to develop and optimise image and video processing algorithms. Image consumption using mobile devices has become increasingly popular, given the availability of smartphones capable of producing and consuming HDR images along with advances in high-speed wireless communication networks. One of the most critical issues associated with mobile HDR image delivery services concerns how to maximise the QoE of the delivered content for users. An open research question therefore addresses how HDR images with different types of content perform on mobile phones. Traditionally, evaluation of the perceived quality of multimedia content is conducted using subjective opinion tests (i.e., explicitly), such as Mean Opinion Scores (MOS). However, it is difficult for the user to link the quality they are experiencing to the quality scale. Moreover, MOS does not give an insight into how the user feels at a physiological level in response to satisfaction or dissatisfaction with the perceived quality. To address this issue, measures that can be taken directly (implicitly) from the participant have now begun to attract interest. The electroencephalogram (EEG) is a promising approach that can be used to assess quality related processes implicitly. However, implicit QoE approaches are still at an early stage and further research is necessary to fully understand the nature of the recorded neural signals and their associations with user-perceived quality. Nevertheless, the EEG is expected to provide additional and complementary information that will aid understanding of the human perception of content. Furthermore, it has the potential to facilitate real-time monitoring of QoE without the need for explicit rating activities. The main aim of this project was therefore to assess the QoE of HDR images employing a physiological method and to investigate its potential application in the field of healthcare. This resulted in the following five main contributions to the research literature: 1. A detailed understanding of the relationship between the subjective and objective evaluation of the most popular TMOs used for colour and greyscale HDR images. Different mobile displays and resolutions were therefore presented under normal viewing conditions for the end-user with an LDR display as a reference. Preliminary results show that, compared to computer displays, small screen devices (SSDs) such as those used in smartphones impact the performance of TMOs in that a higher resolution gave more favourable MOS results. 2. The development of a novel Electrophysiology-based QoE assessment of HDR image quality that can be used to predict perceived image quality. This was achieved by investigating the relationships between changes in EEG features and subjective quality test scores (i.e. MOS) for HDR images viewed with SSD. 3. The development of a novel QoE prediction model, based on the above findings. The model can predict user acceptability and satisfaction for various mobile HDR image scenarios based on delta-beta coupling. Subjective quality tests were conducted to develop and evaluate the model, where the HDR image quality was predicted in terms of MOS. 4. The development of a new method of detecting a colour vision deficiency (CVD) using EEG and HDR images. The results suggest that this method may provide an accurate way to detect CVD with high sensitivity and specificity (close to 100%). Potentially, the method may facilitate the development of a low-cost tool suitable for CVD diagnosis in younger people. 5. The development of an approach that enhances the quality of dental x-ray images. This uses the concepts of QoE in HDR images without re-exposing patients to ionising radiation, thus improving patient care. Potentially, the method provides the basis for an intelligent model that accurately predicts the quality of dental images. Such a model can be embedded into a tool to automatically enhance poor quality dental images.

The electroencephalogram (EEG) reflects summated, slow post-synaptic potentials of cortical neurons. Sensory, motor or cognitive events (such as a fast-rising sensory stimulus, a brisk self-paced movement or a stimulus-triggered cognitive task) can elicit transient changes in the ongoing human EEG, called event-related potentials (ERPs). ERPs are widely used in clinical practice, and believed to reflect the activity of the sensory system activated by the stimulus (for example, laser-evoked potentials are used to substantiate the neuropathic nature of clinical pain conditions). When ERPs are elicited by pairs or trains of stimuli delivered at short inter-stimulus intervals (ISIs), the magnitude of the ERP elicited by the repeated stimuli is markedly reduced, a phenomenon known as response decrement. While the interval between two consecutive stimuli becomes longer, the reduced response is recovered. Thus, this phenomenon has been traditionally interpreted in terms of neural refractoriness of generators of ERPs ("neural refractoriness hypothesis"). This thesis, however, challenges this neural refractoriness hypothesis by describing the results of manipulating the preceding events of the eliciting stimulus. The first study examined the effect of variable and short ISIs on sensory ERPs, delivering trains of auditory and electrical stimuli with random ISIs ranging from 100 to 1000ms. In the second study, pairs of laser stimuli were presented in two comparable conditions. In the constant condition, the ISI was identical across trials in each block, while in the variable condition, the ISI was variable across trials. By directly comparing ERPs elicited by laser stimulation, this study aimed to explore whether lack of saliency in the eliciting stimulus could explain the response decrement during stimulus repetition. Finally, the third study tested the hypothesis that the reduced eliciting ERPs would recover if saliency were introduced by changing the modality of the preceding event. Thus, trains of three stimuli (S1-S2-S3) with 1s ISI were presented; S2 was either same or different in modality as S1 and S3 in each block. Results from these three experiments demonstrate that this "refractoriness hypothesis" does not hold, and suggest that the magnitude of ERPs is only partly related to the magnitude of the incoming sensory input, and instead largely reflects neural activities triggered by salient events in the sensory environment. These results are important for the correct interpretation of ERPs in both physiological and clinical studies.

NeuroGaze is a novel Virtual Reality (VR) interface that integrates electroencephalogram (EEG) and eye tracking technologies to enhance user interaction within virtual environments (VEs). Diverging from traditional VR input devices, NeuroGaze allows users to select objects in a VE through gaze direction and cognitive intent captured via EEG signals. The research assesses the performance of the NeuroGaze system against conventional input devices such as VR controllers and eye gaze combined with hand gestures. The experiment, conducted with 20 participants, evaluates task completion time, accuracy, cognitive load through the NASA-TLX surveys, and user preference through a post-evaluation survey. Results indicate that while NeuroGaze presents a learning curve, evidenced by longer average task durations, it potentially offers a more accurate selection method with lower cognitive load, as suggested by its lower error rate and significant differences in the physical demand and temporal NASA-TLX subscale scores. This study highlights the viability of incorporating biometric inputs for more accessible and less demanding VR interactions. Future work aims to explore a multimodal EEG-Functional near-infrared spectroscopy (fNIRS) input device, further develop machine learning models for EEG signal classification, and extend system capabilities to dynamic object selection, highlighting the progressive direction for the use of Brain-Computer Interfaces (BCI) in virtual environments.

A brain computer interface (BCI) is a system which connects neural signals to a computer system. They have been used for controlling systems including robotics, on-screen computer control such as mouse movement, typing, and synthesizing audio signals. Invasive, or implanted, systems are often long-term medical solutions, or used for research where very clear signal is required. Non-invasive systems usually rely on exterior signals gathered through a headset using one or more electrode sensors. These signals are composed of sums of neuron activation potentials from brain activity and can be used to determine particular aspects of brain function. All BCIs rely on neural feedback of some form; the most effective systems combine multiple forms of feedback. The proposed BCI design will control audio playback through a connection to a non-invasive EEG sensor headset worn by the user. This design will take a conclusive step toward better integration of this technology for more effective control of systems in the future. It would be helpful to any consumer who wants to control media play back hands free, as well as patients unable to use hands for control. Its advantages include a low price point and simple implementation compared to medical and research grade equipment. In summary, the proposed BCI audio playback control system is a simple yet vital step toward a truly robust implementation of such a system on a larger scale.

Augmented Reality (AR) head-mounted display (HMD) provides users with an immersive virtual experience in the real world. The portability of this technology affords various information display options for construction workers that are not possible otherwise. The information delivered via an interactive user interface provides an innovative method to display complex building instructions, which is more intuitive and accessible compared with traditional paper documentations. However, there are still challenges hindering the practical usage of this technology at the construction jobsite. As a technical restriction, current AR HMD products have a limited field of view (FOV) compared to the human vision range. It leads to an uncertainty of how the obstructed view of display will affect construction workers' perception of hazards in their surrounding area. Similarly, the information displayed to workers requires rigorous testing and evaluation to make sure that it does not lead to information overload. Therefore, it is essential to comprehensively evaluate the impacts of using AR HMD from both perspectives of task performance and cognitive performance. This dissertation aims to bridge the gap in understanding the cognitive impacts of using AR HMD in construction assembly tasks. Specifically, it focuses on answering the following two questions: (1) How are task performance and cognitive skills affected by AR displays under complex working conditions? (2) How are moment-to-moment changes of mental workload captured and evaluated during construction assembly tasks? To answer these questions, this dissertation proposed two experiments. The first study tests two AR displays (conformal and tag-along) and paper instruction under complex working conditions, involving different framing scales and interference settings. Subjective responses are collected and analyzed to evaluate overall mental workload and situation awareness. The second study focuses on exploring an electroencephalogram (EEG) based approach for moment-to-moment capture and evaluation of mental workload. It uncovers the cognitive change on the time domain and provides room for further quantitative analyzing on mental workload. Especially, two frameworks of mental workload prediction are proposed by using (1) Long Short-Term Memory (LSTM) and (2) one-dimensional Convolutional Neural Network (1D CNN)-LSTM for forecasting EEG signal and, classifying task conditions and mental workload levels respectively. The approaches are tested to be effective and reliable for predicting and recognizing subjects' mental workload during assembly. In brief, this research contributes to the existing knowledge with an assessment of AR HMD use in construction assembly, including task performance evaluation and both subjective and physiological measurements for cognitive skills.

Recognition and continuous monitoring of human emotions is a key problem that is spread across multiple research disciplines like electrical engineering, computer science, neuroscience, and cognitive science. Instead of the widely used method of approaching this problem from the perspective of computer vision i.e. by tracking user's facial expressions, we take a multi-modal approach. This approach utilizes various bio-sensing modalities in addition to computer vision for assessing and classifying human affects. In the process, we address various limitations—hardware and software—of such bio-sensing modalities. We evaluate the developed framework on real-world applications ranging from watching emotional multimedia content (such as videos) to driving an automobile. Our holistic framework contributes toward (1) the development of a novel multi-modal bio-sensing headset that is capable of recording, monitoring, and tracking various bio-signals in real-time, (2) the development of algorithms that utilize signal processing as well as deep learning tools for various bio-sensor and vision-based modalities for emotion recognition, (3) the utilization of such algorithms in a wide range of applications namely emotion classification, studying emotion elicitation, and monitoring driver's awareness, and (4) the performance comparison of various sensor modalities for the above applications when they are used individually or in fusion with each other. Although the above multi-modal sensor platform has been evaluated in this thesis dissertation on affective computing applications only, it is generalizable. Thus, this platform can work in various applications in the field of human-computer interaction that requires the use of bio-sensing modalities.

In this dissertation I propose a new source imaging algorithm which uses surface non-invasive measurements such as EEG and MEG to estimate underlying brain activities. I employ sparse signal processing techniques (imposing sparsity in multiple domains) as well as an iterative reweighting scheme to come up with an algorithm which objectively and without the need of applying any subjective thresholds, yields extended solutions that not only precisely estimates the location of underlying sources of activity in the brain, but also provides a high quality estimate of the underlying sources’ extent and size. This algorithm is formulated as a convex optimization problem. I have also proposed a scheme to further develop this algorithm to become suitable for imaging sources that evolve over time, basically providing a spatio-temporal estimate of underlying brain activity. In this manner an efficient algorithm that can image dynamic underlying brain networks is developed. The main application this algorithm was motivated by and validated in, is imaging the epileptogenic tissue in focal epilepsy patients. It is shown in this dissertation through analyzing inter-ictal spikes and ictal signals from the EEG recordings of focal epilepsy patients and subsequently comparing it to clinical findings in these patients that the proposed algorithm works well in real-world applications and clinical settings. These clinical findings included the surgically resected volume and seizure onset zone identified by intra-cranial studies; our estimated epileptogenic tissue matched these clinical findings well. While this algorithm was developed for and tested in this particular application, i.e. epileptic activity imaging, it is a general source imaging algorithm and many other applications are also possible, as will be pointed out throughout this dissertation.

The use of electroencephalogram (EEG), an electrophysiological monitoring method for recording the brain activity, for authentication has attracted the interest of researchers for over a decade. In addition to exhibiting qualities of biometric-based authentication, they are revocable, impossible to mimic, and resistant to coercion attacks. However, EEG signals carry a wealth of information about an individual and can reveal private information about the user. This brings significant privacy issues to EEG-based authentication systems as they have access to raw EEG signals. This thesis proposes a privacy-preserving EEG-based authentication system that preserves the privacy of the user by not revealing the raw EEG signals while allowing the system to authenticate the user accurately. In that, perceptual hashing is utilized and instead of raw EEG signals, their perceptually hashed values are used in the authentication process. In addition to describing the authentication process, algorithms to compute the perceptual hash are developed based on two feature extraction techniques. Experimental results show that an authentication system using perceptual hashing can achieve performance comparable to a system that has access to raw EEG signals if enough EEG channels are used in the process. This thesis also presents a security analysis to show that perceptual hashing can prevent information leakage.

Given that mindfulness-based training techniques (MBT) stimulates and pushes ones core cognitive control capacity limits, brain stimulation techniques, such as transcranial direct current stimulation (tDCS), can be used to facilitate the ongoing neural patterns of functional connectivity toward long-lasting neuroplastic change. The current study assessed the combined effects of MBT with right frontal tDCS on cognitive control abilities and their corresponding brain patterns of activation using electroencephalography (EEG) and functional magnetic resonance imaging (fMRI). This study found an enhancement in working memory and sustained attention performance along with changes in the attention-related P3 component and its theta and alpha oscillatory profiles recorded by EEG. Furthermore, a reconfiguration in the chronnectome of large-scale resting-state networks was observed using resting-state fMRI, in addition to task-related changes in the polymodal neural architecture associated with encoding and adaptation, which may bridge the necessary connections from near to far transfer gains.

To implement adaptive aiding in modern aviation systems there is a need for accurate and reliable classification of cognitive workload. Using electroencephalogram (EEG)-derived features, it has been reported that an Artificial Neural Network (ANN) can achieve 95% or higher classification accuracy on the same day for an individual operator, but only 70% or less on a different day. To gain a further insight into this discrepancy, data from a previous study was utilized to study the classification variability. The EEG-derived features were first calculated by spectral power estimation. The variability was then analyzed by performing cognitive workload classification in which different methods of training and testing were used and different classifiers were implemented to compare classification accuracies. The classifiers include an ANN, Adaboost Algorithm, and a t-test method. The results show that when the ANN or Adaboost method is used, the amount of overlapping among training and testing data impacts the classification accuracy significantly. When there is no overlap, all classifiers can only achieve an accuracy of about 70%, with the Adaboost outperforming other classifiers slightly. By allowing some overlap, the accuracy of the ANN or Adaboost method increases significantly. It was concluded that the main source of the classification variability is the inherent variability of the EEG-derived features.

Electroencephalogram (EEG) signals, when recorded within the strong magnetic field of an MR scanner, are subject to various artifacts, of which the ballistocardiogram (BCG) is one of the prominent artifacts affecting the quality of the EEG. The BCG is continuously varying with time and its spectrum overlaps with the EEG spectra, making its suppression a signal processing challenge. A novel method for the identification and removal of this artifact using shape basis functions of the new dilated discrete Hermite transform is investigated in this paper. The BCG artifacts are modeled for every heart beat, using these discrete Hermite basis functions, and are subsequently subtracted from the ongoing EEG. Experimental EEG data was recorded inside and outside a 3 Tesla MRI scanner, from a total of 7 subjects under various experimental conditions. Quantitative assessment of the efficiency of this algorithm was evaluated by adding known BCG templates, at varying Signal to Noise Ratios (SNRs), to the EEG recorded outside the scanner. Significant reduction of the BCG artifact (p

A multi-paradigm approach integrating three novel computational paradigms: wavelet transforms, chaos theory, and artificial neural networks is developed for EEG-based epilepsy diagnosis and seizure detection. This research challenges the assumption that the EEG represents the dynamics of the entire brain as a unified system. It is postulated that the sub-bands yield more accurate information about constituent neuronal activities underlying the EEG. Consequently, certain changes in EEGs not evident in the original full-spectrum EEG may be amplified when each sub-band is analyzed separately. A novel wavelet-chaos methodology is presented for analysis of EEGs and delta, theta, alpha, beta, and gamma sub-bands of EEGs for detection of seizure and epilepsy. The methodology is applied to three different groups of EEGs: healthy subjects, epileptic subjects during a seizure-free interval (interictal), and epileptic subjects during a seizure (ictal). Two potential markers of abnormality quantifying the non-linear chaotic EEG dynamics are discovered: the correlation dimension and largest Lyapunov exponent. A novel wavelet-chaos-neural network methodology is developed for EEG classification. Along with the aforementioned two parameters, the standard deviation (quantifying the signal variance) is employed for EEG representation. It was discovered that a particular mixed-band feature space consisting of nine parameters and LMBPNN result in the highest classification accuracy (96.7%). To increase the robustness of classification, a novel principal component analysis-enhanced cosine radial basis function neural network classifier is developed. The rearrangement of the input space along the principal components of the data improves the classification accuracy of the cosine radial basis function neural network employed in the second stage significantly. The new classifier is as accurate as LMBPNN and is twice as robust. Next, biologically realistic artificial neural networks are developed to reach the next milestone in artificial intelligence. First, an efficient spiking neural network (SNN) model is presented using three training algorithms: SpikeProp, QuickProp, and RProp. Three measures of performance are investigated: number of convergence epochs, computational efficiency, and classification accuracy. Next, a new Multi-Spiking Neural Network (MuSpiNN) and supervised learning algorithm (Multi-SpikeProp) are developed. Finally, the models are applied to the epilepsy and seizure detection problems to achieve high classification accuracies.