Search

An affective brain-computer interface (aBCI) is a direct communication pathway between human brain and computer, via which the computer tries to recognize the affective states of its user and respond accordingly. As aBCI introduces personal affective factors into human-computer interaction, it could potentially enrich the user’s experience during the interaction. Successful emotion recognition plays a key role in such a system. The state-of-the-art aBCIs leverage machine learning techniques which consist in acquiring affective electroencephalogram (EEG) signals from the user and calibrating the classifier to the affective patterns of the user. Many studies have reported satisfactory recognition accuracy using this paradigm. However, affective neural patterns are volatile over time even for the same subject. The recognition accuracy cannot be maintained if the usage of aBCI prolongs without recalibration. Existing studies have overlooked the performance evaluation of aBCI during long-term use. In this paper, we propose SAFE—an EEG dataset for stable affective feature selection. The dataset includes multiple recording sessions spanning across several days for each subject. Multiple sessions across different days were recorded so that the long-term recognition performance of aBCI can be evaluated. Based on this dataset, we demonstrate that the recognition accuracy of aBCIs deteriorates when re-calibration is ruled out during long-term usage. Then, we propose a stable feature selection method to choose the most stable affective features, for mitigating the accuracy deterioration to a lesser extent and maximizing the aBCI performance in the long run. We invite other researchers to test the performance of their aBCI algorithms on this dataset, and especially to evaluate the long-term performance of their methods.

This paper describes an open access electroencephalography (EEG) data set for multitasking mental workload activity induced by a single-session simultaneous capacity (SIMKAP) experiment with 48 subjects. To validate the database, EEG spectral activity was evaluated with EEGLAB and the significant channels and activities for the experiment are highlighted. Classification performance was evaluated by training a support vector regression model on selected features from neighborhood component analysis based on a nine-point workload rating scale. With a reduced feature dimension, 69% classification accuracy was obtained for 3 identified workload levels from the rating scale with Cohen's kappa of 0.46. Accurate discrimination of mental workload is a desirable outcome in the field of operator performance analysis and BCI development; thus, we hope that our provided database and analyses can contribute to future investigations in this research field.

Affective brain-computer interface (aBCI) introduces personal affective factors to human-computer interaction. The state-of-the-art aBCI tailors its classifier to each individual user to achieve accurate emotion classification. A subject-independent classifier that is trained on pooled data from multiple subjects generally leads to inferior accuracy, due to the fact that electroencephalography patterns vary from subject to subject. Transfer learning or domain adaptation techniques have been leveraged to tackle this problem. Existing studies have reported successful applications of domain adaptation techniques on SEED dataset. However, little is known about the effectiveness of the domain adaptation techniques on other affective datasets or in a cross-dataset application. In this paper, we focus on a comparative study on several state-of-the-art domain adaptation techniques on two datasets: 1) DEAP and 2) SEED. We demonstrate that domain adaptation techniques can improve the classification accuracy on both datasets, but not so effective on DEAP as on SEED. Then, we explore the efficacy of domain adaptation in a cross-dataset setting when the data are collected under different environments using different devices and experimental protocols. Here, we propose to apply domain adaptation to reduce the intersubject variance as well as technical discrepancies between datasets, and then train a subject-independent classifier on one dataset and test on the other. Experiment results show that using domain adaptation technique in a transductive adaptation setting can improve the accuracy significantly by 7.25%-13.40% compared to the baseline accuracy where no domain adaptation technique is used. National Research Foundation (NRF) Accepted version This work was supported by the National Research Foundation, Prime Minister’s Office, Singapore, through Its International Research Centers in Singapore Funding Initiative.

Study Objectives: Automated sleep staging has been previously limited by a combination of clinical and physiological heterogeneity. Both factors are in principle addressable with large data sets that enable robust calibration. However, the impact of sample size remains uncertain. The objectives are to investigate the extent to which machine learning methods can approximate the performance of human scorers when supplied with sufficient training cases and to investigate how staging performance depends on the number of training patients, contextual information, model complexity, and imbalance between sleep stage proportions. Methods: A total of 102 features were extracted from six electroencephalography (EEG) channels in routine polysomnography. Two thousand nights were partitioned into equal (n = 1000) training and testing sets for validation. We used epoch-by-epoch Cohen's kappa statistics to measure the agreement between classifier output and human scorer according to American Academy of Sleep Medicine scoring criteria. Results: Epoch-by-epoch Cohen's kappa improved with increasing training EEG recordings until saturation occurred (n = similar to 300). The kappa value was further improved by accounting for contextual (temporal) information, increasing model complexity, and adjusting the model training procedure to account for the imbalance of stage proportions. The final kappa on the testing set was 0.68. Testing on more EEG recordings leads to kappa estimates with lower variance. Conclusion: Training with a large data set enables automated sleep staging that compares favorably with human scorers. Because testing was performed on a large and heterogeneous data set, the performance estimate has low variance and is likely to generalize broadly.

Haptic interaction is a form of a user-computer interaction where physical forces are delivered to the user via vibrations, displacements and rotations of special haptic devices. When quality of the experience of the haptic interaction is assessed, mostly subjective tests using various questionnaires are performed. We proposed novel neurocognitive tools for assessing both overall experience of the haptic interaction, as well as particular time-stamped activities. Our assessment tools are based on recognition of emotions and stress obtained from Electroencephalograms (EEG). We used them in a feasibility study on adding haptic interaction to Skype video conversation.

Neuroscience-based or neuroscience-informed design is a new application area of Brain-Computer Interaction (BCI). It takes its roots in study of human well-being in architecture, human factors study in engineering and manufacturing including neuroergonomics. In traditional human factors studies and/or well-being study, mental workload, stress, and emotion are obtained through questionnaires that are administered upon completion of some task and/or the whole experiment. Recent advances in BCI research allow for using Electroencephalogram (EEG) based brain state recognition algorithms to assess the interaction between brain and human performance. We propose and develop an EEG-based system CogniMeter to monitor and analyze human factors measurements of newly designed software/hardware systems and/or working places. Machine learning techniques are applied to the EEG data to recognize levels of mental workload, stress and emotions during each task. The EEG is used as a tool to monitor and record the brain states of subjects during human factors study experiments. We describe two applications of CogniMeter system: human performance assessment in maritime simulator and EEG-based human factors evaluation in Air Traffic Control (ATC) workplace. By utilizing the proposed EEG-based system, true understanding of subjects working patterns can be obtained. Based on the analyses of the objective real time EEG-based data together with the subjective feedback from the subjects, we are able to reliably evaluate current systems/hardware and/or working place design and refine new concepts and design of future systems.

Created intentionally or spontaneously, cyberworlds are information spaces and communities that immensely augment the way we interact, participate in business and receive information throughout the world. This paper reports position statements presented at the plenary panel of the 2015th International Conference on Cyberworlds. First, the problems of enhancing creativity in cyberworlds using new interfaces are considered. It follows by the discussions on using biometric interfaces in on-line services. Finally, the challenges of using brain-computer interfaces and emotion recognition using electroencephalograms are considered.

Mental workload can be recognized from Electroencephalogram (EEG) and can be used to assess mental efforts of the user performing different tasks. In this work, we designed and implemented an experiment for mental workload recognition related to no-task, visual task, auditory task and multitask performance. The Simultaneous Capacity SIMKAP test was used to induce different levels of mental workload related to multitasking in 12 subjects. EEG data was collected with Emotiv device, processed and analyzed using power, statistical, fractal dimension (FD) features with Support Vector Machine (SVM) and k-Nearest Neighbors (k-NN) classifiers. The best accuracy of 90.39% for 2 classes and 80.09% for 4 classes using SVM was achieved when statistical and FD feature combination was used. The proposed algorithm can be applied for mental workload monitoring.

EEG-based technology is widely used in serious game design since more wireless headsets that meet consumer criteria for wearability, price, portability, and ease-of-use are coming to the market. Originally, such technologies were mostly used in different medical applications, Brain Computer Interfaces (BCI) and neurofeedback games. The algorithms adopted in such applications are mainly based on power spectrum analysis, which may not be fully revealing the nonlinear complexity of the brain activities. In this paper, we first review neurofeedback games, EEG processing methods, and algorithms, and then propose a new nonlinear fractal dimension based approach to neurofeedback implementation targeting EEG-based serious games design. Only one channel is used in the proposed concentration quantification algorithm. The developed method was compared with other methods used for the concentration level recognition in neurofeedback games. The result analysis demonstrated an efficiency of the proposed approach. We designed and implemented new EEG-based 2D and 3D neurofeedback games that make the process of brain training more enjoyable.

Visual interfaces are very important for human interactions in cyberworlds. Visual spatio-temporal querying should be one of the basic tools for data mining and retrieval in cyberworlds. In this paper, we propose a novel function-based query model for arbitrary shape spatio-temporal querying. The queries are defined as geometric shapes changing over time. In our model, data are interpreted geometrically as multidimensional points with time dimension or as moving points. The queries are formulated with geometric objects and operations over them to form the query solid changing over time. The proposed query model allows us to pose arbitrary shape spatio-temporal range queries. With the uniform geometric model we integrate visual mining and querying of time-dependent data employing 3D visualization tools. It allows for creating an intuitive visual interface using 2D projections of 3D query shapes. Our approach combines visualization of spatio-temporal data with visualization of the range query formulation employing very compact function-based query model. The implemented visual query system and its visual interface are proposed and described. An example of application of the system in analysis of simulation results in molecular dynamics is considered.

Surgical training is one of the most promising areas in medicine where 3-D computer graphics and virtual reality techniques are emerging. Orthopedic surgery is a discipline requiring appreciation and understanding of complex 3-dimensional bony structures and their relationships to nerves, blood vessels and other vital structures. Learning these spatial skills requires a lengthy period and much practice. In this paper, we present a software simulator which was developed to aid in the understanding of the complex 3-dimensional relationships between bones and implants. The developed software cuts down the learning curve and allows for better and more precise surgery by letting the surgeon practice the surgery in a virtual environment before undertaking the actual procedure.

Everyone experiences stress in life. Moderate stress can be beneficial to human, however, excessive stress is harmful to the health. To monitor stress, different methods can be used. In this work, an algorithm for stress level recognition from Electroencephalogram (EEG) is proposed. To validate the algorithm, an experiment is designed and carried out with 9 subjects. A Stroop colour-word test is used as a stressor to induce 4 levels of stress, and the EEG data are recorded during the experiment. Different feature combinations and classifiers are proposed and analyzed. By combining fractal dimension and statistical features and using Support Vector Machine (SVM) as the classifier, four levels of stress can be recognized with an average accuracy of 67.06%, three levels of stress can be recognized with an accuracy of 75.22%, and two levels of stress can be recognized with an accuracy of 85.71%. The algorithm is integrated into the system CogniMeter for stress state monitoring. Stress level of the user is visualized on the meter in real time. The system can be applied for stress monitoring of air traffic controllers, operators, etc.

Multitasking is a prevalent phenomenon in our daily lives. Certain occupations, especially in the aviation industry, consider proficient multitasking as a key skill set in their hiring process for pilot or air traffic controller candidates. There is a growing interest in the testing and training of the multitasking ability, with in house software or commercial psychological products, usually implemented in a static task battery format. In this paper, we propose a 3D game, Multitask In Neurofeedback Driving (MIND) for training and testing of the multitasking ability. The game is developed using the Unreal 3 game engine and incorporates neurofeedback, a technique used in the training of human cognitive abilities, to further enhance the potential benefits of the training procedure. The tasks used in the multitasking condition are inspired by various psychological tests and implemented in a manner that attempts to simulate the general cognitive processes required for multitasking while driving a vehicle or piloting an aircraft. The game comes in three variants, single task condition, multitasking condition and multitasking with neurofeedback condition, for the purpose of validating the training outcomes in future studies.

This paper explores the changes in EEG when subjects performed a modified Test of Variables of Attention (TOVA), compared to open eye resting (baseline) state. To recognize these two different brain states, two machine learning algorithms, i.e. extreme learning machine (ELM) and support vector machine (SVM), were applied and compared, using 3 statistical features and 4 power spectral density per channel. The results showed that using all 14 channels, ELM and SVM achieved similar test accuracy of 94.6% and 95.1% respectively (McNemar’s test p = 0.8 > 0.05). Using recursive channel selection, 9 channels (ELM) and 8 channels (SVM) were selected from 14 channels. After channel selection, ELM outperformed SVM significantly (McNemar’s test p = 0.0005 < 0.01) with average test accuracy of 95.0% and 92.5% respectively. The channel rank of each subject was weighted and merged using analytic hierarchical process to obtain a cross-subject ranking, which revealed the close correlation between TOVA and the visual pathway in brain.

Drivers’ high workload caused by distractions has become one of the major concerns for road safety. This paper presents a data-driven method using machine learning algorithms to detect high workload caused by surrogate in-vehicle (IV) secondary tasks performed in an on-road experiment with real traffic. The data were collected using an instrumented vehicle while drivers performed two types of secondary tasks: visual-manual and auditory-vocal tasks. Two types of machine learning methods, support vector machine (SVM) and extreme learning machine (ELM), were applied to detect drivers’ workload via drivers’ visual behaviour (i.e. eye movements) data alone, as well as visual plus driving performance data. The results suggested that both methods can detect drivers’ workload at high accuracy, with ELM outperformed SVM in most cases. We found that for visual intensive workload, using drivers’ visual data alone achieveed an accuracy close to using the combination information from both visual and driving performance data. This study proves that machine learning methods can be used for real driving applications.

This article discusses the problems of teaching computer graphics and shape modeling in large and distributed classes using visual cyberworlds—shared information worlds on the Web. Cyberworlds allow for providing personal mentoring to the students with different cultural and educational backgrounds. The Virtual Campus of Nanyang Technological University is such a cyberworld, which is being used for teaching computer graphics and shape modeling. A part of this cyberworld is the Virtual Shape Modeling Laboratory. It is used by the computer graphics students for designing geometric shapes defined with analytical formulas. Augmenting the existing ways of electronic education with cyberworlds appears to be useful which was proved by the final exam results and overall attitude of the students.

In recent years, the usage of 3D cyberworlds for educational purposes has increased. The metaphors behind the visual design of such cyberworlds are quite diverse, from replication of real universities, art museums and scientific labs to non-existing fictitious places. In this paper, we focus specifically on “cybercampuses”, i.e. virtual worlds representing real educational institutions such as universities and schools. Based on the results of a case study that we have performed, this paper provides an initial set of requirements for a cybercampus representing an existing university. In this connection, we analyze place metaphors and associated design features of the Virtual Campus of Nanyang Technological University in Singapore, discuss the correspondence between the identified metaphors and associated educational goals, and provide directions for further development. Finally, we outline the major challenges for the future evolution of cybercampuses in the context of organizational, social and technological development.

The paper presents a novel technique of nonlinear spectral analysis. This technique is based on the concept of generalized entropy of a given probability distribution, known as the Rényi entropy. This concept allows defining generalized fractal dimension of encephalogram (EEG) and determining fractal spectra of encephalographic signals. These spectra contain information of both frequency and amplitude characteristics of EEG and can be used together with well-accepted techniques of EEG analysis as an enhancement of the latter. Powered by volume visualization of the brain activity, the method provides new clues for understanding the mental processes in humans.

Continuously increasing power of computers makes it possible to model dynamic behaviour of the systems studied by experimental nanoscience (e.g., single-molecule mechanics, electron and atomic force microscopy) with explicit all-atom presentation by applying various molecular dynamics approximations. Computer simulation in molecular dynamics usually results in gigabytes of timedependent data. On the other hand, computational and querying tools could help to analyze such information. We propose a novel geometric querying method for data analysis in molecular dynamics. This method is based on a novel geometric model for querying of spatio-temporal data. In our model, data from database or file are interpreted geometrically as multidimensional points with time dimension. A geometric query is a query solid of any shape specified by its parameters, location and time. These queries are formulated with geometric objects and operations over them to form the query solid changing over time. The geometric objects and operations are described with implicit functions. We propose spatio-temporal queries for data analysis in molecular dynamics that allow formulation of new hypotheses of nanoscale structures and interactions between molecules. The geometric queries in our model can be considered as motion classifiers. The resulting datasets are coordinates of molecules and atoms inside the particular shape changing over time. With the uniform geometric model we integrate visual mining and querying of time-dependent data employing 3D visualization tools. This query system should be useful in detailed studies on structure and movement of large molecule complexes of interest in various nanotechnology applications.

Recently, new interactive devices such as hap tic devices became available for game development. 6DOF hap tic devices give the user an opportunity "to feel" the simulated virtual environment in the way similar to the real world. Hap tic-based interaction can add a new dimension to "serious games" development. The user can "feel" objects surfaces and complex objects interaction forces in a 3D virtual environment. In this paper, we propose two hap tic-based serious games. In the first "T Puzzle" game, the user can "feel" the weight of objects, rotate and move the 3D puzzle pieces in the virtual world. This game can be used to improve the user's spatial abilities. In the second "Mol Docking" game, the player can feel interaction forces between molecular systems and learn the process of molecular docking in collaborative virtual environment.

Nowadays, the human computer interfaces can be designed to be adaptive and emotion-enabled. The recognized emotions of the user can help make the user’s experience more complete, more engaging, less stressful or more stressful depending on the target of the applications. Such affective human-computer interfaces are getting more attention from researchers and engineers. EEG signals are used to recognize emotions of the user in real time. We describe a real-time emotion recognition algorithm that is used to personalize different applications according to the user’s current emotions. The algorithm is subject-dependent and needs a training session before running the application. Two EEG-enabled games and one adaptive advertisement based on the algorithm are designed and implemented. One game is the “Bar” game where a difficulty level of the game is adapted based on the player’s score and emotions. Another game is the “Girl Twins” one where the player’s emotions are monitored in real time, and an emotional companion is implemented as the girl twins avatars whose behaviour changes according to the user’s emotions. An adaptive advertising movie is designed and implemented as well. Here, the real-time emotion recognition algorithm is used to adjust the scenes of the advertisement based on the current emotion recognized.

Medicine is one of the most promising areas for emerging computer graphics and virtual reality techniques. VR training simulators let surgeons practice on virtual body tissue and get the same feedback they would experience in performing a real operation. Hybrid VR systems permit medical practitioners to view the patient overlaid with 3D data sets derived from 3D scanners, thus providing doctors and surgeons with pseudo X-ray vision. While currently available immersive VR surgery systems usually require expensive hardware and software, we developed a desktop VR orthopedic surgery training system that can run on commonly available personal computers.

Emotion recognition from Electroencephalogram (EEG) rapidly gains interest from research community. Two affective EEG databases are presented in this paper. Two experiments are conducted to set up the databases. Audio and visual stimuli are used to evoke emotions during the experiments. The stimuli are selected from IADS and IAPS databases.14 subjects participated in each experiment. Emotiv EEG device is used for the data recording. The EEG data are rated by the participants with arousal, valence, and dominance levels. The correlation between powers of different EEG bands and the affective ratings is studied. The results agree with the literature findings and analyses of benchmark DEAP database that proves the reliability of the two databases. Similar brain patterns of emotions are obtained between the established databases and the benchmark database. A SVM-based emotion recognition algorithm is proposed and applied to both databases and the benchmark database. Use of a Fractal Dimension feature in combination with statistical and Higher Order Crossings (HOC) features gives us results with the best accuracy. Up to 8 emotions can be recognized. The accuracy is consistent between the established databases and the benchmark database.

Nowadays, advertising is a part of our daily life driving consumer behavior, social behavior, personal preferences, etc. As volumes of advertisements increase people become more immune to different types of advertisements. To make the advertisements more efficient is a challenging problem. It is confirmed in the experiments that the emotions felt by the participants during the viewing of an advertisement influence on the effectiveness of the advertisement. In this paper, we propose an emotion-enabled algorithm that can be used to personalize an advertising movie according to the user's current emotions to make the advertisement more efficient. Electroencephalogram (EEG) signals are used to recognize emotions of the user in real time. The proposed emotion-enabled algorithm can adjust the scene of the movie based on the real-time emotion feedback. An advertising movie that applies the emotion-enabled algorithm is designed and implemented.

Emotions play an important role in human communications. A new "emotional dimension" could be added to human-computer interfaces to make communication more intuitive. Generally, emotions could be recognized from text, voice, facial expression, gesture and/or from biosignals (pulse, EEG, temperature, etc.). EEG is electroencephalogram signals of the human brain and could reveal the "inner" real human emotions. EEG-based emotion recognition is relatively new area, and most of the works in this area focus on off-line emotion recognition. Recently, wireless and portable devices came to the market that made possible to add EEG-based interaction to human-computer interfaces and develop the real-time applications. We present a novel work on real-time EEG-based emotion recognition and emotion-enabled applications for entertainment such as emotional avatars, emotion-enabled animation, and emotion-enabled personalized web-based player.

These tutorials discuss the following: cyberspace terrorism; EEG-enabled personalized digital experience; and nontraditional modelling.

Haptic device enable the user to manipulate the molecules and feel interactions during the docking process in virtual environment on the computer. Implementation of force-torque feedback allows the user to have more realistic experience during force simulation and find the optimum docking positions faster. In this paper, we propose a haptic rendering algorithm for biomolecular docking with force-torque feedback. It enables the user to experience six degree-of-freedom (DOF) haptic manipulation in molecular docking process. The linear smoothing method was proposed to improve stability of the haptic rendering during molecular docking. Collaborative docking with two devices was implemented.

Since emotions play an important role in the daily life of human beings, the need and importance of automatic emotion recognition has grown with increasing role of human computer interface applications. Emotion recognition could be done from the text, speech, facial expression or gesture. In this paper, we concentrate on recognition of "inner" emotions from electroencephalogram (EEG) signals. We propose real-time fractal dimension based algorithm of quantification of basic emotions using Arousal-Valence emotion model. Two emotion induction experiments with music stimuli and sound stimuli from International Affective Digitized Sounds (IADS) database were proposed and implemented. Finally, the real-time algorithm was proposed, implemented and tested to recognize six emotions such as fear, frustrated, sad, happy, pleasant and satisfied. Real-time applications were proposed and implemented in 3D virtual environments. The user emotions are recognized and visualized in real time on his/her avatar adding one more so-called "emotion dimension" to human computer interfaces. An EEG-enabled music therapy site was proposed and implemented. The music played to the patients helps them deal with problems such as pain and depression. An EEG-based web-enable music player which can display the music according to the user's current emotion states was designed and implemented.

Haptic devices enable the user to manipulate the molecules and feel interactions during the docking process in virtual environment on the computer. Implementation of torque feedback allows the user to have more realistic experience during force simulation and find the optimum docking positions faster. In this paper, we propose a haptic rendering algorithm for biomolecular docking with torque force. It enables the user to experience six degree-of-freedom (DOF) haptic manipulation in docking process. The linear smoothing method was proposed to improve stability of the haptic rendering during molecular docking.

Recently, EEG-based technology has become more popular in “serious” games designs and developments since new wireless headsets that meet consumer demand for wear ability, price, portability and ease-of-use are coming to the market. Originally, EEG-based technologies were used in neurofeedback games and brain-computer interfaces. Now, such technologies could be used in entertainment, e-learning and new medical applications. In this paper, we review on neurofeedback game designs and algorithms, and propose design, algorithm, and implementation of new EEG-based 2D and 3D concentration games. Possible future medical applications of the games are discussed.

When teaching subjects richly infused with mathematics, in particular geometry, topology and shape modeling, there is a frequent problem that the learners are not able to "visualize" the attendant theoretical concepts. It is important, therefore, to constantly illustrate the associated theories with practical visual exercises, which are preferably to be done in collaboration with other learners to allow them to discuss possible approaches to the problem and to consult with the instructor, virtually or face-to-face. We have proposed an approach that would allow for solving mathematical problems while being immersed within shared virtual 3D collaborative environments. Only mathematical formulas are used by the learners for immediate interactive definition of geometry, appearance and physical property of the shapes being created in the virtual environment. We target learners and educators who are studying subjects rich in mathematics and geometry, or teaching them at the secondary or tertiary level or doing research on these topics. The process allows the learners to see and feel the geometric meaning of mathematics, thus making it less abstract and more perceptual and tangible. We also see ways of incorporating beneficial uses of immersive virtual environments into traditional courses at Universities which might benefit from 3D visualization.

Visual haptic-based biomolecular docking systems could be used for both research and e-learning in research intensive disciplines such as biology, physical chemistry, molecular medicine, biophysics, structural biology, bioinformatics, etc. The assembly of molecules in a three-dimensional space or molecular docking is used for rational drug design where a ligand docks onto a receptor. The computer-aided design systems allow a real-time interactive visualization and manipulation of molecules in virtual environment. These techniques help the user to understand molecular interactions. In recent years, besides the visualization techniques, there has been increasing interest in using haptic interfaces to facilitate the exploration and analysis of molecular docking. Haptic device enables the users to manipulate the molecules and feel its interaction during the docking process in virtual experiment on computer. In this paper, we describe a visual haptic-based biomolecular docking system that we developed for research in helix-helix docking and propose its application in e-learning. We also describe haptic-based collaborative e-learning scenarios.

Visual clustering should be one of the basic tools for time-dependent data analysis in cyberworlds. This paper describes a novel approach to spatial clustering and boundary detection based on geometric modeling and visualization. Datasets and boundaries of clusters are visualized as 3D points and surfaces of reconstructed solids changing over time. Our approach applies the concepts of geometric solid modeling and uses density as clustering criteria that comes from traditional density-based clustering techniques. Visual clustering allows the user to analyze results of clustering the data changing over time and to interactively choose appropriate parameters.

Visual interfaces are very important for human interactions in Cyberworlds. Visual querying should be one of the basic tools for data mining and retrieval in Cyberworlds. In this paper, function-based modeling of spatio-temporal range queries is described. It allows for creating an intuitive visual interface using 2D projection of 3D query shapes. The proposed approach combines visualization of time-dependent data with visualization of the range query formulation employing very compact function-based query model. It is also shown that the proposed geometric model makes constructing the queries on spatio-temporal attributes more intuitive, and allows for posing temporal range queries of any shape.

Visualization techniques increase the user involvement in the interactive process of data mining and querying of spatio-temporal data. This paper describes a novel geometric approach to clustering and querying of spatio-temporal data. We propose the uniform geometric model based on function representation of solids to cluster and query time-dependent data. Clustering and querying are integrated with visualization techniques in one GUI. First, visual clustering with blobby model allows the user to see the result of clustering on the screen for different time points and/or time intervals and set the appropriate parameters interactively. After that, the user gets the data of clusters for the chosen time frames. Then, the user can visually query the cluster/clusters he/she is interested in with geometric primitive solids which currently are cubes, spheres/ellipsoids, cylinders, etc. Geometric operations of union, intersection and/or subtraction can be performed over the geometric primitive solids to get the final query shape. The user visually clusters spatio-temporal data and queries the clusters with geometric shapes through graphics interface accessing dynamically 3D projections of multidimensional points from database, warehouses or files.With the uniform geometric model of the clustering and querying of spatio-temporal data, 3D visualization tools can be naturally incorporated in one system to allow the user to visualize and query clusters changing over time.

In the recent years, the usage of 3D cyberworlds for educational purposes has increased. The metaphors behind the design of virtual places are quite diverse, from replication of real universities to art museums and scientific labs. Based on the results of a case study we have performed, this paper provides an initial set of requirements for a cyberworld representing an existing university. In this connection, we analyze place metaphors and associated design features of the virtual campus of Nanyang Technological University in Singapore in the context of related work. Finally, we discuss the correspondence between the identified metaphors and associated educational goals, providing directions for further development of the virtual campus

Temporal databases and data warehouses are essential components of intelligent information systems in cyberworlds. We describe a geometric model for querying time-dependent data in databases and warehouses. An implementation and application of the model for querying of results of numerical simulation in molecular dynamics is discussed. Data are interpreted geometrically as multidimensional points with time dimension. A geometric query is a query solid of any shape specified by its parameters, location and time. These queries are formulated with geometric objects and operations over them to form the query solid. The geometric objects and operations are described with implicit functions. With the uniform geometric model for querying time-dependent data, 3D visualization tools can be naturally incorporated into the molecular dynamics visualization system to pose the queries.

Databases and warehouses play significant role in real world providing the human with the necessary information for solving problems in all areas of life from marketing to bioinformatics. Therefore, databases and warehouses are essential components of all intelligent information systems in cyberworlds. This paper describes a geometric approach to clustering and querying data in databases and warehouses. Data are interpreted geometrically as multidimensional points. A geometric definition of the cluster as a multidimensional solid defined with implicit functions is introduced. A query window is a query solid of any shape specified by its location. The queries are formulated with geometric objects and operations over them. The geometric objects and operations are described with implicit functions. With the uniform geometric model of the clustering and querying, 3D visualization tools can be naturally incorporated in one system that allows us to visualize and query clusters in 3D space. The user clusters the data and poses the queries through a graphics interface accessing dynamically multidimensional points and solids.

This paper describes a geometric query model for dynamic exploring multidimensional data. An application of the model for solving the problem of meterials selection in product design is discussed. Data from database are interpreted geometrically as multidimensional points. A query window is a query solid of any shape specified by its location. The queries are formulated with geometric objects and operations over them. The geometric objects and operations are described with implicit functions. The process of query specification is visualized. The user poses the queries through graphics interface accessing dynamically multidimensional points, geometric primitives and applying geometric operations over them.

Attention is constantly required in many daily life tasks. Attention-related behavior, such as driving distraction, has been reported as a major reason in traffic accidents. Therefore, the recognition of attention can enhance task performance. Electroencephalogram (EEG) is used to study attention, since it provides a direct measure of the brain activity with high temporal resolution at 1-10ms. In this thesis, we study the recognition of selective attention and sustained attention (vigilance) using machine learning. We start by proposing an experiment which aims to recognize unattended and attended conditions induced by Test of Variables of Attention (TOVA), using EEG features supported by the event-related potential (ERP) literature. However, ERP does not work for real-time applications where the external stimuli (event) required by ERP cannot be controlled. In face of the low signal-to-noise ratio (SNR) in the non-ERP approach, we propose Channel Selection with Different Features (CSDF) algorithm, which selects channels with their own different feature sets, as well as restricts features to as few channels as possible. Using CSDF, 83 out of 868 features are selected to distinguish the unattended and attended conditions. The accuracy 94.3% (±5.6%) is the best compared to other feature selection and channel selection algorithms. Based on CSDF, we find that the first and second order difference in the left parietal and temporal lobes, as well as the Higuchi fractal dimension and mean signal amplitude in the right frontal lobe, are relevant to selective attention. Unlike selective attention which has discrete conditions such as attended/unattended, the vigilance stages cannot be easily observed. Analogous to sleep stages, we want to define the vigilance stages in open eye and situation-aware state in a subject-independent and data-driven way. In the literature, there are vigilance stage models defined under closed eye. However, the EEG signals are more complex in open eye and situation-aware state. Extreme learning machine autoencoder (ELMAE) is used to learn the EEG spectral features and define the vigilance stages during simulated driving. Results show that ELMAE is an efficient alternative to restricted Boltzmann machine (RBM) in vigilance recognition: ELMAE achieves root mean square error at 0.189 (±0.049), which is better than RBM at 0.195 (±0.046); and training speed significantly faster than RBM. Based on ELMAE, we define three vigilance stages in open eye and situation-aware state. Stage I is high vigilance, where the subject is attentive. Stage II is low vigilance, which is further divided into Stage II.1: drowsiness and difficulty in attention allocation; and Stage II.2: distraction instead of falling asleep. A possible explanation for stage II.2 is that, the environment contains not enough external stimuli to keep the open eye and situation-aware state, so that the brain performs vigilance regulation to seek external stimuli, and hence leading to distraction. A major limitation of ELMAE is that it learns nonsparse hidden weights and features. Analogous to feature selection using CSDF, we want to learn sparse features so that the useful information is restricted to a few nonzero features to achieve better interpretability. To address this issue, a novel bio-inspired algorithm, joint weight-delay spike-timing dependent plasticity (joint STDP), is proposed for learning sparse hidden weights and EEG spectral features. Compared to other nonsparse and sparse algorithms, joint STDP achieves the highest level of sparseness in both learned features and hidden weights significantly. On the other hand, joint STDP has slightly larger root mean square error at 0.206 (±0.061), due to the trade-off between performance and sparseness. Extensive experiments and comparisons are carried out to evaluate all the proposed algorithms. The experimental results confirm the advantages of the proposed algorithms, hence making contribution to EEG-based attention research. This thesis is interdisciplinary and includes several fields such as machine learning, computational neuroscience, brain-computer interface and psychology. Doctor of Philosophy (IGS)

To deal with various types of data from diverse areas, where the amount of data is still increasing dramatically, data mining has become one of the fastest growing fields in computer industry in last decade. Data mining aims at discovery of knowledge implicitly stored in information repositories. Among all data mining tasks, clustering is an approach to distinguish different groups of data (clusters) without given class labels. It is also named unsupervised learning in artificial intelligence (AI) research field. In this study, we focus on clustering methods for analyzing spatial data, especially on low dimensional data. Applications in large spatial databases, multimedia, point-based graphics, etc brought new requirements for spatial clustering such as automatic discovering of arbitrary shaped and/or non-homogeneous clusters, detecting various types of outliers and building cluster boundaries, etc. To fulfill such new requirements generated from different spatial applications we proposed and implemented three novel algorithms. Our research is concentrated on 2D algorithms implementation. In the study when the connectivity between data points is the most concern, we propose novel clustering algorithms based on specially constructed adaptive functions. The adaptive function-based clustering method applies functions as influence functions to simulate a relationship between data points in data set. Different features of the whole data set can be described by the field function which is built by the sum of influence functions of each data point. The clustering procedure is executed according to threshold value of the field function. The introduced adaptive parameters of influence functions are employed to localize the field function based on both local and global data distribution. DOCTOR OF PHILOSOPHY (EEE)