Search

Stroke is the leading cause of adult disability. Robot-assisted rehabilitation systems show great promise for motor recovery after a stroke. In this work, we present a gazecontrolled robotic system for upper limb rehabilitation. Subjects perform a painting task in virtual reality. We designed a novel and challenging painting task to encourage motivation and engagement, as these are critical factors in treatment efficacy. Because the robotic system can be programmed to provide varying amounts of assistance or resistance to the subject, it can be applied to a wide range of patients at different phases of recovery. We describe here the system configured in two modes: resistive control and hierarchical control. The former is designed for later stages of recovery, where the patient's impaired limb has recovered some function. It can be configured to provide varying degrees of resistance by adjusting the properties of an admittance controller. The latter targets patients in more acute phases, where the impaired limb is less responsive. It provides a combination of assistive and corrective control. We pilot tested our system on 10 able-bodied subjects. Our results show that the system can provide varying degrees of resistive control, and that the integration of high level control modulated by gaze can improve engagement. These results suggest that the system may provide a more engaging environment for a wide range of rehabilitative therapies than currently available.

Gaze-based interfaces are especially useful for people with disabilities involving the upper limbs or hands. Typically, users select from a number of options (e.g. letters or commands) displayed on a screen by gazing at the desired option. However, in some applications, e.g. gaze-based driving, it may be dangerous to direct gaze away from the environment towards a separate display. In addition, a purely gaze based interface can present a high cognitive load to users, as gaze is not normally used for selection and/or control, but rather for other purposes, such as information gathering. To address these issues, this paper presents a cost-effective multi-modal system for gaze based driving which combines appearance-based gaze estimates derived from webcam images with push button inputs that trigger command execution. This system uses an intuitive "direct interface", where users determine the direction of motion by gazing in the corresponding direction in the environment. We have implemented the system for both wheelchair control and robotic teleoperation. The use of our system should provide substantial benefits for patients with severe motor disabilities, such as ALS, by providing them with a more natural and affordable method of wheelchair control. We compare the performance of our system to the more conventional and common "indirect" system where gaze is used to select commands from a separate display, showing that our system enables faster and more efficient navigation.

Recent research has shown that design principles inherent in a Japanese style garden can reduce measures of stress in both healthy and dementia patients. However, it was not clear how subjects' visual interaction with the scene affected their physiological responses. To address that, we developed a novel non-invasive system to collect synchronized measurements of eye gaze and physiological indicators of sympathetic neural activity: the electrocardiogram, the blood volume pulse and the galvanic skin response, as subjects view a garden environment. We characterized the visual engagement of subjects using the average fixation duration, the saccade amplitude and the gaze transition entropy. We find a statistically significant positive correlation between gaze transition entropy and mean heart rate change. Our results suggest that the visual engagement of subjects with their environment may influence their physiological responses to it: more engagement may lead to more relaxation. Our results also highlight the importance of taking into account the detailed spatio-temporal characteristics of the gaze trajectory.

Previous studies have demonstrated that exposure to a Japanese garden is a non-pharmacological measure to improve the behavioral symptoms of elderly people with dementia, and that Japanese gardens are significantly more effective than other environments. However, it is not clear whether Japanese gardens have similar effects in the young. To address this open question, we measured the physiological responses of university students when viewing a Japanese garden, and compared them to the same students' responses when viewing a control space. We measured three physiological indicators of autonomous nervous system (ANS) activity: the electrocardiograph (ECG), the blood volume pulse (BVP) and the galvanic skin response (GSR). Our results suggest that the Japanese garden does not have as calming an effect on younger subjects as observed previously in elderly subjects. However, students did respond more positively to the Japanese garden than to an unstructured space. Ambient temperature was found to be a critical factor affecting heart rate and heart rate variability, but not other measures.

Face recognition plays an import role in our daily lives. However, computer face recognition performance degrades dramatically with the presence of variations in illumination, head pose and occlusion. In contrast, the human brain can recognize target faces over a much wider range of conditions. In this paper, we investigate target face detection through electroencephalography (EEG). We address the problem of single-trial target-face detection in a rapid serial visual presentation (RSVP) paradigm. Whereas most previous approaches used support vector machines (SVMs), we use a convolutional neural network (CNN) to classify EEG signals when subjects view target and non-target face stimuli. The CNN outperforms the SVM algorithm, which is commonly used for event-related-potential (ERP) detection. We also compare the difference in performance when using animal stimuli. The proposed system can be potentially used in rapid face recognition system.

Eye tracking systems are typically divided into two categories: remote and mobile. Remote systems, where the eye tracker is located near the object being viewed by the subject, have the advantage of being less intrusive, but are typically used for tracking gaze points on fixed two dimensional (2D) computer screens. Mobile systems such as eye tracking glasses, where the eye tracker are attached to the subject, are more intrusive, but are better suited for cases where subjects are viewing objects in the three dimensional (3D) environment. In this paper, we describe how remote gaze tracking systems developed for 2D computer screens can be used to track gaze points in a 3D environment. The system is non-intrusive. It compensates for small head movements by the user, so that the head need not be stabilized by a chin rest or bite bar. The system maps the 3D gaze points of the user onto 2D images from a scene camera and is also located remotely from the subject. Measurement results from this system indicate that it is able to estimate gaze points in the scene camera to within one degree over a wide range of head positions.

This paper addresses the problem of estimating gaze location in the 3D environment using a remote eye tracker. Instead of relying only on data provided by the eye tracker, we investigate how to integrate gaze direction with the point-cloud-based representation of the scene provided by a Kinect sensor. The algorithm first combines the gaze vectors for the two eyes provided by the eye tracker into a single gaze vector emanating from a point in between the two eyes. The gaze target in the three dimensional environment is then identified by finding the point in the 3D point cloud that is closest to the gaze vector. Our experimental results demonstrate that the estimate of the gaze target location provided by this method is significantly better than that provided when considering gaze information alone. It is also better than two other methods for integrating point cloud information: (1) finding the 3D point closest to the gaze location as estimated by triangulating the gaze vectors from the two eyes, and (2) finding the 3D point with smallest average distance to the two gaze vectors considered individually. The proposed method has an average error of 1.7 cm in a workspace of 25 × 23 × 24 cm located at a distance of 60 cm from the user.

We describe a hybrid brain computer interface that integrates gaze information from an eye tracker with brain activity information measured by electroencephalography (EEG). Users explicitly control the end effector of a robot arm to move in one of four directions using motor imagery to perform a pick and place task. Measurements of the natural eye gaze behavior of subjects is used to infer the instantaneous intent of the users based on the past gaze trajectory. This information is integrated with the output of the EEG classifier and contextual information about the environment probabilistically using Bayesian inference. Our experiments demonstrate that subjects can achieve 100% task completion within three minutes and that the integration of EEG and gaze information significantly improves performance over either cue in isolation.

We describe a two stage hidden Markov model based algorithm for inferring target identity in a 2D cursor control task where subjects are instructed to use a joystick to steer a cursor towards a target while avoiding obstacles. The first stage of the model converts a regularly sampled gaze trajectory into a sequence of fixations. The second stage then makes a determination of the end target in the cursor control task based on this sequence of fixations. In contrast to prior work, this two stage model allows for more accurate modelling of the natural eye gaze behavior of subjects, which in turn leads to increased accuracy and speed of target identification. This work demonstrates the importance of accurate gaze modelling, and paves the way for more natural and reliable hybrid Brain Computer Interfaces.