Search

Monitoring the mental workload of operators is of paramount importance in space telerobotic training and other teleoperation tasks. Instead of the estimation of task-specific workload, this article aims at investigating the impact of two significant confounding factors (time-pressure and latency) on space teleoperation and explored the use of eye-tracking technology for factor-induced mental workload estimation and performance evaluation. Ten subjects teleoperated a Canadarm2 robot to complete a complex on-orbit assembly task in our photo-realistic training simulator while wearing a head-mounted eye-tracker. To understand how time-pressure and latency influence eye-tracking features works, we first performed the statistical analysis on various features with respect to a single factor and across multiple groups. Next, eye-tracking features extracted from segment data and trial data is used to identify the mental workload induced by confounding factors, which can be used for developing personalized training programs and guaranteeing safe teleoperation. Furthermore, to improve the recognition performance using segment data, we propose the activity ratio and time ratio to characterize the informative segments. Finally, the relationship between simulator-defined performance measures and eye-tracking features is examined. Results show that fixation duration, saccade frequency and duration, pupil diameter, and index of pupillary activity are significant features that can be used in both factor-induced mental workload estimation and task performance evaluation.

Objective. Brain–computer interfaces (BCI) are becoming important tools for assistive technology, particularly through the use of motor imagery (MI) for aiding task completion. However, most existing methods of MI classification have been applied in a trial-wise fashion, with window sizes of approximately 2 s or more. Application of this type of classifier could cause a delay when switching between MI events. Approach. In this study, state-of-the-art classification methods for motor imagery are assessed offline with considerations for real-time and self-paced control, and a convolutional long-short term memory (C-LSTM) network based on filter bank common spatial patterns (FBCSP) is proposed. In addition, the effects of several methods of data augmentation on different classifiers are explored. Main results. The results of this study show that the proposed network achieves adequate results in distinguishing between different control classes, but both considered deep learning models are still less reliable than a Riemannian MDM classifier. In addition, controlled skewing of the data and the explored data augmentation methods improved the average overall accuracy of the classifiers by 14.0% and 5.3%, respectively. Significance. This manuscript is among the first to attempt combining convolutional and recurrent neural network layers for the purpose of MI classification, and is also one of the first to provide an in-depth comparison of various data augmentation methods for MI classification. In addition, all of these methods are applied on smaller windows of data and with consideration to ambient data, which provides a more realistic test bed for real-time and self-paced control.

Assistive Robotic Manipulators (ARMs) play an important role for people with upper-limb disabilities and the elderly by helping them complete Activities of Daily Living (ADLs). However, as the objects to handle in ADLs differ in size, shape and manipulation constraints, many two or three fingered end-effectors of ARMs have difficulty robustly interacting with these objects. In this paper, we propose vision-based control of a 5-fingered manipulator (Schunk SVH), automatically changing its approach based on object classification using computer vision combined with deep learning. The control method is tested in a simulated environment and achieves a more robust grasp with the properly shaped five-fingered hand than with a comparable three-fingered gripper (Barrett Hand) using the same control sequence. In addition, the final optimal grasp pose (x, y, and $\theta$) is learned through a deep regressor in the penultimate stage of the grasp. This method correctly identifies the optimal grasp pose in 78.35% of cases when considering all three parameters for an object included in the training set, but in a different setting than that of the training set.

The compliant nature of exosuits makes them ideal for providing assistance to complex joints like the shoulder. Exosuits require soft, compact and accurate sensing units for reliable feedback control. In this work, we introduce an OpenSim simulation-based prototype of a proprioception-inspired sensing framework for a multi-DoF shoulder exosuit. The prototype is used to study the feasibility of the sensing system concept to accurately track multiple degrees of freedom (DoFs) of the shoulder simultaneously. The sensing system fuses data from 4 custom string poten-tiometers (SPs), that work together to sense the joint angles at the shoulder. The tendon-routing of the SP modules in the exosuit is proprioception-inspired and based on the organization of the muscles influencing shoulder movement. The sensor fusion/mapping of the simulation data from multi-sensor space to joint space is a multivariate multiple regression problem and was solved using Multi-Layer Perceptron (MLP) & Long Short-Term Memory (LSTM) neural networks. A simulation of the framework in OpenSim on 200,000 random shoulder movements achieved a root mean square error (RMSE) of $\approx 0.2^{\mathrm{o}}$ when trained on 70,000 random movements and tested on 130,000 random movements in both DoFs simultaneously.

Traditional methods of training a Brain-Computer Interface (BCI) on motor imagery (MI) data generally involve multiple intensive sessions. The initial sessions produce simple prompts to users, while later sessions additionally provide realtime feedback to users, allowing for human adaptation to take place. However, this protocol only permits the BCI to update between sessions, with little real-time evaluation of how the classifier has improved. To solve this problem, we propose an adaptive BCI training framework which will update the classifier in real time to provide more accurate feedback to the user on 4-class motor imagery data. This framework will require only one session to fully train a BCI to a given subject. Three variations of an adaptive Riemannian BCI were implemented and compared on data from both our own recorded datasets and the commonly used BCI Competition IV Dataset 2a. Results indicate that the fastest and least computationally expensive adaptive BCI was able to correctly classify motor imagery data at a rate 5.8% higher than when using a standard protocol with limited data. In addition it was confirmed that the adaptive BCI automatically improved its performance as more data became available.

A significant fraction of the world population is plagued by neuromuscular disorders which have no cure other than symptomatic management. An increasingly aging world population would inevitably lead to a further rise in these numbers. The management of the manifestations of many neuromuscular diseases ranging from Parkinson’s disease to stroke has been an active research topic in robotics since the 1960s. Pivotal advances in sensing, actuation, energy sources, and computing technologies have facilitated an increased penetration of wearable robotics into patient rehabilitation and assistance. Furthermore, breakthroughs in research areas including material sciences and new actuation schemes are further accelerating the development of wearable robotics. The purpose of this chapter is to provide a review of robotic devices for upper-limb rehabilitation and assistance. A discussion of the state-of-the-art in design, actuation, and intention-sensing technologies is presented. The chapter also outlines challenges hindering the clinical translation of these technologies and highlights future research opportunities.

To functionally aid patients suffering from neurological disorder, a 3 degrees-of-freedom (DoF) upper limb wearable robot is presented (Fig. 1). In order to provide seamless user assistance, the intention of the wearer must be determined. As a sensing mechanism, electromyographic (EMG) signals have commonly been used to estimate human movement. In this study, the effectiveness of movement recognition using a generalized 8-port EMG sensor (Myo Armband) around the forearm was evaluated. Four fundamental movements of the arm (wrist flexion/extension and forearm pronation/supination) were classified using a neural network (NN) with a single hidden layer. The classification method was optimized through analysis of pre-processing algorithms and window size (0.25 to 1 second) to reduce computational expense and maintain classification accuracy. Through these accomplishments, significant groundwork has been provided for the development of a robust and non-invasive solution to tremor of the upper limb.