Search

Estimation of hand kinematics from surface electromyographic (sEMG) signals provides a non-invasive human-machine interface. This approach is usually subject-specific, so that the training on one individual does not generalise to different subjects. In this paper, we propose a method based on Bidirectional Encoder Representation from Transformers (BERT) structure to predict the movement of hands from the root mean square (RMS) feature of the sEMG signal following μ -law normalization. The method was tested for within-subject and cross-subject conditions. We trained the model with two hard sample mining methods, Gradient Harmonizing Mechanism (GHM) and Online Hard Sample Mining (OHEM). The proposed method was compared with classic approaches, including long short-term memory (LSTM) and Temporal Convolutional Network (TCN) as well as a recent method called Long Exposure Convolutional Memory Network (LE-ConvMN). Correlation coefficient (CC), normalized root mean square error (NRMSE) and time costs were used as performance metrics. Our method (sBERT-OHEM) achieved state-of-the-art performance in cross-subject evaluation as well as high performance in subject-specific tests on the Ninapro dataset. The above tests are based on the same randomly selected 10 subjects. Generally, in the cross-subject situation, with the increasing of the subjects' number, it unavoidably leads to the decline of the performance. While the performance of our method on 38 subjects was significantly higher than the other methods on 10 subjects in cross-subject conditions, which further verified the advantage of our methods.

We propose a neuromorphic framework to process the activity of human spinal motor neurons for movement intention recognition. This framework is integrated into a non-invasive interface that decodes the activity of motor neurons innervating intrinsic and extrinsic hand muscles. One of the main limitations of current neural interfaces is that machine learning models cannot exploit the efficiency of the spike encoding operated by the nervous system. Spiking-based pattern recognition would detect the spatio-temporal sparse activity of a neuronal pool and lead to adaptive and compact implementations, eventually running locally in embedded systems. Emergent Spiking Neural Networks (SNN) have not yet been used for processing the activity of in-vivo human neurons. Here we developed a convolutional SNN to process a total of 467 spinal motor neurons whose activity was identified in 5 participants while executing 10 hand movements. The classification accuracy approached 0.95 ±0.14 for both isometric and non-isometric contractions. These results show for the first time the potential of highly accurate motion intent detection by combining non-invasive neural interfaces and SNN.

Ultrasound (US) muscle image series can be used for peripheral human-machine interfacing based on global features, or even on the decomposition of US images into the contributions of individual motor units (MUs). With respect to state-of-the-art surface electromyography (sEMG), US provides higher spatial resolution and deeper penetration depth. However, the accuracy of current methods for direct US decomposition, even at low forces, is relatively poor. These methods are based on linear mathematical models of the contributions of MUs to US images. Here, we test the hypothesis of linearity by comparing the average velocity twitch profiles of MUs when varying the number of other concomitantly active units. We observe that the velocity twitch profile has a decreasing peak-to-peak amplitude when tracking the same target motor unit at progressively increasing contraction force levels, thus with an increasing number of concomitantly active units. This observation indicates non-linear factors in the generation model. Furthermore, we directly studied the impact of one MU on a neighboring MU, finding that the effect of one source on the other is not symmetrical and may be related to unit size. We conclude that a linear approximation is partly limiting the decomposition methods to decompose full velocity twitch trains from velocity images, highlighting the need for more advanced models and methods for US decomposition than those currently employed.

Virtual environments provide a safe and accessible way to test innovative technologies for controlling wearable robotic devices. However, to simulate devices that support walking, such as powered prosthetic legs, it is not enough to model the hardware without its user. Predictive locomotion synthesizers can generate the movements of a virtual user, with whom the simulated device can be trained or evaluated. We implemented a Deep Reinforcement Learning based motion controller in the MuJoCo physics engine, where autonomy over the humanoid model was shared between the simulated user and the control policy of an active prosthesis. Despite not optimising the controller to match experimental dynamics, realistic torque profiles and ground reaction force curves were produced by the agent. A data-driven and continuous representation of user intent was used to simulate a Human Machine Interface that controlled a transtibial prosthesis in a non-steady state walking setting. The continuous intent representation was shown to mitigate the need for compensatory gait patterns from their virtual users and halve the rate of tripping. Co-adaptation was identified as a potential challenge for training human-in-the-loop prosthesis control policies. The proposed framework outlines a way to explore the complex design space of robot-assisted gait, promoting the transfer of the next generation of intent driven controllers from the lab to real-life scenarios.

Objective: We propose a tactile-induced-oscillation approach to reduce the calibration time in somatosensory brain-computer interfaces (BCI)., Methods: Based on the similarity between tactile induced event-related desynchronization (ERD) and imagined sensation induced ERD activation, we extensively evaluated BCI performance when using a conventional and a novel calibration strategy. In the conventional calibration, the tactile imagined data was used, while in the sensory calibration model sensory stimulation data was used. Subjects were required to sense the tactile stimulus when real tactile was applied to the left or right wrist and were required to perform imagined sensation tasks in the somatosensory BCI paradigm., Results: The sensory calibration led to a significantly better performance than the conventional calibration when tested on the same imagined sensation dataset ( [Formula: see text]=10.89, P=0.0038), with an average 5.1% improvement in accuracy. Moreover, the sensory calibration was 39.3% faster in reaching a performance level of above 70% accuracy., Conclusion: The proposed approach of using tactile ERD from the sensory cortex provides an effective way of reducing the calibration time in a somatosensory BCI system., Significance: The tactile stimulation would be specifically useful before BCI usage, avoiding excessive fatigue when the mental task is difficult to perform. The tactile ERD approach may find BCI applications for patients or users with preserved afferent pathways.

A proportion of users cannot achieve adequate brain-computer interface (BCI) control. The diversity of BCI modalities provides a way to solve this emerging issue. Here, we investigate the accuracy of a somatosensory BCI based on sensory imagery (SI). During the SI tasks, subjects were instructed to imagine a tactile sensation and to maintain the attention on the corresponding hand, as if there was tactile stimulus on the skin of the wrist. The performance across 106 healthy subjects in left- and right-hand SI discrimination was 78.9±13.2%. In 70.7% of the subjects the performance was above 70%. The SI task induced a contralateral cortical activation, and high-density EEG source localization showed that the real tactile stimulation and imagined tactile stimulation shared similar cortical activations within the somatosensory cortex. The somatosensory BCI based on SI provides a new signal modality for independent BCI development. Moreover, a combination of SI and other BCI modalities, such as motor imagery, may provide new avenues for further improving BCI usage and applicability, especially in those subjects unable to attain adequate BCI control with conventional BCI modalities.

Transcutaneous electrical stimulation has been applied in tremor suppression applications. Out-of-phase stimulation strategies applied above or below motor threshold result in a significant attenuation of pathological tremor. For stimulation to be properly timed, the varying phase relationship between agonist-antagonist muscle activity during tremor needs to be accurately estimated in real-time. Here we propose an online tremor phase and frequency tracking technique for the customized control of electrical stimulation, based on a phase-locked loop (PLL) system applied to the estimated neural drive to muscles. Surface electromyography signals were recorded from the wrist extensor and flexor muscle groups of 13 essential tremor patients during postural tremor. The EMG signals were pre-processed and decomposed online and offline via the convolution kernel compensation algorithm to discriminate motor unit spike trains. The summation of motor unit spike trains detected for each muscle was bandpass filtered between 3 to 10 Hz to isolate the tremor related components of the neural drive to muscles. The estimated tremorogenic neural drive was used as input to a PLL that tracked the phase differences between the two muscle groups. The online estimated phase difference was compared with the phase calculated offline using a Hilbert Transform as a ground truth. The results showed a rate of agreement of 0.88 ± 0.22 between offline and online EMG decomposition. The PLL tracked the phase difference of tremor signals in real-time with an average correlation of 0.86 ± 0.16 with the ground truth (average error of 6.40° ± 3.49°). Finally, the online decomposition and phase estimation components were integrated with an electrical stimulator and applied in closed-loop on one patient, to representatively demonstrate the working principle of the full tremor suppression system. The results of this study support the feasibility of real-time estimation of the phase of tremorogenic neural drive to muscles, providing a methodology for future tremor-suppression neuroprostheses.

This work is motivated by the recent advances in Deep Neural Networks (DNNs) and their widespread applications in human-machine interfaces. DNNs have been recently used for detecting the intended hand gesture through the processing of surface electromyogram (sEMG) signals. Objective: Although DNNs have shown superior accuracy compared to conventional methods when large amounts of data are available for training, their performance substantially decreases when data are limited. Collecting large datasets for training may be feasible in research laboratories, but it is not a practical approach for real-life applications. The main objective of this work is to design a modern DNN-based gesture detection model that relies on minimal training data while providing high accuracy. Methods: We propose the novel Few-Shot learning- Hand Gesture Recognition (FS-HGR) architecture. Few-shot learning is a variant of domain adaptation with the goal of inferring the required output based on just one or a few training observations. The proposed FS-HGR generalizes after seeing very few observations from each class by combining temporal convolutions with attention mechanisms. This allows the meta-learner to aggregate contextual information from experience and to pinpoint specific pieces of information within its available set of inputs. Data Source & Summary of Results: The performance of FS-HGR was tested on the second and fifth Ninapro databases, referred to as the DB2 and DB5, respectively. The DB2 consists of 50 gestures (rest included) from 40 healthy subjects. The Ninapro DB5 contains data from 10 healthy participants performing a total of 53 different gestures (rest included). The proposed approach for the Ninapro DB2 led to 85.94% classification accuracy on new repetitions with few-shot observation (5-way 5-shot), 81.29% accuracy on new subjects with few-shot observation (5-way 5-shot), and 73.36% accuracy on new gestures with few-shot observation (5-way 5-shot). Moreover, the proposed approach for the Ninapro DB5 led to 64.65% classification accuracy on new subjects with few-shot observation (5-way 5-shot).

A better understanding of neural pain processing and of the development of pain over time, is critical to identify objective measures of pain and to evaluate the effect of pain alleviation therapies. One issue is, that the brain areas known to be related to pain processing are not exclusively responding to painful stimuli, and the neuronal activity is also influenced by other brain areas. Functional connectivity reflects synchrony or covariation of activation between groups of neurons. Previous studies found changes in connectivity days or weeks after pain induction. However, less in known on the temporal development of pain. Our objective was therefore to investigate the interaction between the anterior cingulate cortex (ACC) and primary somatosensory cortex (SI) in the hyperacute (minute) and sustained (hours) response in an animal model of neuropathic pain. Intra-cortical local field potentials (LFP) were recorded in 18 rats. In 10 rats the spared nerve injury model was used as an intervention. The intra-cortical activity was recorded before, immediately after, and three hours after the intervention. The interaction was quantified as the calculated correlation and coherence. The results from the intervention group showed a decrease in correlation between ACC and SI activity, which was most pronounced in the hyperacute phase but a longer time frame may be required for plastic changes to occur. This indicated that both SI and ACC are involved in hyperacute pain processing.

Electromyography (EMG) is a source of neural information for controlling neuroprosthetic devices. To enhance the information content of conventional bipolar EMG, high-density EMG systems include tens to hundreds of closely spaced electrodes that non-invasively record the electrical activity of muscles with high spatial resolution. Despite the advantages of relying on multiple signal sources, however, variations in electrode-skin contact impedance and noise remain challenging for multichannel myocontrol systems. These spatial and temporal non-stationarities negatively impact the control accuracy and therefore substantially limit the clinical viability of high-density EMG techniques. Here, we propose an adaptive algorithm for automatic artefact/noise detection and attenuation for high-density EMG control. The method infers the presence of noise in each EMG channel by spectro-temporal measures of signal similarity. These measures are then used for establishing a scoring system based on an adaptive weighting and reinforcement formulation. The method was experimentally tested as a pre-processing step for a multi-class discrimination problem of 4-digit activation. The approach was proven to enhance the discriminative information content of high-density EMG signals, as well as to attenuate non-stationary artefacts, with improvements in accuracy and robustness of the classification.

Shortcomings in capabilities of below-knee (transtibial) prostheses, compared to their biological counterparts, still cause medical complications and functional deficit to millions of amputees around the world. Although active (powered actuation) transtibial prostheses have the potential to bridge these gaps, the current control solutions limit their efficacy. Here we describe the development of a novel interface for two degrees-of-freedom position and stiffness control for below-knee amputees. The developed algorithm for the interface relies entirely on muscle electrical signals from the lower leg. The algorithm was tested for voluntary position and stiffness control in eight able-bodied and two transtibial amputees and for voluntary stiffness control with foot position estimation while walking in eight able-bodied and one transtibial amputee. The results of the voluntary control experiment demonstrated a promising target reaching success rate, higher for amputees compared to the able-bodied individuals (82.5% and 72.5% compared to 72.5% and 68.1% for the position and position and stiffness matching tasks respectively). Further, the algorithm could provide the means to control four stiffness levels during walking in both amputee and able-bodied individuals while providing estimates of foot kinematics (gait cycle cross-correlation >75% for the sagittal and >90% for the frontal plane and gait cycle root mean square error <7.5° in sagittal and <3° in frontal plane for able-bodied and amputee individuals across three walking speeds). The results from the two experiments demonstrate the feasibility of using this novel algorithm for online control of multiple degrees of freedom and of their stiffness in lower limb prostheses.

Dexterous upper limb myoelectric prostheses can, to some extent, restore the motor functions lost after an amputation. However, ensuring the reliability of myoelectric control is still an open challenge. In this paper, we propose a classification method that exploits the regularity in muscle activation patterns (uniform scaling) across different force levels within a given movement class. This assumption leads to a simple training procedure, using training data collected at single contraction intensity for each movement class. The proposed method was compared to the widely accepted benchmark [linear discriminant analysis (LDA) classifier] using off-line and online evaluation. The off-line classification errors obtained with the new method were either lower or higher than LDA depending upon the chosen feature set. In the online evaluation, the new classification method was operated using amplitude-EMG features and compared to the state-of-the-art LDA classifier combined with the time domain feature set. The online evaluation was performed in 11 able-bodied and one amputee subject using a set of four functional tasks mimicking daily-life activities. The tasks assessed the dexterity (e.g., switching between functions) and robustness of control (e.g., handling heavy objects). With the new classification scheme, the amputee performed better in all functional tasks, whereas the able-bodied subjects performed significantly better in three out of four functional tasks. Overall, the novel method outperformed the state-of-the-art approach (LDA) while utilizing less training data and a smaller feature set. The proposed method is, therefore, a simple but effective and robust classification scheme, convenient for online implementation and clinical use.

In this paper, we investigated the performance of a multi-class brain-computer interface (BCI). The BCI system is based on the concept of somatosensory attentional orientation (SAO), in which the user shifts and maintains somatosensory attention by imagining the sensation of tactile stimulation of a body part. At the beginning of every trial, a vibration stimulus (200 ms) informed the subjects to prepare for the task. Four SAO tasks were performed following randomly presented cues: SAO of the left hand (SAO-LF), SAO of the right hand (SAO-RT), bilateral SAO (SAO-BI), and SAO suppressed or idle state (SAO-ID). Analysis of the event-related desynchronization and synchronization (ERD/ERS) in the EEG indicated that the four SAO tasks had different somatosensory cortical activation patterns. SAO-LF and SAO-RT exhibited stronger contralateral ERD, whereas bilateral ERD activation was indicative of SAO-BI, and bilateral ERS activation was associated with SAO-ID. By selecting the frequency bands and/or optimal classes, classification accuracy of the system reached 85.2%±11.2% for two classes, 69.5%±16.2% for three classes, and 55.9%±15.8% for four classes. The results validated a multi-class BCI system based on SAO, on a single trial basis. Somatosensory attention to different body parts induces diverse oscillatory dynamics within the somatosensory area of the brain, and the proposed SAO paradigm provided a new approach for a multiple-class BCI that is potentially stimulus independent.

This paper evaluated the psychophysical properties of subdermal electrical stimulation to investigate its feasibility in providing sensory feedback for limb prostheses. The detection threshold (DT), pain threshold (PT), just noticeable difference (JND), as well as the elicited sensation quality, comfort, intensity, and location were assessed in 16 healthy volunteers during stimulation of the ventral and dorsal forearm with subdermal electrodes. Moreover, the results were compared with those obtained from transcutaneous electrical stimulation. Despite a lower DT and PT, subdermal stimulation attained a greater relative dynamic range (i.e., PT/DT) and significantly smaller JNDs for stimulation amplitude. Muscle twitches and movements were more commonly elicited by surface stimulation, especially at the higher stimulation frequencies, whereas the pinprick sensation was more often reported with subdermal stimulation. Less comfort was perceived in subdermal stimulation of the ventral forearm at the highest tested stimulation frequency of 100 Hz. In summary, subdermal electrical stimulation was demonstrated to be able to produce similar sensation quality as transcutaneous stimulation and outperformed the latter in terms of energy efficiency and sensitivity. These results suggest that stimulation through implantable subdermal electrodes may lead to an efficient and compact sensory feedback system for substituting the lost sense in amputees.

We proposed a multi-class tactile brain-computer interface that utilizes stimulus-induced oscillatory dynamics. It was hypothesized that somatosensory attention can modulate tactile-induced oscillation changes, which can decode different sensation attention tasks. Subjects performed four tactile attention tasks, prompted by cues presented in random order and while both wrists were simultaneously stimulated: 1) selective sensation on left hand (SS-L); 2) selective sensation on right hand (SS-R); 3) bilateral selective sensation; and 4) selective sensation suppressed or idle state (SS-S). The classification accuracy between SS-L and SS-R (79.9 ± 8.7%) was comparable with that of a previous tactile BCI system based on selective sensation. Moreover, the accuracy could be improved to an average of 90.3 ± 4.9% by optimal class-pair and frequency-band selection. Three-class discrimination had an accuracy of 75.2 ± 8.3%, with the best discrimination reached for the classes SS-L, SS-R, and SS-S. Finally, four classes were classified with an accuracy of 59.4 ± 7.3%. These results show that the proposed system is a promising new paradigm for multi-class BCI.

A large proportion of users do not achieve adequate control using current non-invasive brain-computer interfaces (BCIs). This issue has being coined "BCI-Illiteracy" and is observed among different BCI modalities. Here, we compare the performance and the BCI-illiteracy rate of a tactile selective sensation (SS) and motor imagery (MI) BCI, for a large subject samples. We analyzed 80 experimental sessions from 57 subjects with two-class SS protocols. For SS, the group average performance was 79.8 ± 10.6%, with 43 out of the 57 subjects (75.4%) exceeding the 70% BCI-illiteracy threshold for left- and right-hand SS discrimination. When compared with previous results, this tactile BCI outperformed all other tactile BCIs currently available. We also analyzed 63 experimental sessions from 43 subjects with two-class MI BCI protocols, where the group average performance was 77.2 ± 13.3%, with 69.7% of the subjects exceeding the 70% performance threshold for left- and right-hand MI. For within-subject comparison, the 24 subjects who participated to both the SS and MI experiments, the BCI performance was superior with SS than MI especially in beta frequency band (p < 0.05), with enhanced R 2 discriminative information in the somatosensory cortex for the SS modality. Both SS and MI showed a functional dissociation between lower alpha ([8 10] Hz) and upper alpha ([10 13] Hz) bands, with BCI performance significantly better in the upper alpha than the lower alpha (p < 0.05) band. In summary, we demonstrated that SS is a promising BCI modality with low BCI illiteracy issue and has great potential in practical applications reaching large population.

We prove the feasibility of decomposing high density surface EMG signals from forearm muscles in non-isometric wrist motor tasks of normally limbed and limb-deficient individuals with the perspective of using the decoded neural information for prosthesis control. For this purpose, we recorded surface EMG signals during motions of three degrees of freedom of the wrist in seven normally limbed subjects and two patients with limb deficiency. The signals were decomposed into individual motor unit activity with a convolutive blind source separation algorithm. On average, for each subject, 16 ± 7 motor units were identified per motor task. The discharge timings of these motor units were estimated with an accuracy > 85%. Moreover, the activity of 6 ± 5 motor units per motor task was consistently detected in all repetitions of the same task. The joint angle at which motor units were first identified was 62.5 ± 26.4% of the range of motion, indicating a prevalence in the identification of high threshold motor units. These findings prove the feasibility of accurate identification of the neural drive to muscles in contractions relevant for myoelectric control, allowing the development of a new generation of myocontrol methods based on motor unit spike trains.

Human motor control relies on a combination of feedback and feedforward strategies. The aim of this study was to longitudinally investigate artificial somatosensory feedback and feedforward control in the context of grasping with myoelectric prosthesis. Nine amputee subjects performed routine grasping trials, with the aim to produce four levels of force during four blocks of 60 trials across five days. The electrotactile force feedback was provided in the second and third block using multipad electrode and spatial coding. The first baseline and last validation block (open-loop control) evaluated the effects of long- (across sessions) and short-term (within session) learning, respectively. The outcome measures were the absolute error between the generated and target force, and the number of force saturations. The results demonstrated that the electrotactile feedback improved the performance both within and across sessions. In the validation block, the performance did not significantly decrease and the quality of open-loop control (baseline) improved across days, converging to the performance characterizing closed-loop control. This paper provides important insights into the feedback and feedforward processes in prosthesis control, contributing to the better understanding of the role and design of feedback in prosthetic systems.

We propose a novel decomposition method for electromyographic signals based on blind source separation. Using the cyclostationary properties of motor unit action potential trains (MUAPt), it is shown that the MUAPt can be decomposed by joint diagonalization of the cyclic spatial correlation matrix of the observations. After modeling the source signals, we provide the proof of orthogonality of the sources and of their delayed versions in a cyclostationary context. We tested the proposed method on simulated signals and showed that it can decompose up to six sources with a probability of correct detection and classification >95%, using only eight recording sites. Moreover, we tested the method on experimental multi-channel signals recorded with thin-film intramuscular electrodes, with a total of 32 recording sites. The rate of agreement of the decomposed MUAPt with those obtained by an expert using a validated tool for decomposition was >93%.

Distinctive EEG signals from the motor and somatosensory cortex are generated during mental tasks of motor imagery (MI) and somatosensory attentional orientation (SAO). In this paper, we hypothesize that a combination of these two signal modalities provides improvements in a brain-computer interface (BCI) performance with respect to using the two methods separately, and generate novel types of multi-class BCI systems. Thirty two subjects were randomly divided into a Control-Group and a Hybrid-Group. In the Control-Group, the subjects performed left and right hand motor imagery (i.e., L-MI and R-MI). In the Hybrid-Group, the subjects performed the four mental tasks (i.e., L-MI, R-MI, L-SAO, and R-SAO). The results indicate that combining two of the tasks in a hybrid manner (such as L-SAO and R-MI) resulted in a significantly greater classification accuracy than when using two MI tasks. The hybrid modality reached 86.1% classification accuracy on average, with a 7.70% increase with respect to MI ( ), and 7.21% to SAO ( ) alone. Moreover, all 16 subjects in the hybrid modality reached at least 70% accuracy, which is considered the threshold for BCI illiteracy. In addition to the two-class results, the classification accuracy was 68.1% and 54.1% for the three-class and four-class hybrid BCI. Combining the induced brain signals from motor and somatosensory cortex, the proposed stimulus-independent hybrid BCI has shown improved performance with respect to individual modalities, reducing the portion of BCI-illiterate subjects, and provided novel types of multi-class BCIs.

Providing somatosensory feedback to the user of a myoelectric prosthesis is an important goal since it can improve the utility as well as facilitate the embodiment of the assistive system. Most often, the grasping force was selected as the feedback variable and communicated through one or more individual single channel stimulation units (e.g., electrodes, vibration motors). In the present study, an integrated, compact, multichannel solution comprising an array electrode and a programmable stimulator was presented. Two coding schemes (15 levels), spatial and mixed (spatial and frequency) modulation, were tested in able-bodied subjects, psychometrically and in force control with routine grasping and force tracking using real and simulated prosthesis. The results demonstrated that mixed and spatial coding, although substantially different in psychometric tests, resulted in a similar performance during both force control tasks. Furthermore, the ideal, visual feedback was not better than the tactile feedback in routine grasping. To explain the observed results, a conceptual model was proposed emphasizing that the performance depends on multiple factors, including feedback uncertainty, nature of the task and the reliability of the feedforward control. The study outcomes, specific conclusions and the general model, are relevant for the design of closed-loop myoelectric prostheses utilizing tactile feedback.

We propose and test a novel brain-computer interface (BCI) based on imagined tactile sensation. During an imagined tactile sensation, referred to as somatosensory attentional orientation (SAO), the subject shifts and maintains somatosensory attention on a body part, e.g., left or right hand. The SAO can be detected from EEG recordings for establishing a communication channel. To test for the hypothesis that SAO on different body parts can be discriminated from EEG, 14 subjects were assigned to a group who received an actual sensory stimulation (STE-Group), and 18 subjects were assigned to the SAO only group (SAO-Group). In single trials, the STE-Group received tactile stimulation first (both wrists simultaneously stimulated), and then maintained the attention on the selected body part (without stimulation). The same group also performed the SAO task first and then received the tactile stimulation. Conversely, the SAO-Group performed SAO without any stimulation, neither before nor after the SAO. In both the STE-Group and SAO-Group, it was possible to identify the SAO-related oscillatory activation that corresponded to a contralateral event-related desynchronization (ERD) stronger than the ipsilateral ERD. Discriminative information, represented as R 2 , was found mainly on the somatosensory area of the cortex. In the STE-Group, the average classification accuracy of SAO was 83.6%, and it was comparable with tactile BCI based on selective sensation (paired-t test, P > 0.05 ). In the SAO-Group the average online performance was 75.7%. For this group, after frequency band selection the offline performance reached 82.5% on average, with ≥ 80% for 12 subjects and ≥ 95% for four subjects. Complementary to tactile sensation, the SAO does not require sensory stimulation, with the advantage of being completely independent from the stimulus.

The amplitude of the surface EMG (sEMG) is commonly estimated by rectification or other nonlinear transformations, followed by smoothing (low-pass linear filtering). Although computationally efficient, this approach leads to an estimation accuracy with a limited theoretical signal-to-noise ratio (SNR). Since sEMG amplitude is one of the most relevant features for myoelectric control, its estimate has become one of the limiting factors for the performance of myoelectric control applications, such as powered prostheses. In this study, we present a recursive nonlinear estimator of sEMG amplitude based on Bayesian filtering. Furthermore, we validate the advantage of the proposed Bayesian filter over the conventional linear filters through an online simultaneous and proportional control (SPC) task, performed by eight able-bodied subjects and three below-elbow limb deficient subjects. The results demonstrated that the proposed Bayesian filter provides significantly more accurate SPC, particularly for the patients, when compared with conventional linear filters. This result presents a major step toward accurate prosthetic control for advanced multi-function prostheses.

Fundamental changes over time of surface EMG signal characteristics are a challenge for myocontrol algorithms controlling prosthetic devices. These changes are generally caused by electrode shifts after donning and doffing, sweating, additional weight or varying arm positions, which results in a change of the signal distribution-a scenario often referred to as covariate shift. A substantial decrease in classification accuracy due to these factors hinders the possibility to directly translate EMG signals into accurate myoelectric control patterns outside laboratory conditions. To overcome this limitation, we propose the use of supervised adaptation methods. The approach is based on adapting a trained classifier using a small calibration set only, which incorporates the relevant aspects of the nonstationarities, but requires only less than 1 min of data recording. The method was tested first through an offline analysis on signals acquired across 5 days from seven able-bodied individuals and four amputees. Moreover, we also conducted a three day online experiment on eight able-bodied individuals and one amputee, assessing user performance and user-ratings of the controllability. Across different testing days, both offline and online performance improved significantly when shrinking the training model parameters by a given estimator towards the calibration set parameters. In the offline data analysis, the classification accuracy remained above 92% over five days with the proposed approach, whereas it decreased to 75% without adaptation. Similarly, in the online study, with the proposed approach the performance increased by 25% compared to a test without adaptation. These results indicate that the proposed methodology can contribute to improve robustness of myoelectric pattern recognition methods in daily life applications.

The detection of voluntary motor intention from EEG has been applied to closed-loop brain-computer interfacing (BCI). The movement-related cortical potential (MRCP) is a low frequency component of the EEG signal, which represents movement intention, preparation, and execution. In this study, we aim at detecting MRCPs from single-trial EEG traces. For this purpose, we propose a detector based on a discriminant manifold learning method, called locality sensitive discriminant analysis (LSDA), and we test it in both online and offline experiments with executed and imagined movements. The online and offline experimental results demonstrated that the proposed LSDA approach for MRCP detection outperformed the Locality Preserving Projection (LPP) approach, which was previously shown to be the most accurate algorithm so far tested for MRCP detection. For example, in the online tests, the performance of LSDA was superior than LPP in terms of a significant reduction in false positives (FP) (passive FP: 1.6 ±0.9/min versus 2.9 ±1.0/min, p = 0.002, active FP: 2.2 ±0.8/min versus 2.7 ±0.6/min , p = 0.03 ), for a similar rate of true positives. In conclusion, the proposed LSDA based MRCP detection method is superior to previous approaches and is promising for developing patient-driven BCI systems for motor function rehabilitation as well as for neuroscience research.

In this study, we present a novel multi-class brain-computer interface (BCI) for communication and control. In this system, the information processing is shared by the algorithm (computer) and the user (human). Specifically, an electro-tactile cycle was presented to the user, providing the choice (class) by delivering timely sensory input. The user discriminated these choices by his/her endogenous sensory ability and selected the desired choice with an intuitive motor task. This selection was detected by a fast brain switch based on real-time detection of movement-related cortical potentials from scalp EEG. We demonstrated the feasibility of such a system with a four-class BCI, yielding a true positive rate of  ∼ 80% and  ∼ 70%, and an information transfer rate of  ∼ 7 bits/min and  ∼ 5 bits/min, for the movement and imagination selection command, respectively. Furthermore, when the system was extended to eight classes, the throughput of the system was improved, demonstrating the capability of accommodating a large number of classes. Combining the endogenous sensory discrimination with the fast brain switch, the proposed system could be an effective, multi-class, gaze-independent BCI system for communication and control applications.

Pattern recognition and regression methods applied to the surface EMG have been used for estimating the user intended motor tasks across multiple degrees of freedom (DOF), for prosthetic control. While these methods are effective in several conditions, they are still characterized by some shortcomings. In this study we propose a methodology that combines these two approaches for mutually alleviating their limitations. This resulted in a control method capable of context-dependent movement estimation that switched automatically between sequential (one DOF at a time) or simultaneous (multiple DOF) prosthesis control, based on an online estimation of signal dimensionality. The proposed method was evaluated in scenarios close to real-life situations, with the control of a physical prosthesis in applied tasks of varying difficulties. Test prostheses were individually manufactured for both able-bodied and transradial amputee subjects. With these prostheses, two amputees performed the Southampton Hand Assessment Procedure test with scores of 58 and 71 points. The five able-bodied individuals performed standardized tests, such as the box&block and clothes pin test, reducing the completion times by up to 30%, with respect to using a state-of-the-art pure sequential control algorithm. Apart from facilitating fast simultaneous movements, the proposed control scheme was also more intuitive to use, since human movements are predominated by simultaneous activations across joints. The proposed method thus represents a significant step towards intelligent, intuitive and natural control of upper limb prostheses.

Myoelectric control offers a direct interface between human intent and various robotic applications through recorded muscle activity. Traditional control schemes realize this interface through direct mapping or pattern recognition techniques. The former approach provides reliable control at the expense of functionality, while the latter increases functionality at the expense of long-term reliability. An alternative approach, using concepts of motor learning, provides session-independent simultaneous control, but previously relied on consistent electrode placement over biomechanically independent muscles. This paper extends the functionality and practicality of the motor learning-based approach, using high-density electrode grids and muscle synergy-inspired decomposition to generate control inputs with reduced constraints on electrode placement. The method is demonstrated via real-time simultaneous and proportional control of a 4-DoF myoelectric interface over multiple days. Subjects showed learning trends consistent with typical motor skill learning without requiring any retraining or recalibration between sessions. Moreover, they adjusted to physical constraints of a robot arm after learning the control in a constraint-free virtual interface, demonstrating robust control as they performed precision tasks. The results demonstrate the efficacy of the proposed man-machine interface as a viable alternative to conventional control schemes for myoelectric interfaces designed for long-term use.

Electrocutaneous stimulation is a promising approach to provide sensory feedback to amputees, and thus close the loop in upper limb prosthetic systems. However, the stimulation introduces artifacts in the recorded electromyographic (EMG) signals, which may be detrimental for the control of myoelectric prostheses. In this study, artifact blanking with three data segmentation approaches was investigated as a simple method to restore the performance of pattern recognition in prosthesis control (eight motions) when EMG signals are corrupted by stimulation artifacts. The methods were tested over a range of stimulation conditions and using four feature sets, comprising both time and frequency domain features. The results demonstrated that when stimulation artifacts were present, the classification performance improved with blanking in all tested conditions. In some cases, the classification performance with blanking was at the level of the benchmark (artifact-free data). The greatest pulse duration and frequency that allowed a full performance recovery were 400 μs and 150 Hz, respectively. These results show that artifact blanking can be used as a practical solution to eliminate the negative influence of the stimulation artifact on EMG pattern classification in a broad range of conditions, thus allowing to close the loop in myoelectric prostheses using electrotactile feedback.

Functional replacement of upper limbs by means of dexterous prosthetic devices remains a technological challenge. While the mechanical design of prosthetic hands has advanced rapidly, the human-machine interfacing and the control strategies needed for the activation of multiple degrees of freedom are not reliable enough for restoring hand function successfully. Machine learning methods capable of inferring the user intent from EMG signals generated by the activation of the remnant muscles are regarded as a promising solution to this problem. However, the lack of robustness of the current methods impedes their routine clinical application. In this study, we propose a novel algorithm for controlling multiple degrees of freedom sequentially, inherently proportionally and with high robustness, allowing a good level of prosthetic hand function. The control algorithm is based on the spatial linear combinations of amplitude-related EMG signal features. The weighting coefficients in this combination are derived from the optimization criterion of the common spatial patterns filters which allow for maximal discriminability between movements. An important component of the study is the validation of the method which was performed on both able-bodied and amputee subjects who used physical prostheses with customized sockets and performed three standardized functional tests mimicking daily-life activities of varying difficulty. Moreover, the new method was compared in the same conditions with one clinical/industrial and one academic state-of-the-art method. The novel algorithm outperformed significantly the state-of-the-art techniques in both subject groups for tests that required the activation of more than one degree of freedom. Because of the evaluation in real time control on both able-bodied subjects and final users (amputees) wearing physical prostheses, the results obtained allow for the direct extrapolation of the benefits of the proposed method for the end users. In conclusion, the method proposed and validated in real-life use scenarios, allows the practical usability of multifunctional hand prostheses in an intuitive way, with significant advantages with respect to previous systems.

Tremor is one of the most prevalent movement disorders. There is a large proportion of patients (around 25%) in whom current treatments do not attain a significant tremor reduction. This paper proposes a tremor suppression strategy that detects tremor from the electromyographic signals of the muscles from which tremor originates and counteracts it by delivering electrical stimulation to the antagonist muscles in an out of phase manner. The detection was based on the iterative Hilbert transform and stimulation was delivered above the motor threshold (motor stimulation) and below the motor threshold (sensory stimulation). The system was tested on six patients with predominant wrist flexion/extension tremor (four with Parkinson disease and two with Essential tremor) and led to an average tremor reduction in the range of 46%-81% and 35%-48% across five patients when using the motor and sensory stimulation, respectively. In one patient, the system did not attenuate tremor. These results demonstrate that tremor attenuation might be achieved by delivering electrical stimulation below the motor threshold, preventing muscle fatigue and discomfort for the patients, which sets the basis for the development of an alternative treatment for tremor.

Reach-to-grasp tasks have become popular paradigms for exploring the neural origin of hand and arm movements. This is typically investigated by correlating limb kinematic with electrophysiological signals from intracortical recordings. However, it has never been investigated whether reach and grasp movements could be well expressed in the muscle domain and whether this could bring improvements with respect to current joint domain-based task representations. In this study, we trained two macaque monkeys to grasp 50 different objects, which resulted in a high variability of hand configurations. A generic musculoskeletal model of the human upper extremity was scaled and morphed to match the specific anatomy of each individual animal. The primate-specific model was used to perform 3-D reach-to-grasp simulations driven by experimental upper limb kinematics derived from electromagnetic sensors. Simulations enabled extracting joint angles from 27 degrees of freedom and the instantaneous length of 50 musculotendon units. Results demonstrated both a more compact representation and a higher decoding capacity of grasping tasks when movements were expressed in the muscle kinematics domain than when expressed in the joint kinematics domain. Accessing musculoskeletal variables might improve our understanding of cortical hand-grasping areas coding, with implications in the development of prosthetics hands.

Research on pattern recognition for myoelectric control has usually focused on a small number of electromyography (EMG) channels because of better clinical acceptability and low computational load with respect to multi-channel EMG. However, recently, high density (HD) EMG technology has substantially improved, also in practical usability, and can thus be applied in myocontrol. HD EMG provides several closely spaced recordings in multiple locations over the skin surface. This study considered the use of HD EMG for controlling upper limb prostheses, based on pattern recognition. In general, robustness and reliability of classical pattern recognition systems are influenced by electrode shift in dons and doff, and by the presence of malfunctioning channels. The aim of this study is to propose a new approach to attenuate these issues. The HD EMG grid of electrodes is an ensemble of sensors that records data spatially correlated. The experimental variogram, which is a measure of the degree of spatial correlation, was used as feature for classification, contrary to previous approaches that are based on temporal or frequency features. The classification based on the variogram was tested on seven able-bodied subjects and one subject with amputation, for the classification of nine and seven classes, respectively. The performance of the proposed approach was comparable with the classic methods based on time-domain and autoregressive features (average classification accuracy over all methods ∼ 95% for nine classes). However, the new spatial features demonstrated lower sensitivity to electrode shift ( ± 1 cm) with respect to the classic features . When even just one channel was noisy, the classification accuracy dropped by ∼ 10% for all methods. However, the new method could be applied without any retraining to a subset of high-quality channels whereas the classic methods require retraining when some channels are omitted. In conclusion, the new spatial feature space proposed in this study improved the robustness to electrode number and shift in myocontrol with respect to previous approaches.

Closing the control loop by providing sensory feedback to the user of a prosthesis is an important challenge, with major impact on the future of prosthetics. Developing and comparing closed-loop systems is a difficult task, since there are many different methods and technologies that can be used to implement each component of the system. Here, we present a test bench developed in Matlab Simulink for configuring and testing the closed-loop human control system in standardized settings. The framework comprises a set of connected generic blocks with normalized inputs and outputs, which can be customized by selecting specific implementations from a library of predefined components. The framework is modular and extensible and it can be used to configure, compare and test different closed-loop system prototypes, thereby guiding the development towards an optimal system configuration. The use of the test bench was demonstrated by investigating two important aspects of closed-loop control: performance of different electrotactile feedback interfaces (spatial versus intensity coding) during a pendulum stabilization task and feedforward methods (joystick versus myocontrol) for force control. The first experiment demonstrated that in the case of trained subjects the intensity coding might be superior to spatial coding. In the second experiment, the control of force was rather poor even with a stable and precise control interface (joystick), demonstrating that inherent characteristics of the prosthesis can be an important limiting factor when considering the overall effectiveness of the closed-loop control. The presented test bench is an important instrument for investigating different aspects of human manual control with sensory feedback.

In closed-loop control of grasping by hand prostheses, the feedback information sent to the user is usually the actual controlled variable, i.e., the grasp force. Although this choice is intuitive and logical, the force production is only the last step in the process of grasping. Therefore, this study evaluated the performance in controlling grasp strength using a hand prosthesis operated through a complete grasping sequence while varying the feedback variables (e.g., closing velocity, grasping force), which were provided to the user visually or through vibrotactile stimulation. The experiments were conducted on 13 volunteers who controlled the Otto Bock Sensor Hand Speed prosthesis. Results showed that vibrotactile patterns were able to replace the visual feedback. Interestingly, the experiments demonstrated that direct force feedback was not essential for the control of grasping force. The subjects were indeed able to control the grip strength, predictively, by estimating the grasping force from the prosthesis velocity of closing. Therefore, grasping without explicit force feedback is not completely blind, contrary to what is usually assumed. In our study we analyzed grasping with a specific prosthetic device, but the outcomes are also applicable for other devices, with one or more degrees-of-freedom. The necessary condition is that the electromyography (EMG) signal directly and proportionally controls the velocity/grasp force of the hand, which is a common approach among EMG controlled prosthetic devices. The results provide important indications on the design of closed-loop EMG controlled prosthetic systems.

This paper addresses the sequential decoding of intramuscular single-channel electromyographic (EMG) signals to extract the activity of individual motor neurons. A hidden Markov model is derived from the physiological generation of the EMG signal. The EMG signal is described as a sum of several action potentials (wavelet) trains, embedded in noise. For each train, the time interval between wavelets is modeled by a process that parameters are linked to the muscular activity. The parameters of this process are estimated sequentially by a Bayes filter, along with the firing instants. The method was tested on some simulated signals and an experimental one, from which the rates of detection and classification of action potentials were above 95% with respect to the reference decomposition. The method works sequentially in time, and is the first to address the problem of intramuscular EMG decomposition online. It has potential applications for man-machine interfacing based on motor neuron activities.

Despite not recording directly from neural cells, the surface electromyogram (EMG) signal contains information on the neural drive to muscles, i.e., the spike trains of motor neurons. Using this property, myoelectric control consists of the recording of EMG signals for extracting control signals to command external devices, such as hand prostheses. In commercial control systems, the intensity of muscle activity is extracted from the EMG and used for single degrees of freedom activation (direct control). Over the past 60 years, academic research has progressed to more sophisticated approaches but, surprisingly, none of these academic achievements has been implemented in commercial systems so far. We provide an overview of both commercial and academic myoelectric control systems and we analyze their performance with respect to the characteristics of the ideal myocontroller. Classic and relatively novel academic methods are described, including techniques for simultaneous and proportional control of multiple degrees of freedom and the use of individual motor neuron spike trains for direct control. The conclusion is that the gap between industry and academia is due to the relatively small functional improvement in daily situations that academic systems offer, despite the promising laboratory results, at the expense of a substantial reduction in robustness. None of the systems so far proposed in the literature fulfills all the important criteria needed for widespread acceptance by the patients, i.e. intuitive, closed-loop, adaptive, and robust real-time ( 200 ms delay) control, minimal number of recording electrodes with low sensitivity to repositioning, minimal training, limited complexity and low consumption. Nonetheless, in recent years, important efforts have been invested in matching these criteria, with relevant steps forwards.

Targeted muscle reinnervation (TMR) redirects nerves that have lost their target, due to amputation, to remaining muscles in the region of the stump with the intent of establishing intuitive myosignals to control a complex prosthetic device. In order to directly recover the neural code underlying an attempted limb movement, in this paper, we present the decomposition of high-density surface electromyographic (EMG) signals detected from three TMR patients into the individual motor unit spike trains. The aim was to prove, for the first time, the feasibility of decoding the neural drive that would reach muscles of the missing limb in TMR patients, to show the accuracy of the decoding, and to demonstrate the representativeness of the pool of extracted motor units. Six to seven flexible EMG electrode grids of 64 electrodes each were mounted over the reinnervated muscles of each patient, resulting in up to 448 EMG signals. The subjects were asked to attempt elbow extension and flexion, hand open and close, wrist extension and flexion, wrist pronation and supination, of their missing limb. The EMG signals were decomposed using the Convolution Kernel Compensation technique and the decomposition accuracy was evaluated with a signal-based index of accuracy, called pulse-to-noise ratio (PNR). The results showed that the spike trains of 3 to 27 motor units could be identified for each task, with a sensitivity of the decomposition > 90%, as revealed by PNR. The motor unit discharge rates were within physiological values of normally innervated muscles. Moreover, the detected motor units showed a high degree of common drive so that the set of extracted units per task was representative of the behavior of the population of active units. The results open a path for a new generation of human-machine interfaces in which the control signals are extracted from noninvasive recordings and the obtained neural information is based directly on the spike trains of motor neurons.

Previous research proposed the extraction of myoelectric control signals by linear factorization of multi-channel electromyogram (EMG) recordings from forearm muscles. This paper further analyses the theoretical basis for dimensionality reduction in high-density EMG signals from forearm muscles. Moreover, it shows that the factorization of muscular activation patterns in weights and activation signals by non-negative matrix factorization (NMF) is robust with respect to the channel configuration from where the EMG signals are obtained. High-density surface EMG signals were recorded from the forearm muscles of six individuals. Weights and activation signals extracted offline from 10 channel configurations with varying channel numbers (6, 8, 16, 192 channels) were highly similar. Additionally, the method proved to be robust against electrode shifts in both transversal and longitudinal direction with respect to the muscle fibers. In a second experiment, six subjects directly used the activation signals extracted from high-density EMG for online goal-directed control tasks involving simultaneous and proportional control of two degrees-of-freedom of the wrist. The synergy weights for this control task were extracted from a reference configuration and activation signals were calculated online from the reference configuration as well as from the two shifted configurations, simulating electrode shift. Despite the electrode shift, the task completion rate, task completion time, and execution efficiency were generally not statistically different among electrode configurations. Online performances were also mostly similar when using either 6, 8, or 16 EMG channels. The robustness of the method to the number and location of channels, proved both offline and online, indicates that EMG signals recorded from forearm muscles can be approximated as linear instantaneous mixtures of activation signals and justifies the use of linear factorization algorithms for extracting, in a minimally supervised way, control signals for simultaneous multi-degree of freedom prosthesis control.

We propose an approach for online simultaneous and proportional myoelectric control of two degrees-of-freedom (DoF) of the wrist, using surface electromyographic signals. The method is based on the nonnegative matrix factorization (NMF) of the wrist muscle activation to extract low-dimensional control signals translated by the user into kinematic variables. This procedure does not need a training set of signals for which the kinematics is known (labeled dataset) and is thus unsupervised (although it requires an initial calibration without labeled signals). The estimated control signals using NMF are used to directly control two DoFs of wrist. The method was tested on seven subjects with upper limb deficiency and on seven able-bodied subjects. The subjects performed online control of a virtual object with two DoFs to achieve goal-oriented tasks. The performance of the two subject groups, measured as the task completion rate, task completion time, and execution efficiency, was not statistically different. The approach was compared, and demonstrated to be superior to the online control by the industrial state-of-the-art approach. These results show that this new approach, which has several advantages over the previous myoelectric prosthetic control systems, has the potential of providing intuitive and dexterous control of artificial limbs for amputees.

In this paper, we present a systematic analysis of the relationship between the accuracy of the mapping between EMG and hand kinematics and the control performance in goal-oriented tasks of three simultaneous and proportional myoelectric control algorithms: nonnegative matrix factorization (NMF), linear regression (LR), and artificial neural networks (ANN). The purpose was to investigate the impact of the precision of the kinematics estimation by a myoelectric controller for accurately complete goal-directed tasks. Nine naïve subjects performed a series of goal-directed myoelectric control tasks using the three algorithms, and their online performance was characterized by 6 indexes. The results showed that, although the three algorithms' mapping accuracies were significantly different, their online performance was similar. Moreover, for LR and ANN, the offline performance was not correlated to any of the online performance indexes, and only a weak correlation was found with three of them for NMF . We conclude that for reliable simultaneous and proportional myoelectric control, it is not necessary to achieve high accuracy in the mapping between EMG and kinematics. Rather, good online myoelectric control is achieved by the continuous interaction and adaptation of the user with the myoelectric controller through feedback (visual in the current study). Control signals generated by EMG with rather poor association with kinematic variables can still be fully exploited by the user for precise control. This conclusion explains the possibility of accurate simultaneous and proportional control over multiple degrees of freedom when using unsupervised algorithms, such as NMF.