Search

The Eighth International Brain-Computer Interface (BCI) Meeting was held June 7–9th, 2021 in a virtual format. The conference continued the BCI Meeting series’ interactive nature with 21 workshops covering topics in BCI (also called brain-machine interface) research. As in the past, workshops covered the breadth of topics in BCI. Some workshops provided detailed examinations of specific methods, hardware, or processes. Others focused on specific BCI applications or user groups. Several workshops continued consensus building efforts designed to create BCI standards and increase the ease of comparisons between studies and the potential for meta-analysis and large multi-site clinical trials. Ethical and translational considerations were both the primary topic for some workshops or an important secondary consideration for others. The range of BCI applications continues to expand, with more workshops focusing on approaches that can extend beyond the needs of those with physical impairments. This paper summarizes each workshop, provides background information and references for further study, presents an overview of the discussion topics, and describes the conclusion, challenges, or initiatives that resulted from the interactions and discussion at the workshop.

Brain–computer interface (BCI) systems may require the user to perform a set of mental tasks, such as imagining different types of motion. The performance demonstrated on these tasks varies...

Quantifying and evaluating properly the performances is a critical issue in BCI experiments. The choice of the most adapted metrics can be difficult because they are specific to the experimental paradigm, task control, and data. In the current study evaluation criteria for closed-loop adaptive dynamic and hierarchical discrete-continuous brain-computer interfaces are examined. The challenges such as imbalanced multi-class bias for discrete decoding (classification), online computing cost and dynamic analysis of results in time are considered. There are two main objectives of the study: identifying the best suited performances metrics according to the requirements and combining several levels of evaluation in the whole BCI system (decoder and patient oriented). The main novelty of the study is the combination of several common metrics with new temporal metrics and a time dynamic approach which allows adequately reflect the performance of adaptive dynamic and hierarchical discrete-continuous brain-computer interfaces BCI systems. Additional response time and blocking error patterns reveal complementary information for BCI system performance evaluation. Developed criteria are applied to performance evaluation of 8-dimensional exoskeleton control by tetraplegic patient for both decoder and user-centered metrics.

Brain source imaging and time frequency mapping (TFM) are commonly used in magneto/electro encephalography (M/EEG) imaging. However, these methods suffer from important limitations. Source imaging is based on an ill-posed inverse problem leading to instability of source localization solutions, has a limited capacity to localize high frequency oscillations and loses its robustness for induced responses (ill-defined trigger). The drawback of TFM is that it involves independent analysis of signals from a number of frequency bands, and from co-localized sensors. In the present article, a regression-based multi-sensor space&ndash, time&ndash, frequency analysis (MSA) approach, which integrates co-localized sensors and/or multi-frequency information, is proposed. To estimate task-specific brain activations, MSA uses cross-validated, shifted, multiple Pearson correlation, calculated from the time&ndash, frequency transformed brain signal and the binary signal of stimuli. The results are projected from the sensor space onto the cortical surface. To assess MSA performance, the proposed method was compared to the weighted minimum norm estimate (wMNE) source imaging method, in terms of spatial selectivity and robustness against an ill-defined trigger. Magnetoencephalography (MEG) recordings were performed in fourteen subjects during two motor tasks: finger tapping and elbow flexion/extension. In particular, our results show that the MSA approach provides good localization performance when compared to wMNE and statistically significant improvement of robustness against ill-defined trigger.

Objective.The evaluation of the long-term stability of ElectroCorticoGram (ECoG) signals is an important scientific question as new implantable recording devices can be used for medical purposes such as Brain-Computer Interface (BCI) or brain monitoring.Approach.The long-term clinical validation of wireless implantable multi-channel acquisition system for generic interface with neurons (WIMAGINE), a wireless 64-channel epidural ECoG recorder was investigated. The WIMAGINE device was implanted in two quadriplegic patients within the context of a BCI protocol. This study focused on the ECoG signal stability in two patients bilaterally implanted in June 2017 (P1) and in November 2019 (P2).Methods. The ECoG signal was recorded at rest prior to each BCI session resulting in a 32 month and in a 14 month follow-up for P1 and P2 respectively. State-of-the-art signal evaluation metrics such as root mean square (RMS), the band power (BP), the signal to noise ratio (SNR), the effective bandwidth (EBW) and the spectral edge frequency (SEF) were used to evaluate stability of signal over the implantation time course. The time-frequency maps obtained from task-related motor activations were also studied to investigate the long-term selectivity of the electrodes.Mainresults.Based on temporal linear regressions, we report a limited decrease of the signal average level (RMS), spectral distribution (BP) and SNR, and a remarkable steadiness of the EBW and SEF. Time-frequency maps obtained during motor imagery, showed a high level of discrimination 1 month after surgery and also after 2 years.Conclusions.The WIMAGINE epidural device showed high stability over time. The signal evaluation metrics of two quadriplegic patients during 32 months and 14 months respectively provide strong evidence that this wireless implant is well-suited for long-term ECoG recording.Significance.These findings are relevant for the future of implantable BCIs, and could benefit other patients with spinal cord injury, amyotrophic lateral sclerosis, neuromuscular diseases or drug-resistant epilepsy.

Summary Background Approximately 20% of traumatic cervical spinal cord injuries result in tetraplegia. Neuroprosthetics are being developed to manage this condition and thus improve the lives of patients. We aimed to test the feasibility of a semi-invasive technique that uses brain signals to drive an exoskeleton. Methods We recruited two participants at Clinatec research centre, associated with Grenoble University Hospital, Grenoble, France, into our ongoing clinical trial. Inclusion criteria were age 18–45 years, stability of neurological deficits, a need for additional mobility expressed by the patient, ambulatory or hospitalised monitoring, registration in the French social security system, and signed informed consent. The exclusion criteria were previous brain surgery, anticoagulant treatments, neuropsychological sequelae, depression, substance dependence or misuse, and contraindications to magnetoencephalography (MEG), EEG, or MRI. One participant was excluded because of a technical problem with the implants. The remaining participant was a 28-year-old man, who had tetraplegia following a C4–C5 spinal cord injury. Two bilateral wireless epidural recorders, each with 64 electrodes, were implanted over the upper limb sensorimotor areas of the brain. Epidural electrocorticographic (ECoG) signals were processed online by an adaptive decoding algorithm to send commands to effectors (virtual avatar or exoskeleton). Throughout the 24 months of the study, the patient did various mental tasks to progressively increase the number of degrees of freedom. Findings Between June 12, 2017, and July 21, 2019, the patient cortically controlled a programme that simulated walking and made bimanual, multi-joint, upper-limb movements with eight degrees of freedom during various reach-and-touch tasks and wrist rotations, using a virtual avatar at home (64·0% [SD 5·1] success) or an exoskeleton in the laboratory (70·9% [11·6] success). Compared with microelectrodes, epidural ECoG is semi-invasive and has similar efficiency. The decoding models were reusable for up to approximately 7 weeks without recalibration. Interpretation These results showed long-term (24-month) activation of a four-limb neuroprosthetic exoskeleton by a complete brain–machine interface system using continuous, online epidural ECoG to decode brain activity in a tetraplegic patient. Up to eight degrees of freedom could be simultaneously controlled using a unique model, which was reusable without recalibration for up to about 7 weeks. Funding French Atomic Energy Commission, French Ministry of Health, Edmond J Safra Philanthropic Foundation, Fondation Motrice, Fondation Nanosciences, Institut Carnot, Fonds de Dotation Clinatec.

This article deals with the long-term preclinical validation of WIMAGINE® (Wireless Implantable Multi-channel Acquisition system for Generic Interface with Neurons), a 64-channel wireless implantable recorder that measures the electrical activity at the cortical surface (electrocorticography, ECoG). The WIMAGINE® implant was designed for chronic wireless neuronal signal acquisition, to be used e.g., as an intracranial Brain-Computer Interface (BCI) for severely motor-impaired patients. Due to the size and shape of WIMAGINE®, sheep appeared to be the best animal model on which to carry out long-term in vivo validation. The devices were implanted in two sheep for a follow-up period of 10 months, including idle state cortical recordings and Somato-Sensory Evoked Potential (SSEP) sessions. ECoG and SSEP demonstrated relatively stable behavior during the 10-month observation period. Information recorded from the SensoriMotor Cortex (SMC) showed an SSEP phase reversal, indicating the cortical site of the sensorimotor activity was retained after 10 months of contact. Based on weekly recordings of raw ECoG signals, the effective bandwidth was in the range of 230 Hz for both animals and remarkably stable over time, meaning preservation of the high frequency bands valuable for decoding of the brain activity using BCIs. The power spectral density (in dB/Hz), on a log scale, was of the order of 2.2, -4.5 and -18 for the frequency bands (10-40), (40-100), and (100-200) Hz, respectively. The outcome of this preclinical work is the first long-term in vivo validation of the WIMAGINE® implant, highlighting its ability to record the brain electrical activity through the dura mater and to send wireless digitized data to the external base station. Apart from local adhesion of the dura to the skull, the neurosurgeon did not face any difficulty in the implantation of the WIMAGINE® device and post-mortem analysis of the brain revealed no side effect related to the implantation. We also report on the reliability of the system; including the implantable device, the antennas module and the external base station.

Motor Brain-Computer Interfaces (BCIs) are systems that allow severely motor-impaired patients to use their brain activity to interact with their environment. Electrocorticography (ECoG) arrays may be profitably used to develop safe and chronic motor BCI systems. BCI signal processing pipelines generally include neuronal signal pre-processing, feature extraction and classification/regression. The article presents a comparative study addressing the problem of neural feature classification in asynchronous multi-limb ECoG-driven BCIs. Several conventional classifiers often reported in the BCI literature were coupled with two preprocessing techniques and with a conventional feature extraction approach. They were compared to artificial neural network (ANN) end-to-end classifiers which mimic conventional BCI signal processing pipelines. Different initializations of ANNs were particularly studied. The comparison study was carried out using publicly available datasets (BCI competition IV).

Brain-Computer Interfaces (BCIs) are systems which translate brain neural activity into commands for external devices. BCI users generally alternate between No-Control (NC) and Intentional Control (IC) periods. NC/IC discrimination is crucial for clinical BCIs, particularly when they provide neural control over complex effectors such as exoskeletons. Numerous BCI decoders focus on the estimation of continuously-valued limb trajectories from neural signals. The integration of NC support into continuous decoders is investigated in the present article. Most discrete/continuous BCI hybrid decoders rely on static state models which don't exploit the dynamic of NC/IC state succession. A hybrid decoder, referred to as Markov Switching Linear Model (MSLM), is proposed in the present article. The MSLM assumes that the NC/IC state sequence is generated by a first-order Markov chain, and performs dynamic NC/IC state detection. Linear continuous movement models are probabilistically combined using the NC and IC state posterior probabilities yielded by the state decoder. The proposed decoder is evaluated for the task of asynchronous wrist position decoding from high dimensional space-time-frequency ElectroCorticoGraphic (ECoG) features in monkeys. The MSLM is compared with another dynamic hybrid decoder proposed in the literature, namely a Switching Kalman Filter (SKF). A comparison is additionally drawn with a Wiener filter decoder which infers NC states by thresholding trajectory estimates. The MSLM decoder is found to outperform both the SKF and the thresholded Wiener filter decoder in terms of False Positive Ratio and NC/IC state detection error. It additionally surpasses the SKF with respect to the Pearson Correlation Coefficient and Root Mean Squared Error between true and estimated continuous trajectories.

Among biomedical signals, repetitive or quasi-periodic signals are particularly widespread. While the periodic component is still presented these signals are characterized by period variations (fundamental frequency, amplitude, etc.). The lack of synchronization or phase shifts results in variations in similar segments’ durations, nominally identical signals demonstrate a variation at peak retention times, etc. The inverse methods of oscillation theory were proposed recently as a tool to solve the problems of modelling of repetitive signals with phase shift. In the article, the inverse method of oscillation theory is considered as a tool to solve the problems of supervised and non-supervised classification, and filtering of repetitive signals with phase shift. Examples of application are presented.

International audience; BackgroundBrain Computer Interface (BCI) studies are performed in an increasing number of applications. Questions are raised about electrodes, data processing and effectors. Experiments are needed to solve these issues. ObjectiveTo develop a simple BCI set-up to easier studies for improving the mathematical tools to process the ECoG to control an effector. MethodWe designed a simple BCI using transcranial electrodes (17 screws, three mechanically linked to create a common reference, 14 used as recording electrodes) to record Electro-Cortico-Graphic (ECoG) neuronal activities in rodents. The data processing is based on an online self-paced non-supervised (asynchronous) BCI paradigm. N-way partial least squares algorithm together with Continuous Wavelet Transformation of ECoG recordings detect signatures related to motor activities. Signature detection in freely moving rats may activate external effectors during a behavioral task, which involved pushing a lever to obtain a reward. ResultsAfter routine training, we showed that peak brain activity preceding a lever push (LP) to obtain food reward was located mostly in the cerebellar cortex with a higher correlation coefficient, suggesting a strong postural component and also in the occipital cerebral cortex. Analysis of brain activities provided a stable signature in the high gamma band (approximate to 180Hz) occurring within 1500 msec before the lever push approximately around -400 msec to -500 msec. Detection of the signature from a single cerebellar cortical electrode triggers the effector with high efficiency (68% Offline and 30% Online) and rare false positives per minute in sessions about 30 minutes and up to one hour (approximate to 2 online and offline). ConclusionsIn summary, our results are original as compared to the rest of the literature, which involves rarely rodents, a simple BCI set-up has been developed in rats, the data show for the first time long-term, up to one year, unsupervised online control of an effector.

A tensor-input/tensor-output Recursive Exponentially Weighted N-Way Partial Least Squares (REW-NPLS) regression algorithm is proposed for high dimension multi-way (tensor) data treatment and adaptive modeling of complex processes in real-time. The method unites fast and efficient calculation schemes of the Recursive Exponentially Weighted PLS with the robustness of tensor-based approaches. Moreover, contrary to other multi-way recursive algorithms, no loss of information occurs in the REW-NPLS. In addition, the Recursive-Validation method for recursive estimation of the hyper-parameters is proposed instead of conventional cross-validation procedure. The approach was then compared to state-of-the-art methods. The efficiency of the methods was tested in electrocorticography (ECoG) and magnetoencephalography (MEG) datasets. The algorithms are implemented in software suitable for real-time operation. Although the Brain-Computer Interface applications are used to demonstrate the methods, the proposed approaches could be efficiently used in a wide range of tasks beyond neuroscience uniting complex multi-modal data structures, adaptive modeling, and real-time computational requirements.

In the current paper the decoding algorithms for motor-related BCI systems for continuous upper limb trajectory prediction are considered. Two methods for the smooth prediction, namely Sobolev and Polynomial Penalized Multi-Way Partial Least Squares (PLS) regressions, are proposed. The methods are compared to the Multi-Way Partial Least Squares and Kalman Filter approaches. The comparison demonstrated that the proposed methods combined the prediction accuracy of the algorithms of the PLS family and trajectory smoothness of the Kalman Filter. In addition, the prediction delay is significantly lower for the proposed algorithms than for the Kalman Filter approach. The proposed methods could be applied in a wide range of applications beyond neuroscience.

The goal of the CLINATEC® Brain Computer Interface (BCI) Project is to improve tetraplegic subjects' quality of life by allowing them to interact with their environment through the control of effectors, such as an exoskeleton. The BCI platform is based on a wireless 64-channel ElectroCorticoGram (ECoG) recording implant WIMAGINE®, designed for long-term clinical application, and a BCI software environment associated to a 4-limb exoskeleton EMY (Enhancing MobilitY). Innovative ECoG signal decoding algorithms will allow the control of the exoskeleton by the subject's brain activity. Currently, the whole BCI platform was tested in real-time in preclinical experiments carried out in nonhuman primates. In these experiments, the exoskeleton arm was controlled by means of the decoded neuronal activity.

The goal of the CLINATEC® Brain Computer Interface (BCI) Project is to improve tetraplegic subjects' quality of life by allowing them to interact with their environment through the control of effectors, such as an exoskeleton. The BCI platform is based on a wireless 64-channel ElectroCorticoGram (ECoG) recording implant WIMAGINE®, designed for long-term clinical application, and a BCI software environment associated to a 4-limb exoskeleton EMY (Enhancing MobilitY). Innovative ECoG signal decoding algorithms will allow the control of the exoskeleton by the subject's brain activity. Currently, the whole BCI platform was tested in real-time in preclinical experiments carried out in nonhuman primates. In these experiments, the exoskeleton arm was controlled by means of the decoded neuronal activity.

Long-term BCI based on chronic electrocorticogram (ECoG) is one of the challenges of BCI project at CLINATEC ® /LETI/CEA. Brain signal decoding algorithms are crucial to convert the preserved tetraplegia patient's brain activity into motor command for an exoskeleton. Robust model identification methods have been designed in CLINATEC ® , and have already provided relevant models of dependence between neural activity and intended movement when applied on ECoG non-human primate (NHP) data. To prepare for BCI clinical application in patients, preliminary studies on human data have been carried out using non-invasive magnetoencephalography (MEG). Five able-bodied subjects participated in the study. Each subject performed two 4-minute sessions for each of the following paradigms: execution of real stereotypical movements after disappearance of a specific visual cue (1), execution of self-paced stereotypical movements (2). Additionally, cue-paced motor imaginary (3) was performed by one of the subjects. The stereotypical movements used for the three paradigms consisted in rectilinear trajectories of the hand from an idle position, and back (along axis x , y and z ). Hand coordinates were tracked during real hand movement sessions. Scaled real movement templates were used as estimations of the imagined movements for paradigm (3). Brain activity was recorded by a MEG Elekta Neuromag system. Thirty-six sensors were selected over the subject's sensorimotor area. Time-frequency representations of the corresponding signals were computed by means of continuous wavelet transforms. A model between these brain activity features and hand coordinates was identified using Partial Least Squares combined with Generalized Linear Model. Cross-validation was performed to assess model accuracy. Preliminary results yield promising average correlation levels between predicted and real movement, suggesting that CLINATEC ® model identification methods are relevant for hand movement decoding from MEG signals in human subjects. Experiments in progress focus on closed-loop decoding sessions in which subjects aim at controlling avatar movement based on real-time decoding of their brain sensorimotor activity. This is the last validation step before going to tetraplegia patients with ECoG signals.

Recently, the N-way partial least squares (NPLS) approach was reported as an effective tool for neuronal signal decoding and brain–computer interface (BCI) system calibration. This method simultaneously analyzes data in several domains. It combines the projection of a data tensor to a low dimensional space with linear regression. In this paper the L1-Penalized NPLS is proposed for sparse BCI system calibration, allowing uniting the projection technique with an effective selection of subset of features. The L1-Penalized NPLS was applied for the binary self-paced BCI system calibration, providing selection of electrodes subset. Our BCI system is designed for animal research, in particular for research in non-human primates. (Some figures may appear in colour only in the online journal)

Brain-computer interfaces (BCIs) include stimulators, infusion devices, and neuroprostheses. They all belong to functional neurosurgery. Deep brain stimulators (DBS) are widely used for therapy and are in need of innovative evolutions. Robotized exoskeletons require BCIs able to drive up to 26 degrees of freedom (DoF). We report the nanomicrotechnology development of prototypes for new 3D DBS and for motor neuroprostheses. For this complex project, all compounds have been designed and are being tested. Experiments were performed in rats and primates for proof of concepts and development of the electroencephalogram (EEG) recognition algorithm.Various devices have been designed. (A) In human, a programmable multiplexer connecting five tetrapolar (20 contacts) electrodes to one DBS channel has been designed and implanted bilaterally into STN in two Parkinsonian patients. (B) A 50-mm diameter titanium implant, telepowered, including a radioset, emitting ECoG data recorded by a 64-electrode array using an application-specific integrated circuit, is being designed to be implanted in a 50-mm trephine opening. Data received by the radioreceiver are processed through an original wavelet-based Iterative N-way Partial Least Square algorithm (INPLS, CEA patent). Animals, implanted with ECoG recording electrodes, had to press a lever to obtain a reward. The brain signature associated to the lever press (LP) was detected online by ECoG processing using INPLS. This detection allowed triggering the food dispenser.(A) The 3D multiplexer allowed tailoring the electrical field to the STN. The multiplication of the contacts affected the battery life and suggested different implantation schemes. (B) The components of the human implantable cortical BCI are being tested for reliability and toxicology to meet criteria for chronicle implantation in 2012. (C) In rats, the algorithm INPLS could detect the cortical signature with an accuracy of about 80% of LPs on the electrodes with the best correlation coefficient (located over the cerebellar cortex), 1% of the algorithm decisions were false positives. We aim to pilot effectors with DoF up to 3 in monkeys.We have designed multielectrodes wireless implants to open the way for BCI ECoG-driven effectors. These technologies are also used to develop new generations of brain stimulators, either cortical or for deep targets. This chapter is aimed at illustrating that BCIs are actually the daily background of DBS, that the evolution of the method involves a growing multiplicity of targets and indications, that new technologies make possible and simpler than before to design innovative solutions to improve DBS methodology, and that the coming out of BCI-driven neuroprostheses for compensation of motor and sensory deficits is a natural evolution of functional neurosurgery.

Brain–computer interfaces (BCIs) include stimulators, infusion devices, and neuroprostheses. They all belong to functional neurosurgery. Deep brain stimulators (DBS) are widely used for therapy and are in need of innovative evolutions. Robotized exoskeletons require BCIs able to drive up to 26 degrees of freedom (DoF). We report the nanomicrotechnology development of prototypes for new 3D DBS and for motor neuroprostheses. For this complex project, all compounds have been designed and are being tested. Experiments were performed in rats and primates for proof of concepts and development of the electroencephalogram (EEG) recognition algorithm. Methods: Various devices have been designed. (A) In human, a programmable multiplexer connecting five tetrapolar (20 contacts) electrodes to one DBS channel has been designed and implanted bilaterally into STN in two Parkinsonian patients. (B) A 50-mm diameter titanium implant, telepowered, including a radioset, emitting ECoG data recorded by a 64-electrode array using an application-specific integrated circuit, is being designed to be implanted in a 50-mm trephine opening. Data received by the radioreceiver are processed through an original wavelet-based Iterative N-way Partial Least Square algorithm (INPLS, CEA patent). Animals, implanted with ECoG recording electrodes, had to press a lever to obtain a reward. The brain signature associated to the lever press (LP) was detected online by ECoG processing using INPLS. This detection allowed triggering the food dispenser. Results: (A) The 3D multiplexer allowed tailoring the electrical field to the STN. The multiplication of the contacts affected the battery life and suggested different implantation schemes. (B) The components of the human implantable cortical BCI are being tested for reliability and toxicology to meet criteria for chronicle implantation in 2012. (C) In rats, the algorithm INPLS could detect the cortical signature with an accuracy of about 80% of LPs on the electrodes with the best correlation coefficient (located over the cerebellar cortex), 1% of the algorithm decisions were false positives. We aim to pilot effectors with DoF up to 3 in monkeys. Conclusion: We have designed multielectrodes wireless implants to open the way for BCI ECoG-driven effectors. These technologies are also used to develop new generations of brain stimulators, either cortical or for deep targets. This chapter is aimed at illustrating that BCIs are actually the daily background of DBS, that the evolution of the method involves a growing multiplicity of targets and indications, that new technologies make possible and simpler than before to design innovative solutions to improve DBS methodology, and that the coming out of BCI-driven neuroprostheses for compensation of motor and sensory deficits is a natural evolution of functional neurosurgery.

A spinal cord injury interrupts the communication between the brain and the region of the spinal cord that produces walking, leading to paralysis1,2. Here, we restored this communication with a digital bridge between the brain and spinal cord that enabled an individual with chronic tetraplegia to stand and walk naturally in community settings. This brain–spine interface (BSI) consists of fully implanted recording and stimulation systems that establish a direct link between cortical signals3 and the analogue modulation of epidural electrical stimulation targeting the spinal cord regions involved in the production of walking4–6. A highly reliable BSI is calibrated within a few minutes. This reliability has remained stable over one year, including during independent use at home. The participant reports that the BSI enables natural control over the movements of his legs to stand, walk, climb stairs and even traverse complex terrains. Moreover, neurorehabilitation supported by the BSI improved neurological recovery. The participant regained the ability to walk with crutches overground even when the BSI was switched off. This digital bridge establishes a framework to restore natural control of movement after paralysis.

Brain-computer interfaces (BCIs) may significantly improve tetraplegic patients' quality of life by creating an alternative communication path between humans and the environment, potentially compensating for motor function loss. This thesis focuses on ECoG-based BCI systems that showed a high potential to provide efficient communication while being less invasive than intracortical recordings. In particular, we explored problem of continuous 3D hand translation decoding in a tetraplegic patient. In this case, most studies use linear models that may be too simple to analyze brain processes and may suffer from low decoding accuracy. Models based on deep learning (DL) have been proven effective in various tasks and thus emerge as a potential solution to create a robust brain signals representation. In this thesis, we studied the potential of DL-based methods for hand translation decoding from ECoG signals.First, we evaluated several DL models to predict 3D hand translation from ECoG time-frequency features. The analysis was performed on a dataset recorded with a tetraplegic subject in the BCI and Tetraplegia clinical trial (NCT02550522). We started the investigation with a multilayer perceptron taking vectorized features as input. Then, we proposed convolutional neural networks (CNN), which take matrix-organized inputs approximating the spatial arrangement of the electrodes. In addition, we investigated the usefulness of long short-term memory (LSTM) to analyze temporal information. Results showed that CNN-based architectures performed better than the current state-of-the-art multilinear model on the analyzed ECoG dataset. The best architecture used a CNN-based model to analyze the spatial representation of time-frequency features followed by LSTM exploiting the sequential character of the desired hand trajectory. Compared to the multilinear model, DL-based solutions increased average cosine similarity by up to 60%.In the case of BCI, access to large datasets is limited because recordings are time-consuming and tiring. To investigate the influence of the dataset size on the decoding performance, we compared the learning curve characteristics of DL and multilinear models evaluated in the previous step. The training dataset size was gradually increased from 5 to 140 minutes of signal in different computational experiments, focusing on dataset size requirements and patient adaptation effects. Our results revealed that DL models have a learning curve profile similar to the multilinear model, increasing performance for almost all training dataset sizes. This result validates the DL-based models as a good candidate for real-life applications. We also observed increased data quality for recordings performed later, indicating improved patient ability to generate meaningful patterns.DL proved its usefulness for computer vision, primarily in the case of end-to-end learning. It enabled extracting more powerful representations trained for a specific task and removed the step of hand-crafted feature extraction. We evaluated methods using raw ECoG signals as a natural extension of hand-crafted feature analysis. In the data processing pipeline evaluated so far, continuous wavelet transform was used to extract time-frequency representation, which can be seen as a convolution between a set of wavelet filters and the ECoG signal. In this setup, the gradient w.r.t. filters coefficients can be computed, and thus the whole network can be trained within an end-to-end scheme. The parameters of wavelet filters were optimized end-to-end to see potential profit from adjusting the parameters to this specific problem. The results showed only minor or no benefit from training the wavelets in terms of cosine similarity, while end-to-end models require more computational power. This may suggest that training first layer parameters may be less beneficial and more challenging in the case of ECoG-based BCI.; Les interfaces cerveau-machines (ICMs) peuvent améliorer considérablement la qualité de vie des patients tétraplégiques en créant une voie de communication alternative et en compensant la perte de la fonction motrice. Cette thèse se concentre sur les systèmes ICM basés sur l'ECoG qui ont montré un fort potentiel pour fournir une communication efficace, tout en étant moins invasif que les enregistrements intracorticaux. Cependant, la plupart des systèmes actuels souffrent d'une faible précision de décodage. Les modèles basés sur l'apprentissage profond (DL) se sont avérés efficaces dans de nombreuses tâches et apparaissent donc comme une solution possible pour créer une représentation robuste des signaux cérébraux. Dans cette thèse, nous avons étudié le potentiel des méthodes basées sur l'apprentissage profond pour le décodage de la translation de la main à partir de signaux ECoG.Tout d'abord, nous avons évalué plusieurs modèles de DL sur des données de l'essai clinique BCI et Tétraplégie (NCT02550522) enregistrées avec un sujet tétraplégique, pour prédire la translation de la main en 3D à partir des caractéristiques temps-fréquence des signaux ECoG. Nous avons commencé l'étude avec un perceptron multicouche prenant en entrée des caractéristiques vectorisées. Ensuite, nous avons proposé des réseaux neuronaux convolutifs (CNN), qui prennent des entrées organisées en matrices se rapprochant de la disposition spatiale des électrodes, et des cellulles long short-term memory (LSTM) pour analyser les informations temporelles. Les résultats ont montré que les architectures basées sur les réseaux neuronaux convolutifs sont plus performantes que le modèle multilinéaire actuel sur l'ensemble des données ECoG analysées, avec une augmentation de la similarité cosinus moyenne allant jusqu'à 60 %. La meilleure architecture utilisait un modèle CNN pour analyser la représentation spatiale des caractéristiques temps-fréquence, suivi d'un LSTM exploitant le caractère séquentiel des données.Dans le cas des ICM, l'accès à de grands jeux de données est limité car les enregistrements sont longs et fatigants pour le patient. Pour étudier l'impact de la taille du jeu de données sur les performances de décodage, la quantité de données d'entraînement a été progressivement augmentée de 5 à 140 minutes de signal dans différentes expériences de calcul, en se concentrant sur les exigences de taille du jeu de données et les effets d'adaptation du patient. Nos résultats ont révélé que les modèles de DL augmentent les performances pour presque toutes les tailles de jeu de données d'entraînement. Ce résultat valide les modèles de DL comme bons candidats pour les applications de la vie réelle. Nous avons également observé une meilleure qualité des données pour les enregistrements effectués plus tardivement, ce qui indique un progrès du patient pour générer des signaux informatifs.Le DL a prouvé son utilité pour la vision par ordinateur, principalement dans le cas de l'apprentissage de bout en bout, permettant l'extraction de représentations plus adaptées à une tâche spécifique. Nous avons évalué des méthodes utilisant des signaux ECoG bruts en tant qu'extension naturelle de l'analyse des caractéristiques créées à la main. Les paramètres de la transformée en ondelettes continue utilisée pour extraire la représentation temps-fréquence ont été optimisés de bout en bout pour voir le bénéfice potentiel de l'ajustement des paramètres à ce problème spécifique. Les résultats ont montré qu'il n'y avait que peu ou pas de bénéfice à entraîner les ondelettes en termes de similarité en cosinus, alors que les modèles de bout en bout nécessitent plus de puissance de calcul. Cela peut suggérer que l'entraînement des paramètres de la première couche peut être moins bénéfique et plus difficile dans le cas d'une BCI basée sur l'ECoG.

Brain-computer interfaces (BCIs) transform neural signals into commands for effectors. They are mainly used as tools for functional compensation of impaired functions in disabled subjects. The Clinatec clinical trial “BCI and tetraplegia” aims at providing a proof of concept of long-term functional compensation of upper and lower limbs motor deficits in tetraplegics using a BCI. The clinical trial showed promising results for compensation of motor functions. However, the BCI’s usability remains to be improved. The decoder used to estimate the user’s intention from the processed neural signal (control decoder) must be fitted before the BCI can be used. This is done during dedicated training sessions, during which the user is directed to perform specific motor imagery tasks. Training sessions have to be held regularly in order to update the control decoder due to degradation of performances with time. This thesis proposes to limit the negative impact of training sessions. In this work, the control decoder is trained and updated using inferred labels instead of labels acquired during a dedicated training session. The labels are inferred using the output of the control decoder and neural correlates of task performance. In order to be usable in state-of-the-art motor BCIs, the adaptation process must be possible for control decoders with multiple discrete or continuous outputs. We argue that adaptation of a control decoder with multiple continuous outputs is best done using neural correlates of task performance that can be decoded continuously in time. Additionally, these neural correlates should be detected in the sensorimotor cortex due to the position of the implanted neural acquisition system. Using multiple ECoG datasets from a tetraplegic enrolled in the Clinatec BCI clinical trial, we show in this thesis that it is possible to detect such continuous in time neural correlates of task performance from the sensorimotor cortex, that control decoder labels can be inferred using these neural correlates, and finally that these labels can be used to successfully train decoders for discrete or continuous control. The subject used motor imagery to control a binary avatar (Runner MI dataset), a virtual exoskeleton with four discrete motor states (Exo dataset) or a hand-shaped cursor on a two-dimensional screen (Cursor dataset). In five-fold cross-validations, the mean and standard deviation of the area under the receiver operating characteristic curve (AUC of the ROC) for the decoder of neural correlates of task performance were 0.6225±0.0429 in the Runner MI dataset, 0.5677±0.0427 in the Exo dataset and 0.6570±0.0188 in the Cursor dataset. In a pseudo cross-validation simulating online use, the accuracy of the estimated labels was 64.9% in the Runner dataset and 64.5% in the Exo dataset. In the Cursor dataset, 63.3% of the estimated labels were less than 60° away from the actual labels. The AUC of the ROC of the control decoder was 0.6360±0.0958 in the Runner MI dataset when trained auto-adaptively compared to 0.8958±0.0153 when trained supervisedly, and to a chance level of 0.5007±0.0691. The multiclass generalization of the AUC of the ROC of the control decoder was 0.7595±0.0278 in the Exo dataset when trained auto-adaptively compared to 0.8177±0.0301 when trained supervisedly, and to a chance level of 0.5163±0.0580. In the Cursor dataset, the cosine similarity was 0.1589±0.0668 when trained auto-adaptively compared to 0.2107±0.0664 when trained in a classical supervised manner, and a chance level of -0.0231±0.0327. These results are promising for the future of auto-adaptive complex motor BCIs., Les Interfaces cerveau machine (ICMs) transforment les signaux neuronaux en commande pour des effecteurs. Elles sont principalement utilisées pour compenser les déficits d’handicapés moteurs. L’essai clinique « BCI et tétraplégie » de Clinatec a pour but the fournir une preuve de concept the compensation fonctionnelle chronique des déficits moteurs des quatre membres de tétraplégiques au moyen d’ICMs. Bien que les résultats obtenus soient prometteurs, l’usabilité des ICMs utilisées reste améliorable. Le décodeur utilisé pour transformer les signaux neuronaux en commandes (décodeur de contrôle) doit par exemple être entrainé avant de pouvoir utiliser l’ICM. Ceci est fait durant des séances d’entrainement spécifiques, durant lesquelles le patient doit effectuer des tâches d’imagerie motrices imposés. Ces séances d’entrainement constituent un temps mort et doivent être effectuées régulièrement afin de compenser les pertes de performances qui surviennent avec le temps. Cette thèse vise à limiter les effets négatifs des séances d’entrainements. Pour cela, le décodeur de contrôle est entrainé et mis à jour avec des labels estimés au lieu de labels obtenus lors de séances d’entrainement. Les labels sont estimés grâce à la sortie du décodeur de control ainsi qu’un décodeur de corrélats neuronaux de performance de tâche. Afin d’être utilisable avec les ICMs motrices les plus performantes, l’adaptation doit être faisable pour les décodeurs de contrôle qui ont plusieurs sorties continues. Nous suggérons qu’une telle adaptation requiert des corrélats neuronaux de performances de tâche acquis de manière continuelle dans le temps. Il est aussi nécessaire de détecter ces corrélats dans le cortex sensorimoteur à cause de la position des implants. Dans cette thèse, nous montrons sur plusieurs jeux de données ECoG d’un tétraplégique qu’il est possible de détecter continuellement des corrélats neuronaux de performance de tâche depuis le cortex sensorimoteur, que les labels des décodeurs de contrôle peuvent être estimés grâce à ces corrélats et enfin que ces labels peuvent être utilisés pour l’entrainement de décodeurs avec des sorties discrètes ou continues. Le patient a utilisé l’imagerie motrice pour contrôler un avatar binaire (jeu de données Runner MI), un exosquelette virtuel avec quatre états moteur discrets (jeu de données Exo) ou un curseur en deux dimensions continues sur un écran d’ordinateur (jeu de données Cursor). Lors de validations croisées à 5 blocs, la moyenne et la déviation standard de l’aire sous la courbe (AUC) ROC du décodeur de corrélats neuronaux de performance de tâche était de 0.6225±0.0429, 0.5677±0.0427 et 0.6570±0.0188 pour les jeux de données Runner MI, Exo et Cursor respectivement. Dans une simulation d’utilisation en ligne de de cette méthode d’adaptation, la précision des labels estimés était de 64.9% et 64.5% pour les jeux de données Runner MI et Exo. Pour le jeu de données Cursor, 63% des labels estimés étaient à moins de 60° d’écart des vrais labels. Dans cette simulation avec le jeu de donnée Runner MI, l’AUC de la ROC du décodeur de contrôle était de 0.6360±0.0958 quand entrainé de manière auto-adaptative, comparé à 0.8958±0.0153 pour un entrainement supervisé et à un niveau de chance de 0.5007±0.0691. Pour le jeu de données Exo, la généralisation multi classe de l’AUC de la ROC était 0.7595±0.0278 pour un entrainement auto-adaptative, comparé à 0.8177±0.0301 pour un entrainement supervisé et à un niveau de chance de 0.5163±0.0580. Pour le jeu de données Cursor, la similarité cosinus était 0.1589±0.0668 pour un entrainement auto-adaptative, comparé à 0.2107±0.0664 pour un entrainement supervisé et à un niveau de chance de -0.0231±0.0327. Ces résultats sont prometteurs pour le développement des ICMs motrices complexes auto-adaptatives.

Brain-computer interfaces (BCIs) transform neural signals into commands for effectors. They are mainly used as tools for functional compensation of impaired functions in disabled subjects. The Clinatec clinical trial “BCI and tetraplegia” aims at providing a proof of concept of long-term functional compensation of upper and lower limbs motor deficits in tetraplegics using a BCI. The clinical trial showed promising results for compensation of motor functions. However, the BCI’s usability remains to be improved. The decoder used to estimate the user’s intention from the processed neural signal (control decoder) must be fitted before the BCI can be used. This is done during dedicated training sessions, during which the user is directed to perform specific motor imagery tasks. Training sessions have to be held regularly in order to update the control decoder due to degradation of performances with time. This thesis proposes to limit the negative impact of training sessions. In this work, the control decoder is trained and updated using inferred labels instead of labels acquired during a dedicated training session. The labels are inferred using the output of the control decoder and neural correlates of task performance. In order to be usable in state-of-the-art motor BCIs, the adaptation process must be possible for control decoders with multiple discrete or continuous outputs. We argue that adaptation of a control decoder with multiple continuous outputs is best done using neural correlates of task performance that can be decoded continuously in time. Additionally, these neural correlates should be detected in the sensorimotor cortex due to the position of the implanted neural acquisition system. Using multiple ECoG datasets from a tetraplegic enrolled in the Clinatec BCI clinical trial, we show in this thesis that it is possible to detect such continuous in time neural correlates of task performance from the sensorimotor cortex, that control decoder labels can be inferred using these neural correlates, and finally that these labels can be used to successfully train decoders for discrete or continuous control. The subject used motor imagery to control a binary avatar (Runner MI dataset), a virtual exoskeleton with four discrete motor states (Exo dataset) or a hand-shaped cursor on a two-dimensional screen (Cursor dataset). In five-fold cross-validations, the mean and standard deviation of the area under the receiver operating characteristic curve (AUC of the ROC) for the decoder of neural correlates of task performance were 0.6225±0.0429 in the Runner MI dataset, 0.5677±0.0427 in the Exo dataset and 0.6570±0.0188 in the Cursor dataset. In a pseudo cross-validation simulating online use, the accuracy of the estimated labels was 64.9% in the Runner dataset and 64.5% in the Exo dataset. In the Cursor dataset, 63.3% of the estimated labels were less than 60° away from the actual labels. The AUC of the ROC of the control decoder was 0.6360±0.0958 in the Runner MI dataset when trained auto-adaptively compared to 0.8958±0.0153 when trained supervisedly, and to a chance level of 0.5007±0.0691. The multiclass generalization of the AUC of the ROC of the control decoder was 0.7595±0.0278 in the Exo dataset when trained auto-adaptively compared to 0.8177±0.0301 when trained supervisedly, and to a chance level of 0.5163±0.0580. In the Cursor dataset, the cosine similarity was 0.1589±0.0668 when trained auto-adaptively compared to 0.2107±0.0664 when trained in a classical supervised manner, and a chance level of -0.0231±0.0327. These results are promising for the future of auto-adaptive complex motor BCIs., Les Interfaces cerveau machine (ICMs) transforment les signaux neuronaux en commande pour des effecteurs. Elles sont principalement utilisées pour compenser les déficits d’handicapés moteurs. L’essai clinique « BCI et tétraplégie » de Clinatec a pour but the fournir une preuve de concept the compensation fonctionnelle chronique des déficits moteurs des quatre membres de tétraplégiques au moyen d’ICMs. Bien que les résultats obtenus soient prometteurs, l’usabilité des ICMs utilisées reste améliorable. Le décodeur utilisé pour transformer les signaux neuronaux en commandes (décodeur de contrôle) doit par exemple être entrainé avant de pouvoir utiliser l’ICM. Ceci est fait durant des séances d’entrainement spécifiques, durant lesquelles le patient doit effectuer des tâches d’imagerie motrices imposés. Ces séances d’entrainement constituent un temps mort et doivent être effectuées régulièrement afin de compenser les pertes de performances qui surviennent avec le temps. Cette thèse vise à limiter les effets négatifs des séances d’entrainements. Pour cela, le décodeur de contrôle est entrainé et mis à jour avec des labels estimés au lieu de labels obtenus lors de séances d’entrainement. Les labels sont estimés grâce à la sortie du décodeur de control ainsi qu’un décodeur de corrélats neuronaux de performance de tâche. Afin d’être utilisable avec les ICMs motrices les plus performantes, l’adaptation doit être faisable pour les décodeurs de contrôle qui ont plusieurs sorties continues. Nous suggérons qu’une telle adaptation requiert des corrélats neuronaux de performances de tâche acquis de manière continuelle dans le temps. Il est aussi nécessaire de détecter ces corrélats dans le cortex sensorimoteur à cause de la position des implants. Dans cette thèse, nous montrons sur plusieurs jeux de données ECoG d’un tétraplégique qu’il est possible de détecter continuellement des corrélats neuronaux de performance de tâche depuis le cortex sensorimoteur, que les labels des décodeurs de contrôle peuvent être estimés grâce à ces corrélats et enfin que ces labels peuvent être utilisés pour l’entrainement de décodeurs avec des sorties discrètes ou continues. Le patient a utilisé l’imagerie motrice pour contrôler un avatar binaire (jeu de données Runner MI), un exosquelette virtuel avec quatre états moteur discrets (jeu de données Exo) ou un curseur en deux dimensions continues sur un écran d’ordinateur (jeu de données Cursor). Lors de validations croisées à 5 blocs, la moyenne et la déviation standard de l’aire sous la courbe (AUC) ROC du décodeur de corrélats neuronaux de performance de tâche était de 0.6225±0.0429, 0.5677±0.0427 et 0.6570±0.0188 pour les jeux de données Runner MI, Exo et Cursor respectivement. Dans une simulation d’utilisation en ligne de de cette méthode d’adaptation, la précision des labels estimés était de 64.9% et 64.5% pour les jeux de données Runner MI et Exo. Pour le jeu de données Cursor, 63% des labels estimés étaient à moins de 60° d’écart des vrais labels. Dans cette simulation avec le jeu de donnée Runner MI, l’AUC de la ROC du décodeur de contrôle était de 0.6360±0.0958 quand entrainé de manière auto-adaptative, comparé à 0.8958±0.0153 pour un entrainement supervisé et à un niveau de chance de 0.5007±0.0691. Pour le jeu de données Exo, la généralisation multi classe de l’AUC de la ROC était 0.7595±0.0278 pour un entrainement auto-adaptative, comparé à 0.8177±0.0301 pour un entrainement supervisé et à un niveau de chance de 0.5163±0.0580. Pour le jeu de données Cursor, la similarité cosinus était 0.1589±0.0668 pour un entrainement auto-adaptative, comparé à 0.2107±0.0664 pour un entrainement supervisé et à un niveau de chance de -0.0231±0.0327. Ces résultats sont prometteurs pour le développement des ICMs motrices complexes auto-adaptatives.