Search

The Eighth International Brain-Computer Interface (BCI) Meeting was held June 7–9th, 2021 in a virtual format. The conference continued the BCI Meeting series’ interactive nature with 21 workshops covering topics in BCI (also called brain-machine interface) research. As in the past, workshops covered the breadth of topics in BCI. Some workshops provided detailed examinations of specific methods, hardware, or processes. Others focused on specific BCI applications or user groups. Several workshops continued consensus building efforts designed to create BCI standards and increase the ease of comparisons between studies and the potential for meta-analysis and large multi-site clinical trials. Ethical and translational considerations were both the primary topic for some workshops or an important secondary consideration for others. The range of BCI applications continues to expand, with more workshops focusing on approaches that can extend beyond the needs of those with physical impairments. This paper summarizes each workshop, provides background information and references for further study, presents an overview of the discussion topics, and describes the conclusion, challenges, or initiatives that resulted from the interactions and discussion at the workshop.

IntroductionMotor Brain–Computer Interfaces (BCIs) create new communication pathways between the brain and external effectors for patients with severe motor impairments. Control of complex effectors such as robotic arms or exoskeletons is generally based on the real-time decoding of high-resolution neural signals. However, high-dimensional and noisy brain signals pose challenges, such as limitations in the generalization ability of the decoding model and increased computational demands.MethodsThe use of sparse decoders may offer a way to address these challenges. A sparsity-promoting penalization is a common approach to obtaining a sparse solution. BCI features are naturally structured and grouped according to spatial (electrodes), frequency, and temporal dimensions. Applying group-wise sparsity, where the coefficients of a group are set to zero simultaneously, has the potential to decrease computational time and memory usage, as well as simplify data transfer. Additionally, online closed-loop decoder adaptation (CLDA) is known to be an efficient procedure for BCI decoder training, taking into account neuronal feedback. In this study, we propose a new algorithm for online closed-loop training of group-wise sparse multilinear decoders using Lp-Penalized Recursive Exponentially Weighted N-way Partial Least Square (PREW-NPLS). Three types of sparsity-promoting penalization were explored using Lpwith p = 0., 0.5, and 1.ResultsThe algorithms were tested offline in a pseudo-online manner for features grouped by spatial dimension. A comparison study was conducted using an epidural ECoG dataset recorded from a tetraplegic individual during long-term BCI experiments for controlling a virtual avatar (left/right-hand 3D translation). Novel algorithms showed comparable or better decoding performance than conventional REW-NPLS, which was achieved with sparse models. The proposed algorithms are compatible with real-time CLDA.DiscussionThe proposed algorithm demonstrated good performance while drastically reducing the computational load and the memory consumption. However, the current study is limited to offline computation on data recorded with a single patient, with penalization restricted to the spatial domain only.

Brain–computer interfaces (BCIs) translate brain signals into commands to external effectors, and mainly target severely disabled users. The usability of BCIs may be improved by reducing their major constraints, such as the necessity for special training sessions to initially calibrate and later keep up to date the neural signal decoders. In this study, we show that it is possible to train and update BCI decoders during free use of motor BCIs. In addition to the neural signal decoder controlling effectors (control decoder), one more classifier is proposed to detect neural correlates of BCI motor task performances (MTP). MTP decoders reveal whether the actions performed by BCI effectors matched the user’s intentions. The combined outputs of MTP and control decoders allow forming training datasets to update the control decoder online and in real time during free use of BCIs. The usability of the proposed auto-adaptive BCI (aaBCI) is demonstrated for two principle BCIs paradigms: with discrete outputs (4 classes BCI, virtual 4-limb exoskeleton), and with continuous outputs (cursor 2D control). The proof of concept was performed in an online simulation study using an ECoG dataset collected from a tetraplegic during a BCI clinical trial. The control decoder reached a multiclass area under the ROC curve of 0.7404 using aaBCI, compared to a chance level of 0.5173 and to 0.8187 for supervised training for the multiclass BCI, and a cosine similarity of 0.1211 using aaBCI, compared to a chance level of 0.0036 and to 0.2002 for supervised training for the continuous BCI.

Brain–computer interface (BCI) systems may require the user to perform a set of mental tasks, such as imagining different types of motion. The performance demonstrated on these tasks varies...

Quantifying and evaluating properly the performances is a critical issue in BCI experiments. The choice of the most adapted metrics can be difficult because they are specific to the experimental paradigm, task control, and data. In the current study evaluation criteria for closed-loop adaptive dynamic and hierarchical discrete-continuous brain-computer interfaces are examined. The challenges such as imbalanced multi-class bias for discrete decoding (classification), online computing cost and dynamic analysis of results in time are considered. There are two main objectives of the study: identifying the best suited performances metrics according to the requirements and combining several levels of evaluation in the whole BCI system (decoder and patient oriented). The main novelty of the study is the combination of several common metrics with new temporal metrics and a time dynamic approach which allows adequately reflect the performance of adaptive dynamic and hierarchical discrete-continuous brain-computer interfaces BCI systems. Additional response time and blocking error patterns reveal complementary information for BCI system performance evaluation. Developed criteria are applied to performance evaluation of 8-dimensional exoskeleton control by tetraplegic patient for both decoder and user-centered metrics.

Brain source imaging and time frequency mapping (TFM) are commonly used in magneto/electro encephalography (M/EEG) imaging. However, these methods suffer from important limitations. Source imaging is based on an ill-posed inverse problem leading to instability of source localization solutions, has a limited capacity to localize high frequency oscillations and loses its robustness for induced responses (ill-defined trigger). The drawback of TFM is that it involves independent analysis of signals from a number of frequency bands, and from co-localized sensors. In the present article, a regression-based multi-sensor space&ndash, time&ndash, frequency analysis (MSA) approach, which integrates co-localized sensors and/or multi-frequency information, is proposed. To estimate task-specific brain activations, MSA uses cross-validated, shifted, multiple Pearson correlation, calculated from the time&ndash, frequency transformed brain signal and the binary signal of stimuli. The results are projected from the sensor space onto the cortical surface. To assess MSA performance, the proposed method was compared to the weighted minimum norm estimate (wMNE) source imaging method, in terms of spatial selectivity and robustness against an ill-defined trigger. Magnetoencephalography (MEG) recordings were performed in fourteen subjects during two motor tasks: finger tapping and elbow flexion/extension. In particular, our results show that the MSA approach provides good localization performance when compared to wMNE and statistically significant improvement of robustness against ill-defined trigger.

Objective. The article aims at addressing 2 challenges to step motor brain-computer interface (BCI) out of laboratories: asynchronous control of complex bimanual effectors with large numbers of degrees of freedom, using chronic and safe recorders, and the decoding performance stability over time without frequent decoder recalibration. Approach. Closed-loop adaptive/incremental decoder training is one strategy to create a model stable over time. Adaptive decoders update their parameters with new incoming data, optimizing the model parameters in real time. It allows cross-session training with multiple recording conditions during closed loop BCI experiments. In the article, an adaptive tensor-based recursive exponentially weighted Markov-switching multi-linear model (REW-MSLM) decoder is proposed. REW-MSLM uses a mixture of expert (ME) architecture, mixing or switching independent decoders (experts) according to the probability estimated by a ‘gating’ model. A Hidden Markov model approach is employed as gating model to improve the decoding robustness and to provide strong idle state support. The ME architecture fits the multi-limb paradigm associating an expert to a particular limb or action. Main results. Asynchronous control of an exoskeleton by a tetraplegic patient using a chronically implanted epidural electrocorticography (EpiCoG) recorder is reported. The stable over a period of six months (without decoder recalibration) eight-dimensional alternative bimanual control of the exoskeleton and its virtual avatar is demonstrated. Significance. Based on the long-term (>36 months) chronic bilateral EpiCoG recordings in a tetraplegic (ClinicalTrials.gov, NCT02550522), we addressed the poorly explored field of asynchronous bimanual BCI. The new decoder was designed to meet to several challenges: the high-dimensional control of a complex effector in experiments closer to real-world behavior (point-to-point pursuit versus conventional center-out tasks), with the ability of the BCI system to act as a stand-alone device switching between idle and control states, and a stable performance over a long period of time without decoder recalibration.

Objective.The evaluation of the long-term stability of ElectroCorticoGram (ECoG) signals is an important scientific question as new implantable recording devices can be used for medical purposes such as Brain-Computer Interface (BCI) or brain monitoring.Approach.The long-term clinical validation of wireless implantable multi-channel acquisition system for generic interface with neurons (WIMAGINE), a wireless 64-channel epidural ECoG recorder was investigated. The WIMAGINE device was implanted in two quadriplegic patients within the context of a BCI protocol. This study focused on the ECoG signal stability in two patients bilaterally implanted in June 2017 (P1) and in November 2019 (P2).Methods. The ECoG signal was recorded at rest prior to each BCI session resulting in a 32 month and in a 14 month follow-up for P1 and P2 respectively. State-of-the-art signal evaluation metrics such as root mean square (RMS), the band power (BP), the signal to noise ratio (SNR), the effective bandwidth (EBW) and the spectral edge frequency (SEF) were used to evaluate stability of signal over the implantation time course. The time-frequency maps obtained from task-related motor activations were also studied to investigate the long-term selectivity of the electrodes.Mainresults.Based on temporal linear regressions, we report a limited decrease of the signal average level (RMS), spectral distribution (BP) and SNR, and a remarkable steadiness of the EBW and SEF. Time-frequency maps obtained during motor imagery, showed a high level of discrimination 1 month after surgery and also after 2 years.Conclusions.The WIMAGINE epidural device showed high stability over time. The signal evaluation metrics of two quadriplegic patients during 32 months and 14 months respectively provide strong evidence that this wireless implant is well-suited for long-term ECoG recording.Significance.These findings are relevant for the future of implantable BCIs, and could benefit other patients with spinal cord injury, amyotrophic lateral sclerosis, neuromuscular diseases or drug-resistant epilepsy.

Summary Background Approximately 20% of traumatic cervical spinal cord injuries result in tetraplegia. Neuroprosthetics are being developed to manage this condition and thus improve the lives of patients. We aimed to test the feasibility of a semi-invasive technique that uses brain signals to drive an exoskeleton. Methods We recruited two participants at Clinatec research centre, associated with Grenoble University Hospital, Grenoble, France, into our ongoing clinical trial. Inclusion criteria were age 18–45 years, stability of neurological deficits, a need for additional mobility expressed by the patient, ambulatory or hospitalised monitoring, registration in the French social security system, and signed informed consent. The exclusion criteria were previous brain surgery, anticoagulant treatments, neuropsychological sequelae, depression, substance dependence or misuse, and contraindications to magnetoencephalography (MEG), EEG, or MRI. One participant was excluded because of a technical problem with the implants. The remaining participant was a 28-year-old man, who had tetraplegia following a C4–C5 spinal cord injury. Two bilateral wireless epidural recorders, each with 64 electrodes, were implanted over the upper limb sensorimotor areas of the brain. Epidural electrocorticographic (ECoG) signals were processed online by an adaptive decoding algorithm to send commands to effectors (virtual avatar or exoskeleton). Throughout the 24 months of the study, the patient did various mental tasks to progressively increase the number of degrees of freedom. Findings Between June 12, 2017, and July 21, 2019, the patient cortically controlled a programme that simulated walking and made bimanual, multi-joint, upper-limb movements with eight degrees of freedom during various reach-and-touch tasks and wrist rotations, using a virtual avatar at home (64·0% [SD 5·1] success) or an exoskeleton in the laboratory (70·9% [11·6] success). Compared with microelectrodes, epidural ECoG is semi-invasive and has similar efficiency. The decoding models were reusable for up to approximately 7 weeks without recalibration. Interpretation These results showed long-term (24-month) activation of a four-limb neuroprosthetic exoskeleton by a complete brain–machine interface system using continuous, online epidural ECoG to decode brain activity in a tetraplegic patient. Up to eight degrees of freedom could be simultaneously controlled using a unique model, which was reusable without recalibration for up to about 7 weeks. Funding French Atomic Energy Commission, French Ministry of Health, Edmond J Safra Philanthropic Foundation, Fondation Motrice, Fondation Nanosciences, Institut Carnot, Fonds de Dotation Clinatec.

The Brain Computer Interface (BCI) is a technology that provides non-muscular communication channels to utilize the external devices. The signal from the brain is recorded, then classified. To classify, time-frequency features are frequently extracted using Continuous Wavelet Transform (CWT). However, the CWT is computationally heavy. This becomes the bottleneck of the real-time processing of BCI systems. We introduce a new method to solve the CWT using piecewise polynomials approximation over mother wavelets. The implementation shows that our method, piecewise polynomials based CWT (PCWT), is clearly better than the state of the art computation using the Fast Fourier Transform in the smaller buffer, which is preferable in the real-time systems. The proposed method can apply to other applications and should provide an effective approach to the real-time signal processing.

This article deals with the long-term preclinical validation of WIMAGINE® (Wireless Implantable Multi-channel Acquisition system for Generic Interface with Neurons), a 64-channel wireless implantable recorder that measures the electrical activity at the cortical surface (electrocorticography, ECoG). The WIMAGINE® implant was designed for chronic wireless neuronal signal acquisition, to be used e.g., as an intracranial Brain-Computer Interface (BCI) for severely motor-impaired patients. Due to the size and shape of WIMAGINE®, sheep appeared to be the best animal model on which to carry out long-term in vivo validation. The devices were implanted in two sheep for a follow-up period of 10 months, including idle state cortical recordings and Somato-Sensory Evoked Potential (SSEP) sessions. ECoG and SSEP demonstrated relatively stable behavior during the 10-month observation period. Information recorded from the SensoriMotor Cortex (SMC) showed an SSEP phase reversal, indicating the cortical site of the sensorimotor activity was retained after 10 months of contact. Based on weekly recordings of raw ECoG signals, the effective bandwidth was in the range of 230 Hz for both animals and remarkably stable over time, meaning preservation of the high frequency bands valuable for decoding of the brain activity using BCIs. The power spectral density (in dB/Hz), on a log scale, was of the order of 2.2, -4.5 and -18 for the frequency bands (10-40), (40-100), and (100-200) Hz, respectively. The outcome of this preclinical work is the first long-term in vivo validation of the WIMAGINE® implant, highlighting its ability to record the brain electrical activity through the dura mater and to send wireless digitized data to the external base station. Apart from local adhesion of the dura to the skull, the neurosurgeon did not face any difficulty in the implantation of the WIMAGINE® device and post-mortem analysis of the brain revealed no side effect related to the implantation. We also report on the reliability of the system; including the implantable device, the antennas module and the external base station.

Motor Brain-Computer Interfaces (BCIs) are systems that allow severely motor-impaired patients to use their brain activity to interact with their environment. Electrocorticography (ECoG) arrays may be profitably used to develop safe and chronic motor BCI systems. BCI signal processing pipelines generally include neuronal signal pre-processing, feature extraction and classification/regression. The article presents a comparative study addressing the problem of neural feature classification in asynchronous multi-limb ECoG-driven BCIs. Several conventional classifiers often reported in the BCI literature were coupled with two preprocessing techniques and with a conventional feature extraction approach. They were compared to artificial neural network (ANN) end-to-end classifiers which mimic conventional BCI signal processing pipelines. Different initializations of ANNs were particularly studied. The comparison study was carried out using publicly available datasets (BCI competition IV).

Brain-Computer Interfaces (BCIs) are systems which translate brain neural activity into commands for external devices. BCI users generally alternate between No-Control (NC) and Intentional Control (IC) periods. NC/IC discrimination is crucial for clinical BCIs, particularly when they provide neural control over complex effectors such as exoskeletons. Numerous BCI decoders focus on the estimation of continuously-valued limb trajectories from neural signals. The integration of NC support into continuous decoders is investigated in the present article. Most discrete/continuous BCI hybrid decoders rely on static state models which don't exploit the dynamic of NC/IC state succession. A hybrid decoder, referred to as Markov Switching Linear Model (MSLM), is proposed in the present article. The MSLM assumes that the NC/IC state sequence is generated by a first-order Markov chain, and performs dynamic NC/IC state detection. Linear continuous movement models are probabilistically combined using the NC and IC state posterior probabilities yielded by the state decoder. The proposed decoder is evaluated for the task of asynchronous wrist position decoding from high dimensional space-time-frequency ElectroCorticoGraphic (ECoG) features in monkeys. The MSLM is compared with another dynamic hybrid decoder proposed in the literature, namely a Switching Kalman Filter (SKF). A comparison is additionally drawn with a Wiener filter decoder which infers NC states by thresholding trajectory estimates. The MSLM decoder is found to outperform both the SKF and the thresholded Wiener filter decoder in terms of False Positive Ratio and NC/IC state detection error. It additionally surpasses the SKF with respect to the Pearson Correlation Coefficient and Root Mean Squared Error between true and estimated continuous trajectories.

Among biomedical signals, repetitive or quasi-periodic signals are particularly widespread. While the periodic component is still presented these signals are characterized by period variations (fundamental frequency, amplitude, etc.). The lack of synchronization or phase shifts results in variations in similar segments’ durations, nominally identical signals demonstrate a variation at peak retention times, etc. The inverse methods of oscillation theory were proposed recently as a tool to solve the problems of modelling of repetitive signals with phase shift. In the article, the inverse method of oscillation theory is considered as a tool to solve the problems of supervised and non-supervised classification, and filtering of repetitive signals with phase shift. Examples of application are presented.

Brain-Computer Interfaces (BCIs) are systems that establish a direct communication pathway between the users' brain activity and external effectors. They offer the potential to improve the quality of life of motor-impaired patients. Motor BCIs aim to permit severely motor-impaired users to regain limb mobility by controlling orthoses or prostheses. In particular, motor BCI systems benefit patients if the decoded actions reflect the users' intentions with an accuracy that enables them to efficiently interact with their environment. One of the main challenges of BCI systems is to adapt the BCI's signal translation blocks to the user to reach a high decoding accuracy. This paper will review the literature of data-driven and user-specific transducer design and identification approaches and it focuses on internally-paced motor BCIs. In particular, continuous kinematic biomimetic and mental-task decoders are reviewed. Furthermore, static and dynamic decoding approaches, linear and non-linear decoding, offline and real-time identification algorithms are considered. The current progress and challenges related to the design of clinical-compatible motor BCI transducers are additionally discussed.

International audience; BackgroundBrain Computer Interface (BCI) studies are performed in an increasing number of applications. Questions are raised about electrodes, data processing and effectors. Experiments are needed to solve these issues. ObjectiveTo develop a simple BCI set-up to easier studies for improving the mathematical tools to process the ECoG to control an effector. MethodWe designed a simple BCI using transcranial electrodes (17 screws, three mechanically linked to create a common reference, 14 used as recording electrodes) to record Electro-Cortico-Graphic (ECoG) neuronal activities in rodents. The data processing is based on an online self-paced non-supervised (asynchronous) BCI paradigm. N-way partial least squares algorithm together with Continuous Wavelet Transformation of ECoG recordings detect signatures related to motor activities. Signature detection in freely moving rats may activate external effectors during a behavioral task, which involved pushing a lever to obtain a reward. ResultsAfter routine training, we showed that peak brain activity preceding a lever push (LP) to obtain food reward was located mostly in the cerebellar cortex with a higher correlation coefficient, suggesting a strong postural component and also in the occipital cerebral cortex. Analysis of brain activities provided a stable signature in the high gamma band (approximate to 180Hz) occurring within 1500 msec before the lever push approximately around -400 msec to -500 msec. Detection of the signature from a single cerebellar cortical electrode triggers the effector with high efficiency (68% Offline and 30% Online) and rare false positives per minute in sessions about 30 minutes and up to one hour (approximate to 2 online and offline). ConclusionsIn summary, our results are original as compared to the rest of the literature, which involves rarely rodents, a simple BCI set-up has been developed in rats, the data show for the first time long-term, up to one year, unsupervised online control of an effector.

A tensor-input/tensor-output Recursive Exponentially Weighted N-Way Partial Least Squares (REW-NPLS) regression algorithm is proposed for high dimension multi-way (tensor) data treatment and adaptive modeling of complex processes in real-time. The method unites fast and efficient calculation schemes of the Recursive Exponentially Weighted PLS with the robustness of tensor-based approaches. Moreover, contrary to other multi-way recursive algorithms, no loss of information occurs in the REW-NPLS. In addition, the Recursive-Validation method for recursive estimation of the hyper-parameters is proposed instead of conventional cross-validation procedure. The approach was then compared to state-of-the-art methods. The efficiency of the methods was tested in electrocorticography (ECoG) and magnetoencephalography (MEG) datasets. The algorithms are implemented in software suitable for real-time operation. Although the Brain-Computer Interface applications are used to demonstrate the methods, the proposed approaches could be efficiently used in a wide range of tasks beyond neuroscience uniting complex multi-modal data structures, adaptive modeling, and real-time computational requirements.

In the current paper the decoding algorithms for motor-related BCI systems for continuous upper limb trajectory prediction are considered. Two methods for the smooth prediction, namely Sobolev and Polynomial Penalized Multi-Way Partial Least Squares (PLS) regressions, are proposed. The methods are compared to the Multi-Way Partial Least Squares and Kalman Filter approaches. The comparison demonstrated that the proposed methods combined the prediction accuracy of the algorithms of the PLS family and trajectory smoothness of the Kalman Filter. In addition, the prediction delay is significantly lower for the proposed algorithms than for the Kalman Filter approach. The proposed methods could be applied in a wide range of applications beyond neuroscience.

Brain-Computer Interfaces (BCIs) are systems which convert brain neural activity into commands for external devices. BCI users generally alternate between No Control (NC) and Intentional Control (IC) periods. Numerous motor-related BCI decoders focus on the prediction of continuously-valued limb trajectories from neural signals. Although NC/IC discrimination is crucial for clinical BCIs, continuous decoders rarely support NC periods. Integration of NC support in continuous decoders is investigated in the present article. Two discrete/continuous hybrid decoders are compared for the task of asynchronous wrist position decoding from ElectroCorticoGraphic (ECoG) signals in monkeys. One static and one dynamic decoder, namely a Switching Linear (SL) decoder and a Switching Kalman Filter (SKF), are evaluated on high dimensional time-frequency-space ECoG signal representations. The SL decoder was found to outperform the SKF for both NC/IC class detection and trajectory modeling.

The goal of the CLINATEC® Brain Computer Interface (BCI) Project is to improve tetraplegic subjects' quality of life by allowing them to interact with their environment through the control of effectors, such as an exoskeleton. The BCI platform is based on a wireless 64-channel ElectroCorticoGram (ECoG) recording implant WIMAGINE®, designed for long-term clinical application, and a BCI software environment associated to a 4-limb exoskeleton EMY (Enhancing MobilitY). Innovative ECoG signal decoding algorithms will allow the control of the exoskeleton by the subject's brain activity. Currently, the whole BCI platform was tested in real-time in preclinical experiments carried out in nonhuman primates. In these experiments, the exoskeleton arm was controlled by means of the decoded neuronal activity.

The goal of the CLINATEC® Brain Computer Interface (BCI) Project is to improve tetraplegic subjects' quality of life by allowing them to interact with their environment through the control of effectors, such as an exoskeleton. The BCI platform is based on a wireless 64-channel ElectroCorticoGram (ECoG) recording implant WIMAGINE®, designed for long-term clinical application, and a BCI software environment associated to a 4-limb exoskeleton EMY (Enhancing MobilitY). Innovative ECoG signal decoding algorithms will allow the control of the exoskeleton by the subject's brain activity. Currently, the whole BCI platform was tested in real-time in preclinical experiments carried out in nonhuman primates. In these experiments, the exoskeleton arm was controlled by means of the decoded neuronal activity.

Long-term BCI based on chronic electrocorticogram (ECoG) is one of the challenges of BCI project at CLINATEC ® /LETI/CEA. Brain signal decoding algorithms are crucial to convert the preserved tetraplegia patient's brain activity into motor command for an exoskeleton. Robust model identification methods have been designed in CLINATEC ® , and have already provided relevant models of dependence between neural activity and intended movement when applied on ECoG non-human primate (NHP) data. To prepare for BCI clinical application in patients, preliminary studies on human data have been carried out using non-invasive magnetoencephalography (MEG). Five able-bodied subjects participated in the study. Each subject performed two 4-minute sessions for each of the following paradigms: execution of real stereotypical movements after disappearance of a specific visual cue (1), execution of self-paced stereotypical movements (2). Additionally, cue-paced motor imaginary (3) was performed by one of the subjects. The stereotypical movements used for the three paradigms consisted in rectilinear trajectories of the hand from an idle position, and back (along axis x , y and z ). Hand coordinates were tracked during real hand movement sessions. Scaled real movement templates were used as estimations of the imagined movements for paradigm (3). Brain activity was recorded by a MEG Elekta Neuromag system. Thirty-six sensors were selected over the subject's sensorimotor area. Time-frequency representations of the corresponding signals were computed by means of continuous wavelet transforms. A model between these brain activity features and hand coordinates was identified using Partial Least Squares combined with Generalized Linear Model. Cross-validation was performed to assess model accuracy. Preliminary results yield promising average correlation levels between predicted and real movement, suggesting that CLINATEC ® model identification methods are relevant for hand movement decoding from MEG signals in human subjects. Experiments in progress focus on closed-loop decoding sessions in which subjects aim at controlling avatar movement based on real-time decoding of their brain sensorimotor activity. This is the last validation step before going to tetraplegia patients with ECoG signals.

Recently, the N-way partial least squares (NPLS) approach was reported as an effective tool for neuronal signal decoding and brain–computer interface (BCI) system calibration. This method simultaneously analyzes data in several domains. It combines the projection of a data tensor to a low dimensional space with linear regression. In this paper the L1-Penalized NPLS is proposed for sparse BCI system calibration, allowing uniting the projection technique with an effective selection of subset of features. The L1-Penalized NPLS was applied for the binary self-paced BCI system calibration, providing selection of electrodes subset. Our BCI system is designed for animal research, in particular for research in non-human primates. (Some figures may appear in colour only in the online journal)

Brain-computer interfaces (BCIs) include stimulators, infusion devices, and neuroprostheses. They all belong to functional neurosurgery. Deep brain stimulators (DBS) are widely used for therapy and are in need of innovative evolutions. Robotized exoskeletons require BCIs able to drive up to 26 degrees of freedom (DoF). We report the nanomicrotechnology development of prototypes for new 3D DBS and for motor neuroprostheses. For this complex project, all compounds have been designed and are being tested. Experiments were performed in rats and primates for proof of concepts and development of the electroencephalogram (EEG) recognition algorithm.Various devices have been designed. (A) In human, a programmable multiplexer connecting five tetrapolar (20 contacts) electrodes to one DBS channel has been designed and implanted bilaterally into STN in two Parkinsonian patients. (B) A 50-mm diameter titanium implant, telepowered, including a radioset, emitting ECoG data recorded by a 64-electrode array using an application-specific integrated circuit, is being designed to be implanted in a 50-mm trephine opening. Data received by the radioreceiver are processed through an original wavelet-based Iterative N-way Partial Least Square algorithm (INPLS, CEA patent). Animals, implanted with ECoG recording electrodes, had to press a lever to obtain a reward. The brain signature associated to the lever press (LP) was detected online by ECoG processing using INPLS. This detection allowed triggering the food dispenser.(A) The 3D multiplexer allowed tailoring the electrical field to the STN. The multiplication of the contacts affected the battery life and suggested different implantation schemes. (B) The components of the human implantable cortical BCI are being tested for reliability and toxicology to meet criteria for chronicle implantation in 2012. (C) In rats, the algorithm INPLS could detect the cortical signature with an accuracy of about 80% of LPs on the electrodes with the best correlation coefficient (located over the cerebellar cortex), 1% of the algorithm decisions were false positives. We aim to pilot effectors with DoF up to 3 in monkeys.We have designed multielectrodes wireless implants to open the way for BCI ECoG-driven effectors. These technologies are also used to develop new generations of brain stimulators, either cortical or for deep targets. This chapter is aimed at illustrating that BCIs are actually the daily background of DBS, that the evolution of the method involves a growing multiplicity of targets and indications, that new technologies make possible and simpler than before to design innovative solutions to improve DBS methodology, and that the coming out of BCI-driven neuroprostheses for compensation of motor and sensory deficits is a natural evolution of functional neurosurgery.

Brain–computer interfaces (BCIs) include stimulators, infusion devices, and neuroprostheses. They all belong to functional neurosurgery. Deep brain stimulators (DBS) are widely used for therapy and are in need of innovative evolutions. Robotized exoskeletons require BCIs able to drive up to 26 degrees of freedom (DoF). We report the nanomicrotechnology development of prototypes for new 3D DBS and for motor neuroprostheses. For this complex project, all compounds have been designed and are being tested. Experiments were performed in rats and primates for proof of concepts and development of the electroencephalogram (EEG) recognition algorithm. Methods: Various devices have been designed. (A) In human, a programmable multiplexer connecting five tetrapolar (20 contacts) electrodes to one DBS channel has been designed and implanted bilaterally into STN in two Parkinsonian patients. (B) A 50-mm diameter titanium implant, telepowered, including a radioset, emitting ECoG data recorded by a 64-electrode array using an application-specific integrated circuit, is being designed to be implanted in a 50-mm trephine opening. Data received by the radioreceiver are processed through an original wavelet-based Iterative N-way Partial Least Square algorithm (INPLS, CEA patent). Animals, implanted with ECoG recording electrodes, had to press a lever to obtain a reward. The brain signature associated to the lever press (LP) was detected online by ECoG processing using INPLS. This detection allowed triggering the food dispenser. Results: (A) The 3D multiplexer allowed tailoring the electrical field to the STN. The multiplication of the contacts affected the battery life and suggested different implantation schemes. (B) The components of the human implantable cortical BCI are being tested for reliability and toxicology to meet criteria for chronicle implantation in 2012. (C) In rats, the algorithm INPLS could detect the cortical signature with an accuracy of about 80% of LPs on the electrodes with the best correlation coefficient (located over the cerebellar cortex), 1% of the algorithm decisions were false positives. We aim to pilot effectors with DoF up to 3 in monkeys. Conclusion: We have designed multielectrodes wireless implants to open the way for BCI ECoG-driven effectors. These technologies are also used to develop new generations of brain stimulators, either cortical or for deep targets. This chapter is aimed at illustrating that BCIs are actually the daily background of DBS, that the evolution of the method involves a growing multiplicity of targets and indications, that new technologies make possible and simpler than before to design innovative solutions to improve DBS methodology, and that the coming out of BCI-driven neuroprostheses for compensation of motor and sensory deficits is a natural evolution of functional neurosurgery.

In the article tensor-input/tensor-output blockwise Recursive N-way Partial Least Squares (RNPLS) regression is considered. It combines the multi-way tensors decomposition with a consecutive calculation scheme and allows blockwise treatment of tensor data arrays with huge dimensions, as well as the adaptive modeling of time-dependent processes with tensor variables. In the article the numerical study of the algorithm is undertaken. The RNPLS algorithm demonstrates fast and stable convergence of regression coefficients. Applied to Brain Computer Interface system calibration, the algorithm provides an efficient adjustment of the decoding model. Combining the online adaptation with easy interpretation of results, the method can be effectively applied in a variety of multi-modal neural activity flow modeling tasks.

A spinal cord injury interrupts the communication between the brain and the region of the spinal cord that produces walking, leading to paralysis1,2. Here, we restored this communication with a digital bridge between the brain and spinal cord that enabled an individual with chronic tetraplegia to stand and walk naturally in community settings. This brain–spine interface (BSI) consists of fully implanted recording and stimulation systems that establish a direct link between cortical signals3 and the analogue modulation of epidural electrical stimulation targeting the spinal cord regions involved in the production of walking4–6. A highly reliable BSI is calibrated within a few minutes. This reliability has remained stable over one year, including during independent use at home. The participant reports that the BSI enables natural control over the movements of his legs to stand, walk, climb stairs and even traverse complex terrains. Moreover, neurorehabilitation supported by the BSI improved neurological recovery. The participant regained the ability to walk with crutches overground even when the BSI was switched off. This digital bridge establishes a framework to restore natural control of movement after paralysis.

Brain-computer interfaces (BCIs) may significantly improve tetraplegic patients' quality of life by creating an alternative communication path between humans and the environment, potentially compensating for motor function loss. This thesis focuses on ECoG-based BCI systems that showed a high potential to provide efficient communication while being less invasive than intracortical recordings. In particular, we explored problem of continuous 3D hand translation decoding in a tetraplegic patient. In this case, most studies use linear models that may be too simple to analyze brain processes and may suffer from low decoding accuracy. Models based on deep learning (DL) have been proven effective in various tasks and thus emerge as a potential solution to create a robust brain signals representation. In this thesis, we studied the potential of DL-based methods for hand translation decoding from ECoG signals.First, we evaluated several DL models to predict 3D hand translation from ECoG time-frequency features. The analysis was performed on a dataset recorded with a tetraplegic subject in the BCI and Tetraplegia clinical trial (NCT02550522). We started the investigation with a multilayer perceptron taking vectorized features as input. Then, we proposed convolutional neural networks (CNN), which take matrix-organized inputs approximating the spatial arrangement of the electrodes. In addition, we investigated the usefulness of long short-term memory (LSTM) to analyze temporal information. Results showed that CNN-based architectures performed better than the current state-of-the-art multilinear model on the analyzed ECoG dataset. The best architecture used a CNN-based model to analyze the spatial representation of time-frequency features followed by LSTM exploiting the sequential character of the desired hand trajectory. Compared to the multilinear model, DL-based solutions increased average cosine similarity by up to 60%.In the case of BCI, access to large datasets is limited because recordings are time-consuming and tiring. To investigate the influence of the dataset size on the decoding performance, we compared the learning curve characteristics of DL and multilinear models evaluated in the previous step. The training dataset size was gradually increased from 5 to 140 minutes of signal in different computational experiments, focusing on dataset size requirements and patient adaptation effects. Our results revealed that DL models have a learning curve profile similar to the multilinear model, increasing performance for almost all training dataset sizes. This result validates the DL-based models as a good candidate for real-life applications. We also observed increased data quality for recordings performed later, indicating improved patient ability to generate meaningful patterns.DL proved its usefulness for computer vision, primarily in the case of end-to-end learning. It enabled extracting more powerful representations trained for a specific task and removed the step of hand-crafted feature extraction. We evaluated methods using raw ECoG signals as a natural extension of hand-crafted feature analysis. In the data processing pipeline evaluated so far, continuous wavelet transform was used to extract time-frequency representation, which can be seen as a convolution between a set of wavelet filters and the ECoG signal. In this setup, the gradient w.r.t. filters coefficients can be computed, and thus the whole network can be trained within an end-to-end scheme. The parameters of wavelet filters were optimized end-to-end to see potential profit from adjusting the parameters to this specific problem. The results showed only minor or no benefit from training the wavelets in terms of cosine similarity, while end-to-end models require more computational power. This may suggest that training first layer parameters may be less beneficial and more challenging in the case of ECoG-based BCI.; Les interfaces cerveau-machines (ICMs) peuvent améliorer considérablement la qualité de vie des patients tétraplégiques en créant une voie de communication alternative et en compensant la perte de la fonction motrice. Cette thèse se concentre sur les systèmes ICM basés sur l'ECoG qui ont montré un fort potentiel pour fournir une communication efficace, tout en étant moins invasif que les enregistrements intracorticaux. Cependant, la plupart des systèmes actuels souffrent d'une faible précision de décodage. Les modèles basés sur l'apprentissage profond (DL) se sont avérés efficaces dans de nombreuses tâches et apparaissent donc comme une solution possible pour créer une représentation robuste des signaux cérébraux. Dans cette thèse, nous avons étudié le potentiel des méthodes basées sur l'apprentissage profond pour le décodage de la translation de la main à partir de signaux ECoG.Tout d'abord, nous avons évalué plusieurs modèles de DL sur des données de l'essai clinique BCI et Tétraplégie (NCT02550522) enregistrées avec un sujet tétraplégique, pour prédire la translation de la main en 3D à partir des caractéristiques temps-fréquence des signaux ECoG. Nous avons commencé l'étude avec un perceptron multicouche prenant en entrée des caractéristiques vectorisées. Ensuite, nous avons proposé des réseaux neuronaux convolutifs (CNN), qui prennent des entrées organisées en matrices se rapprochant de la disposition spatiale des électrodes, et des cellulles long short-term memory (LSTM) pour analyser les informations temporelles. Les résultats ont montré que les architectures basées sur les réseaux neuronaux convolutifs sont plus performantes que le modèle multilinéaire actuel sur l'ensemble des données ECoG analysées, avec une augmentation de la similarité cosinus moyenne allant jusqu'à 60 %. La meilleure architecture utilisait un modèle CNN pour analyser la représentation spatiale des caractéristiques temps-fréquence, suivi d'un LSTM exploitant le caractère séquentiel des données.Dans le cas des ICM, l'accès à de grands jeux de données est limité car les enregistrements sont longs et fatigants pour le patient. Pour étudier l'impact de la taille du jeu de données sur les performances de décodage, la quantité de données d'entraînement a été progressivement augmentée de 5 à 140 minutes de signal dans différentes expériences de calcul, en se concentrant sur les exigences de taille du jeu de données et les effets d'adaptation du patient. Nos résultats ont révélé que les modèles de DL augmentent les performances pour presque toutes les tailles de jeu de données d'entraînement. Ce résultat valide les modèles de DL comme bons candidats pour les applications de la vie réelle. Nous avons également observé une meilleure qualité des données pour les enregistrements effectués plus tardivement, ce qui indique un progrès du patient pour générer des signaux informatifs.Le DL a prouvé son utilité pour la vision par ordinateur, principalement dans le cas de l'apprentissage de bout en bout, permettant l'extraction de représentations plus adaptées à une tâche spécifique. Nous avons évalué des méthodes utilisant des signaux ECoG bruts en tant qu'extension naturelle de l'analyse des caractéristiques créées à la main. Les paramètres de la transformée en ondelettes continue utilisée pour extraire la représentation temps-fréquence ont été optimisés de bout en bout pour voir le bénéfice potentiel de l'ajustement des paramètres à ce problème spécifique. Les résultats ont montré qu'il n'y avait que peu ou pas de bénéfice à entraîner les ondelettes en termes de similarité en cosinus, alors que les modèles de bout en bout nécessitent plus de puissance de calcul. Cela peut suggérer que l'entraînement des paramètres de la première couche peut être moins bénéfique et plus difficile dans le cas d'une BCI basée sur l'ECoG.

Brain-computer interfaces (BCIs) transform neural signals into commands for effectors. They are mainly used as tools for functional compensation of impaired functions in disabled subjects. The Clinatec clinical trial “BCI and tetraplegia” aims at providing a proof of concept of long-term functional compensation of upper and lower limbs motor deficits in tetraplegics using a BCI. The clinical trial showed promising results for compensation of motor functions. However, the BCI’s usability remains to be improved. The decoder used to estimate the user’s intention from the processed neural signal (control decoder) must be fitted before the BCI can be used. This is done during dedicated training sessions, during which the user is directed to perform specific motor imagery tasks. Training sessions have to be held regularly in order to update the control decoder due to degradation of performances with time. This thesis proposes to limit the negative impact of training sessions. In this work, the control decoder is trained and updated using inferred labels instead of labels acquired during a dedicated training session. The labels are inferred using the output of the control decoder and neural correlates of task performance. In order to be usable in state-of-the-art motor BCIs, the adaptation process must be possible for control decoders with multiple discrete or continuous outputs. We argue that adaptation of a control decoder with multiple continuous outputs is best done using neural correlates of task performance that can be decoded continuously in time. Additionally, these neural correlates should be detected in the sensorimotor cortex due to the position of the implanted neural acquisition system. Using multiple ECoG datasets from a tetraplegic enrolled in the Clinatec BCI clinical trial, we show in this thesis that it is possible to detect such continuous in time neural correlates of task performance from the sensorimotor cortex, that control decoder labels can be inferred using these neural correlates, and finally that these labels can be used to successfully train decoders for discrete or continuous control. The subject used motor imagery to control a binary avatar (Runner MI dataset), a virtual exoskeleton with four discrete motor states (Exo dataset) or a hand-shaped cursor on a two-dimensional screen (Cursor dataset). In five-fold cross-validations, the mean and standard deviation of the area under the receiver operating characteristic curve (AUC of the ROC) for the decoder of neural correlates of task performance were 0.6225±0.0429 in the Runner MI dataset, 0.5677±0.0427 in the Exo dataset and 0.6570±0.0188 in the Cursor dataset. In a pseudo cross-validation simulating online use, the accuracy of the estimated labels was 64.9% in the Runner dataset and 64.5% in the Exo dataset. In the Cursor dataset, 63.3% of the estimated labels were less than 60° away from the actual labels. The AUC of the ROC of the control decoder was 0.6360±0.0958 in the Runner MI dataset when trained auto-adaptively compared to 0.8958±0.0153 when trained supervisedly, and to a chance level of 0.5007±0.0691. The multiclass generalization of the AUC of the ROC of the control decoder was 0.7595±0.0278 in the Exo dataset when trained auto-adaptively compared to 0.8177±0.0301 when trained supervisedly, and to a chance level of 0.5163±0.0580. In the Cursor dataset, the cosine similarity was 0.1589±0.0668 when trained auto-adaptively compared to 0.2107±0.0664 when trained in a classical supervised manner, and a chance level of -0.0231±0.0327. These results are promising for the future of auto-adaptive complex motor BCIs., Les Interfaces cerveau machine (ICMs) transforment les signaux neuronaux en commande pour des effecteurs. Elles sont principalement utilisées pour compenser les déficits d’handicapés moteurs. L’essai clinique « BCI et tétraplégie » de Clinatec a pour but the fournir une preuve de concept the compensation fonctionnelle chronique des déficits moteurs des quatre membres de tétraplégiques au moyen d’ICMs. Bien que les résultats obtenus soient prometteurs, l’usabilité des ICMs utilisées reste améliorable. Le décodeur utilisé pour transformer les signaux neuronaux en commandes (décodeur de contrôle) doit par exemple être entrainé avant de pouvoir utiliser l’ICM. Ceci est fait durant des séances d’entrainement spécifiques, durant lesquelles le patient doit effectuer des tâches d’imagerie motrices imposés. Ces séances d’entrainement constituent un temps mort et doivent être effectuées régulièrement afin de compenser les pertes de performances qui surviennent avec le temps. Cette thèse vise à limiter les effets négatifs des séances d’entrainements. Pour cela, le décodeur de contrôle est entrainé et mis à jour avec des labels estimés au lieu de labels obtenus lors de séances d’entrainement. Les labels sont estimés grâce à la sortie du décodeur de control ainsi qu’un décodeur de corrélats neuronaux de performance de tâche. Afin d’être utilisable avec les ICMs motrices les plus performantes, l’adaptation doit être faisable pour les décodeurs de contrôle qui ont plusieurs sorties continues. Nous suggérons qu’une telle adaptation requiert des corrélats neuronaux de performances de tâche acquis de manière continuelle dans le temps. Il est aussi nécessaire de détecter ces corrélats dans le cortex sensorimoteur à cause de la position des implants. Dans cette thèse, nous montrons sur plusieurs jeux de données ECoG d’un tétraplégique qu’il est possible de détecter continuellement des corrélats neuronaux de performance de tâche depuis le cortex sensorimoteur, que les labels des décodeurs de contrôle peuvent être estimés grâce à ces corrélats et enfin que ces labels peuvent être utilisés pour l’entrainement de décodeurs avec des sorties discrètes ou continues. Le patient a utilisé l’imagerie motrice pour contrôler un avatar binaire (jeu de données Runner MI), un exosquelette virtuel avec quatre états moteur discrets (jeu de données Exo) ou un curseur en deux dimensions continues sur un écran d’ordinateur (jeu de données Cursor). Lors de validations croisées à 5 blocs, la moyenne et la déviation standard de l’aire sous la courbe (AUC) ROC du décodeur de corrélats neuronaux de performance de tâche était de 0.6225±0.0429, 0.5677±0.0427 et 0.6570±0.0188 pour les jeux de données Runner MI, Exo et Cursor respectivement. Dans une simulation d’utilisation en ligne de de cette méthode d’adaptation, la précision des labels estimés était de 64.9% et 64.5% pour les jeux de données Runner MI et Exo. Pour le jeu de données Cursor, 63% des labels estimés étaient à moins de 60° d’écart des vrais labels. Dans cette simulation avec le jeu de donnée Runner MI, l’AUC de la ROC du décodeur de contrôle était de 0.6360±0.0958 quand entrainé de manière auto-adaptative, comparé à 0.8958±0.0153 pour un entrainement supervisé et à un niveau de chance de 0.5007±0.0691. Pour le jeu de données Exo, la généralisation multi classe de l’AUC de la ROC était 0.7595±0.0278 pour un entrainement auto-adaptative, comparé à 0.8177±0.0301 pour un entrainement supervisé et à un niveau de chance de 0.5163±0.0580. Pour le jeu de données Cursor, la similarité cosinus était 0.1589±0.0668 pour un entrainement auto-adaptative, comparé à 0.2107±0.0664 pour un entrainement supervisé et à un niveau de chance de -0.0231±0.0327. Ces résultats sont prometteurs pour le développement des ICMs motrices complexes auto-adaptatives.

Brain-computer interfaces (BCIs) transform neural signals into commands for effectors. They are mainly used as tools for functional compensation of impaired functions in disabled subjects. The Clinatec clinical trial “BCI and tetraplegia” aims at providing a proof of concept of long-term functional compensation of upper and lower limbs motor deficits in tetraplegics using a BCI. The clinical trial showed promising results for compensation of motor functions. However, the BCI’s usability remains to be improved. The decoder used to estimate the user’s intention from the processed neural signal (control decoder) must be fitted before the BCI can be used. This is done during dedicated training sessions, during which the user is directed to perform specific motor imagery tasks. Training sessions have to be held regularly in order to update the control decoder due to degradation of performances with time. This thesis proposes to limit the negative impact of training sessions. In this work, the control decoder is trained and updated using inferred labels instead of labels acquired during a dedicated training session. The labels are inferred using the output of the control decoder and neural correlates of task performance. In order to be usable in state-of-the-art motor BCIs, the adaptation process must be possible for control decoders with multiple discrete or continuous outputs. We argue that adaptation of a control decoder with multiple continuous outputs is best done using neural correlates of task performance that can be decoded continuously in time. Additionally, these neural correlates should be detected in the sensorimotor cortex due to the position of the implanted neural acquisition system. Using multiple ECoG datasets from a tetraplegic enrolled in the Clinatec BCI clinical trial, we show in this thesis that it is possible to detect such continuous in time neural correlates of task performance from the sensorimotor cortex, that control decoder labels can be inferred using these neural correlates, and finally that these labels can be used to successfully train decoders for discrete or continuous control. The subject used motor imagery to control a binary avatar (Runner MI dataset), a virtual exoskeleton with four discrete motor states (Exo dataset) or a hand-shaped cursor on a two-dimensional screen (Cursor dataset). In five-fold cross-validations, the mean and standard deviation of the area under the receiver operating characteristic curve (AUC of the ROC) for the decoder of neural correlates of task performance were 0.6225±0.0429 in the Runner MI dataset, 0.5677±0.0427 in the Exo dataset and 0.6570±0.0188 in the Cursor dataset. In a pseudo cross-validation simulating online use, the accuracy of the estimated labels was 64.9% in the Runner dataset and 64.5% in the Exo dataset. In the Cursor dataset, 63.3% of the estimated labels were less than 60° away from the actual labels. The AUC of the ROC of the control decoder was 0.6360±0.0958 in the Runner MI dataset when trained auto-adaptively compared to 0.8958±0.0153 when trained supervisedly, and to a chance level of 0.5007±0.0691. The multiclass generalization of the AUC of the ROC of the control decoder was 0.7595±0.0278 in the Exo dataset when trained auto-adaptively compared to 0.8177±0.0301 when trained supervisedly, and to a chance level of 0.5163±0.0580. In the Cursor dataset, the cosine similarity was 0.1589±0.0668 when trained auto-adaptively compared to 0.2107±0.0664 when trained in a classical supervised manner, and a chance level of -0.0231±0.0327. These results are promising for the future of auto-adaptive complex motor BCIs., Les Interfaces cerveau machine (ICMs) transforment les signaux neuronaux en commande pour des effecteurs. Elles sont principalement utilisées pour compenser les déficits d’handicapés moteurs. L’essai clinique « BCI et tétraplégie » de Clinatec a pour but the fournir une preuve de concept the compensation fonctionnelle chronique des déficits moteurs des quatre membres de tétraplégiques au moyen d’ICMs. Bien que les résultats obtenus soient prometteurs, l’usabilité des ICMs utilisées reste améliorable. Le décodeur utilisé pour transformer les signaux neuronaux en commandes (décodeur de contrôle) doit par exemple être entrainé avant de pouvoir utiliser l’ICM. Ceci est fait durant des séances d’entrainement spécifiques, durant lesquelles le patient doit effectuer des tâches d’imagerie motrices imposés. Ces séances d’entrainement constituent un temps mort et doivent être effectuées régulièrement afin de compenser les pertes de performances qui surviennent avec le temps. Cette thèse vise à limiter les effets négatifs des séances d’entrainements. Pour cela, le décodeur de contrôle est entrainé et mis à jour avec des labels estimés au lieu de labels obtenus lors de séances d’entrainement. Les labels sont estimés grâce à la sortie du décodeur de control ainsi qu’un décodeur de corrélats neuronaux de performance de tâche. Afin d’être utilisable avec les ICMs motrices les plus performantes, l’adaptation doit être faisable pour les décodeurs de contrôle qui ont plusieurs sorties continues. Nous suggérons qu’une telle adaptation requiert des corrélats neuronaux de performances de tâche acquis de manière continuelle dans le temps. Il est aussi nécessaire de détecter ces corrélats dans le cortex sensorimoteur à cause de la position des implants. Dans cette thèse, nous montrons sur plusieurs jeux de données ECoG d’un tétraplégique qu’il est possible de détecter continuellement des corrélats neuronaux de performance de tâche depuis le cortex sensorimoteur, que les labels des décodeurs de contrôle peuvent être estimés grâce à ces corrélats et enfin que ces labels peuvent être utilisés pour l’entrainement de décodeurs avec des sorties discrètes ou continues. Le patient a utilisé l’imagerie motrice pour contrôler un avatar binaire (jeu de données Runner MI), un exosquelette virtuel avec quatre états moteur discrets (jeu de données Exo) ou un curseur en deux dimensions continues sur un écran d’ordinateur (jeu de données Cursor). Lors de validations croisées à 5 blocs, la moyenne et la déviation standard de l’aire sous la courbe (AUC) ROC du décodeur de corrélats neuronaux de performance de tâche était de 0.6225±0.0429, 0.5677±0.0427 et 0.6570±0.0188 pour les jeux de données Runner MI, Exo et Cursor respectivement. Dans une simulation d’utilisation en ligne de de cette méthode d’adaptation, la précision des labels estimés était de 64.9% et 64.5% pour les jeux de données Runner MI et Exo. Pour le jeu de données Cursor, 63% des labels estimés étaient à moins de 60° d’écart des vrais labels. Dans cette simulation avec le jeu de donnée Runner MI, l’AUC de la ROC du décodeur de contrôle était de 0.6360±0.0958 quand entrainé de manière auto-adaptative, comparé à 0.8958±0.0153 pour un entrainement supervisé et à un niveau de chance de 0.5007±0.0691. Pour le jeu de données Exo, la généralisation multi classe de l’AUC de la ROC était 0.7595±0.0278 pour un entrainement auto-adaptative, comparé à 0.8177±0.0301 pour un entrainement supervisé et à un niveau de chance de 0.5163±0.0580. Pour le jeu de données Cursor, la similarité cosinus était 0.1589±0.0668 pour un entrainement auto-adaptative, comparé à 0.2107±0.0664 pour un entrainement supervisé et à un niveau de chance de -0.0231±0.0327. Ces résultats sont prometteurs pour le développement des ICMs motrices complexes auto-adaptatives.

Brain-computer interfaces (BCIs) are systems that allow the control of external devices from the brain’s neural signals without neuromuscular activation. Among the various applications, functional compensation and rehabilitation of individuals suffering from severe motor disabilities (with motor BCIs) has always been a focus for BCI research. Brain signals are translated, through signal processing steps, into orders realized by an effector which returns feedbacks (visual, tactile, proprioceptive…) to the patient giving him back some mobility and autonomy. Nevertheless, numerous challenges to translate BCI from offline experiments based on healthy subjects recordings to daily life applications for disable patients. Even though BCI decoding highlights good control performance during specific task such as center out experiments, the development of decoder for online asynchronous decoding, stable during long period, is still one of the BCI community claim. Moreover, a lack of studies and algorithms on multi-limb effector control were highlighted.Relying on the “BCI and Tetraplegia” clinical trial of CEA/LETI/CLINATEC, the development of new decoders for real-time closed-loop adaptive asynchronous multi-limb is addressed in the present doctoral thesis. Recursive exponentially weighted Markov switching multi-linear model (REW-MSLM) was designed to handle complex / high dimensional multi-limb effector control with online closed-loop calibration of the decoding model.Based on a mixture of expert architecture, REW-MSLM allows a tetraplegic patient who underwent bilateral epidural electrocorticographic (ECoG) arrays implantation of chronic wireless implants (WIMAGINE) 8D control of a whole body exoskeleton over several months without model recalibration. The patient was able to perform alternative 3D left and right hand translations and 1D left and right wrist rotations with high accuracy and during long period without any model recalibration. Experiments with higher controlled dimensions and other effectors such as wheelchair have also been tested and highlighted promising results. This PhD thesis aims to present new innovative adaptive BCI decoder adapted to multi-limb decoding for clinical applications and highlights the interest of such decoder in the perspective of the current state-of-the-art., Les interfaces cerveau-Machine (ICM) sont des systèmes permettant de traduire l’activité cérébrale d’un individu via un ordinateur afin d’effectuer des tâches nécessitant normalement une action des nerfs périphériques et/ou musculaires. En se basant sur l’essai clinique de CEA/LETI/CLINATEC, nommé « BCI et Tétraplégie », les recherches de thèse présentées dans ce manuscrit répondent aux challenges de l’intégration ICM dans la vie quotidienne, à savoir le contrôle complexe asynchrone d’effecteurs multi-membres et l’adaptation en temps réel des algorithmes de décodage durant des expériences en closed-loop. Pour répondre aux challenges de l’essai clinique, l’algorithme incrémental adapatif en temps réel Recursive exponentially weighted Markov switching multi-linear model (REW-MSLM) a été créé. L’algorithme REWMSLM repose sur une architecture du type Mixture d’Experts (ME). Les MEs combinent plusieurs décodeurs continus dit «experts » dont les prédictions sont pondérées par un modèle discretnommé «gate » . Les modèles des experts sont estimés via l’algorithme REW-NPLS tandis que l emodèle de gate est un modèle de markov caché (HMM). L’algorithme REW-MSLM a permis à un patient tétraplégique de contrôler un exosquelette 4 membres avec 8 degrés de liberté via ledécodage de signaux électrocorticographiques (ECoG) enregistrés avec deux implants épiduraux sans fil nommé WIMAGINE. Dans une même expérience, avec un même modèle, le patient aréalisé des tâches de contrôle alternatif des mouvements du bras gauche et du bras droit dansl’espace 3D et de rotation 1D des poignets gauche et droit. Le contrôle du patient durant ces tâches est resté très stable, même durant des expériences 6 mois après la fin de la mise à jour du modèle.Par exemple, les performances de contrôle réalisées 0 à 37 jours après la dernière mise à jour dumodèle ont montré un score de réussite de 71 ± 12% et 99 ± 2% pour les tâches de contrôle 3Ddes mouvements des mains et les tâches de contrôle 1D de rotation des poignets. Lesperformances de contrôle réalisées de 0 à 167 jours après la dernière mise à jour du modèle ontmontré un score de réussite de 67 ± 21% et 93 ± 12% pour les tâches de contrôle 3D desmouvements des mains et les tâches de contrôle 1D de rotation des poignets. Dans le but d’êtreintégré dans REW-MSLM, de nouveaux algorithmes, à savoir le (Automatic) Penalized RecursiveExponentially Weighted N-way Partial Least Squares (PREW-NPLS et APREW-NPLS) et uneversion du Modèle de Markov Caché Hiérarchique (H2M2), ont été imaginés et testés en offline.Lp-PREW-NPLS est un algorithme incrémental adaptatif permettant une pénalisation pargroupes du modèle de décodage suivant la norme/pseudo norme Lp=0, 0.5 ou 1. Pour uneutilisation en temps réel, l’algorithme PREW-NPLS nécessite de déterminer avec une étudeoffline préliminaire l’hyperparamètre de pénalisation optimal. APREW-NPLS permet de comparer en temps réel plusieurs modèles avec des hyperparamètres de pénalisation différents afin de l’optimiser en temps réel durant l’expérience. Les algorithmes PREW-NPLS et APREWNPLSont été créés pour réduire la dimension de l’espace des caractéristiques et améliorer lesperformances de décodage. H2M2, quant à lui est un classifieur dynamique similaire aux modèles de type HMM mais avec une structure hiérarchique. La structure hiérarchique est répartie encouches avec les états des couches inférieures dépendants des états des couches supérieures.L’algorithme H2M2 a été conçu dans le but d’améliorer la réactivité du modèle de classification de REW-MSLM (gating). Les résultats de décodage neural direct des signaux épiduraux ECoG obtenus nous poussent à diversifier l’utilisation de ces algorithmes à d’autres domaines.

Brain-computer interfaces (BCIs) are systems that allow the control of external devices from the brain’s neural signals without neuromuscular activation. Among the various applications, functional compensation and rehabilitation of individuals suffering from severe motor disabilities (with motor BCIs) has always been a focus for BCI research. Brain signals are translated, through signal processing steps, into orders realized by an effector which returns feedbacks (visual, tactile, proprioceptive…) to the patient giving him back some mobility and autonomy. Nevertheless, numerous challenges to translate BCI from offline experiments based on healthy subjects recordings to daily life applications for disable patients. Even though BCI decoding highlights good control performance during specific task such as center out experiments, the development of decoder for online asynchronous decoding, stable during long period, is still one of the BCI community claim. Moreover, a lack of studies and algorithms on multi-limb effector control were highlighted.Relying on the “BCI and Tetraplegia” clinical trial of CEA/LETI/CLINATEC, the development of new decoders for real-time closed-loop adaptive asynchronous multi-limb is addressed in the present doctoral thesis. Recursive exponentially weighted Markov switching multi-linear model (REW-MSLM) was designed to handle complex / high dimensional multi-limb effector control with online closed-loop calibration of the decoding model.Based on a mixture of expert architecture, REW-MSLM allows a tetraplegic patient who underwent bilateral epidural electrocorticographic (ECoG) arrays implantation of chronic wireless implants (WIMAGINE) 8D control of a whole body exoskeleton over several months without model recalibration. The patient was able to perform alternative 3D left and right hand translations and 1D left and right wrist rotations with high accuracy and during long period without any model recalibration. Experiments with higher controlled dimensions and other effectors such as wheelchair have also been tested and highlighted promising results. This PhD thesis aims to present new innovative adaptive BCI decoder adapted to multi-limb decoding for clinical applications and highlights the interest of such decoder in the perspective of the current state-of-the-art., Les interfaces cerveau-Machine (ICM) sont des systèmes permettant de traduire l’activité cérébrale d’un individu via un ordinateur afin d’effectuer des tâches nécessitant normalement une action des nerfs périphériques et/ou musculaires. En se basant sur l’essai clinique de CEA/LETI/CLINATEC, nommé « BCI et Tétraplégie », les recherches de thèse présentées dans ce manuscrit répondent aux challenges de l’intégration ICM dans la vie quotidienne, à savoir le contrôle complexe asynchrone d’effecteurs multi-membres et l’adaptation en temps réel des algorithmes de décodage durant des expériences en closed-loop. Pour répondre aux challenges de l’essai clinique, l’algorithme incrémental adapatif en temps réel Recursive exponentially weighted Markov switching multi-linear model (REW-MSLM) a été créé. L’algorithme REWMSLM repose sur une architecture du type Mixture d’Experts (ME). Les MEs combinent plusieurs décodeurs continus dit «experts » dont les prédictions sont pondérées par un modèle discretnommé «gate » . Les modèles des experts sont estimés via l’algorithme REW-NPLS tandis que l emodèle de gate est un modèle de markov caché (HMM). L’algorithme REW-MSLM a permis à un patient tétraplégique de contrôler un exosquelette 4 membres avec 8 degrés de liberté via ledécodage de signaux électrocorticographiques (ECoG) enregistrés avec deux implants épiduraux sans fil nommé WIMAGINE. Dans une même expérience, avec un même modèle, le patient aréalisé des tâches de contrôle alternatif des mouvements du bras gauche et du bras droit dansl’espace 3D et de rotation 1D des poignets gauche et droit. Le contrôle du patient durant ces tâches est resté très stable, même durant des expériences 6 mois après la fin de la mise à jour du modèle.Par exemple, les performances de contrôle réalisées 0 à 37 jours après la dernière mise à jour dumodèle ont montré un score de réussite de 71 ± 12% et 99 ± 2% pour les tâches de contrôle 3Ddes mouvements des mains et les tâches de contrôle 1D de rotation des poignets. Lesperformances de contrôle réalisées de 0 à 167 jours après la dernière mise à jour du modèle ontmontré un score de réussite de 67 ± 21% et 93 ± 12% pour les tâches de contrôle 3D desmouvements des mains et les tâches de contrôle 1D de rotation des poignets. Dans le but d’êtreintégré dans REW-MSLM, de nouveaux algorithmes, à savoir le (Automatic) Penalized RecursiveExponentially Weighted N-way Partial Least Squares (PREW-NPLS et APREW-NPLS) et uneversion du Modèle de Markov Caché Hiérarchique (H2M2), ont été imaginés et testés en offline.Lp-PREW-NPLS est un algorithme incrémental adaptatif permettant une pénalisation pargroupes du modèle de décodage suivant la norme/pseudo norme Lp=0, 0.5 ou 1. Pour uneutilisation en temps réel, l’algorithme PREW-NPLS nécessite de déterminer avec une étudeoffline préliminaire l’hyperparamètre de pénalisation optimal. APREW-NPLS permet de comparer en temps réel plusieurs modèles avec des hyperparamètres de pénalisation différents afin de l’optimiser en temps réel durant l’expérience. Les algorithmes PREW-NPLS et APREWNPLSont été créés pour réduire la dimension de l’espace des caractéristiques et améliorer lesperformances de décodage. H2M2, quant à lui est un classifieur dynamique similaire aux modèles de type HMM mais avec une structure hiérarchique. La structure hiérarchique est répartie encouches avec les états des couches inférieures dépendants des états des couches supérieures.L’algorithme H2M2 a été conçu dans le but d’améliorer la réactivité du modèle de classification de REW-MSLM (gating). Les résultats de décodage neural direct des signaux épiduraux ECoG obtenus nous poussent à diversifier l’utilisation de ces algorithmes à d’autres domaines.

Brain-Computer Interfaces (BCI) are systems that allow severely motor-impaired patients to use their brain activity to control external devices, for example upper-limb prostheses in the case of motor BCIs. The user's intentions are estimated by applying a decoder on neural features extracted from the user's brain activity. Signal processing challenges specific to the clinical deployment of motor BCI systems are addressed in the present doctoral thesis, namely asynchronous mono-limb or sequential multi-limb decoding and accurate decoding during active control states. A switching decoder, namely a Markov Switching Linear Model (MSLM), has been developed to limit spurious system activations, to prevent parallel limb movements and to accurately decode complex movements.The MSLM associates linear models with different possible control states, e.g. activation of a specific limb, specific movement phases. Dynamic state detection is performed by the MSLM, and the probability of each state is used to weight the linear models. The performance of the MSLM decoder was assessed for asynchronous wrist and multi-finger trajectory reconstruction from electrocorticographic signals. It was found to outperform previously reported decoders for the limitation of spurious activations during no-control periods and permitted to improve decoding accuracy during active periods., Les Interfaces Cerveau-Machine (ICM) sont des systèmes qui permettent à des patients souffrant d'un handicap moteur sévère d'utiliser leur activité cérébrale pour contrôler des effecteurs, par exemple des prothèses des membres supérieurs dans le cas d'ICM motrices. Les intentions de mouvement de l'utilisateur sont estimées en appliquant un décodeur sur des caractéristiques extraites de son activité cérébrale. Des challenges spécifiques au déploiement clinique d'ICMs motrices ont été considérés, à savoir le contrôle mono-membre ou séquentiel multi-membre asynchrone et précis. Un décodeur, le Markov Switching Linear Model (MSLM), a été développé pour limiter les activations erronées de l'ICM, empêcher des mouvements parallèles des effecteurs et décoder avec précision des mouvements complexes. Le MSLM associe des modèles linéaires à différents états possibles, e.g. le contrôle d'un membre spécifique ou une phase de mouvement particulière. Le MSLM réalise une détection d'état dynamique, et les probabilités des états sont utilisées pour pondérer les modèles linéaires.La performance du décodeur MSLM a été évaluée pour la reconstruction asynchrone de trajectoires de poignet et de doigts à partir de signaux electrocorticographiques. Il a permis de limiter les activations erronées du système et d'améliorer la précision du décodage du signal cérébral.

Brain Computer Interface (BCI) is a system for translation of brain neural activity into commands for external devices. This study was undertaken as a step toward the fully autonomous (self-paced) BCI functioning in natural environment which is of crucial importance for BCI clinical applications. To model the natural environment binary self-paced BCI experiments were carried out in freely moving animals. In comparison to the previous works, the long-term experimental sessions were carried out, which better comply with the real-life applications requirements. The main goal of the study was to discriminate the specific neuronal pattern related to the animal's control action against background brain activity of freely-moving animal. To achieve the necessary level of selectivity the Multi-Way Analysis was chosen since it provides a simultaneous signal processing in several domains, namely, temporal, frequency and spatial. To improve the capacity of the generic Multy-Way PLS approach for treatment of high-dimensional data, the Iterative NPLS algorithm is introduced in the current study. Having lower memory requirements it provides huge datasets treatment, allows high resolution, preserves the accuracy of the generic algorithm, and demonstrates better robustness. For adaptive calibration of BCI system the Recursive NPLS algorithm is proposed. Finally, the Penalized NPLS algorithm is developed for effective selection of feature subsets, namely, for subset of electrodes. The proposed algorithms were tested on artificial and real datasets. They demonstrated performance which either suppress or is comparable with one of the generic NPLS algorithm. Their computational efficiency is acceptable for the real-time applications. Developed algorithms were applied for calibration of the BCI system and were used in the real-time close-loop binary BCI experiments in animals. The proposed methods represent a prospective approach for further development of a human BCI system.; Une Interface Cerveau-Machine (ICM) est un système permettant de transformer l'activité neurale du cerveau en une commande d'effecteurs externes. Cette étude correspond à une étape vers une ICM totalement autonome fonctionnant dans un environnement naturel ce qui est d'une importance cruciale pour les futures applications cliniques d'une ICM. Pour représenter l'environnement naturel, des expériences avec une ICM binaire asynchrone ont été réalisées avec des animaux libres de se mouvoir. En comparaison avec les études précédentes, des expériences sur le long terme ont été réalisées, ce qui est plus conforme aux exigences des applications de la vie réelle. L'objectif principal de cette étude est de différencier le modèle spécifique neuronal lié à l'intention d'action de l'activité de fond du cerveau chez des animaux libres de tous mouvements. Pour atteindre le niveau nécessaire de sélectivité, l'analyse Multi-Voies PLS a été choisie sachant qu'elle fournit simultanément un traitement du signal dans plusieurs domaines, à savoir, temporel, fréquentiel et spatial. Pour améliorer la capacité de l'approche générique Multi-Voies PLS pour le traitement de données à grandes dimensions, l'algorithme « Itérative NPLS » est introduit dans notre travail. En ayant des besoins plus faibles en mémoire, cet algorithme fournit des traitements de grands ensembles de données, permet une résolution élevée, préserve l'exactitude de l'algorithme générique et démontre une meilleure robustesse. Pour la calibration adaptative d'un système ICM, l'algorithme récursif NPLS est proposé. Finalement, l'algorithme pénalisé NPLS est développé pour la sélection efficace d'un sous-ensemble de fonctions, à savoir, un sous-ensemble d'électrodes. Les algorithmes proposés ont été testés sur des ensembles de données artificielles et réelles. Ils ont démontré une performance qui est comparable à celle d'un algorithme générique NPLS. Leur efficacité de calcul est acceptable pour les applications en temps réel. Les algorithmes développés ont été appliqués à la calibration d'un système ICM et ont été utilisés dans des expériences d'ICM avec bouclage en temps réel chez des animaux. Enfin, les méthodes proposées représentent une approche prospective pour de futurs développements de systèmes ICM humains.

Modular architecture is a hallmark of many brain circuits. Particularly, in the cerebral cortex it has been observed that reciprocal connections are often present between functionally interconnected areas that are hierarchically organized. Evolutionary development is another distinctive characteristic of living species, even the simplest viruses are capable to adapt to better fit new environmental conditions. Having hierarchical architectures and evolutionary features in mind, we build unique and novel simulation framework, which allows us to model and to study evolving hierarchically organized circuits of modules of spiking neural networks. Each module is characterized by embedded neural development and expression of spike timing dependent plasticity. Cell death, synaptic plasticity and projection pruning, embedded in the neural model, drive the build-up of auto-associative links within each module, which generate an areal activity that reflect the changes in the corresponding functional connectivity within and between neuronal modules. Bio-electric activity of each module is recorded by means of virtual electrodes and these signals, called electrochipograms (EChG), are analyzed by time and frequency domain methods in order to find general patterns of emerging behavior. Beside time and frequency domain analysis methods, a novel robust non-linear structural regression approach is proposed to provide researchers with more powerful tools specially adapted to the data typically used in the domain. We tested the effect of an external stimulus at fixed frequency fed to a sensory module, which pro jecting its activity to two hierarchically organized parallel pathways. We found that modeled circuits manifest behavior similar in certain aspects to that of real brains. We show evidence that all networks of modules are able to maintain long patterns of activity associated with the stimulus offset. These findings bring new insights to the understanding of EEG-like signals, both real and virtual. The findings prove that the approach is successful and could be extended to model cognitive and behavioral processes in the brains., L'architecture modulaire est une caractéristique distinctive des circuits cérébraux. En particulier, il a été observé l'existence de connexions réciproques entre des zones fonctionnellement interconnectées dans le cortex, et qui par ailleurs sont hiérarchiquement organisées. De plus, le développement évolutif est une autre caractéristique distinctive des espèces vivantes ; même les virus sont capables d'adaptation pour mieux répondre à de nouvelles conditions environnementales. En tenant compte de ces deux importants aspects, nous avons construit un nouvel et unique outil de simulation permettant de modéliser et d'étudier l'évolution des circuits multi-modulaires hiérarchiques. Dans ce modèle, chaque module est représenté par des réseaux de neurones impulsionels et caractérisé à la fois par des changements d'activités neurales imbriquées et par la plasticité synaptique. La morte cellulaire, la plasticité synaptique et l'apoptose intégrés dans le modèle créent des liens auto-associatifs au sein des modules. Ces liens peuvent générer une activité zonale qui reflète l'évolution de la connectivité fonctionnelle à l'intérieur comme à l'extérieur des modules, et donc entre les plusieurs modules neuronaux. L'activité bioélectrique de chaque module est enregistrée au moyen des électrodes virtuelles. Les signaux, electrochipogrammes (EChG), sont analysés par les méthodes fréquentiels et les méthodes de potentiels évoqués afin de trouver des généralités dans le comportement émergeant. En plus de ces méthodes conventionnelles, nous proposons une nouvelle approche de régression non-linéaire structurelle afin de fournir des outils plus puissants et mieux adaptés aux données habituellement analysées dans ce domaine. Nous avons donc testé l'effet d'un stimulus externe sur le développement de liens fonctionnels d'un réseau neuronaux. Le circuit est structuré hiérarchiquement avec un unique module sensoriel et d'autres modules constitués de deux voies parallèles organisées aussi de façon hiérarchique. Nos résultats montrent que les circuits modélisés manifestent un comportement similaire que les circuits biologiques réels. En particulier, tous les éléments du circuit peuvent traiter et maintenir des patterns d'activité liés à la disparition du stimulus. Les résultats obtenus dans nos expériences apportent un éclairage sur les processus émergents et coordonnés de l'activité électrique enregistrée par des EEG de circuits inter-corticaux hiérarchiques et évolutifs qui sont artificiels ou réels. Plus généralement, notre approche concernant les signaux EEG pourrait être étendue à la modélisation d'une vaste variété des processus cognitifs et comportementaux.