Search

Pneumatic artificial muscle (PAM) has been widely used in rehabilitation and other fields as a flexible and safe actuator. In this paper, a PAM-actuated wearable exoskeleton robot is developed for upper limb rehabilitation. However, accurate modeling and control of the PAM are difficult due to complex hysteresis. To solve this problem, this paper proposes an active neural network method for hysteresis compensation, where a neural network (NN) is utilized as the hysteresis compensator and unscented Kalman filtering is used to estimate the weights and approximation error of the NN in real time. Compared with other inversion-based methods, the NN is directly used as the hysteresis compensator without needing inversion. Additionally, the proposed method does not require pre-training of the NN since the weights can be dynamically updated. To verify the effectiveness and robustness of the proposed method, a series of experiments have been conducted on the self-built exoskeleton robot. Compared with other popular control methods, the proposed method can track the desired trajectory faster, and tracking accuracy is gradually improved through iterative learning and updating.

Early diagnosis and intervention of depression promote complete recovery, with its traditional clinical assessments depending on the diagnostic scales, clinical experience of doctors and patient cooperation. Recent researches indicate that functional near-infrared spectroscopy (fNIRS) based on deep learning provides a promising approach to depression diagnosis. However, collecting large fNIRS datasets within a standard experimental paradigm remains challenging, limiting the applications of deep networks that require more data. To address these challenges, in this paper, we propose an fNIRS-driven depression recognition architecture based on cross-modal data augmentation (fCMDA), which converts fNIRS data into pseudo-sequence activation images. The approach incorporates a time-domain augmentation mechanism, including time warping and time masking, to generate diverse data. Additionally, we design a stimulation task-driven data pseudo-sequence method to map fNIRS data into pseudo-sequence activation images, facilitating the extraction of spatial-temporal, contextual and dynamic characteristics. Ultimately, we construct a depression recognition model based on deep classification networks using the imbalance loss function. Extensive experiments are performed on the two-class depression diagnosis and five-class depression severity recognition, which reveal impressive results with accuracy of 0.905 and 0.889, respectively. The fCMDA architecture provides a novel solution for effective depression recognition with limited data.

In recent years, the steady-state visual evoked potentials (SSVEP) based brain control method has been employed to help people with disabilities because of its advantages of high information transmission rate and low training time. However, the existing SSVEP brain control methods cannot adapt to dynamic or unstructured environments. Moreover, the recognition accuracy from the conventional decoding algorithm still needs to improve. To address the above problems, this study proposed a steady-state hybrid visual evoked potentials (SSHVEP) paradigm using the grasping targets in their environment to improve the connection between the subjects' and their dynamic environments. Moreover, a novel EEG decoding method, using the multivariate variational mode decomposition (MVMD) algorithm for adaptive sub-band division and convolutional neural network (CNN) for target recognition, was applied to improve the decoding accuracy of the SSHVEPs. 18 subjects participated in the offline and online experiments. The offline accuracy across 18 subjects by the 9-target SSHVEP paradigm was up to 95.41 ± 2.70 %, which is a 5.80% improvement compared to the conventional algorithm. To further validate the performance of the proposed method, the brain-controlled grasping robot system using the SSHVEP paradigm was built. The average accuracy reached 93.21 ± 10.18 % for the online experiment. All the experimental results demonstrated the effectiveness of the brain-computer interaction method based on the SSHVEP paradigm and the MVMD combined CNN algorithm studied in this paper.

The steady-state visual evoked potential (SSVEP) has become one of the most prominent BCI paradigms with high information transfer rate, and has been widely applied in rehabilitation and assistive applications. This paper proposes a least-square (LS) unified framework to summarize the correlation analysis (CA)-based SSVEP spatial filtering methods from a machine learning perspective. Within this framework, the commonalities and differences between various spatial filtering methods appear apparent, the interpretation of computational factors becomes intuitive, and spatial filters can be determined by solving a generalized optimization problem with non-linear and regularization items. Moreover, the proposed LS framework provides the foundation of utilizing the knowledge behind these spatial filtering methods in further classification/regression model designs. Through a comparative analysis of existing representative spatial filtering methods, recommendations are made for the superior and robust design strategies. These recommended strategies are further integrated to fill the research gaps and demonstrate the ability of the proposed LS framework to promote algorithmic improvements, resulting in five new spatial filtering methods. This study could offer significant insights in understanding the relationships between various design strategies in the spatial filtering methods from the machine learning perspective, and would also contribute to the development of the SSVEP recognition methods with high performance.

Timely diagnosis of mild traumatic brain injury (mTBI) remains challenging due to the rapid recovery of acute symptoms and the absence of evidence of injury in static neuroimaging scans. Furthermore, while longitudinal tracking of mTBI is essential in understanding how the diseases progresses/regresses over time for enhancing personalized patient care, a standardized approach for this purpose is not yet available. Recent functional neuroimaging studies have provided evidence of brain function alterations following mTBI, suggesting mTBI-detection models can be built based on these changes. Most of these models, however, rely on manual feature engineering, but the optimal set of features for detecting mTBI may be unknown. Data-driven approaches, on the other hand, may uncover hidden relationships in an automated manner, making them suitable for the problem of mTBI detection. This paper presents a data-driven framework based on Siamese Convolutional Neural Network (SCNN) to detect mTBI and to monitor the recovery state from mTBI over time. The proposed framework is tested on the cortical images of Thy1-GCaMP6s mice, obtained via widefield calcium imaging, acquired in a longitudinal study. Results show that the proposed model achieves a classification accuracy of 96.5%. To track the state of the injured brain over time, a reference distance map is constructed, which together with the SCNN model, are employed to assess the recovery state in subsequent sessions after injury, revealing that the recovery progress varies among subjects. The promising results of this work suggest that a similar approach could be potentially applicable for monitoring recovery from mTBI, in humans.

Various measures have been proposed to quantify upper-limb use through wrist-worn inertial measurement units. The two most popular traditional measures of upper-limb use - thresholded activity counts (TAC) and the gross movement (GM) score suffer from high sensitivity and low specificity, and vice versa. We previously proposed a hybrid version of these two measures - the GMAC - that showed better overall detection performance than TAC and GM. In this paper, we answer two critical questions to improve the GMAC measure's usefulness: (a) can it be implemented using only the accelerometer data? (b) what are its optimal parameter values? Here, we propose a modified GMAC using only the accelerometer data and optimize its parameters to develop: (a) a generic measure that is both limb- and subject-independent, and (b) limb-specific measures that were only subject-independent. The optimized GMAC showed better detection performance than the previous GMAC and surprisingly had comparable performance to the best-performing machine learning-based measure (random forest inter-subject model). In hemiparetic data, its performance was similar to the previous GMAC and the random forest inter-subject model; the limb-specific GMAC measure, however, had a better performance than the generic measure. The optimized limb-specific GMAC is a simple, interpretable alternative to a machine learning-based inter-subject model. The optimized GMAC can be a valuable measure for offline or real-time detection and feedback of upper limb use. The preliminary results of this study, based on a small dataset, need to be validated on a larger dataset to evaluate its generalizability.

Many challenges exist in the study of using orthotics, exoskeletons or exosuits as tools for rehabilitation and assistance of healthy people in daily activities due to the requirements of portability and safe interaction with the user and the environment. One approach to dealing with these challenges is to design a control system that can be deployed in a portable device to identify the relationships that exist between the gait variables and gait cycle for different locomotion modes. In order to estimate the knee and ankle angles in the sagittal plane for different locomotion modes, a novel multimodal feature-decoupled kinematic estimation system consisting of a multimodal locomotion classifier and an optimal joint angle estimator is proposed in this paper. The multi-source information output from different conventional primary models are fused by assigning the non-fixed weight. To improve the performance of the primary models, a data augmentation module based on the time-frequency domain analysis method is designed. The results show that the inclusion of the data augmentation module and multi-source information fusion modules has improved the classification accuracy to 98.56% and kinematic estimation performance (PCC) to 0.904 (walking), 0.956 (running), 0.899 (stair ascent), 0.851 (stair descent), respectively. The kinematic estimation quality is generally higher for faster speed (running) or proximal joint (knee) compared to other modes and ankle. The limitations and advantages of the proposed approach are discussed. Based on our findings, the multimodal kinematic estimation system has potential in facilitating the deployment for human-in-loop control of lower-limb intelligent assistive devices.

Modeling the muscle response to functional electrical stimulation (FES) is an important step during model-based FES control system design. The Hammerstein structure is widely used in simulating this nonlinear biomechanical response. However, a fixed relationship cannot cope well with the time-varying property of muscles and muscle fatigue. In this paper, we proposed an adaptive Hammerstein model to predict ankle joint torque induced by electrical stimulation, which used variable forgetting factor recursive least squares (VFFRLS) method to update the model parameters. To validate the proposed model, ten healthy individuals were recruited for short-duration FES experiments, ten for long-duration FES experiments, and three stroke patients for both. The isometric ankle dorsiflexion torque induced by FES was measured, and then the test performance of the fixed-parameter Hammerstein model, the adaptive Hammerstein model based on fixed forgetting factor recursive least squares (FFFRLS) and the adaptive Hammerstein model based on VFFRLS was compared. The goodness of fit, root mean square error, peak error and success rate were applied to evaluate the accuracy and stability of the model. The results indicate a significant improvement in both the accuracy and stability of the proposed adaptive model compared to the fixed-parameter model and the adaptive model based on FFFRLS. The proposed adaptive model enhances the ability of the model to cope with muscle changes.

The olfactory system enables humans to smell different odors, which are closely related to emotions. The high temporal resolution and non-invasiveness of Electroencephalogram (EEG) make it suitable to objectively study human preferences for odors. Effectively learning the temporal dynamics and spatial information from EEG is crucial for detecting odor-induced emotional valence. In this paper, we propose a deep learning architecture called Temporal Attention with Spatial Autoencoder Network (TASA) for predicting odor-induced emotions using EEG. TASA consists of a filter-bank layer, a spatial encoder, a time segmentation layer, a Long Short-Term Memory (LSTM) module, a multi-head self-attention (MSA) layer, and a fully connected layer. We improve upon the previous work by utilizing a two-phase learning framework, using the autoencoder module to learn the spatial information among electrodes by reconstructing the given input with a latent representation in the spatial dimension, which aims to minimize information loss compared to spatial filtering with CNN. The second improvement is inspired by the continuous nature of the olfactory process; we propose to use LSTM-MSA in TASA to capture its temporal dynamics by learning the intercorrelation among the time segments of the EEG. TASA is evaluated on an existing olfactory EEG dataset and compared with several existing deep learning architectures to demonstrate its effectiveness in predicting olfactory-triggered emotional responses. Interpretability analyses with DeepLIFT also suggest that TASA learns spatial-spectral features that are relevant to olfactory-induced emotion recognition.

Robot-assisted motor training is applied for neurorehabilitation in stroke patients, using motor imagery (MI) as a representative paradigm of brain-computer interfaces to offer real-life assistance to individuals facing movement challenges. However, the effectiveness of training with MI may vary depending on the location of the stroke lesion, which should be considered. This paper introduces a multi-task electroencephalogram-based heterogeneous ensemble learning (MEEG-HEL) specifically designed for cross-subject training. In the proposed framework, common spatial patterns were used for feature extraction, and the features according to stroke lesions are shared and selected through sequential forward floating selection. The heterogeneous ensembles were used as classifiers. Nine patients with chronic ischemic stroke participated, engaging in MI and motor execution (ME) paradigms involving finger tapping. The classification criteria for the multi-task were established in two ways, taking into account the characteristics of stroke patients. In the cross-subject session, the first involved a direction recognition task for two-handed classification, achieving a performance of 0.7419 (±0.0811) in MI and 0.7061 (±0.1270) in ME. The second task focused on motor assessment for lesion location, resulting in a performance of 0.7457 (±0.1317) in MI and 0.6791 (±0.1253) in ME. Comparing the specific-subject session, except for ME on the motor assessment task, performance on both tasks was significantly higher than the cross-subject session. Furthermore, classification performance was similar to or statistically higher in cross-subject sessions compared to baseline models. The proposed MEEG-HEL holds promise in improving the practicality of neurorehabilitation in clinical settings and facilitating the detection of lesions.

As an essential cognitive function, attention has been widely studied and various indices based on EEG have been proposed for its convenience and easy availability for real-time attention monitoring. Although existing indices based on spectral power of empirical frequency bands are able to describe the attentional state in some way, the reliability still needs to be improved. This paper proposed a subject-specific attention index based on the weighted spectral power. Unlike traditional indices, the ranges of frequency bands are not empirical but obtained from subject-specific change patterns of spectral power of electroencephalograph (EEG) to overcome the great inter-subject variance. In addition, the contribution of each frequency component in the frequency band is considered different. Specifically, the ratio of power spectral density (PSD) function in attentional and inattentional state is utilized to calculate the weight to enhance the effectiveness of the proposed index. The proposed subject-specific attention index based on the weighted spectral power is evaluated on two open datasets including EEG data of a total of 44 subjects. The results of the proposed index are compared with 3 traditional attention indices using various statistical analysis methods including significance tests and distribution variance measurements. According to the experimental results, the proposed index can describe the attentional state more accurately. The proposed index respectively achieves accuracies of 86.21% and 70.00% at the 1% significance level in both the t-test and Wilcoxon rank-sum test for two datasets, which obtains improvements of 41.38% and 20.00% compared to the best result of the traditional indices. These results indicate that the proposed index provides an efficient way to measure attentional state.

The existing surface electromyography-based pattern recognition system (sEMG-PRS) exhibits limited generalizability in practical applications. In this paper, we propose a stacked weighted random forest (SWRF) algorithm to enhance the long-term usability and user adaptability of sEMG-PRS. First, the weighted random forest (WRF) is proposed to address the issue of imbalanced performance in standard random forests (RF) caused by randomness in sampling and feature selection. Then, the stacking is employed to further enhance the generalizability of WRF. Specifically, RF is utilized as the base learner, while WRF serves as the meta-leaning layer algorithm. The SWRF is evaluated against classical classification algorithms in both online experiments and offline datasets. The offline experiments indicate that the SWRF achieves an average classification accuracy of 89.06%, outperforming RF, WRF, long short-term memory (LSTM), and support vector machine (SVM). The online experiments indicate that SWRF outperforms the aforementioned algorithms regarding long-term usability and user adaptability. We believe that our method has significant potential for practical application in sEMG-PRS.

Electroencephalogram (EEG) signals are widely utilized in the field of cognitive workload decoding (CWD). However, when the recognition scenario is shifted from subject-dependent to subject-independent or spans a long period, the accuracy of CWD deteriorates significantly. Current solutions are either dependent on extensive training datasets or fail to maintain clear distinctions between categories, additionally lacking a robust feature extraction mechanism. In this paper, we tackle these issues by proposing a Bi-Classifier Joint Domain Adaptation (BCJDA) model for EEG-based cross-time and cross-subject CWD. Specifically, the model consists of a feature extractor, a domain discriminator, and a Bi-Classifier, containing two sets of adversarial processes for domain-wise alignment and class-wise alignment. In the adversarial domain adaptation, the feature extractor is forced to learn the common domain features deliberately. The Bi-Classifier also fosters the feature extractor to retain the category discrepancies of the unlabeled domain, so that its classification boundary is consistent with the labeled domain. Furthermore, different adversarial distance functions of the Bi-Classifier are adopted and evaluated in this model. We conduct classification experiments on a publicly available BCI competition dataset for recognizing low, medium, and high cognitive workload levels. The experimental results demonstrate that our proposed BCJDA model based on cross-gradient difference maximization achieves the best performance.

Physical therapists play a crucial role in guiding patients through effective and safe rehabilitation processes according to medical guidelines. However, due to the therapist-patient imbalance, it is neither economical nor feasible for therapists to provide guidance to every patient during recovery sessions. Automated assessment of physical rehabilitation can help with this problem, but accurately quantifying patients' training movements and providing meaningful feedback poses a challenge. In this paper, an Expert-knowledge-based Graph Convolutional approach is proposed to automate the assessment of the quality of physical rehabilitation exercises. This approach utilizes experts' knowledge to improve the spatial feature extraction ability of the Graph Convolutional module and a Gated pooling module for feature aggregation. Additionally, a Transformer module is employed to capture long-range temporal dependencies in the movements. The attention scores and weight matrix obtained through this approach can serve as interpretability tools to help therapists understand the assessment model and assist patients in improving their exercises. The effectiveness of the proposed method is verified on the KIMORE dataset, achieving state-of-the-art performance compared to existing models. Experimental results also illustrate the interpretability of the method in both spatial and temporal dimensions.

Electroencephalography (EEG) signals are the brain signals acquired using the non-invasive approach. Owing to the high portability and practicality, EEG signals have found extensive application in monitoring human physiological states across various domains. In recent years, deep learning methodologies have been explored to decode the intricate information embedded in EEG signals. However, since EEG signals are acquired from humans, it has issues with acquiring enormous amounts of data for training the deep learning models. Therefore, previous research has attempted to develop pre-trained models that could show significant performance improvement through fine-tuning when data are scarce. Nonetheless, existing pre-trained models often struggle with constraints, such as the necessity to operate within datasets of identical configurations or the need to distort the original data to apply the pre-trained model. In this paper, we proposed the domain-free transformer, called DFformer, for generalizing the EEG pre-trained model. In addition, we presented the pre-trained model based on DFformer, which is capable of seamless integration across diverse datasets without necessitating architectural modification or data distortion. The proposed model achieved competitive performance across motor imagery and sleep stage classification datasets. Notably, even when fine-tuned on datasets distinct from the pre-training phase, DFformer demonstrated marked performance enhancements. Hence, we demonstrate the potential of DFformer to overcome the conventional limitations in pre-trained model development, offering robust applicability across a spectrum of domains.

Sleep staging serves as a fundamental assessment for sleep quality measurement and sleep disorder diagnosis. Although current deep learning approaches have successfully integrated multimodal sleep signals, enhancing the accuracy of automatic sleep staging, certain challenges remain, as follows: 1) optimizing the utilization of multi-modal information complementarity, 2) effectively extracting both long- and short-range temporal features of sleep information, and 3) addressing the class imbalance problem in sleep data. To address these challenges, this paper proposes a two-stream encode-decoder network, named TSEDSleepNet, which is inspired by the depth sensitive attention and automatic multi-modal fusion (DSA2F) framework. In TSEDSleepNet, a two-stream encoder is used to extract the multiscale features of electrooculogram (EOG) and electroencephalogram (EEG) signals. And a self-attention mechanism is utilized to fuse the multiscale features, generating multi-modal saliency features. Subsequently, the coarser-scale construction module (CSCM) is adopted to extract and construct multi-resolution features from the multiscale features and the salient features. Thereafter, a Transformer module is applied to capture both long- and short-range temporal features from the multi-resolution features. Finally, the long- and short-range temporal features are restored with low-layer details and mapped to the predicted classification results. Additionally, the Lovász loss function is applied to alleviate the class imbalance problem in sleep datasets. Our proposed method was tested on the Sleep-EDF-39 and Sleep-EDF-153 datasets, and it achieved classification accuracies of 88.9% and 85.2% and Macro-F1 scores of 84.8% and 79.7%, respectively, thus outperforming conventional traditional baseline models. These results highlight the efficacy of the proposed method in fusing multi-modal information. This method has potential for application as an adjunct tool for diagnosing sleep disorders.

This study presents a novel high density surface electromyography (EMG) decomposition method, named as 2CFastICA, because it incorporates two key algorithms: kernel constrained FastICA and correlation constrained FastICA. The former focuses on overcoming the local convergence of FastICA without requiring the peel-off strategy used in the progressive FastICA peel-off (PFP) framework. The latter further refines the output of kernel constrained FastICA by correcting possible erroneous or missed spikes. The two constrained FastICA algorithms supplement each other to warrant the decomposition performance. The 2CFastICA method was validated using simulated surface EMG signals with different motor unit numbers and signal to noise ratios (SNRs). Two source validation was also performed by simultaneous high density surface EMG and intramuscular EMG recordings, showing a matching rate (MR) of (97.2 ± 3.5)% for 170 common motor units. In addition, a different form of two source validation was also conducted taking advantages of the high density surface EMG characteristics of patients with amyotrophic lateral sclerosis, showing a MR of (99.4 ± 0.9)% for 34 common motor units from interference and sparse datasets. Both simulation and experimental results indicate that 2CFastICA can achieve similar decomposition performance to PFP. However, the efficiency of decomposition can be greatly improved by 2CFastICA since the complex signal processing procedures associated with the peel-off strategy are not required any more. Along with this paper, we also provide the MATLAB open source code of 2CFastICA for high density surface EMG decomposition.

Machine vision and artificial intelligence hold promise across healthcare applications. In this paper, we focus on the emerging research direction of infant action recognition, and we specifically consider the task of reaching which is an important developmental milestone. We develop E-babyNet, a lightweight yet effective neural-network-based framework for infant action recognition that leverages the spatial and temporal correlation of bounding boxes of infants' hands and objects to reach for to determine the onset and offset of the reaching action. E-babyNet consists of two main layers based on two LSTM and a Bidirectional LSTM (BiLSTM) model, respectively. The first layer provides a pre-evaluation of the reaching action for each hand by providing onset and offset keyframes. Then, the biLSTM model merges the previous outputs to deliver an outcome of the reaching actions detection for each frame including the reaching hand. We evaluated our approach against four other lightweight structures using a dataset comprising 5,865 annotated images resulting in 16,337 bounding boxes from 375 distinctive infant reaching actions performed while sitting by different subjects in unconstrained (home/clinic) environments. Results illustrate the effectiveness of our approach and ability to provide reliable reaching action detection and offer onset and offset keyframes with a precision of one frame. Moreover, the biLSTM layer can handle the transition between reaching actions and help reduce false detections.

Machine learning (ML) has been instrumental in solving complex problems and significantly advancing different areas of our lives. Decision tree-based methods have gained significant popularity among the diverse range of ML algorithms due to their simplicity and interpretability. This paper presents a comprehensive overview of decision trees, including the core concepts, algorithms, applications, their early development to the recent high-performing ensemble algorithms and their mathematical and algorithmic representations, which are lacking in the literature and will be beneficial to ML researchers and industry experts. Some of the algorithms include classification and regression tree (CART), Iterative Dichotomiser 3 (ID3), C4.5, C5.0, Chi-squared Automatic Interaction Detection (CHAID), conditional inference trees, and other tree-based ensemble algorithms, such as random forest, gradient-boosted decision trees, and rotation forest. Their utilisation in recent literature is also discussed, focusing on applications in medical diagnosis and fraud detection.

Single-image super-resolution (SISR) techniques have found wide applications in semiconductor defect inspection. Enhancing image resolution to improve inspection sensitivity and accuracy holds great significance. A novel SISR algorithm, called cross-convolutional residual network (CCRN), is proposed in this study. CCRN comprises a cross-convolutional module (CCM), which incorporates a cross-sharing mechanism that facilitates the fusion of features from different stages, enabling the extraction of more information from the image. Moreover, a global residual aggregation structure (GRA) is introduced. GRA captures and transfers different levels of residual features acquired from learning each CCM to the reconstruction layer. Experimental results demonstrate that the proposed SR algorithm outperforms existing state-of-the-art SR algorithms in terms of both visual and quantitative metrics when applied to optical, SEM, and TEM images of microfluidic chips, CMOS image sensors, and quantum dots, respectively. Additionally, CCRN significantly improves the accuracy of defect classification and inspection of unpatterned wafers, as evaluated using the WM-811K dataset. Notably, an increase in local defection testing accuracy from 79.00% to 89.00% and an improvement in classification accuracy from 93.69% to 96.06% are achieved. These findings underscore the potential applications of the proposed algorithm in improving semiconductor defect inspection and classification accuracies. [ABSTRACT FROM AUTHOR]