Integration of Convolutional Neural Network and Vision Transformer for gesture recognition using sEMG.

Currently, gesture recognition primarily utilizes Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) among deep learning methods. However, the unique spatial and temporal features of surface Electromyography (sEMG) signals render these methods insufficient for effective feature extraction. To tackle this challenge, this paper proposes a novel model named CNN-VIT, which integrates the architecture of CNN and Vision Transformers (VIT) with a weighted mechanism. This innovative model combines the strengths of CNN and VIT to comprehensively exploit the motion information encoded in sEMG signals, subsequently improving the accuracy and reliability of gesture recognition. To validate the algorithm's practical efficacy, we conducted experiments on the Ninapro DB2 Exercise B dataset, followed by tests on the DB-MYO dataset, which was collected using a myoelectric data bracelet. Additionally, we performed real-time prediction experiments to further assess the model's performance. Results demonstrate a classification accuracy of 83.05%, 90.40%, and 85.00%, affirming the superior classification performance of CNN-VIT. [Display omitted] • We propose a novel model named CNN-VIT, which integrates the architecture of CNN and Vision Transformers (VIT) with a weighted mechanism. • Our model combines the strengths of CNN and VIT to comprehensively exploit the motion information encoded in sEMG signals, achieving excellent performance in gesture recognition. • We propose a weighting mechanism named ECSA. This mechanism enhances the representational power of the features by assigning weights to each channel of the feature maps. • The ECSA mechanism not only captures the attention along the width of the image data but also maps the positional information of various sEMG signal channels onto the feature maps. [ABSTRACT FROM AUTHOR]