Search

Brain–computer interfaces (BCIs) based on motor imagery (MI) enable the disabled to interact with the world through brain signals. To meet demands of real-time, stable, and diverse interactions, it is crucial to develop lightweight networks that can accurately and reliably decode multi-class MI tasks. In this paper, we introduce BrainGridNet, a convolutional neural network (CNN) framework that integrates two intersecting depthwise CNN branches with 3D electroencephalography (EEG) data to decode a five-class MI task. The BrainGridNet attains competitive results in both the time and frequency domains, with superior performance in the frequency domain. As a result, an accuracy of 80.26 percent and a kappa value of 0.753 are achieved by BrainGridNet, surpassing the state-of-the-art (SOTA) model. Additionally, BrainGridNet shows optimal computational efficiency, excels in decoding the most challenging subject, and maintains robust accuracy despite the random loss of 16 electrode signals. Finally, the visualizations demonstrate that BrainGridNet learns discriminative features and identifies critical brain regions and frequency bands corresponding to each MI class. The convergence of BrainGridNet's strong feature extraction capability, high decoding accuracy, steady decoding efficacy, and low computational costs renders it an appealing choice for facilitating the development of BCIs. [ABSTRACT FROM AUTHOR]

Image registration is a fundamental problem in computer vision and robotics. Recently, learning-based image registration methods have made great progress. However, these methods are sensitive to abnormal transformation and have insufficient robustness, which leads to more mismatched points in the actual environment. In this paper, we propose a new registration framework based on ensemble learning and dynamic adaptive kernel. Specifically, we first use a dynamic adaptive kernel to extract deep features at the coarse level to guide fine-level registration. Then we added an adaptive feature pyramid network based on the integrated learning principle to realize the fine-level feature extraction. Through different scale, receptive fields, not only the local geometric information of each point is considered, but also its low texture information at the pixel level is considered. According to the actual registration environment, fine features are adaptively obtained to reduce the sensitivity of the model to abnormal transformation. We use the global receptive field provided in the transformer to obtain feature descriptors based on these two levels. In addition, we use the cosine loss directly defined on the corresponding relationship to train the network and balance the samples, to achieve feature point registration based on the corresponding relationship. Extensive experiments on object-level and scene-level datasets show that the proposed method outperforms existing state-of-the-art techniques by a large margin. More critically, it has the best generalization ability in unknown scenes with different sensor modes. [ABSTRACT FROM AUTHOR]

Recently, deep convolutional neural networks (CNNs) have been widely explored in single image super-resolution (SISR) and achieved remarkable progress. However, most of the existing CNN-based SISR networks with a single-stream structure fail to make full use of the multi-scale features of low-resolution (LR) image. While those multi-scale SR models often integrate the information with different receptive fields by means of linear fusion, which leads to the redundant feature extraction and hinders the reconstruction performance of the network. To address both issues, in this paper, we propose a non-linear perceptual multi-scale network (NLPMSNet) to fuse the multi-scale image information in a non-linear manner. Specifically, a novel non-linear perceptual multi-scale module (NLPMSM) is developed to learn more discriminative multi-scale feature correlation by using high-order channel attention mechanism, so as to adaptively extract image features at different scales. Besides, we present a multi-cascade residual nested group (MC-RNG) structure, which uses a global multi-cascade mechanism to organize multiple local residual nested groups (LRNG) to capture sufficient non-local hierarchical context information for reconstructing high-frequency details. LRNG uses a local residual nesting mechanism to stack NLPMSMs, which aims to form a more effective residual learning mechanism and obtain more representative local features. Experimental results show that, compared with the state-of-the-art SISR methods, the proposed NLPMSNet performs well in both quantitative metrics and visual quality with a small number of parameters. [ABSTRACT FROM AUTHOR]

In this paper, a novel dual-channel system for multi-class text emotion recognition has been proposed, and a novel technique to explain its training & predictions has been developed. The architecture of the proposed system contains the embedding module, dual-channel module, emotion classification module, and explainability module. The embedding module extracts the textual features from the input sentences in the form of embedding vectors using the pre-trained Bidirectional Encoder Representations from Transformers (BERT) model. Then the embedding vectors are fed as the inputs to the dual-channel network containing two network channels made up of convolutional neural network (CNN) and bidirectional long short term memory (BiLSTM) network. The intuition behind using CNN and BiLSTM in both the channels was to harness the goodness of the convolutional layer for feature extraction and the BiLSTM layer to extract text's order and sequence-related information. The outputs of both channels are in the form of embedding vectors which are concatenated and fed to the emotion classification module. The proposed system's architecture has been determined by thorough ablation studies, and a framework has been developed to discuss its computational cost. The emotion classification module learns and projects the emotion embeddings on a hyperplane in the form of clusters. The proposed explainability technique explains the training and predictions of the proposed system by analyzing the inter & intra-cluster distances and the intersection of these clusters. The proposed approach's consistent accuracy, precision, recall, and F1 score results for ISEAR, Aman, AffectiveText, and EmotionLines datasets, ensure its applicability to diverse texts. [ABSTRACT FROM AUTHOR]

Heart sound classification plays a vital role in the early detection of cardiovascular disorders, especially for small primary health care clinics. Despite that much progress has been made for heart sound classification in recent years, most of them are based on conventional segmented features and shallow structure based classifiers. These conventional acoustic representation and classification methods may be insufficient in characterizing heart sound, and generally suffer from a degraded performance due to the complicated and changeable cardiac acoustic environment. In this paper, we propose a new heart sound classification method based on improved Mel-frequency cepstrum coefficient (MFCC) features and convolutional recurrent neural networks. The Mel-frequency cepstrums are firstly calculated without dividing the heart sound signal. A new improved feature extraction scheme based on MFCC is proposed to elaborate the dynamic characteristics among consecutive heart sound signals. Finally, the MFCC-based features are fed to a deep convolutional and recurrent neural network (CRNN) for feature learning and later classification task. The proposed deep learning framework can take advantage of the encoded local characteristics extracted from the convolutional neural network (CNN) and the long-term dependencies captured by the recurrent neural network (RNN). Comprehensive studies on the performance of different network parameters and different network connection strategies are presented in this paper. Performance comparisons with state-of-the-art algorithms are given for discussions. Experiments show that, for the two-class classification problem (pathological or non-pathological), a classification accuracy of 98% has been achieved on the 2016 PhysioNet/CinC Challenge database. • We propose a new MFCC-based heart sound classification method. • A new improved MFCC feature extraction scheme is developed. • CNN and RNN are combined for heart sound classification. • We show promising performance on benchmark PCG databases. [ABSTRACT FROM AUTHOR]

An increasing need of running Convolutional Neural Network (CNN) models on mobile devices encourages the studies on efficient and lightweight neural network model. In this paper, an Inverse Residual Multi-Branch Network named IremulbNet is proposed to solve the problem of insufficient classification accuracy in existing lightweight network models. The core module of this model is to reconstruct an inverse residual structure, in which a special feature fusion method, multi-branch feature extraction, and depthwise separable convolution techniques are used to improve the classification accuracy. The performance of model is tested using image databases. Experimental results show that for the fine-grained image dataset Imagenet-woof, IremulbNet achieved 10.9%, 12.2%, and 15.3% higher accuracy than that of MobileNet V3, ShuffleNet V2, and PeleeNet, respectively. Moreover, it can reduce inference time (GPU) about 42.09% and 75.56% compared to classic ResNet50 and DenseNet121. • A lightweight IremulbNet is proposed to achieve high accuracy image recognition with a small number of parameters and operations. • A special feature fusion method, multi-branch feature extraction, and depthwise separable convolution are used to improve the performance of the network. • A significant improvement in recognition accuracy can be obtained in image classification tasks with low computation load. [ABSTRACT FROM AUTHOR]

Vision transformers (ViTs) have become one of the dominant frameworks for vision tasks in recent years because of their ability to efficiently capture long-range dependencies in image recognition tasks using self-attention. In fact, both CNNs and ViTs have advantages and disadvantages in vision tasks, and some studies suggest that the use of both may be an effective way to balance performance and computational cost. In this paper, we propose a new hybrid network based on CNN and transformer, using CNN to extract local features and transformer to capture long-distance dependencies. We also proposed a new feature map resolution reduction based on Discrete Cosine Transform and self-attention, named DCT-Attention Down-sample (DAD). Our DctViT-L achieves 84.8% top-1 accuracy on ImageNet 1K, far outperforming CMT, Next-ViT, SpectFormer and other state-of-the-art models, with lower computational costs. Using DctViT-B as the backbone, RetinaNet can achieve 46.8% mAP on COCO val2017, which improves mAP by 2.5% and 1.1% with less calculation cost compared with CMT-S and SpectFormer as the backbone. • We propose a DAD block for feature map compression and feature extraction using DCT. • We design new convolutional blocks and transformer blocks. • We propose a CNN and Transformer hybrid neural network family DctViT. • We experimentally demonstrate the superior performance of the DctViT network. [ABSTRACT FROM AUTHOR]

The SSVEP-based paradigm serves as a prevalent approach in the realm of brain–computer interface (BCI). However, the processing of multi-channel electroencephalogram (EEG) data introduces challenges due to its non-Euclidean characteristic, necessitating methodologies that account for inter-channel topological relations. In this paper, we introduce the Dynamic Decomposition Graph Convolutional Neural Network (DDGCNN) designed for the classification of SSVEP EEG signals. Our approach incorporates layerwise dynamic graphs to address the oversmoothing issue in Graph Convolutional Networks (GCNs), employing a dense connection mechanism to mitigate the gradient vanishing problem. Furthermore, we enhance the traditional linear transformation inherent in GCNs with graph dynamic fusion, thereby elevating feature extraction and adaptive aggregation capabilities. Our experimental results demonstrate the effectiveness of proposed approach in learning and extracting features from EEG topological structure. The results shown that DDGCNN outperforms other state-of-the-art (SOTA) algorithms reported on two datasets (Dataset 1: 54 subjects, 4 targets, 2 sessions; Dataset 2: 35 subjects, 40 targets). Additionally, we showcase the implementation of DDGCNN in the context of synchronized BCI robotic fish control. This work represents a significant advancement in the field of EEG signal processing for SSVEP-based BCIs. Our proposed method processes SSVEP time domain signals directly as an end-to-end system, making it easy to deploy. The code is available at https://github.com/zshubin/DDGCNN. [ABSTRACT FROM AUTHOR]

A novel convolutional neural network is proposed for local prior feature embedding and imbalanced dataset modeling for multi-channel time-varying signal classification. This model consists of a single-channel signal feature parallel extraction unit, a multi-channel signal feature integration unit, a local feature embedding and feature similarity measurement unit, a full connection layer, and a Softmax classifier. An algorithm combining dynamic clustering and sliding window was used to select segments signals with typical local features in each pattern class, forming a typical local feature set. The one-dimensional CNNs were used to extract features from the single-channel signal in parallel, a comprehensive feature matrix of the multi-channel signal and the local feature matrix templates were produced. Using the method of external embedding, based on the sliding window and dynamic time warping (DTW) algorithm, the local feature similarities between the local feature template of each pattern class and the comprehensive feature sub-matrix of the input signal were measured, and the maximum values were selected to construct a local feature similarity vector in order. The information fusion was realized through a full connection layer. The proposed methodology can extract and represent both global and local signals features, strengthen the role of prior local feature in classification and improve the modeling properties of imbalanced datasets. A comprehensive learning algorithm is presented in this paper. The classification diagnosis of cardiovascular disease based on 12-lead ECG signals was used as a verification experiment. Results showed that the accuracy and generalization for the proposed technique were significantly improved. • Local prior feature knowledge embedding, model structure and data constraints. • The proposed model has the properties of feature combination and fusion analysis. • Improve the distinguishing ability of different pattern features. • Model small-scale and imbalanced datasets and maintain diversity of signal features. [ABSTRACT FROM AUTHOR]

The non-invasive electroencephalogram (EEG) signals enable humans to communicate with devices and have control over them, this process requires precise classification and identification of those signals. The recent revolution of deep learning has empowered both feature extraction and classification in a joint manner of different data types. However, deep learning is a data learning approach that demands a large number of training samples. Whilst, the EEG research field lacks a large amount of data which restricts the use of deep learning within this field. This paper proposes a novel augmentation method for enlarging EEG datasets. Our CutCat augmentation method generates trials from inter- and intra-subjects and trials. The method relies on cutting a specific period from an EEG trial and concatenating it with a period from another trial from the same subject or different subjects. The method has been tested on shallow and deep convolutional neural networks (CNN) for the classification of motor imagery (MI) EEG data. Two input formulation types images and time-series have been used as input to the neural networks. Short-time Fourier transform (STFT) is used for generating training images from the time-series signals. The experimental results demonstrate that the proposed augmentation method is a promising strategy for handling the classification of small-scale datasets. Classification results on two EEG datasets show advancement in comparison with the results of state-of-the-art researches. [ABSTRACT FROM AUTHOR]