Search

In recent years, deep learning has already been applied to English lip-reading. However, Chinese lip-reading starts late and lacks relevant dataset, and the recognition accuracy is not ideal. Therefore, this paper proposes a new hybrid neural network model to establish a Chinese lip-reading system. In this paper, we integrate the attention mechanism into both CNN and RNN. Specifically, we add the convolutional block attention module (CBAM) to the ResNet50 neural network, which enhances its ability to capture the small differences among the mouth patterns of similarly pronounced words in Chinese, improving the performance of feature extraction in the convolution process. We also add the time attention mechanism to the GRU neural network, which helps to extract the features among consecutive lip motion images. Considering the effects of the moments before and after on the current moment in the lip-reading process, we assign more weights to the key frames, which makes the features more representative. We further validate our model through experiments on our self-built dataset. Our experiments show that using convolutional block attention module (CBAM) in the Chinese lip-reading model can accurately recognize Chinese numbers 0–9 and some frequently used Chinese words. Compared with other lip-reading systems, our system has better performance and higher recognition accuracy. [ABSTRACT FROM AUTHOR]

• A method is proposed to analyze the radio spectral bias from fre- quency perspective, based on a multi-spectral attention mechanism with DCT for learning-based frequency components selection, termed as FLANs. This attention mechanism is the general case of conventional global aver- age pooling (GAP) and leverage identical structures of the popular neural networks. • We propose a simple yet effective learning-based dynamic frequency selection module in FLANs. The module consists of multiple learning-based switches to identify the trivial frequency components based on the DCT coefficients for static removal during inference. • To the best of our knowledge, this is the first work that compromises the merits of hand-crafted feature mechanism and deep learning in frequency domain for AMC. Extensive experiments have been conducted to validate the superiority of FLANs for automatic modulation classification over a wide variety of state-of-the-art methods on RADIOML 2018.01A dataset. Deep neural networks have been recently applied in automatic modulation classification task and achieved remarkable success. However, Existing neural networks mainly focus on the purely data-driven architecture design, and fail to explore the hand-crafted feature mechanisms which are particularly significant for radio signal presentation in wireless communication. Inspired by digital signal processing theories, we propose frequency learning attention networks (FLANs) to analyze the radio spectral bias from frequency perspective, based on a multi-spectral attention mechanism for learning-based frequency components selection. FLANs are the general case of classical global average pooling and leverage identical structures of the popular neural networks. Extensive experiments have been conducted to validate the superiority of FLANs for automatic modulation classification over a wide variety of state-of-the-art methods on RADIOML 2018.01A dataset. [ABSTRACT FROM AUTHOR]