Search

Image recognition is an important task in computer vision with broad applications. In recent years, with the advent of deep learning, lightweight convolutional neural network (CNN) has brought new opportunities for image recognition, which allows high-performance recognition algorithms to run on resource-constrained devices with strong representation and generalization capabilities. This paper first presents an overview of several classical lightweight CNN models. Then, a comprehensive review is provided on recent image recognition techniques using lightweight CNN. According to the strategies applied to optimize image recognition performance, existing methods are classified into three categories: (1) model compression, (2) optimization of lightweight network, and (3) combining Transformer with lightweight network. In addition, some representative methods are tested on three commonly used datasets for performance comparison. Finally, technical challenges and future research trends in this field are discussed. • A comprehensive review of recent advances in lightweight CNN for image recognition. • Classification according to different optimization directions. • Present the evaluation results of different representative lightweight CNN methods. • Discuss future research trends of image recognition. [ABSTRACT FROM AUTHOR]

Moving image recognition has become a well-explored problem in computer vision. However, it is difficult for the traditional convolutional neural network (CNN) model to effectively capture timing information in motion. For better use of video sequence features and to improve the accuracy of action recognition, Therefore, this paper proposes a Three-dimensional CNN (3DCNN) model based on Gated Recurrent Unit (GRU) with an attention mechanism. The model leverages 3DCNN for deep feature extraction from video frames, employs GRU to capture the temporal dynamics of feature sequences and incorporates an attention mechanism to emphasize key frames, which improves moving image recognition. Demonstrating superior accuracy in'Cross-Subject' and'Cross-View' evaluations, our model surpasses standard benchmarks with accuracies of 83.2% and 87.3% respectively. • We combined 3 DCNN and GRU to classify the sequences of the motion images. • The framework adds attention mechanisms to improve recognition capabilities. • Experimental results show that the action recognition accuracy is increased by 6%. [ABSTRACT FROM AUTHOR]

Recently, convolutional neural networks (CNNs), which obtain powerful deep features in an end-to-end manner, have achieved powerful performance in remote sensing scene classification. However, the average or maximum pooling operations defined in the spatial domain and coarser-resolution features with high levels cannot extract reliable features and clear boundaries for small-scale targets in remote sensing scene imagery. This paper attempts to address these problems and proposes a multi-domain sematic high-order network for scene classification, named MSHNet. First, wavelet-spatial and detachable pooling blocks defined in the wavelet and spatial domains are inserted at the end of the convolutional block to learn the features in a more structural fusion manner. Second, multi-scale and multi-resolution semantic embedding modules are proposed to take full advantage of the complementary information and effectively maintain the spatial structures of learned deep features. Third, we employ a factorized bilinear coding approach to obtain compact and discriminative second-order features. MSHNet is thoroughly evaluated on two publicly available benchmarks, i.e., AID (Aerial Image Dataset) and NWPU-RESISC45 (Northwestern Polytechnical University-Remote Sensing Image Scene Classification 45). The extensive results illustrate that our MSHNet is competitive with other related multi-scale deep neural networks. [Display omitted] • Wavelet-spatial and detachable pooling maintain structures of learned features. • Factorized bilinear coding method with semantic features captures rich statistics. • MSHNet obtains competitive performance on NWPU-RESISC45 and AID datasets. [ABSTRACT FROM AUTHOR]

Convolutional networks (ConvNets) are computationally expensive but well known for their performance on image data. One way to reduce their complexity is to explore inherited data sparsity. However, since the gradients involved in ConvNets require dynamic updates, applying data sparsity in the training step is not straightforward. Dictionary-based learning methods can be useful since they encode the original data in a sparse form. This paper proposes a new dictionary-based training paradigm for ConvNets by exploiting redundancy in the training data while keeping the distinctive features intact. The ConvNet is then trained on the reduced, sparse dataset. The new approach significantly reduces the training time without compromising accuracy. To the best of our knowledge, this is the first implementation of ConvNet on dictionary-based sparse training data. The proposed method is validated on three publicly available datasets â€"MNIST, USPS, and MNIST FASHION. The experimental results show a significant reduction of 4.5 times in the overall computational burden of vanilla ConvNet for all the datasets. Whereas the accuracy is intact at 97.21% for MNIST, 96.81% for USPS, and 88.4% for FASHION datasets. These results are comparable to state-of-the-art algorithms, such as ResNet-{18,34,50}, trained on the full training dataset. • ConvNets are powerful for images but computationally expensive. • Paper proposes dictionary-based training for ConvNets exploiting sparsity. • The proposed method preserves orthogonal features and reduces training time. • Results: 4.5x reduction in computational burden. • High accuracy: 97.21% MNIST, 96.81% USPS, 88.4% FASHION. [ABSTRACT FROM AUTHOR]

Hypercomplex convolutional neural networks (HCNNs) have recently been used to improve deep learning architectures due to their ability to share weights across input channels and thus improve the cohesiveness of learned representations within the layers. Much work has been done to study the effect of hypercomplex representation in CNNs. However, to date, none of these models have used fully hypercomplex architectures, meaning that some of the layers in these earlier networks used conventional real-valued calculations that did not engage cross-channel weight sharing. Here, we introduce and study full hypercomplex CNNs by ensuring all layers perform hypercomplex calculations. For the earlier HCNNs, the real-valued layers are found in: (I) in the front end of the networks, (II) in the back end of the networks, and (III) in the residual blocks of the networks. The present research examines the performance of HCNNs when the modules mentioned above are replaced with hypercomplex equivalents. These representational networks have outperformed and shown state-of-the-art results compared to previous hypercomplex models (on some specific datasets). A disadvantage of these representational networks is that they consume high computational costs. To reduce this cost, novel separable hypercomplex networks (SHNNs) are proposed. They are created by factoring a quaternion convolutional module into two consecutive separable vectormap convolutional modules. As the successive layers of deep HCNNs perform local hierarchical grouping on increasingly abstract features, these groupings are responsible for the long-distance interaction problems found in the dense layers. To handle these problems, researchers have applied attention mechanisms in hypercomplex space. This paper offers a perspective on the basic concepts of representational networks and attention-based representational networks with the focus on (1) reviewing deep full HCNNs, including hypercomplex-based fully connected dense layer; (2) analyzing hypercomplex-based separable concept; (3) describing how the hypercomplex spatial layer of residual bottleneck block is replaced with an attention layer; and (4) providing a comprehensive summary comparing the performance of recent deep HCNNs, deep convolutional neural networks, and original attention based networks. We compare the performance of the networks in terms of accuracy and the number of trainable parameters on several image classification datasets. • Hypercomplex CNNs with and without attention introduce cross-channel weight sharing. • Cross-channel weight sharing enables discovery of a new class of input correlations. • Hypercomplex networks can be further optimized without loss of accuracy. • Toward this goal, separable hypercomplex operations are introduced. [ABSTRACT FROM AUTHOR]

• A novel model for improving identification of Diabetic Foot Ulcers. • A bottleneck attention module focused on channel-wise and spatial important features. • Experimental results show that the proposed model outperforms state-of-the-art models. Diabetic Foot Ulcer (DFU) is a severe complication of diabetes, and it may cause lower limb amputation. However, the manual diagnosis of DFU is a complicated and expensive process. The primary objective of this work is to design an efficient Convolutional Neural Network (CNN) approach to identify DFU. Therefore, a novel CNN-based approach (AESPNet) is proposed in this paper, where convolution layers are stacked together in a parallel fashion and with an intermediate attention module to perform DFU vs -normal skin classification. The AESPNet consists of 2 blocks, where varying-sized kernel convolution layers are connected in a parallel fashion for better local and global feature abstraction. A bottleneck attention module is associated after every concatenation operation in the network. The Stochastic Gradient Descent (SGD) (with momentum) optimizer with 1 e - 2 learning rate is used to train the proposed network on a privately accessed DFU dataset. The results of the proposed approach are compared with other standard CNN-based schemes, namely AlexNet, VGG16, DenseNet121, and InceptionV3. It has been found that the proposed AESPNet outperforms state-of-the-art schemes with a sensitivity score of 98.44% and 0.98 F1-Scores. [ABSTRACT FROM AUTHOR]

• SIID-CNN detects signal modes' interference region in spectrogram images using CNN. • SIID-CNN treats interference detection as 3-classes image classification problem. • The training set relies on a local linear model for modes instantaneous frequency. • SIID-CNN classification is accurate and robust to the presence of moderate noise. Ridge curves retrieval in time–frequency (TF) domains is fundamental in many application fields as they convey most of information concerning the instantaneous frequencies of non-stationary signals. However, it represents a very hard task in the case of multicomponent signals having non-separable modes as they generate interference in TF domains. A preliminary detection of these interference regions may be then useful for the definition of effective strategies for ridge curve recovery. This paper introduces SIID-CNN (Spectrogram Image Interference Detection via CNN) , that is a novel approach based on machine learning for the automatic detection of interference regions in spectrogram images. Each spectrogram sample is suitably classified as interference, single mode or background by accounting for its relative information. Some critical problems, such as the training set size and the type of examples to use for populating the training set, are dealt with. Experimental results show that a local linear model for spectrogram image and a small training set can guarantee good classification rates for a wide class of non-stationary signals, even in the presence of moderate noise. [ABSTRACT FROM AUTHOR]

In the last few years, the multi-modal emotion recognition has become an important research issue in the affective computing community due to its wide range of applications that include mental disease diagnosis, human behavior understanding, human machine/robot interaction or autonomous driving systems. In this paper, we introduce a novel end-to-end multimodal emotion recognition methodology, based on audio and visual fusion designed to leverage the mutually complementary nature of features while maintaining the modality-specific information. The proposed method integrates spatial, channel and temporal attention mechanisms into a visual 3D convolutional neural network (3D-CNN) and temporal attention into an audio 2D convolutional neural network (2D-CNN) to capture the intra-modal features characteristics. Further, the inter-modal information is captured with the help of an audio-video (A-V) cross-attention fusion technique that effectively identifies salient relationships across the two modalities. Finally, by considering the semantic relations between the emotion categories, we design a novel classification loss based on an emotional metric constraint that guides the attention generation mechanisms. We demonstrate that by exploiting the relations between the emotion categories our method yields more discriminative embeddings, with more compact intra-class representations and increased inter-class separability. The experimental evaluation carried out on the RAVDESS (The Ryerson Audio-Visual Database of Emotional Speech and Song), and CREMA-D (Crowd-sourced Emotional Multimodal Actors Dataset) datasets validates the proposed methodology, which leads to average accuracy scores of 89.25% and 84.57%, respectively. In addition, when compared to state-of-the-art techniques, the proposed solution shows superior performances, with gains in accuracy ranging in the [1.72%, 11.25%] interval. • A multimodal emotion recognition framework with various self-attention mechanisms. • An audio-video fusion strategy which uses cross-attention. • A learnable emotional metric that extends the traditional triplet loss function. • An extensive objective evaluation is performed on RAVDESS and CREMA-D datasets. [ABSTRACT FROM AUTHOR]

Recent work has shown that convolutional neural networks (CNN) can achieve state of the art if the datasets are well built. However, the existing convolutional layer is affected by various datasets with the inevitable problems of local abnormal features, i.e., illumination intensity and feature interaction. This paper replaces the convolutional layer with a batch covariance layer (BCL) to locate the category-related region unaffected by the problems. The BCL is regarded as a 3D covariance operation, which calculates the correlation between the kernels and feature maps in kernel size of all channels. Forward propagation, backward propagation, gradient updating, and testing procedure of the BCL are described. The comparison between BCL and convolutional layer shows the ability of BCL to reduce the influence of illumination intensity and feature interaction for discriminating and generating tasks. Complexity analysis shows that BCL can improve the accuracy with a thimbleful time consumption increase. Besides, the batch covariance neural network (BCovNN) is extended from the CNN by replacing the convolutional layer with BCL. Ablation experiment verifies the improvement of BCovNN is provided by BCL separately. BCovNN is evaluated on several popular datasets (i.e., MNIST, STL-10, CIFAR-10, and ImageNet) for image recognition and PASCAL VOC (2007 and 2012) datasets for object localization. Experimental results reveal that BCovNN achieves significant improvements over the corresponding CNN. • Batch covariance layer is proposed to replace convolutional layer for improvement. • BCL is regarded as a 3D covariance layer normalizing the kernels in batch size. • BCL mitigates abnormal local features for discriminating and generating tasks. • BCL can improve CNN's performance with thimbleful complexity increase. [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel framework to segment 3D human motion capture data into distinct behaviors. First, in preprocessing, we build a normalized pose space by eliminating translation and orientation from the 3D poses. We then transform these normalized 3D poses into 2D RGB images, and as a result, we simplify the task of motion segmentation as image classification and recognition. Furthermore, we identify the most significant joints of the skeleton that contribute substantially to executing a motion and get benefits from them by assigning them more weights. The weight allocation to the specific joint has been done purely based on its deviation capability. Finally, each motion is encoded into compact visual representation by exploiting RGB images with weighted joints. We adopt a transfer learning approach to extract a fixed-size feature vector using off-the-shelf deep Convolutional Neural Network (CNN), Alexnet, after fine-tuning. We develop a Kd-tree on these highly descriptive feature vectors to retrieve the nearest neighbors. Based on a similarity measure, we classify the motion segments and ultimately place the cuts on the ongoing motion sequences. We perform extensive experiments to evaluate our proposed approach on popular Motion Capture (MoCap) datasets, CMU and HDM05. Our approach almost outperforms all other state-of-the-art methods, and the results highlight the capabilities of our proposed scheme for effective segmentation. • We propose a novel data-driven approach for the segmentation of motion capture data. • We encode x, y, and z coordinates of a joint into an RGB image pixel. In this way, we develop an image that represents the complete motion. • We adopt a transfer learning approach and use a pre-trained deep CNN architecture, "Alexnet" to extract the fixed-size features. • We build Kd-tree on these fixed-size features to search and retrieve nearest neighbors from the input query frames. • Based on the retrieved nearest neighbors, the actions are classified, and the segmentation is performed. [ABSTRACT FROM AUTHOR]