Search

Scene classification and recognition have always been one of the most challenging tasks of scene understanding due to the inherent ambiguity in visual scenes. The core of scene classification and recognition tasks is scene representation. Deep learning advances in computer vision, especially deep CNNs, have significantly improved scene representation in the last decade. Deep convolutional features extracted from deep CNNs provide discriminative representations of the images and are widely used in various computer vision tasks, such as scene classification. Deep convolutional features capture the appearance characteristics of the image and the spatial information about different image regions. Meanwhile, the semantic and context information obtained from high-level concepts about scene images, such as objects and their relationships, can significantly contribute to identifying scene images. Therefore, in this paper, we divide visual scenes into two categories, object-based and layout-based. Object-based scenes are scenes that have scene-specific objects and, based on those objects, can be described and identified. In contrast, the layout-based scenes do not have scene-specific objects and are described and identified based on the appearance and layout of the image. This paper proposes a new neural network model for representing and classifying visual scenes, which we call G-CNN (GNN-CNN). The proposed model includes two modules, feature extraction and feature fusion, and the feature extraction module composes of visual and semantic branches. The visual branch is responsible for extracting deep CNN features from the image, and the semantic branch is responsible for extracting semantic GNN features from the scene graph corresponding to the image. The feature fusion module is a novel two-stream neural network that fuses the CNN and GNN feature vectors to produce a comprehensive representation of the scene image. Finally, a fully-connected classifier classified the obtained comprehensive feature vector into one of the pre-defined categories. The proposed model has been evaluated on three benchmark scene datasets, UIUC Sports, MIT67, and SUN397, and obtained classification accuracy of 99.91%, 96.01%, and 85.32%, respectively. In addition, a new dataset named Scene40, which has been introduced in our previous paper, is also used for further evaluation of the proposed method. The comparison results based on classification accuracy criteria show that the proposed model can outperform the best previous methods on three benchmark scene datasets. [ABSTRACT FROM AUTHOR]

Defect detection systems based on machine vision have been widely used as an essential part of intelligent manufacturing systems. However, in traditional object detection methods that rely on images as input, differences in defect areas, blurred images, and complex background interference can seriously impair detection accuracy. To meet these challenges, this paper proposed a dual-path neural network based on shearlet transform (STDPNet) by taking advantage of shearlet transform in multi-scale analysis and combining it with the improved object detection algorithm proposed in this paper. First, images are multi-scale and multi-directional decomposed with shearlet transform, and multi-directional sub-band information is input to the detection network instead of image information. Then, this paper proposed a dual-path object detection network for the differences between different frequency bands and introduced a transfer learning strategy between paths to improve the model performance. Finally, the training results on the NEU surface defect public dataset show that the mean average precision of STDPNet achieves 86.81% at a detection speed of 44.45 f/s, which exceeds that of Faster R-CNN by 12%. Experiments on different datasets prove that the accuracy is significantly superior to other models, and the proposed method is more advantageous compared to other models in large, fuzzy, and indistinguishable defect types. [ABSTRACT FROM AUTHOR]

To equip convolutional neural networks (CNNs) with explainability, it is essential to interpret how opaque models make specific decisions, understand what causes the errors, improve the architecture design, and identify unethical biases in the classifiers. This paper introduces ADVISE, a new explainability method that quantifies and leverages the relevance of each unit of the feature map to provide better visual explanations. To this end, we propose using adaptive bandwidth kernel density estimation to assign a relevance score to each unit of the feature map with respect to the predicted class. We also propose an evaluation protocol to quantitatively assess the visual explainability of CNN models. Our extensive evaluation of ADVISE in image classification tasks using pretrained AlexNet, VGG16, ResNet50, and Xception models on ImageNet shows that our method outperforms other visual explainable methods in quantifying feature-relevance and visual explainability while maintaining competitive time complexity. Our experiments further show that ADVISE fulfils the sensitivity and implementation independence axioms while passing the sanity checks. The implementation is accessible for reproducibility purposes on https://github.com/dehshibi/ADVISE. [ABSTRACT FROM AUTHOR]

This paper presents the first method for synthesizing seasonal transition of terrain textures for an input heightfield. Our method reproduces a seamless transition of terrain textures according to the seasons by learning measured data on the earth using a convolutional neural network. We attribute the main seasonal texture transition to vegetation and snow, and control the texture synthesis not only with the input heightfield but also with the annual temperature and precipitation based on Köppen's climate classification as well as insolation at the location. We found that month-by-month synthesis yields incoherent transitions, while a naïve conditioning with explicit temporal information (e.g., month) degrades generalizability due to the north–south hemisphere difference. To address these issues, we introduce a simple solution—periodic conditioning on the annual data without explicit temporal information. Our experiments reveal that our method can synthesize plausible seasonal transitions of terrain textures. We also demonstrate large-scale texture synthesis by tiling the texture output. [ABSTRACT FROM AUTHOR]

Multi-exposure fusion (MEF) is widely researched and applied to high dynamic range (HDR) imaging, where one of the most challenging problems is the artifacts caused by the motion of objects between input images. Recently, deep learning methods have been widely applied for HDR imaging with excellent results, showing significant advantages. However, many methods cannot avoid artifacts due to inaccurate alignment before merging HDR images. In this paper, we propose an end-to-end network (C-ED-GMNET) with a channel and spatial attention network, an encoder–decoder network and a gradual merging network for generating artifact-free HDR images in dynamic scenes. The attention module consists of two submodules to identify useful features and exclude harmful components in inputs from both channel and spatial dimensions, respectively. The attention-guided feature maps are sent to the encoders for further feature extraction, and then, the outputs are sent to a gradual merging module consisting of two steps to generate deep features progressively. Besides, the differences between the merged image and the original images are identified by applying global residual learning with all the inputs, and the merged image feature is recovered by the decoder to obtain the final HDR image. Quantitative and qualitative experiments on two public datasets show that the proposed C-ED-GMNET produces better results than existing state-of-the-art methods and significantly reduces the runtime due to the encoders which reduce the amount of computation. [ABSTRACT FROM AUTHOR]

Rice which is a staple food crop in most Asian countries mainly suffers from higher yield loss due to different factors, and one of the common factors that affect rice yield is rice leaf disease. Many countries do not allow the plantation of new rice varieties unless they are disease-resistant. Hence, disease control serves as an important factor for rice production and this can be effectively accomplished by identifying the rice disease at an early stage which is required for timely pesticide application and disease control. The rice leaf diseases are mainly inspected by the farmers with their naked eye which is error-prone, time-consuming, and often leads to huge yield loss if predicted wrong. This process is effectively handled by different researchers via the use of computer vision and machine learning techniques. To overcome the drawbacks associated with manual processing and low recognition accuracy of different statistical and machine learning techniques, this paper presents a novel concept that uses 3D 2D CNN for feature extraction and an improved backtracking search (IBS) algorithm-optimized deep generative adversarial network (DGAN) for classification. The 3D2D deep convolutional neural network (DCNN) is formed by integrating the 3D fast-learning block with a 2DCNN for extracting the rice disease features such as lesions and shape. The IBS algorithm-optimized GAN architecture is used for rice disease classification and the IBS algorithm is mainly adopted to overcome the instability and overfitting issues that arise in the DGAN. The proposed IBS-optimized GAN architecture offers improved accuracy of 98.7% which is relatively higher than the existing techniques such as extreme gradient boosting (XGBoost), transfer learning techniques, and support vector machine (SVM). [ABSTRACT FROM AUTHOR]

Unlike 2D images that are represented in regular grids, 3D point clouds are irregular and unordered, hence directly applying convolution neural networks (CNNs) to process point clouds is quite challenging. In this paper, we propose a novel deep neural network named AKNet to achieve point cloud semantic segmentation. The key to our AKNet is the attentive kernel convolution (AKConv), which is a deformed convolution operation for perceiving sufficient local context of 3D scenes. AKConv first constructs the Basic Weight Units that are robust to point's ordering. Then, for capturing the more distinctive local features, the convolution kernels of AKConv are associated with Attentive Weight Units through the self-attentive function acting on Basic Weight Units. Furthermore, 3D point clouds provide richer geometric shape information, which is helpful to recognize objects. However, inputting only raw point features to the convolution function could cause geometric information loss. Thus, we utilize augmented features as input of AKConv. Besides, to preserve the semantic information from the encoding to decoding layers, we introduce the backward encoding (BE) mechanism by utilizing higher-layer semantic features. We conduct experiments on three large-scale point clouds datasets. The experimental results demonstrate that our AKNet outperforms state-of-the-art (SOTA) networks. [ABSTRACT FROM AUTHOR]

Intelligent recognition of traffic signs and markings is an important component of autonomous driving and intelligent transportation systems. It is also an important theoretical basis for autonomous driving path planning. To address the low accuracy and poor real-time performance of the traffic signs and markings detection in complex and multivariate scenes, a lightweight convolutional neural network recognition method in multiple interference scenes is proposed. Firstly, based on Gamma correction and contrast limited histogram equalization algorithm, the traffic signs and markings image under the low illumination condition is enhanced adaptively. Then, the MobileNet-V2 fuse together with DeepLab-V3 + algorithm to segment traffic signs and markings in multiple scenes. Finally, an identification model is established to realize the adaptive recognition of traffic signs and markings, which are based on Lightweight Convolutional Neural Networks (Lw-CNN). Besides, the recognition method proposed in this paper is verified by the CSUST Chinese Traffic Sign Detection Benchmark (CCTSDB). The experimental results show that the Mean Intersection Over Union (MIOU) of the MobileNet-V2 and DeepLab-V3 + algorithm reaches 83.07%, it increased by 25.8% than before image enhancement. The recognition accuracy of algorithm based on lightweight convolutional neural network is 99.92%, higher than the algorithms of MobileNet-V2 and VGG16. [ABSTRACT FROM AUTHOR]

Handwritten character recognition (HCR) is a challenging task because of high intra-class dissimilarity and high inter-class similarity among the character images. Therefore, the features that are used to represent the character images should be such that they maximize the inter-class variations and minimize intra-class variations. In this paper, a multilevel approach for recognition of Meitei Mayek handwritten characters using fusion of features strategy is presented. The proposed methodology has two levels of recognition. The first level employs a simple CNN. A method to identify the structurally similar character classes based on the probability values of the softmax function is also proposed. The fused feature set is employed in the second-level recognition to distinguish between these character classes, thereby enhancing the discriminative power of the fused feature set by reducing the search space in the second level. The fused feature set is a combination of traditional handcrafted feature descriptors and deep features learned by CNN. The fused feature set is fed to an SVM for second-level recognition. The experimental results show a superior performance of the proposed methodology in identifying the structurally similar character classes. This achieves an overall recognition accuracy of 97.08%, which sets the benchmark on the concerned Meitei Mayek handwritten character dataset. The methodology also shows significant results on MNIST, DIDA and CArDIS datasets. [ABSTRACT FROM AUTHOR]

Pupil localization is one of the most critical and essential requirements for eye gaze estimation and eye movement tracking. Because pupil images contain monotonous and uncomplicated information, the dataset uses a single class of labels to describe the image content, and using convolutional neural networks can quickly and accurately identify the pupil position on the input image. On low-resolution images, traditional methods encounter issues of low accuracy and cumbersome design steps. A lightweight pupil localization algorithm is proposed in this paper, utilizing a convolutional neural network (CNN) with additional training samples. The experimental results demonstrate the algorithm’s significant effectiveness in identifying the pupil position within the training set, with the accuracy of pupil position in the test set reaching 97.78%. This provides an evidence of the algorithm’s feasibility for accurately localizing pupils in low-resolution images. [ABSTRACT FROM AUTHOR]