Search

The models based on convolutional neural network have achieved excellent results in image super-resolution by acquiring prior knowledge from a large number of images, but such models still have problems such as the features between layers in the depth network cannot be effectively fused, the number of parameters is too large, and cross-channel feature learning is impossible. Based on this, a deep recursive residual channel attention network (DRRCAN) model was proposed in this paper. To solve the problem that the information between different layers in the deep network cannot be fused effectively, this paper constructs a channel feature fusion module, which can effectively fuse the feature information of different layers. To solve the problem that the parameters increase sharply due to the increase of network depth, recursive blocks are adopted in this paper, which greatly reduces the number of parameters in the deep network. The channel attention is integrated to enable the model to learn features across channels. In addition, to avoid gradient explosion or disappearance, residual modules, long skip connections are introduced to improve the stability and generalization ability of the model. Extensive benchmark evaluations validate the superiority of the proposed DRRCAN model compared with existing algorithms. [ABSTRACT FROM AUTHOR]

Recently, convolutional neural networks (CNNs) have facilitated the rapid development of image super-resolution. Most deep networks are challenging to apply to the real world due to their high cost of memory storage and computational complexity. This paper addresses this issue by proposing a lightweight subpixel sampling network (SSN). Specifically, we use a traditional encoder-decoder structure and replace the deconvolution and pooling layers by subpixel up-sampling and down-sampling without parameters. Subpixel sampling retains more image information than other sampling methods. In addition, we propose parsimonious spatial and channel attention blocks through which multi-scale features are fused and more image textures can be recovered. Through extensive experiments, we validate the effectiveness of subpixel sampling, spatial attention block, and channel attention block. In terms of quantitative metrics and visual quality, our models achieve performance comparable to state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Texture filtering attempts to retain salient structures and remove insignificant textures. In this paper, we propose a highly effective iterative unsupervised deep bilateral texture filtering neural network for texture smoothing. The bilateral texture loss function is introduced to train the model without the ground truth smoothing images for guidance. The proposed model inherits well-known advantages of the bilateral texture filter to capture the texture information effectively. The model is trained solely using the training data, then the predicted outputs are generated iteratively through multiple forward passes. Extensive experiments demonstrate that our proposed iterative unsupervised deep bilateral texture filtering neural network outperforms existing methods in effectively removing textures while preserving the main structures of the image. The results showcase the superior performance of our approach and its ability to achieve high-quality texture smoothing without sacrificing important image features. [ABSTRACT FROM AUTHOR]

Recently, cycle-consistent adversarial network (CycleGAN) has been utilized in image dehazing tasks due to train without paired images. However, the quality of generated dehazed images is low. To enhance the dehazed images, this paper presents an improved CycleGAN model, called DFC-dehaze, in which the convolutional neural network (CNN)-based generator is replaced by the Dehazeformer-t model. To reduce the haze residuals in the recovered images, the local–global discriminator is used to handle locally varying haze. When the generated images are blurred or have unrealistic colors, the discriminator gives them the low scores by the negative sample punishment mechanism. Another, the structural similarity index measurement (SSIM) loss is combined with the cyclic consistent loss to improve the visual quality of the dehazed images. The experimental results demonstrate that the proposed methods outperform other unsupervised methods and yield visual outcomes that are comparable to those produced by supervised methods. [ABSTRACT FROM AUTHOR]

Almost all Vision Transformer-based models need to pre-train on the massive datasets and costly computation. Suppose researchers do not have enough data to train a Vision Transformer-based model or do not have powerful GPUs to implement computation for millions of labeled data. In that case, Vision Transformer-based models have no advantages over CNNs. Swin Transformer is brought forward to figure out these problems by applying the shifted window-based self-attention, which has linear computational complexity. Although Swin Transformer significantly reduces computing costs and works well on mid-size datasets, it still performs not well when it trains on a small-size dataset. In this paper, we propose a hierarchical and data-efficient Transformer based on Swin Transformer, which we call ESwin Transformer. We mainly redesigned the patch embedding module and patch merging module of Swin Transformer. We merely applied some unsophisticated convolutional components to these modules, which significantly improved performance when we trained our model on a small dataset. Our empirical results show that ESwin Transformer trained on CIFAR10/CIFAR100 with no extra data for 300 epochs achieves 97.17 % / 83.78 % accuracy and performs better than Swin Transformer and DeiT in the same training time. [ABSTRACT FROM AUTHOR]

The majority of the current widely used algorithms for fatigue detection rely on shallow learning to extract fatigue characteristics and use a single feature to determine the level of fatigue. The accuracy of detection is greatly affected by individual and environmental differences, and there are certain limitations in complex scenes. To improve the accuracy and real-time performance of the fatigue detection algorithm, a new driver fatigue detection algorithm based on multi-feature fusion is proposed. This paper employs two cameras to capture photos of the driver and the road, respectively, and a lightweight convolutional neural network to extract features from the driver's face, including the eyes, mouth, and head, as well as lane departure features from the road images. The four fatigue features are analyzed and fused to comprehensively detect the driver's fatigue state. The experimental results show that the multi-feature fusion-based driver fatigue detection algorithm can not only detect the driver's fatigue condition accurately but also classify the fatigue state according to the degree of fatigue, which is useful for making effective pre-warning system. [ABSTRACT FROM AUTHOR]

Display technologies have evolved over the years. It is critical to develop practical HDR capturing, processing, and display solutions to bring 3D technologies to the next level. Depth estimation of multi-exposure stereo image sequences is an essential task in the development of cost-effective 3D HDR video content. In this paper, we develop a deep architecture for multi-exposure stereo depth estimation. The proposed architecture has two novel components. First, the stereo matching technique used in traditional stereo depth estimation is revamped. For the stereo depth estimation component of our architecture, a mono-to-stereo transfer learning approach is deployed. The proposed formulation circumvents the cost volume construction requirement, which is replaced by a dual-encoder single-decoder CNN with different weights for feature fusion. EfficientNet-based blocks are used to learn the disparity. Secondly, we combine disparity maps obtained from the stereo images at different exposure levels using a robust disparity feature fusion approach. The disparity maps obtained at different exposures are merged using weight maps calculated for different quality measures. The final predicted disparity map obtained is more robust and retains best features that preserve the depth discontinuities. The proposed CNN offers flexibility to train using standard dynamic range stereo data or with multi-exposure low dynamic range stereo sequences. In terms of performance, the proposed model surpasses state-of-the-art monocular and stereo depth estimation methods, both quantitatively and qualitatively, on challenging Scene flow and differently exposed Middlebury stereo datasets. The architecture performs exceedingly well on complex natural scenes, demonstrating its usefulness for diverse 3D HDR applications. [ABSTRACT FROM AUTHOR]

Analyzing 3D objects without pose priors using neural networks is challenging. In view of the shortcoming that spherical convolutional networks lack the construction of a part–whole hierarchy with rotation equivariance for 3D object recognition with unknown poses, which generates whole rotation-equivariant features that cannot be effectively preserved, a rotation-equivariant part–whole hierarchy spherical vector network is proposed in this paper. In our experiments, we map a 3D object onto the unit sphere, construct an ordered list of vectors from the convolutional layers of the rotation-equivariant spherical convolutional network and then construct a part–whole hierarchy to classify the 3D object using the proposed rotation-equivariant routing algorithm. The experimental results show that the proposed method improves not only the recognition of 3D objects with known poses, but also the recognition of 3D objects with unknown poses compared to previous spherical convolutional neural networks. This finding validates the construction of the rotation-equivariant part–whole hierarchy. [ABSTRACT FROM AUTHOR]

Global contexts are critical to locating salient objects for salient object detection (SOD). However, the convolution operation in CNNs has a local receptive field, which cannot capture long-distance global information. Recent studies have shown that modernized CNN models with large kernel convolution, such as ConvNeXt, can effectively extend the receptive fields. Based on it, this paper explores the potential of large kernel CNN for SOD task. Inspired by the common information between RGB and depth images in salient objects, we propose a ConvNeXt-based Siamese network with shared weight parameters. This structural design can effectively reduce the number of parameters without sacrificing performance. Furthermore, a depth information preprocessing module is proposed to minimize the impact of low-quality depth images on predicted saliency maps. For cross-modal feature interaction, a dynamic fusion module is designed to enhance cross-modal complementarity dynamically. Extensive experiments and evaluation results on six benchmark datasets demonstrate the outstanding performance of the proposed method against 14 state-of-the-art RGB-D methods. Our code will be released at https://github.com/zyh5119232/CSNet. [ABSTRACT FROM AUTHOR]

In recent years, due to the powerful feature extraction capabilities of convolutional neural networks (CNNs), many single-image super-resolution (SISR) methods based on CNN have achieved remarkable results. However, more model parameters and higher computational cost make these methods unsuitable for devices with limited computing power. In this paper, a lightweight multi-dimension feature fusion network (LMDFFN) is proposed to reduce model parameters. In nonlinear mapping module of the LMDFFN, the lightweight feature extraction block (LFEB) and the multi-dimensional feature extraction block (MDFEB) embedded in each LFEB are designed to significantly improve the reconstruction performance. The channel split operation can group the features. Combined with square-kernel convolution kernel and asymmetric convolution kernel, different types of convolution kernels can focus on different types of features, and the grouping mechanism can effectively reduce the complexity of the model. In addition, a channel and spatial attention module (CSAM) dedicated to SISR is proposed to focus on image details and different semantic level in feature maps. Moreover, a lightweight reconstruction module is designed with depth-wise separable convolution, and the designed spatial attention module is integrated into the reconstruction module to focus on more important features in the reconstruction stage. Compared with recent lightweight SISR methods, LMDFFN has better reconstruction accuracy with lower model parameters and computational complexity, and the reconstructed images have higher visual quality. [ABSTRACT FROM AUTHOR]

This paper presents a novel and efficient multitask learning framework for image translation and saliency detection from remote sensing images, which mainly contains the image translation network-weight sharing attention GAN (WSA-GAN) and the salient object detection network-boundary guidance network (BGNet). WSA-GAN can be used to generate a large number of synthetic infrared remote sensing images (IRIs) or optical remote sensing images (ORIs) from the corresponding complementary modality images. Then, a new multimodal context-aware learning is proposed for feature extraction and to coordinate the entanglement of latent features in the multimodal context of ORIs and IRIs. Since convolutional neural networks do not perform well when the object has directional variance, our framework introduces the attention-aware CapsNet (AACNet) to alleviate the problem and enhance the feature expressiveness. In addition, knowledge distillation strategy is introduced in AACNet to reduce the model complexity. Finally, the multiscale feature learning network and the boundary-aware block are designed to generate more accurate saliency detection results with clear boundaries. Experimental results demonstrate that the presented image translation and salient object detection networks outperform other approaches. [ABSTRACT FROM AUTHOR]

Multi-exposure fusion (MEF) is widely researched and applied to high dynamic range (HDR) imaging, where one of the most challenging problems is the artifacts caused by the motion of objects between input images. Recently, deep learning methods have been widely applied for HDR imaging with excellent results, showing significant advantages. However, many methods cannot avoid artifacts due to inaccurate alignment before merging HDR images. In this paper, we propose an end-to-end network (C-ED-GMNET) with a channel and spatial attention network, an encoder–decoder network and a gradual merging network for generating artifact-free HDR images in dynamic scenes. The attention module consists of two submodules to identify useful features and exclude harmful components in inputs from both channel and spatial dimensions, respectively. The attention-guided feature maps are sent to the encoders for further feature extraction, and then, the outputs are sent to a gradual merging module consisting of two steps to generate deep features progressively. Besides, the differences between the merged image and the original images are identified by applying global residual learning with all the inputs, and the merged image feature is recovered by the decoder to obtain the final HDR image. Quantitative and qualitative experiments on two public datasets show that the proposed C-ED-GMNET produces better results than existing state-of-the-art methods and significantly reduces the runtime due to the encoders which reduce the amount of computation. [ABSTRACT FROM AUTHOR]

Convolutional neural networks have become a powerful tool for partial 3D object detection. However, their power has not been fully realized for focusing on global information, which is crucial for object detection. In this paper, we resolve the problem with a multi-feature voxel transformer (MVTr), an architecture that extracts long-range relationship features through self-attention between multi-feature voxels. In general, converting a point cloud to a voxel representation can reduce a lot of computation, but it would take a long process for the attention network to pay attention to the car voxels in a huge 3D real scene. To this end, we propose a semantic voxel module which takes semantic voxels as input and cooperates with a sparse and a non-empty voxel module to extract features. And the semantic voxels are generated from image segmentation and point cloud projection, which only retains a large number of car voxels. To further enlarge the attention range while maintaining a favorable computational, we propose two attention mechanisms for multi-head attention: local attention and stumpy attention. Finally, we propose the fusion attention module, which can add channel attention and spatial attention to the 2D backbone network. MVTr combines the semantic information of the image and the 3D information of the point cloud and can be applied to most 3D object detection tasks. Experimental results on KITTI dataset show that our method is effective, and the precision has significant advantages compared to other similar feature fusion-based methods. [ABSTRACT FROM AUTHOR]

In order to address the problem of low efficiency and accuracy of artificial visual inspection for the quality of toy sets, this paper investigates the defect detection of toy sets based on machine vision. Firstly, an improved U-Net (IU-Net) is developed by introducing the Squeeze-and-Excitation (SE) block into the U-Net and modifying its objective function. Then, a convolutional neural network (CNN) is constructed for feature extraction and defect matching. Finally, a defect detection model for toy sets is created by integrating these two models. In addition, when it comes to a new-variety toy set, the IU-Net model is trained through transfer learning with a small number of new-variety data samples. This can reduce the dependence on new-variety data samples and improve the efficiency of model training process. The experimental results show that the defect detection accuracy of the proposed method is 100% and 99.43% for the whole toy set and single component, respectively, which meets the requirements of industrial automation quality inspection of toy sets. [ABSTRACT FROM AUTHOR]

With the continuous development of 3D data acquisition technology in recent years, it is more and more convenient to obtain the point cloud data of large scenes, which contains a variety of rich information. How to effectively and accurately classify and segment point cloud data of large scenes has become a research hot-spot in the field of computer vision. In this paper, we study the method based on clustering, make full use of the spatial location and context information, and propose a new network framework, FeatureB2SENet. In the 2D and 3D projection feature calculation, we generate a 32 × 32 × 1 feature image for each point and input it into the convolution neural network to process the feature image. Finally, a comprehensive verification analysis is carried out on GML _ A, GML _ B and Vaihingen data sets, which proves that the classification effect is better. [ABSTRACT FROM AUTHOR]

Rice which is a staple food crop in most Asian countries mainly suffers from higher yield loss due to different factors, and one of the common factors that affect rice yield is rice leaf disease. Many countries do not allow the plantation of new rice varieties unless they are disease-resistant. Hence, disease control serves as an important factor for rice production and this can be effectively accomplished by identifying the rice disease at an early stage which is required for timely pesticide application and disease control. The rice leaf diseases are mainly inspected by the farmers with their naked eye which is error-prone, time-consuming, and often leads to huge yield loss if predicted wrong. This process is effectively handled by different researchers via the use of computer vision and machine learning techniques. To overcome the drawbacks associated with manual processing and low recognition accuracy of different statistical and machine learning techniques, this paper presents a novel concept that uses 3D 2D CNN for feature extraction and an improved backtracking search (IBS) algorithm-optimized deep generative adversarial network (DGAN) for classification. The 3D2D deep convolutional neural network (DCNN) is formed by integrating the 3D fast-learning block with a 2DCNN for extracting the rice disease features such as lesions and shape. The IBS algorithm-optimized GAN architecture is used for rice disease classification and the IBS algorithm is mainly adopted to overcome the instability and overfitting issues that arise in the DGAN. The proposed IBS-optimized GAN architecture offers improved accuracy of 98.7% which is relatively higher than the existing techniques such as extreme gradient boosting (XGBoost), transfer learning techniques, and support vector machine (SVM). [ABSTRACT FROM AUTHOR]

Intelligent recognition of traffic signs and markings is an important component of autonomous driving and intelligent transportation systems. It is also an important theoretical basis for autonomous driving path planning. To address the low accuracy and poor real-time performance of the traffic signs and markings detection in complex and multivariate scenes, a lightweight convolutional neural network recognition method in multiple interference scenes is proposed. Firstly, based on Gamma correction and contrast limited histogram equalization algorithm, the traffic signs and markings image under the low illumination condition is enhanced adaptively. Then, the MobileNet-V2 fuse together with DeepLab-V3 + algorithm to segment traffic signs and markings in multiple scenes. Finally, an identification model is established to realize the adaptive recognition of traffic signs and markings, which are based on Lightweight Convolutional Neural Networks (Lw-CNN). Besides, the recognition method proposed in this paper is verified by the CSUST Chinese Traffic Sign Detection Benchmark (CCTSDB). The experimental results show that the Mean Intersection Over Union (MIOU) of the MobileNet-V2 and DeepLab-V3 + algorithm reaches 83.07%, it increased by 25.8% than before image enhancement. The recognition accuracy of algorithm based on lightweight convolutional neural network is 99.92%, higher than the algorithms of MobileNet-V2 and VGG16. [ABSTRACT FROM AUTHOR]

Unlike 2D images that are represented in regular grids, 3D point clouds are irregular and unordered, hence directly applying convolution neural networks (CNNs) to process point clouds is quite challenging. In this paper, we propose a novel deep neural network named AKNet to achieve point cloud semantic segmentation. The key to our AKNet is the attentive kernel convolution (AKConv), which is a deformed convolution operation for perceiving sufficient local context of 3D scenes. AKConv first constructs the Basic Weight Units that are robust to point's ordering. Then, for capturing the more distinctive local features, the convolution kernels of AKConv are associated with Attentive Weight Units through the self-attentive function acting on Basic Weight Units. Furthermore, 3D point clouds provide richer geometric shape information, which is helpful to recognize objects. However, inputting only raw point features to the convolution function could cause geometric information loss. Thus, we utilize augmented features as input of AKConv. Besides, to preserve the semantic information from the encoding to decoding layers, we introduce the backward encoding (BE) mechanism by utilizing higher-layer semantic features. We conduct experiments on three large-scale point clouds datasets. The experimental results demonstrate that our AKNet outperforms state-of-the-art (SOTA) networks. [ABSTRACT FROM AUTHOR]

Anteflexion angle is an essential health indicator to assess physical flexibility, and image recognition has become an effective method to detect physical anteflexion angle. In this paper, we present a physical flexibility detection algorithm under complex backgrounds, based on edge detection with transformers (ED-Former). The algorithm adopts the lightweight self-attention mechanism from the transformer architecture and combines it with an edge detection network. Additionally, we design the angle protector, which is added to the output to reduce the effect of weak edges in the problem image. Then, based on the obtained edge information, we propose a low-complexity ergonomics-based physical regions combination strategy to calculate the anteflexion angle. We select the regions of the shoulder, waist, and knee, and calculate the angle based on the relation between these regions. The experimental results show that the ED-Former has an excellent performance in physical edge detection tasks—notably producing an F1 score of 0.833 and outperforming state-of-the-art CNNs and transformers on these tasks. Furthermore, our network is effective in preventing adversarial attacks. Overall, we find that the average error of anteflexion angle with respect to the true is 1.267°, well within the 5° requirement for physical flexibility detection. [ABSTRACT FROM AUTHOR]

Deep convolutional neural networks can effectively improve the performance of single-image super-resolution reconstruction. Deep networks tend to achieve better performance than others. However, the deep CNNs will lead to a dramatic increase in the size of parameters, limiting its application on embedding and resource-constrained devices, such as smart phone. To address the common problems of blurred image edges, inflexible convolution kernel size selection and slow convergence during training procedure due to redundant network structure in image super-resolution algorithms, this paper proposes a lightweight single-image super-resolution network that fuses multi-level features. The components are mainly two-level nested residual blocks. To better extract features and reduce the number of parameters, each residual block adopts an asymmetric structure. Firstly, it expands twice and then compresses the number of channels twice. Secondly, in the residual block, the feature information of different channels is weighted and fused by adding an autocorrelation weight unit. The quality of the reconstructed image of the proposed method is superior to the existing image super-resolution reconstruction methods in both subjective perception and objective evaluation indicators, and the reconstruction performance is better when the factor is large. [ABSTRACT FROM AUTHOR]

Handwritten character recognition (HCR) is a challenging task because of high intra-class dissimilarity and high inter-class similarity among the character images. Therefore, the features that are used to represent the character images should be such that they maximize the inter-class variations and minimize intra-class variations. In this paper, a multilevel approach for recognition of Meitei Mayek handwritten characters using fusion of features strategy is presented. The proposed methodology has two levels of recognition. The first level employs a simple CNN. A method to identify the structurally similar character classes based on the probability values of the softmax function is also proposed. The fused feature set is employed in the second-level recognition to distinguish between these character classes, thereby enhancing the discriminative power of the fused feature set by reducing the search space in the second level. The fused feature set is a combination of traditional handcrafted feature descriptors and deep features learned by CNN. The fused feature set is fed to an SVM for second-level recognition. The experimental results show a superior performance of the proposed methodology in identifying the structurally similar character classes. This achieves an overall recognition accuracy of 97.08%, which sets the benchmark on the concerned Meitei Mayek handwritten character dataset. The methodology also shows significant results on MNIST, DIDA and CArDIS datasets. [ABSTRACT FROM AUTHOR]

This paper proposes an unconstrained human gaze estimation approach for medium-distance scene based on monocular vision. A recurrent convolution neural network for gaze estimation is designed to construct the mapping relationship from the face image to the 3D gaze vector in the camera space, so as to complete the estimation of the human gaze. Firstly, based on the VICON system and a monocular camera, the face image sequence and gaze vector sequence of human gaze behavior are collected in different situations of medium-distance scenes to construct the time-synchronized gaze estimation dataset. Secondly, the recurrent convolutional gaze estimation neural network is designed, and the head pose feature and eye appearance feature are extracted from single frame image with monocular vision image sequences as input. Thirdly, the extracted gaze features are fused in the spatial dimension and the temporal dimension to estimate the direction of the human gaze. Finally, the cross-validation and experimental evaluation of the estimation errors of the constructed gaze estimation model are carried out. The results show that the Cross-Person experimental average error is 7.65°, and the Person-Specific experimental error is 3.88°. Compared with the Gaze360 method, the proposed approach reduces the average evaluation error of the gaze angle by 5.0% in the Cross-Person experiments and 16.6% with the Person-Specific experiments. These experiments verify the effectiveness of the proposed approach, and it has better generalization ability and robustness. [ABSTRACT FROM AUTHOR]

Pupil localization is one of the most critical and essential requirements for eye gaze estimation and eye movement tracking. Because pupil images contain monotonous and uncomplicated information, the dataset uses a single class of labels to describe the image content, and using convolutional neural networks can quickly and accurately identify the pupil position on the input image. On low-resolution images, traditional methods encounter issues of low accuracy and cumbersome design steps. A lightweight pupil localization algorithm is proposed in this paper, utilizing a convolutional neural network (CNN) with additional training samples. The experimental results demonstrate the algorithm’s significant effectiveness in identifying the pupil position within the training set, with the accuracy of pupil position in the test set reaching 97.78%. This provides an evidence of the algorithm’s feasibility for accurately localizing pupils in low-resolution images. [ABSTRACT FROM AUTHOR]