Search

Forest fire is a severe natural disaster, which leads to the destruction of forest ecology. At present, fire detection technology represented by convolutional neural network is widely used in forest resource protection, which can realize rapid analysis. However, in forest flame and smoke detection tasks, due to continuous expansion of the target range, a better detection effect cannot be achieved. This paper proposes an improved YOLOX method for multi-scale forest fire detection. This method proposes a novel feature pyramid model to reduce the information loss of high-level forest fire feature maps and enhance the representation ability of feature pyramids. Moreover, the method applies a small object data augmentation strategy to enrich the forest fire dataset, making it more suitable for the actual forest fire scene. According to the experimental results, the mAP of the model proposed in this paper reaches 79.64%, which is about 4.89% higher than the baseline network YOLOX. The method improves the accuracy of forest fire detection, reduces false alarms, and is suitable for real scenarios of forest fires. [ABSTRACT FROM AUTHOR]

Deep convolutional neural networks can effectively improve the performance of single-image super-resolution reconstruction. Deep networks tend to achieve better performance than others. However, the deep CNNs will lead to a dramatic increase in the size of parameters, limiting its application on embedding and resource-constrained devices, such as smart phone. To address the common problems of blurred image edges, inflexible convolution kernel size selection and slow convergence during training procedure due to redundant network structure in image super-resolution algorithms, this paper proposes a lightweight single-image super-resolution network that fuses multi-level features. The components are mainly two-level nested residual blocks. To better extract features and reduce the number of parameters, each residual block adopts an asymmetric structure. Firstly, it expands twice and then compresses the number of channels twice. Secondly, in the residual block, the feature information of different channels is weighted and fused by adding an autocorrelation weight unit. The quality of the reconstructed image of the proposed method is superior to the existing image super-resolution reconstruction methods in both subjective perception and objective evaluation indicators, and the reconstruction performance is better when the factor is large. [ABSTRACT FROM AUTHOR]

To equip convolutional neural networks (CNNs) with explainability, it is essential to interpret how opaque models make specific decisions, understand what causes the errors, improve the architecture design, and identify unethical biases in the classifiers. This paper introduces ADVISE, a new explainability method that quantifies and leverages the relevance of each unit of the feature map to provide better visual explanations. To this end, we propose using adaptive bandwidth kernel density estimation to assign a relevance score to each unit of the feature map with respect to the predicted class. We also propose an evaluation protocol to quantitatively assess the visual explainability of CNN models. Our extensive evaluation of ADVISE in image classification tasks using pretrained AlexNet, VGG16, ResNet50, and Xception models on ImageNet shows that our method outperforms other visual explainable methods in quantifying feature-relevance and visual explainability while maintaining competitive time complexity. Our experiments further show that ADVISE fulfils the sensitivity and implementation independence axioms while passing the sanity checks. The implementation is accessible for reproducibility purposes on https://github.com/dehshibi/ADVISE. [ABSTRACT FROM AUTHOR]

Lung nodules are abnormal growths and lesions may exist. Both lungs may have nodules. Most lung nodules are harmless (not cancerous/malignant). Pulmonary nodules are rare in lung cancer. X-rays and CT scans identify the lung nodules. Doctors may term the growth a lung spot, coin lesion, or shadow. It is necessary to obtain properly computed tomography (CT) scans of the lungs to get an accurate diagnosis and a good estimate of the severity of lung cancer. This study aims to design and evaluate a deep learning (DL) algorithm for identifying pulmonary nodules (PNs) using the LUNA-16 dataset and examine the prevalence of PNs using DB-Net. The paper states that a new, resource-efficient deep learning architecture is called for, and it has been given the name of DB-NET. When a physician orders a CT scan, they need to employ an accurate and efficient lung nodule segmentation method because they need to detect lung cancer at an early stage. However, segmentation of lung nodules is a difficult task because of the nodules' characteristics on the CT image as well as the nodules' concealed shape, visual quality, and context. The DB-NET model architecture is presented as a resource-efficient deep learning solution for handling the challenge at hand in this paper. Furthermore, it incorporates the Mish nonlinearity function and the mask class weights to improve segmentation effectiveness. In addition to the LUNA-16 dataset, which contained 1200 lung nodules collected during the LUNA-16 test, the LUNA-16 dataset was extensively used to train and assess the proposed model. The DB-NET architecture surpasses the existing U-NET model by a dice coefficient index of 88.89%, and it also achieves a similar level of accuracy to that of human experts. [ABSTRACT FROM AUTHOR]

Multi-exposure fusion (MEF) is widely researched and applied to high dynamic range (HDR) imaging, where one of the most challenging problems is the artifacts caused by the motion of objects between input images. Recently, deep learning methods have been widely applied for HDR imaging with excellent results, showing significant advantages. However, many methods cannot avoid artifacts due to inaccurate alignment before merging HDR images. In this paper, we propose an end-to-end network (C-ED-GMNET) with a channel and spatial attention network, an encoder–decoder network and a gradual merging network for generating artifact-free HDR images in dynamic scenes. The attention module consists of two submodules to identify useful features and exclude harmful components in inputs from both channel and spatial dimensions, respectively. The attention-guided feature maps are sent to the encoders for further feature extraction, and then, the outputs are sent to a gradual merging module consisting of two steps to generate deep features progressively. Besides, the differences between the merged image and the original images are identified by applying global residual learning with all the inputs, and the merged image feature is recovered by the decoder to obtain the final HDR image. Quantitative and qualitative experiments on two public datasets show that the proposed C-ED-GMNET produces better results than existing state-of-the-art methods and significantly reduces the runtime due to the encoders which reduce the amount of computation. [ABSTRACT FROM AUTHOR]

Chinese calligraphy is one of the excellent expressions of Chinese traditional art. But people without domain knowledge of calligraphy can hardly read, appreciate, or learn this art form, due to it contains many brush strokes with unique shapes and complicate structural topological relationship. In this paper, we explore the solution of text sequence recognition of calligraphy, which is a challenging task because traditional algorithms of text recognition can rarely obtain the satisfied results for the varied styles of calligraphy. Therefore, based on a trainable neural network, this paper proposes an easy recognition method, which combines feature sequence extraction based on DenseNet (Dense Convolutional Network) model, sequence modeling and transcription into a consolidated architecture. Compared with previous algorithms for text recognition, it has two distinctive properties: One is to acquire the artistic features on shapes and structures of different styles of calligraphic characters and the other is to handel sequences in arbitrary lengths without character segmentation. Our experimental results prove that in contrast with several common recognition methods, our method of Chinese calligraphic character in diverse styles demonstrates greater robustness, and the recognition accuracy rate reaches 84.70%. [ABSTRACT FROM AUTHOR]

The experimental purpose of X-ray diffraction is to analyze crystalline material structure at the atomic and molecular levels. Such experiments are known as X-ray crystallography. Traditionally, human experts do it with some domain knowledge. X-ray crystallography is useful in numerous domains, e.g., physics, chemistry, and biology. It is tough to own manual physics of diffraction patterns to see a crystal structure with a colossal data set. A convolutional neural network (CNN) is a deep neural network that maps an input image into a high-dimensional space. CNN produces an affordable function for image classification. This paper uses an extension of the convolutional neural network to predict crystal structure from diffraction patterns. We propose a machine-enabled method to predict crystallographic size and space group from a limited number of XRD patterns for small films. We overcome the problem of scarce data within the development of building materials by combining the learning model of moderately monitored equipment, a physics information-enhancing strategy using data generated from the Inorganic Crystal Structure Database, and test data. We compare our approach with a large variety of typical addition as modern machine learning-based classification techniques for crystal structure prediction. Results show that our proposed system outperforms all the baselines by a significant margin for the crystal structure prediction task. Results also show the impact of the number of layers in the all-convolutional neural network architecture for crystal structure prediction. [ABSTRACT FROM AUTHOR]

Handwritten character recognition (HCR) is a challenging task because of high intra-class dissimilarity and high inter-class similarity among the character images. Therefore, the features that are used to represent the character images should be such that they maximize the inter-class variations and minimize intra-class variations. In this paper, a multilevel approach for recognition of Meitei Mayek handwritten characters using fusion of features strategy is presented. The proposed methodology has two levels of recognition. The first level employs a simple CNN. A method to identify the structurally similar character classes based on the probability values of the softmax function is also proposed. The fused feature set is employed in the second-level recognition to distinguish between these character classes, thereby enhancing the discriminative power of the fused feature set by reducing the search space in the second level. The fused feature set is a combination of traditional handcrafted feature descriptors and deep features learned by CNN. The fused feature set is fed to an SVM for second-level recognition. The experimental results show a superior performance of the proposed methodology in identifying the structurally similar character classes. This achieves an overall recognition accuracy of 97.08%, which sets the benchmark on the concerned Meitei Mayek handwritten character dataset. The methodology also shows significant results on MNIST, DIDA and CArDIS datasets. [ABSTRACT FROM AUTHOR]

Face recognition in visual surveillance systems is important for various applications to identify individuals who are behaving defiantly at the time of an event or for investigation purposes. Despite the dramatic improvements in facial recognition technology in recent years, it is difficult to recognize faces from surveillance feeds due to the presence of multiple people of different scales and orientations. This paper solves the task of low-resolution face recognition by combining exemplary techniques for extracting distinct features. This research utilizes the attributes learned by customized and learnable filters and injected in the training process to better match them with human brain functionality. The Gabor transform aims to convolve a facial image using a range of Gabor filter coefficients at various scales and orientations, resulting in scale and rotation invariant features. The tailored architecture with residual stream aims to enhance functional representation and prevent the gradient of the prediction engine from affecting the backbone network functional map. Experimental analysis is performed on the SCface and TinyFace databases and is reported with an accuracy of 89.21% on the SCface database and 56.68% on the TinyFace database. [ABSTRACT FROM AUTHOR]

Depth-estimation is an important task for autonomous driving, 3D object detection and recognition, scene understanding, and other fields. To improve the quality of depth estimation in high-frequency edge detail area, this paper presents DSS-Net, a dual-stream stereo network, combining a bottom-up steam based on scene understanding and a top-down stream based on parallax local optimization. Firstly, in the bottom-up stream, deep features containing high-level semantic information are extracted by a deep network, and then, a coarse estimate of the disparity is computed according to depth intervals classified based on deep features. In the up-bottom stream, the model uses the high-resolution shallow features with rich details and the initial coarse disparity to construct the local dense matching cost in the parallax neighborhood of each pixel of the initial disparity map, and uses stacked multiple hourglass networks to refine the parallax diagram in several stages. We achieve results on the Scene Flow, KITTI 2012 and 2015 datasets, showing that our method has high precision. [ABSTRACT FROM AUTHOR]

It is a challenging task to segment drivable area of road in automatic driving system. Convolutional neural network has excellent performance in road segmentation. However, the existing segmentation methods only focus on improving the performance of road segmentation under good road conditions, but pay little attention to the performance of road segmentation under severe weather conditions. In this paper, an image enhancement network (IEC-Net) based on CycleGAN is proposed to enhance the diversified features of input images. Firstly, an unsupervised CycleGAN network is trained to feature enhance road images under severe weather conditions, so as to obtain an enhanced image with rich feature information. Secondly, the enhanced image is input into the most advanced semantic segmentation network, so as to realize the segmentation of the drivable area of the road. The experimental results show that the IEC-Net based on CycleGAN can be directly combined with any advanced semantic segmentation network and can not only realize end-to-end training, but also greatly improve the performance of the original semantic segmentation network for road segmentation under severe weather conditions. [ABSTRACT FROM AUTHOR]

Compared with hardware-dependent methods, light field (LF) reconstruction algorithms enable requiring the densely sampled light field (DSLF) more economical and convenient. Most of the current LF reconstruction methods either directly apply multiple convolutional layers to the input views to generate DSLF or warp the input views to novel viewpoints based on the estimated depth and then blend to obtain the DSLF. These two types of methods produce either blurry results in texture areas or distortions near the boundaries of depth discontinuities. In this paper, we propose an end-to-end learning-based approach, which combines the characteristics of the above two methods from a parallel and complementary perspective, to reconstruct a high-quality light field. Our method consists of three sub-networks, i.e., two parallel sub-networks termed as ASR-Net and Warp-Net, and the refine sub-network termed as Refine-Net. ASR-Net directly learns an intermediate DSLF through deep convolutional layers, and Warp-Net warps the input views based on the estimated depth to obtain other intermediate DSLFs that maintain high-frequency texture information. These intermediate DSLFs are adaptively fused through the learned attention maps, and a fusion DSLF with the advantages of two types of intermediate DSLFs is obtained. Finally, the residual map learned from the intermediate DSLFs is added to the fusion DSLF to get a better one. Comprehensive experiments demonstrate the superiority of the proposed method on several LF datasets compared with the state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Center and scale prediction (CSP) is an anchor-free pedestrian detector with good performance. However, there are lots of parameters in the detector, which seriously limits the speed. In this paper, a new network is designed for the improvement of the detector speed, which contains less parameters, named Feature Fusion: Center and Scale Prediction (F-CSP). F-CSP fuses multi-scale feature maps with two efficient feature fusion networks: Feature Pyramid Networks (FPN) and Balanced Feature Pyramid (BFP). Specifically, FPN is used to reduce the channel of feature maps, and BFP is used to fuse multiple feature maps into a single one. This way, the proposed detector achieves competitive accuracy and higher speed on the challenging pedestrian detection benchmark. The performance of F-CSP is demonstrated on the Caltech dataset. Compared with CSP, under the premise of ensuring accuracy, the speed is increased from 45.1 to 32.9 ms/img. [ABSTRACT FROM AUTHOR]

This paper introduces a large-scale Meitei Mayek handwritten character database. It consists of the complete character set of the script. There are a total of 85,124 character images of 55 character classes with 72,330 and 12,794 images in training and test sets, respectively. The present work focuses on collecting the natural handwriting of individuals by carrying out sample collection in two phases: (a) unconstrained handwriting in the form of answer sheets and classroom notes and (b) tabular forms. A total of nearly 500 individuals have contributed in the development of the database. Recognition of the character images in the database is carried out using different feature descriptors with four popular classifiers, namely KNN, Linear Support Vector Classifier, Random Forest and Support Vector Machine. The paper also proposes a convolutional neural network (CNN) model by enhancing a base CNN architecture by optimally tuning the hyperparameters. Experimental results show that the CNN model can be benchmarked against the concerned database with a test accuracy of 95.56%. [ABSTRACT FROM AUTHOR]

With the continuous development of EMG acquisition technology and artificial intelligence technology, EMG signal analysis has been extensively studied in human–computer interaction, rehabilitation training, prosthetic control and remote device control. As hand movements become more and more complex, hand movement recognition based on surface electromyography (sEMG) has become a hotspot. In this paper, by using multi-features fusion-based Long Short-Term Memory convolutional neural network (MFFCNN-LSTM), a continuous hand movement recognition method based on time-domain and time–frequency-spectrum features of forearm sEMG signal is proposed. Ten basic hand movements including rest action are identified. Firstly, the hand movement data is cut from NinaPro db8 dataset to extract effective sEMG signal fragments. Secondly, the empirical Fourier decomposition method is used to denoise the sEMG signals. Thirdly, the time-domain and time–frequency-spectrum features of sEMG signals from different channels are extracted, and sent to two parallel CNN networks to extract the high-dimension features, respectively. Fourthly, the high-dimension features are fused as the input of LSTM, a fully connected layer and a softmax layer to recognize the continuous hand movements. Finally, MFFCNN-LSTM is compared with the support vector machine, CNN and LSTM on the same computer. The experimental results show that the recognition accuracy, sensitivity and specificity of MFFCNN-LSTM on NinaPro db8 dataset are 98.5%, 95.25% and 95.5%, respectively. It has higher recognition accuracy on the five public datasets than other methods. [ABSTRACT FROM AUTHOR]

Detection of surface defects in manufacturing systems is crucial for product quality. Detection of surface defects with high accuracy can prevent financial and time losses. Recently, efforts to develop high-performance automatic surface defect detection systems using computer vision and machine-learning methods have become prominent. In line with this purpose, this paper proposed a novel approach based on Depth-wise Squeeze and Excitation Block-based Efficient-Unet (DSEB-EUNet) for automatic surface defect detection. The proposed model consists of an encoder–decoder, the basic structure of the Unet architecture, and a Depth-wise Squeeze and Excitation Block added to the skip-connection of Unet. First, in the encoder part of the proposed model, low-level and high-level features were obtained by the EfficientNet network. Then, these features were transferred to the Depth-wise Squeeze and Excitation Block. The proposed DSEB based on the combination of Squeeze-Excitation and Depth-wise Separable Convolution enabled to reveal of critical information by weighting the features with a lightweight gating mechanism for surface defect detection. Besides, in the decoder part of the proposed model, the structure called Multi-level Feature Concatenated Block (MFCB) transferred the weighted features to the last layers without losing spatial detail. Finally, pixel-level defect detection was performed using the sigmoid function. The proposed model was tested using three general datasets for surface defect detection. In experimental works, the best F1-scores for MT, DAGM, and AITEX datasets using the proposed DSEB-EUNet architecture were 89.20%, 85.97%, and 90.39%, respectively. These results showed the proposed model outperforms higher performance compared to state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

At present, gastric cancer patients account for a large proportion of all tumor patients. Gastric tumor image segmentation can provide a reliable additional basis for the clinical analysis and diagnosis of gastric cancer. However, the existing gastric cancer image datasets have disadvantages such as small data sizes and difficulty in labeling. Moreover, most existing CNN-based methods are unable to generate satisfactory segmentation masks without accurate labels, which are due to the limited context information and insufficient discriminative feature maps obtained after the consecutive pooling and convolution operations. This paper presents a gastric cancer lesion dataset for gastric tumor image segmentation research. A multiscale boundary neural network (MBNet) is proposed to automatically segment the real tumor area in gastric cancer images. MBNet adopts encoder–decoder architecture. In each stage of the encoder, a boundary extraction refinement module is proposed for obtaining multi granular edge information and refinement firstly. Then, we build a selective fusion module to selectively fuse features from the different stages. By cascading the two modules, the richer context and fine-grained features of each stage are encoded. Finally, the astrous spatial pyramid pooling is improved to obtain the remote dependency relationship of the overall context and the fine spatial structure information. The experimental results show that the accuracy of the model reaches 92.3%, the similarity coefficient (DICE) reaches 86.9%, and the performance of the proposed method on the CVC-ClinicDB and Kvasir-SEG datasets also outperforms existing approaches. [ABSTRACT FROM AUTHOR]

This paper investigates the issue of crowd counting for crowd images. A novel method named two-task convolutional neural network (T 2 CNN) is proposed to simultaneously learn two tasks for crowd counting: the tasks of dense degree classification and density map estimation. Basically, T 2 CNN contains two main modules: dense degree classification (DDC) and density map estimation. Generally, it is sufficient for crowd counting to estimate density maps of images. It is known that images may have different dense degrees and the local regions in the same image have different dense degrees. Thus, dense degree may provide the high-level prior of images and is excepted to help the estimation of density maps. To better estimate density maps of images, T 2 CNN incorporates the DDC module into the task of density map estimation. T 2 CNN takes the scale-adaptive convolutional neural network as the estimator of density maps, which adopts multi-scale layers to tackle the scale and perspective variations that exist in crowd images commonly. To verify the effectiveness and robustness of our model, we conduct extensive experiments on the Shanghaitech, UCF_CC_50 and WorldExpo'10 datasets. Experimental results indicate that T 2 CNN achieves good performance. [ABSTRACT FROM AUTHOR]

We propose a fully automatic approach to restore aged old line drawings. We decompose the task into two subtasks: the line extraction subtask, which aims to extract line fragments and remove the paper texture background, and the restoration subtask, which fills in possible gaps and deterioration of the lines to produce a clean line drawing. Our approach is based on a convolutional neural network that consists of two sub-networks corresponding to the two subtasks. They are trained as part of a single framework in an end-to-end fashion. We also introduce a new dataset consisting of manually annotated sketches by Leonardo da Vinci which, in combination with a synthetic data generation approach, allows training the network to restore deteriorated line drawings. We evaluate our method on challenging 500-year-old sketches and compare with existing approaches with a user study, in which it is found that our approach is preferred 72.7% of the time. [ABSTRACT FROM AUTHOR]

In this paper, a lightweighted Intensive Feature Extrication Deep Network (ExtriDeNet) is proposed for precise hand gesture recognition (HGR). ExtriDeNet primarily consists of two blocks: Intensive Feature Fusion Block (IFFB) and Intensive Feature Assimilation Block (IFAB). IFFB incorporates two different scaled filters 3 × 3 and 5 × 5 to capture contextual features of hands, while IFAB is designed by embedding influential features of IFFB with two extreme minute and high-level feature responses from two receptive fields generated by employing 1 × 1 and 7 × 7 sized filters, respectively. The combination of multiscaled filters enriches the network with the most significant features and enhances the learnability of the network. Thus, the proposed ExtriDeNet efficiently defines the distinctive features of different hand gesture classes and achieves high performance as compared to state-of-the-art HGR approaches. The performance of the proposed network is evaluated on the standard datasets: MUGD, Finger Spelling, OUHands, NUS-I, NUS-II and HGR1 for both subject-dependent and subject-independent scheme. [ABSTRACT FROM AUTHOR]

Human brain is known to have a layered structure and perform not only feedforward process from lower layer to upper layer, but also feedback process from upper layer to lower layer. Neural network is a mathematical model of the function of neurons, and several models are proposed until now. Although neural network imitates the human brain, everyone uses only feedforward process and direct feedback process from upper layer to lower layer is not used in prediction process. Therefore, in this paper, we propose Feedback U-Net using convolutional LSTM. Our model is a segmentation model using convolutional LSTM and feedback process. The output of U-Net at the first round is fed back to the input, and our method re-considers the segmentation result at the second round. By using convolutional LSTM, the features are extracted well based on the features extracted at the first round. On both of the Drosophila cell image and Mouse cell image datasets, our model outperformed conventional U-Net which uses only feedforward process. [ABSTRACT FROM AUTHOR]

Early identification of crop disease can aid the farmers to take timely precautions and countermeasures for its removal. In this paper, a real-time system to identify the type of disease present in a crop based on leaf images using machine learning is proposed. A deep convolutional neural network architecture is proposed to classify the crop disease, and a single shot detector is used for identification and localization of the leaf. These models are deployed on an embedded hardware, Nvidia Jetson TX1, for real-time in-field plant disease detection and identification. The disease classification accuracy achieved is around 96.88%, and the classification results are compared with existing convolutional neural network architectures. Also, the high success rate of the proposed system in the actual field test makes the proposed system a completely deployable system. [ABSTRACT FROM AUTHOR]

High angular resolution light field (LF) enables exciting applications such as depth estimation, virtual reality, and augmented reality. Although many light field angular super-resolution methods have been proposed, the reconstruction problem of LF with a wide-baseline is far from being solved. In this paper, we propose an end-to-end learning-based approach to achieve angular super-resolution of the light field with a wide-baseline. Our model consists of three components. We first train a convolutional neural network to predict the depth map for each sub-aperture view. Then the estimated depth maps are used to warp the input views. In the final component, we first use a convolutional neural network to fuse the initial warped light fields, and then we propose an edge-aware inpainting network to modify the inaccurate pixels in the near-edge regions. Accordingly, we design an EdgePyramid structure that contains multi-scale edges to perform the inpainting of near-edge pixels. Moreover, we introduce a novel loss function to reduce the artifacts and further estimate the similarity in near-edge regions. Experimental results on various light field datasets including large-baseline light field images show that our method outperforms the state-of-the-art light field angular super-resolution methods, especially in the terms of visual performance near edges. [ABSTRACT FROM AUTHOR]

Intelligent video surveillance systems have been used recently for automatic monitoring of human interactions. Although they play a significant role in reducing security concerns, there are many challenges for distinguishing between normal and abnormal behaviors such as crowded environments and camera viewpoint. In this paper, we propose a novel deep violence detection framework based on the specific features derived from handcrafted methods. These features are related to appearance, speed of movement, and representative image and fed to a convolutional neural network (CNN) as spatial, temporal, and spatiotemporal streams. The spatial stream trained the network with each frame in the video to learn environment patterns. The temporal stream contained three consecutive frames to learn motion patterns of violent behavior with a modified differential magnitude of optical flow. Moreover, in spatio-temporal stream, we introduced a discriminative feature with a novel differential motion energy image to represent violent actions more interpretable. This approach covers different aspects of violent behavior by fusing the results of these streams. The proposed CNN network is trained with violence-labeled and normal-labeled frames of 3 Hockey, Movie, and ViF datasets which comprised both crowded and uncrowded situations. The experimental results showed that the proposed deep violence detection approach outperformed state-of-the-art works in terms of accuracy and processing time. [ABSTRACT FROM AUTHOR]

Background: Chromosome analysis plays an important role in detecting genetic disorders. However, it is time-consuming when it is done manually. The first step for an automated solution is removing the background noise in the chromosome images. Denoising is studied by many researchers; however, it is still a challenging task due to contrast issues, blotches, and non-chromosome objects. Methods: In this paper, we proposed a cascaded neural network architecture for denoising G-banding chromosomes images. The proposed method consists of two steps. The first step is the initial segmentation network which combines the capabilities of U-net and residual units. The second step is the classification block, which is implemented in order to automate the denoising process and reduce the pixel losses on the chromosomes. Results: The results showed that the proposed segmentation network achieves a higher dice score compared to state-of-the-art semantic segmentation neural networks, and the classification block greatly reduces the pixel losses on the chromosomes. We tested the proposed denoising method on 84 G-banding chromosome images and achieved a 98.74% dice score. Conclusion: Our automated denoising method outperformed the methods presented in previous studies and state-of-the-art methods. It can help cytogeneticists with repetitive work and provide them more accurate chromosomes for further evaluation. [ABSTRACT FROM AUTHOR]

Visual tracking is the process of estimating the position of an object in a video sequence and plays a very important role in the field of autonomous video processing. Recent work renders that the trackers developed using deep learning techniques such as the convolutional neural network (CNN) exhibits outstanding performances in terms of accuracy and robustness as compared to other state-of-the-art trackers. Multi-domain convolutional neural network (MDNet) is a deep tracker which uses the CNN for estimating the target in each frame of the video sequence. The majority of the tracking challenges could be very easily handled by the MDNet tracker due to its offline training and online tracking features. The offline training stage helps in capturing the target representations into the shared layers of the CNN, while the online tracking uses a large number of probable random samples of bboxes (bounding boxes) around the previous target for estimating the target in the current frame. Once the target is estimated, a process of fine-tuning is performed which will update the weights of shared layers of CNN. The large number of random samples used for target estimation and the huge number of random training samples generated for the fine-tuning during the online stage makes tracking by MDNet computationally complex and slow. The major contribution of this paper is to suggest using guided samples to the input of the CNN rather than random samples. Moreover, it generates a lesser number of highly efficient training samples for the fine-tuning which helps in decreasing the computational complexity of the tracker by half without much compromise on the performance and thus improves the speed of the tracking process. An extensive evaluation has been performed on the proposed Guided MDNet with different datasets like ALOV300++, OTB and VOT, and its performances are measured in terms of metrics like F-score, one-pass evaluation, robustness and accuracy. [ABSTRACT FROM AUTHOR]

In recent years, the deep hashing image retrieval algorithm has become a hot spot in current research. Although the deep hashing algorithm has achieved good results in image retrieval, how to further improve the accuracy of the image retrieval algorithm and reduce the computational complexity of the algorithm, the two basic requirements of the algorithm, need attention in image retrieval. The paper proposes a new Aggregate Deep Fast Supervised Discrete Hashing (ADFSDH) method for highly efficient image retrieval on large-scale datasets. Specifically, in order to improve the algorithm performance, the paper first proposes a new Aggregate Deep Convolutional Neural Network (ADCNN) mode based on VGG16, VGG19 and transfer learning for effective image feature extraction, which contains two different feature extractors in parallel. And then, the paper proposes a new feature fusion method. When our weighted proportion is consistent with the Mean Average Precision results of two different feature extractors, we can obtain the most accurate description of the image. Firstly, in order to save ADCNN required storage space and improve ADCNN image retrieval efficiency, the Fast Supervised Discrete Hashing algorithm after adjusting the parameters is introduced into the ADCNN model. In addition, ADFSDH unifies feature learning and hash coding into the same framework. The proposed method was experimented on three datasets (CIFAR10, MNIST and FD-XJ), and the result shows that it is superior to the current mainstream approaches in image retrieval. [ABSTRACT FROM AUTHOR]

Medical segmentation is a task that pays attention to details. The rapid development of deep learning in image processing technology makes it possible to segment objects accurately on small datasets. In this paper, we propose a hierarchical multi-scale attention network that focuses on the fine-grained parts of the target. Our attention network consists of a hierarchical encoder module with dense connections, a multi-scale module attention to fine-grained parts, and a decoder module. We also combine the weighted cross-entropy loss function based on details and the Dice coefficient loss to increase the sensitivity of fine grains. To verify our module's performance, we carried out a series of comparative experiments on the multi-scale attention module on the DRIVE dataset. We determine the best structure through experiments and compare it with several classical deep learning models. Our experiments show that extracting semantic information of images at an appropriate resolution can also improve the accuracy of detail segmentation. To show the generalization ability of the work, we conducted experiments on different DRIVE, STARE, and CHASE_DB 1 datasets, and our method achieved 0.8802/0.8464/0.8216 in sensitivity performance metric, 0.9756/0.9869/0.9784 in specificity, and 0.9675/0.9657/0.9637 in accuracy. [ABSTRACT FROM AUTHOR]

Some hardware is similar in color and shape between different classes, and some hardware varies within a class, thereby decreasing the accuracy of classification models. The baseline of image classification depends on network architectures. To solve this problem, this paper proposes an improved residual network architecture that achieves high accuracy in hardware classification. This paper proposes a hardware dataset that fills the blank of hardware images. The improved residual network architectures are based on depthwise over-parameterized convolution and pyramidal convolution. This model could obtain information from different levels and channels. Moreover, compared with the novel architectures, our model does not require increasing the computation. Experiments on the hardware dataset show that our neural network architectures achieved better accuracy compared with notable architectures such as ResNet50 and DenseNet121. The classification accuracy of the proposed network is 94.6% and the f1 score is 90.26%. To verify the proposed network architecture, we applied our model in fine-grained datasets; our model achieved the best performance in the same training setting. [ABSTRACT FROM AUTHOR]

Object detection is an important topic for visual data processing in the visual computing area. Although a number of approaches have been studied, it still remains a challenge. There is a suitable way to promote image classifiers by blending training with blended images and corresponding blended labels. However, our experiments show that directly moving existing blending methods from classification to object detection will cause the training process become harder and eventually will lead to a bad performance. Inspired by our discovery, this paper presents a multi-phase blending method with incremental blending intensity to improve the accuracy of object detectors and achieve remarkable improvements. Firstly, to adapt blending method to detection task, we propose a smoothly scheduled and incremental blending intensity to control the degree of multi-phase blending. Based on the above dynamic coefficient, we propose an incremental blending method, in which the blending intensity is smoothly increased from zero to full. Therefore, more complex and various data can be created to achieve the goal of regularizing the network. Secondly, we also design an incremental hybrid loss function to replace the original loss function. The blending intensity in our loss function increases smoothly, which is controlled by our scheduled coefficient. Thirdly, we further discard more negative examples in our multi-phase training process than other typical training methods and processes. By doing so, we can regularize the neural network to enhance generalization capability with data diversity and eventually to improve the accuracy in object detection. Another advantage is that there is no negative effect on evaluation because our method is just applied during the training process. Typical experiments show the proposed method improves the generalization of the detection networks. On PASCAL VOC and MS COCO, our method outperforms the state-of-the-art RFBNet of one-stage detectors for real-time processing. [ABSTRACT FROM AUTHOR]

Multi-label image classification is a fundamental and challenging task in computer vision. Although remarkable success has been achieved by applying CNN–RNN pattern, such method has a slow convergence rate due to the existence of RNN module. Instead of utilizing the RNN modules, this paper proposes a novel channel correlation network which is fully based on convolutional neural network (CNN) to model the label correlations with high training efficiency. By creating a new attention module, the image features obtained by CNN are further convoluted to obtain the correspondence between the label and the channel-wise feature map. Then we use the SE and the convolution operation alternately to eliminate the irrelevant information to better explore the label correlation. Experiments on PASCAL VOC 2007 and MIRFlickr25k show that our model can effectively exploit the dependencies between multiple tags to achieve better performance. [ABSTRACT FROM AUTHOR]

Sign language is a visual language used by persons with hearing and speech impairment to communicate through fingerspellings and body gestures. This paper proposes a framework for automatic sign language recognition without the need of hand segmentation. The proposed method first generates three different types of motion templates: motion history image, dynamic image and our proposed RGB motion image. These motion templates are used to fine-tune three ConvNets trained on ImageNet dataset. Fine-tuning avoids learning all the parameters from scratch, leading to faster network convergence even with a small number of training samples. For combining the output of three ConvNets, we propose a fusion technique based on Kernel-based extreme learning machine (KELM). The features extracted from the last fully connected layer of trained ConvNets are used to train three KELMs, and the final class label is predicted by averaging their scores. The proposed approach is validated on a number of publicly available sign language as well as human action recognition datasets, and state-of-the-art results are achieved. Finally, an Indian sign language dataset is also collected using a thermal camera. The experimental results obtained show that our ConvNet-based deep features along with proposed KELM-based fusion are robust for any type of human motion recognition. [ABSTRACT FROM AUTHOR]

Recently, the prediction of interactions in videos has been an active subject in computer vision. Its goal is to deduce interactions in their early stages. Many approaches have been proposed to predict interaction, but it still remains a challenging problem. In the present paper, features are optical flow fields extracted from video frames using convolutional neural networks. This feature, which is extracted from successive frames, constructs a time series. Then, the problem is modeled in the form of a time series prediction. Prediction of the interaction type is based on matching the time series under experiment with the time series available in the training set. Dynamic time warping provides an optimal match between a pair of time-series data by a nonlinear mapping between two data. Finally, the SVM and KNN classification methods with dynamic time warping distance are used to predict the video label. The results showed that the proposed model improved on standard interaction recognition datasets including the TVHI, BIT, and UT interaction. [ABSTRACT FROM AUTHOR]

With the recent development and application of human–computer interaction systems, facial expression recognition (FER) has become a popular research area. The recognition of facial expression is a difficult problem for existing machine learning and deep learning models because that the images can vary in brightness, background, pose, etc. Deep learning methods also require the support of big data. It does not perform well when the database is small. Feature extraction is very important for FER, even a simple algorithm can be very effective if the extracted features are sufficient to be separable. However, deep learning methods automatically extract features so that some useless features can interfere with useful features. For these reasons, FER is still a challenging problem in computer vision. In this paper, with the aim of coping with few data and extracting only useful features from image, we propose new face cropping and rotation strategies and simplification of the convolutional neural network (CNN) to make data more abundant and only useful facial features can be extracted. Experiments to evaluate the proposed method were performed on the CK+ and JAFFE databases. High average recognition accuracies of 97.38% and 97.18% were obtained for 7-class experiments on the CK+ and JAFFE databases, respectively. A study of the impact of each proposed data processing method and CNN simplification is also presented. The proposed method is competitive with existing methods in terms of training time, testing time, and recognition accuracy. [ABSTRACT FROM AUTHOR]

Stereo image segmentation is the key technology in stereo image editing with the population of stereoscopic 3D media. Most previous methods perform stereo image segmentation on both views relying primarily on per-pixel disparities, which results in the segmentation quality closely connected to the accuracy of the disparities. Therefore, a mechanism to remove the errors of the disparities are highly demanded. To date, there's no such a method yet that can produce accurate disparity maps. In this paper, we propose a novel convolutional neural network (CNN)-based framework, which will automatically propagate the segmentation result from one view to the other. The key problem of accurate stereo image segmentation is the missing of occluded regions. To solve this problem, the CNN architecture is proposed to improve the stereo segmentation performance. In order to address the inevitable inaccuracies problem of the disparities computed from a stereo pair of images, we utilize the coherent disparity propagation that propagates segment result via those pixels with coherent disparities. The pixels by coherent disparity propagation and the high confidence pixels of the object probability map produced by the CNN architecture are then used to generate the initial reliable pixels to perform an energy minimization framework-based segmentation. A comprehensive evaluations and comparisons on Middlebury and Adobe benchmark datasets show the effectiveness of our proposed method in terms of high-quality results, and the robustness against various types of inputs. [ABSTRACT FROM AUTHOR]

Binocular tone mapping is studied in the previous works to generate a fusible pair of LDR images in order to convey more visual content than one single LDR image. However, the existing methods are all based on monocular tone mapping operators. It greatly restricts the preservation of local details and global contrast in a binocular LDR pair. In this paper, we proposed the first binocular tone mapping operator to more effectively distribute visual content to an LDR pair, leveraging the great representability and interpretability of deep convolutional neural network. Based on the existing binocular perception models, novel loss functions are also proposed to optimize the output pairs in terms of local details, global contrast, content distribution, and binocular fusibility. Our method is validated with a qualitative and quantitative evaluation, as well as a user study. Statistics show that our method outperforms the state-of-the-art binocular tone mapping frameworks in terms of both visual quality and time performance. [ABSTRACT FROM AUTHOR]

A recent challenge in computer vision is exploring the cardinality of a relationship among multiple visual entities to answer questions like whether the subjects in a photograph have a kin relationship. This paper tackles kinship recognition from the aging viewpoint in which the system could find the parent of a child where the input image of the parent belongs to the age range that is lower than the child is. Technical contributions of this research are twofold. (1) An efficient discriminative feature space is constructed by proposing kernelized bi-directional PCA to form a topological cubic feature space. Cubic feature space in conjunction with the introduced cubic norm is used to solve the kinship problem. (2) To fill the gap of aging effect in finding a kin relation, a semi-supervised learning paradigm is proposed. To do this, first, the pooling layer of a convolutional neural network is modified to do a soft pooling. Then, the last pooling layer, as a rich feature vector, is fed into density-based spatial clustering of applications with noise algorithm. This pre-classification phase would be useful when there is no aggregation on how many classes should be used in the age group estimation task. Finally, by adding kernel computation to sparse representation classifier, the age classification is done. Evaluation of the proposed method on five publicly available facial kinship datasets shows the superiority of the proposed method over both the state-of-the-art kinship verification methods and what is known as human decision-making. [ABSTRACT FROM AUTHOR]

Indoor semantic segmentation plays a critical role in many applications, such as intelligent robots. However, multi-class recognition is still challenging, especially for pixel-level indoor semantic labeling. In this paper, a novel deep structured model that combines the strengths of the widely used convolutional neural networks (CNNs) and recurrent neural networks (RNNs) is proposed. We first present a multi-information fusion model that utilizes the scene category information to fine-tune the fully convolutional network. Then, to refine the coarse outputs of CNN, the RNN is applied to the final CNN layer so that we can build an end-to-end trainable system. This Graph-RNN is transformed from a conditional random field based on superpixel segmentation graphical modeling that can utilize flexible contextual information of different neighboring regions. The experimental results on the recent large SUN RGB-D dataset demonstrate that the proposed model outperforms existing state-of-the-art methods on the challenging 40 dominant classes task (40.8% mean IU accuracy and 69.1% pixel accuracy). We also evaluate our model on the public NYU depth V2 dataset and achieve remarkable performance. [ABSTRACT FROM AUTHOR]

Face recognition for surveillance remains a complex challenge due to the disparity between low-resolution (LR) face images captured by surveillance cameras and the typically high-resolution (HR) face images in databases. To address this cross-resolution face recognition problem, we propose a two-stage dual-resolution face network to learn more robust resolution-invariant representations. In the first stage, we pre-train the proposed dual-resolution face network using solely HR images. Our network utilizes a two-branch structure and introduces bilateral connections to fuse the high- and low-resolution features extracted by two branches, respectively. In the second stage, we introduce the triplet loss as the fine-tuning loss function and design a training strategy that combines the triplet loss with competence-based curriculum learning. According to the competence function, the pre-trained model can train first from easy sample sets and gradually progress to more challenging ones. Our method achieves a remarkable face verification accuracy of 99.25% on the native cross-quality dataset SCFace and 99.71% on the high-quality dataset LFW. Moreover, our method also enhances the face verification accuracy on the native low-quality dataset. [ABSTRACT FROM AUTHOR]

In video streaming, bandwidth constraints significantly affect client-side video quality. Addressing this, deep neural networks offer a promising avenue for implementing video super-resolution (VSR) at the user end, leveraging advancements in modern hardware, including mobile devices. The principal challenge in VSR is the computational intensity involved in processing temporal/spatial video data. Conventional methods, uniformly processing entire scenes, often result in inefficient resource allocation. This is evident in the over-processing of simpler regions and insufficient attention to complex regions, leading to edge artifacts in merged regions. Our innovative approach employs semantic segmentation and spatial frequency-based categorization to divide each video frame into regions of varying complexity: simple, medium, and complex. These are then processed through an efficient incremental model, optimizing computational resources. A key innovation is the sparse temporal/spatial feature transformation layer, which mitigates edge artifacts and ensures seamless integration of regional features, enhancing the naturalness of the super-resolution outcome. Experimental results demonstrate that our method significantly boosts VSR efficiency while maintaining effectiveness. This marks a notable advancement in streaming video technology, optimizing video quality with reduced computational demands. This approach, featuring semantic segmentation, spatial frequency analysis, and an incremental network structure, represents a substantial improvement over traditional VSR methodologies, addressing the core challenges of efficiency and quality in high-resolution video streaming. [ABSTRACT FROM AUTHOR]

Human action recognition analyzes the behavior in a scene according to the spatiotemporal features carried in image sequences. Existing works suffers from ineffective spatial–temporal feature learning. For short video sequence, the critical challenge is to extract informative spatiotemporal features from a limited-length video. For long video sequences, combining long-range contextual information can improve recognition performance. However, conventional methods primarily consider modeling the action's spatiotemporal features along a single direction, which is difficult to consider context information and ignores the information from the opposite direction. This article proposes a bi-directional network to simulate the bi-directional Long Short-Term Memory (Bi-LSTM) processing of time series data. Specifically, two 3D Convolutional Neural Networks (3D CNNs) extract spatiotemporal features along the forward and backward image sequence of action for each modality individually. After integrating the features of each branch, a dynamic-fusion strategy is applied to obtain a video-level prediction. We conducted comprehensive experiments on the action recognition dataset UCF101 and HMDB51 and achieved 98.0% and 81.4% recognition accuracy, respectively, with a reduction of three quarters of the inputting RGB images. [ABSTRACT FROM AUTHOR]

Most of existing inpainting techniques require to know beforehandwhere those damaged pixels are, i.e., non-blind inpainting methods. However, in many applications, such information may not be readily available. In this paper, we propose a novel blind inpainting method based on a fully convolutional neural network. We term this method as blind inpainting convolutional neural network (BICNN). It purely cascades three convolutional layers to directly learn an end-to-end mapping between a pre-acquired dataset of corrupted/ground truth subimage pairs. Stochastic gradient descent with standard backpropagation is used to train the BICNN. Once the BICNN is learned, it can automatically identify and remove the corrupting patterns from a corrupted image without knowing the specific regions. The learned BICNN takes a corrupted image of any size as input and directly produces a clean output by only one pass of forward propagation. Experimental results indicate that the proposed method can achieve a better inpainting performance than the existing inpainting methods for various corrupting patterns. [ABSTRACT FROM AUTHOR]

Deep supervised hashing has turned up to unravel many large-scale image retrieval challenges. Although deep supervised hashing accomplishes good results for image retrieval process, requisite for further improving the retrieval accuracy always remains the primal focus of interest. In Deep hashing methods, feature representation happens at the outset of the fully connected (FC) layers, causing shortage of spatial information owing to its global nature, whereas deeper pooling layers preserve semantically similar information by retaining the images spatial information, which can result in uplifting the retrieval performance. Hereby, for enhancing the image retrieval accuracy through exploring spatial information, a novel way of deep supervised hashing based on Pooled Feature map (DSHPoolF) is proposed to generate compact hash codes that explore the spatial information by weighing the informative Feature maps from the last pooling layer. This is achieved, firstly, by weighing the last pooling layers Feature map in two ways, namely average–max-based pooling and probability-based pooling strategies. Secondly, informative Feature maps are selected with the help of the weights. In addition to this, the informative Feature maps play a key role in optimizing quantization error together with the loss function and classification errors in a single-step, point-wise ranking manner. This proposed DSHPoolF method is assessed using three datasets (CIFAR-10, MNIST and ImageNet) that unveils primitive outcome in comparison with other existing prominent hash-based methods. [ABSTRACT FROM AUTHOR]

At present, occlusion and similar appearance pose serious challenges to the task of person re-identification. In this work, we propose an efficient multi-scale channel attention network (EMCA) to learn robust and more discriminative features to solve these problems. Specifically, we designed a novel cross-channel attention module (CCAM) in EMCA and placed it after different layers in the backbone. The CCAM includes local cross-channel interaction (LCI) and channel weight integration (CWI). LCI focuses on both the maximum pooling features and the average pooling features to generate channel weights through convolutional layers, respectively. CWI combines the two channel weights to generate richer and more discriminant channel weights. Experiments on four popular person Re-ID datasets (Market-1501, DukeMTMC-ReID, CUHK-03 (detected) and MSMT17) show that the performance of our EMCA is consistently significantly superior to the existing state-of-the-art methods. [ABSTRACT FROM AUTHOR]

In recent years, single-image super-resolution (SISR) methods based on the attention mechanism have been widely explored and achieved remarkable performances. However, most existing networks only explore channel correlations or the spatial long-distance dependences in a single scale while ignoring the mutual guidance of multi-scale information, resulting in the loss of high-frequency information in the reconstructed image. To address this issue, we propose a multi-feature self-attention super-resolution network (MFSN) to embed multi-scale encoding information into the attention mechanism. Specifically, the network consists of a shallow feature extraction subnetwork, a multi-feature alignment subnetwork (MFAN) and a reconstruction subnetwork. The MFAN is composed of an adjacent feature alignment residual block (AFAB) and a dense backward fusion block (DBFB), where AFAB explores multi-scale encoding information using the low-resolution space statistics with larger receptive fields to weight and align the original-scale feature map, so as to extract more discriminative high-frequency features adaptively. Meanwhile, the contrast-aware channel attention module adopts contrast pooling that is more suitable for low-level computer vision tasks to realize the adaptive selection of channel feature in the AFAB. Structurally, the DBFB adopts the backward fusion mechanism to fuse the output of each AFAB, from the deep layer to the shallow layer to make full use of the hierarchical features. Experimental results demonstrate the superiority of our MFSN network in terms of both quantitative metrics and visual quality. [ABSTRACT FROM AUTHOR]

During the last years, many advances have been made in tasks like 3D model retrieval, 3D model classification, and 3D model segmentation. The typical 3D representations such as point clouds, voxels, and polygon meshes are mostly suitable for rendering purposes, while their use for cognitive processes (retrieval, classification, segmentation) is limited due to their high redundancy and complexity. We propose a deep learning architecture to handle 3D models represented as sets of image views as input. Our proposed architecture combines other standard architectures, like Convolutional Neural Networks and autoencoders, for computing 3D model embeddings using sets of image views extracted from the 3D models, avoiding the common view pooling layer approach used in these cases. Our goal is to represent a 3D model as a vector with enough information so it can substitute the 3D model for high-level tasks. Since this vector is a learned representation which tries to capture the relevant information of a 3D model, we show that the embedding representation conveys semantic information that helps to deal with the similarity assessment of 3D objects. We compare our proposed embedding technique with state-of-the-art techniques for 3D Model Retrieval using the ShapeNet and ModelNet datasets. We show that the embeddings obtained with our proposed architecture allow us to obtain a high effectiveness score in both normalized and perturbed versions of the ShapeNet dataset while improving the training and inference times compared to the standard state-of-the-art techniques. [ABSTRACT FROM AUTHOR]

A novel two-task learning method based on an improved convolutional neural network (CNN) using the idea of parameter transfer in transfer learning is proposed, aiming at the problem that a traditional convolutional neural network cannot simultaneously classify two attributes of e-commerce images. The network designed in this method has two channels, and each channel is responsible for learning a unique attribute of the image. First, the network is pre-trained by the channel corresponding to the most important attribute in the image, and the former network parameters are optimized. Then, two channels are used to train the network simultaneously. In the training process, the two learning tasks help each other by sharing parameters, which improves the convergence speed of the network and the generalization ability of the model. Aiming at the problem that there are fewer specific types of e-commerce images in datasets and the problem of class imbalance exists, a method of over-sampling based on the mix-up algorithm is proposed. The relationship between the complexity of the two attributes and the sparse rate of the CNN output feature matrix is studied, and the improved Grad-CAM algorithm is used to visualize and analyze the key areas for classification of two attributes, which improves the interpretability of the network. Experiments show that the proposed CNN method has good classification effect for two-attribute e-commerce images and traditional images. [ABSTRACT FROM AUTHOR]

Artistic media play an important role in recognizing and classifying artworks in many artwork classification works and public artwork databases. We employ deep CNN structure to recognize artistic media from artworks and to classify them into predetermined categories. For this purpose, we define basic artistic media as oilpaint brush, pastel, pencil and watercolor and build artwork image dataset by collecting artwork images from various websites. To build our classifier, we implement various recent deep CNN structures and compare their performances. Among them, we select DenseNet, which shows best performance for recognizing artistic media. Through the human baseline experiment, we show that the performance of our classifier is compatible with that of trained human. Furthermore, we also show that our classifier shows a similar recognition and classification pattern with human in terms of well-classifying media, ill-classifying media, confusing pair and confusing case. We also collect synthesized oilpaint artwork images from fourteen important oilpaint literatures and apply them to our classifier. Our classifier shows a meaningful performance, which will lead to an evaluation scheme for the artistic media simulation techniques of non-photorealistic rendering (NPR) society. [ABSTRACT FROM AUTHOR]

In recent years, with the continuous development of deep learning technology, image super-resolution (SR) has witnessed tremendous progress. Transformer as an emerging deep learning model, its application in the field of image SR is also gradually attracted the attention of researchers. In this paper, we propose parallel coupled convolutional transformer (PCCFormer) architecture for image SR. The proposed design uses transformer and convolutional neural network (CNN) as the structure of the network, thus leveraging the complementary benefits. We use parallel attention transformer to improve feature expression ability of the model. We introduce adaptive convolution residual block to effectively extract the image local information. We introduce a novel concurrent module to aggregate enriched features, including long-range features from transformer and local features captured by CNN. Extensive experiments conducted on benchmark datasets illustrate that PCCFormer surpasses existing image SR methods. For example, our PCCFormer achieves the PSNR of 34.01 dB on Urban100 dataset with scale factor 2. [ABSTRACT FROM AUTHOR]

Recently, deep learning methods have been widely applied to single image super-resolution reconstruction tasks and achieved great improvement in both quantitative and qualitative evaluations. However, most of the existing convolutional neural network-based methods commonly reduce the number of layers or channels to obtain lightweight model. These strategies may reduce the representation ability of informative features and degrade the network performance. To address this issue, we propose a sparse lightweight attention network (SLAN), a novel SISR algorithm to keep informative features between layers. Specially, a sparse attention feature fusion module with lightweight attention and sparse extracting modules is developed to expand feature extracting receptive field and enhance the informative feature extracting ability. To take advantage of the multi-level features and keep fewer multi-adds, cross fusion is used and demonstrated usefully. Extensive experimental results on public datasets demonstrate the superior performance of our proposed SLAN. The average PSNR(dB)/SSIM values of SLAN are about 0.04/0.0004, 0.06/0.0009, and 0.09/0.0018 larger than the competitors under scaling factors of × 2, × 3 and × 4, respectively. Our SLAN benefits from its model size and low computation cost and may be deployed on mobile platforms. [ABSTRACT FROM AUTHOR]